Better lossless compression than PNG with a simpler algorithm

Related tags

Compression Zpng
Overview

Zpng

Small experimental lossless photographic image compression library with a C API and command-line interface.

It's much faster than PNG and compresses better for photographic images. This compressor often takes less than 6% of the time of a PNG compressor and produces a file that is 66% of the size. It was written in just 500 lines of C code thanks to Facebook's Zstd library.

The goal was to see if I could create a better lossless compressor than PNG in just one evening (a few hours) using Zstd and some past experience writing my GCIF library. Zstd is magical.

I'm not expecting anyone else to use this, but feel free if you need some fast compression in just a few hundred lines of C code.

Example results

$ ./ZpngApp.exe -c IMG_0008.jpg IMG_0008.zpng
Compressing IMG_0008.jpg to IMG_0008.zpng
Loaded IMG_0008.jpg in 338.712 msec
Compressed ZPNG in 248.737 msec
ZPNG compression size: 27731886 bytes

$ ./ZpngApp.exe -d IMG_0008.zpng IMG_0008.png
Decompressing IMG_0008.zpng to IMG_0008.png (output will be PNG format)
Decompressed ZPNG in 123.028 msec
Compressed PNG in 4339.82 msec
Wrote decompressed PNG file: IMG_0008.png

$ ll IMG_0008.*
-rw-r--r-- 1 leon 197121 13991058 Jun 25  2017 IMG_0008.jpg
-rw-r--r-- 1 leon 197121 41897485 May  2 22:29 IMG_0008.png
-rw-r--r-- 1 leon 197121 27731886 May  2 22:29 IMG_0008.zpng

FLIF and other formats get better compression ratios but you will wait like 20 seconds for them to finish.

This compressor runs faster than some JPEG decoders! This compressor takes less than 6% of the time of the PNG compressor and produces a file that is 66% of the size.

How it works

This library is similar to PNG in that the image is first filtered, and then submitted to a data compressor. The filtering step is a bit simpler and faster but somehow more effective than the one used in PNG. The data compressor used is Zstd, which makes it significantly faster than PNG to compress and decompress.

Filtering:

(1) Reversible color channel transformation. (2) Split each color channel into a separate color plane. (3) Subtract each color value from the one to its left.

This kind of filtering works great for large photographic images and is very fast.

Experimental results

I ran a few experiments to arrive at this simple codec:

Interleaving is a 1% compression win, and a 0.3% performance win: Not used.

Splitting the data into blocks of 4 at a time actually reduces compression.

No filtering:
Total size = 2629842851
Total compress time = 12942621 usec

Subtract only:
Total size = 1570514796
Total compress time = 16961469 usec

Color filter, followed by subtract:
Total size = 1514724952
Total compress time = 16554638 usec

Subtract, followed by color filter:
Total size = 1514724952
Total compress time = 16376380 usec
Total de-compress time = 6511436 usec
Notes: Order of applying filter does not matter but this way is faster.

Subtract, followed by color filter YUVr from JPEG2000:
Total size = 1506802640
Total compress time = 17169743 usec
Total de-compress time = 7107897 usec
Note: Only 0.5% better compression ratio in trade for performance impact.

Subtract, followed by color filter, splitting into YUV color planes:
Total size = 1486938616
Total compress time = 14514563 usec
Total de-compress time = 6596546 usec
Note: Huge improvement!  Let's call this Zpng!

Credits

Software by Christopher A. Taylor [email protected]

Please reach out if you need support or would like to collaborate on a project.

Issues
  • Is it portable?

    Is it portable?

    I have a deeply rooted distrust to code which uses raw integer types instead of fixed with types as defined in stdint.h. At best, this way of providing code only causes compiler warnings (Example: the size of the buffer is unsigned in the code, but should be size_t, which is usually typedef'd as unsigned long long). My distrust is so deep, that I am compulsed to report the lack of using proper primitive types as an issue.

    opened by ruffianeo 1
  • Try out some other prediction and color filter schemes

    Try out some other prediction and color filter schemes

    Examples:

    https://gist.github.com/kajott/6771de6dfca0255039103e9c4c26e034 https://github.com/catid/gcif/blob/master/decoder/Filters.cpp etc

    Maybe something will be better and about as fast?

    opened by catid 0
Owner
Chris Taylor
Focus on erasure correction coding (ECC/FEC), cryptography, networking libraries, lossless image compression, Azure Kinect
Chris Taylor
Extremely Fast Compression algorithm

LZ4 - Extremely fast compression LZ4 is lossless compression algorithm, providing compression speed > 500 MB/s per core, scalable with multi-cores CPU

lz4 7.4k Aug 6, 2022
Zstandard - Fast real-time compression algorithm

Zstandard, or zstd as short version, is a fast lossless compression algorithm, targeting real-time compression scenarios at zlib-level and better comp

Facebook 17.4k Aug 8, 2022
A simple C library implementing the compression algorithm for isosceles triangles.

orvaenting Summary A simple C library implementing the compression algorithm for isosceles triangles. License This project's license is GPL 2 (as of J

Kevin Matthes 0 Apr 1, 2022
PNGFuse is a cross-platform application that allows you to embed and extract full zlib-compressed files within PNG metadata.

PNGFuse PNGFuse is a portable, lightweight, and cross-platform application written in C++ that allows you to embed and extract full zlib-compressed fi

Eta 3 Dec 29, 2021
Converts common image formats (PNG, JPG, etc.) to GPU-native compressed (BCn, ETC, ASTC) in KTX containers.

Converts common image formats (PNG, JPG, etc.) to GPU-native compressed (BCn, ETC, ASTC) in KTX containers.

Cort 138 Jun 12, 2022
7Zip Windows Client , re-build with better icons.

7zip-BetterIcon 7Zip Windows Client (https://www.7-zip.org ) , re-build with better icons. How to Build Use the Visual Studio 2017 Community or Visual

null 16 Jul 20, 2022
Brotli compression format

SECURITY NOTE Please consider updating brotli to version 1.0.9 (latest). Version 1.0.9 contains a fix to "integer overflow" problem. This happens when

Google 11.4k Aug 13, 2022
Multi-format archive and compression library

Welcome to libarchive! The libarchive project develops a portable, efficient C library that can read and write streaming archives in a variety of form

null 1.8k Aug 13, 2022
LZFSE compression library and command line tool

LZFSE This is a reference C implementation of the LZFSE compressor introduced in the Compression library with OS X 10.11 and iOS 9. LZFSE is a Lempel-

null 1.7k Jul 30, 2022
Small strings compression library

SMAZ - compression for very small strings ----------------------------------------- Smaz is a simple compression library suitable for compressing ver

Salvatore Sanfilippo 1k Aug 11, 2022
A massively spiffy yet delicately unobtrusive compression library.

ZLIB DATA COMPRESSION LIBRARY zlib 1.2.11 is a general purpose data compression library. All the code is thread safe. The data format used by the z

Mark Adler 3.7k Aug 12, 2022
A bespoke sample compression codec for 64k intros

pulsejet A bespoke sample compression codec for 64K intros codec pulsejet lifts a lot of ideas from Opus, and more specifically, its CELT layer, which

logicoma 34 Jul 25, 2022
A variation CredBandit that uses compression to reduce the size of the data that must be trasnmitted.

compressedCredBandit compressedCredBandit is a modified version of anthemtotheego's proof of concept Beacon Object File (BOF). This version does all t

Conor Richard 17 Apr 9, 2022
Data compression utility for minimalist demoscene programs.

bzpack Bzpack is a data compression utility which targets retrocomputing and demoscene enthusiasts. Given the artificially imposed size limits on prog

Milos Bazelides 20 Jul 27, 2022
gzip (GNU zip) is a compression utility designed to be a replacement for 'compress'

gzip (GNU zip) is a compression utility designed to be a replacement for 'compress'

ACM at UCLA 7 Apr 27, 2022
Advanced DXTc texture compression and transcoding library

crunch/crnlib v1.04 - Advanced DXTn texture compression library Public Domain - Please see license.txt. Portions of this software make use of public d

null 748 Aug 10, 2022
Analysing and implementation of lossless data compression techniques like Huffman encoding and LZW was conducted along with JPEG lossy compression technique based on discrete cosine transform (DCT) for Image compression.

PROJECT FILE COMPRESSION ALGORITHMS - Huffman compression LZW compression DCT Aim of the project - Implement above mentioned compression algorithms an

null 1 Dec 14, 2021
Like feh, but better, faster, more image formats, simpler, more lightweight, animation support, and better UI

Like feh, but better, faster, more image formats, simpler, more lightweight, animation support, and better UI

martin 10 Jul 27, 2022
Lossless data compression codec with LZMA-like ratios but 1.5x-8x faster decompression speed, C/C++

LZHAM - Lossless Data Compression Codec Public Domain (see LICENSE) LZHAM is a lossless data compression codec written in C/C++ (specifically C++03),

Rich Geldreich 628 Jul 28, 2022
The “Quite OK Image” format for fast, lossless image compression

The “Quite OK Image” format for fast, lossless image compression

Dominic Szablewski 5.7k Aug 6, 2022
QOY - The "Quite OK YCbCr420A" format for fast, lossless image compression

QOY - The "Quite OK YCbCr420A" format for fast, lossless* image compression ( * colorspace conversion to/from RGBA is lossy, if used ) Single-file MIT

Chainfire 18 May 27, 2022
PyFLAC - Real-time lossless audio compression in Python

A simple Pythonic interface for libFLAC. FLAC stands for Free Lossless Audio Codec, an audio format similar to MP3, but lossless, meaning that audio i

Sonos, Inc. 92 Aug 4, 2022
A C++ data container replicating std::stack functionality but with better performance than standard library containers in a stack context.

plf::stack A data container replicating std::stack functionality but with better performance than standard library containers in a stack context. C++9

Matt Bentley 48 Jul 24, 2022
Hello, I am creating this file to make everyone understand the basis of C++ language which is actually the advanced version of C but better than C because of its OOPs feature.

Hello-in-C++ ?? ?? FOR BEGINNERS IN C++ Hello, I am creating this file to make everyone understand the basics of C++ language which is actually the ad

Ankita Mohan 2 Dec 27, 2021
Turn images into text better than caca/aalib

hiptext hiptext is command line tool for rendering images and videos inside terminals. Dependencies You need clang or gcc >=4.7. You also need to inst

Justine Tunney 668 Aug 10, 2022
Phan Sang 8 Jul 30, 2022
An optimized "RTOS" (more than HAL but less than RTOS) for ROV controling and getting sensor data

Nitori-ROV-OS 一个专门为水下机器人(ROV、AUV)进行优化的实时操作系统,暂命名为 Nitori,中文名 荷取 可以通过修改硬件兼容层(Port)进行移植 预计最初版本支持stm32f407和stm32h750,并在实验室目前的水下机器人中进行部署 系统分为四层,六个主要组件: 硬件

Doublues_G 2 Jan 10, 2022
Fast streaming PNG<->QOI converter with some compression-improving extensions

QOIG Fast streaming PNG<->QOI converter with some compression-improving extensions. Can achieve 1%-10% better compression than QOI without sacrificing

David Rutter 4 May 31, 2022
Shamir’s Secret Sharing Algorithm: Shamir’s Secret Sharing is an algorithm in cryptography created by Adi Shamir. The main aim of this algorithm is to divide secret that needs to be encrypted into various unique parts.

Shamir-s-Secret-Sharing-Algorithm-Cryptography Shamir’s Secret Sharing Algorithm: Shamir’s Secret Sharing is an algorithm in cryptography created by A

Pavan Ananth Sharma 5 Jul 5, 2022