Better lossless compression than PNG with a simpler algorithm

Last update: Dec 23, 2022

Related tags

Overview

Zpng

Small experimental lossless photographic image compression library with a C API and command-line interface.

It's much faster than PNG and compresses better for photographic images. This compressor often takes less than 6% of the time of a PNG compressor and produces a file that is 66% of the size. It was written in just 500 lines of C code thanks to Facebook's Zstd library.

The goal was to see if I could create a better lossless compressor than PNG in just one evening (a few hours) using Zstd and some past experience writing my GCIF library. Zstd is magical.

I'm not expecting anyone else to use this, but feel free if you need some fast compression in just a few hundred lines of C code.

Example results

$ ./ZpngApp.exe -c IMG_0008.jpg IMG_0008.zpng
Compressing IMG_0008.jpg to IMG_0008.zpng
Loaded IMG_0008.jpg in 338.712 msec
Compressed ZPNG in 248.737 msec
ZPNG compression size: 27731886 bytes

$ ./ZpngApp.exe -d IMG_0008.zpng IMG_0008.png
Decompressing IMG_0008.zpng to IMG_0008.png (output will be PNG format)
Decompressed ZPNG in 123.028 msec
Compressed PNG in 4339.82 msec
Wrote decompressed PNG file: IMG_0008.png

$ ll IMG_0008.*
-rw-r--r-- 1 leon 197121 13991058 Jun 25  2017 IMG_0008.jpg
-rw-r--r-- 1 leon 197121 41897485 May  2 22:29 IMG_0008.png
-rw-r--r-- 1 leon 197121 27731886 May  2 22:29 IMG_0008.zpng

FLIF and other formats get better compression ratios but you will wait like 20 seconds for them to finish.

This compressor runs faster than some JPEG decoders! This compressor takes less than 6% of the time of the PNG compressor and produces a file that is 66% of the size.

How it works

This library is similar to PNG in that the image is first filtered, and then submitted to a data compressor. The filtering step is a bit simpler and faster but somehow more effective than the one used in PNG. The data compressor used is Zstd, which makes it significantly faster than PNG to compress and decompress.

Filtering:

(1) Reversible color channel transformation. (2) Split each color channel into a separate color plane. (3) Subtract each color value from the one to its left.

This kind of filtering works great for large photographic images and is very fast.

Experimental results

I ran a few experiments to arrive at this simple codec:

Interleaving is a 1% compression win, and a 0.3% performance win: Not used.

Splitting the data into blocks of 4 at a time actually reduces compression.

No filtering:
Total size = 2629842851
Total compress time = 12942621 usec

Subtract only:
Total size = 1570514796
Total compress time = 16961469 usec

Color filter, followed by subtract:
Total size = 1514724952
Total compress time = 16554638 usec

Subtract, followed by color filter:
Total size = 1514724952
Total compress time = 16376380 usec
Total de-compress time = 6511436 usec
Notes: Order of applying filter does not matter but this way is faster.

Subtract, followed by color filter YUVr from JPEG2000:
Total size = 1506802640
Total compress time = 17169743 usec
Total de-compress time = 7107897 usec
Note: Only 0.5% better compression ratio in trade for performance impact.

Subtract, followed by color filter, splitting into YUV color planes:
Total size = 1486938616
Total compress time = 14514563 usec
Total de-compress time = 6596546 usec
Note: Huge improvement!  Let's call this Zpng!

Credits

Software by Christopher A. Taylor [email protected]

Please reach out if you need support or would like to collaborate on a project.

A massively spiffy yet delicately unobtrusive compression library.

ZLIB DATA COMPRESSION LIBRARY zlib 1.2.11 is a general purpose data compression library. All the code is thread safe. The data format used by the z

4.1k Dec 30, 2022

A bespoke sample compression codec for 64k intros

pulsejet A bespoke sample compression codec for 64K intros codec pulsejet lifts a lot of ideas from Opus, and more specifically, its CELT layer, which

34 Jul 25, 2022

A variation CredBandit that uses compression to reduce the size of the data that must be trasnmitted.

compressedCredBandit compressedCredBandit is a modified version of anthemtotheego's proof of concept Beacon Object File (BOF). This version does all t

18 Sep 22, 2022

Data compression utility for minimalist demoscene programs.

bzpack Bzpack is a data compression utility which targets retrocomputing and demoscene enthusiasts. Given the artificially imposed size limits on prog

20 Jul 27, 2022

gzip (GNU zip) is a compression utility designed to be a replacement for 'compress'

8 Nov 6, 2022

Advanced DXTc texture compression and transcoding library

crunch/crnlib v1.04 - Advanced DXTn texture compression library Public Domain - Please see license.txt. Portions of this software make use of public d

775 Dec 26, 2022

Analysing and implementation of lossless data compression techniques like Huffman encoding and LZW was conducted along with JPEG lossy compression technique based on discrete cosine transform (DCT) for Image compression.

PROJECT FILE COMPRESSION ALGORITHMS - Huffman compression LZW compression DCT Aim of the project - Implement above mentioned compression algorithms an

1 Dec 14, 2021

Like feh, but better, faster, more image formats, simpler, more lightweight, animation support, and better UI

11 Oct 23, 2022

Lossless data compression codec with LZMA-like ratios but 1.5x-8x faster decompression speed, C/C++

LZHAM - Lossless Data Compression Codec Public Domain (see LICENSE) LZHAM is a lossless data compression codec written in C/C++ (specifically C++03),

641 Dec 22, 2022

The “Quite OK Image” format for fast, lossless image compression

6k Dec 30, 2022

QOY - The "Quite OK YCbCr420A" format for fast, lossless image compression

QOY - The "Quite OK YCbCr420A" format for fast, lossless* image compression ( * colorspace conversion to/from RGBA is lossy, if used ) Single-file MIT

19 Oct 1, 2022

PyFLAC - Real-time lossless audio compression in Python

A simple Pythonic interface for libFLAC. FLAC stands for Free Lossless Audio Codec, an audio format similar to MP3, but lossless, meaning that audio i

97 Dec 9, 2022

A C++ data container replicating std::stack functionality but with better performance than standard library containers in a stack context.

plf::stack A data container replicating std::stack functionality but with better performance than standard library containers in a stack context. C++9

47 Sep 11, 2022

Upgraded from Pixar postcard path tracing, instead of printing Pixar, this program print my name, which pose to be more challenging than the original code. The upgraded is also more readable and run 9x faster than the original source code.

SDF-Sphere-Tracing Upgraded from Pixar postcard path tracing, instead of printing Pixar, this program print my name, which pose to be more challenging

17 Dec 29, 2022

An optimized "RTOS" (more than HAL but less than RTOS) for ROV controling and getting sensor data

Nitori-ROV-OS 一个专门为水下机器人（ROV、AUV）进行优化的实时操作系统，暂命名为 Nitori，中文名荷取可以通过修改硬件兼容层（Port）进行移植预计最初版本支持stm32f407和stm32h750，并在实验室目前的水下机器人中进行部署系统分为四层，六个主要组件：硬件

2 Jan 10, 2022

Hello, I am creating this file to make everyone understand the basis of C++ language which is actually the advanced version of C but better than C because of its OOPs feature.

Hello-in-C++ 😄 😄 FOR BEGINNERS IN C++ Hello, I am creating this file to make everyone understand the basics of C++ language which is actually the ad

2 Dec 27, 2021

Turn images into text better than caca/aalib

hiptext hiptext is command line tool for rendering images and videos inside terminals. Dependencies You need clang or gcc =4.7. You also need to inst

688 Dec 31, 2022

Fast streaming PNG-QOI converter with some compression-improving extensions

QOIG Fast streaming PNG-QOI converter with some compression-improving extensions. Can achieve 1%-10% better compression than QOI without sacrificing

3 Oct 3, 2022

Shamir’s Secret Sharing Algorithm: Shamir’s Secret Sharing is an algorithm in cryptography created by Adi Shamir. The main aim of this algorithm is to divide secret that needs to be encrypted into various unique parts.

Shamir-s-Secret-Sharing-Algorithm-Cryptography Shamir’s Secret Sharing Algorithm: Shamir’s Secret Sharing is an algorithm in cryptography created by A

5 Jul 5, 2022

Comments

Is it portable?

I have a deeply rooted distrust to code which uses raw integer types instead of fixed with types as defined in stdint.h. At best, this way of providing code only causes compiler warnings (Example: the size of the buffer is unsigned in the code, but should be size_t, which is usually typedef'd as unsigned long long). My distrust is so deep, that I am compulsed to report the lack of using proper primitive types as an issue.

opened by ruffianeo 1
Try out some other prediction and color filter schemes

Examples:

https://gist.github.com/kajott/6771de6dfca0255039103e9c4c26e034 https://github.com/catid/gcif/blob/master/decoder/Filters.cpp etc

Maybe something will be better and about as fast?

opened by catid 0

Better lossless compression than PNG with a simpler algorithm

Related tags

Overview

Zpng

Example results

How it works

Experimental results

Credits

You might also like...

A massively spiffy yet delicately unobtrusive compression library.

A bespoke sample compression codec for 64k intros

A variation CredBandit that uses compression to reduce the size of the data that must be trasnmitted.

Data compression utility for minimalist demoscene programs.

gzip (GNU zip) is a compression utility designed to be a replacement for 'compress'

Advanced DXTc texture compression and transcoding library

Analysing and implementation of lossless data compression techniques like Huffman encoding and LZW was conducted along with JPEG lossy compression technique based on discrete cosine transform (DCT) for Image compression.

Like feh, but better, faster, more image formats, simpler, more lightweight, animation support, and better UI

Lossless data compression codec with LZMA-like ratios but 1.5x-8x faster decompression speed, C/C++

The “Quite OK Image” format for fast, lossless image compression

QOY - The "Quite OK YCbCr420A" format for fast, lossless image compression

PyFLAC - Real-time lossless audio compression in Python

A C++ data container replicating std::stack functionality but with better performance than standard library containers in a stack context.

Upgraded from Pixar postcard path tracing, instead of printing Pixar, this program print my name, which pose to be more challenging than the original code. The upgraded is also more readable and run 9x faster than the original source code.

An optimized "RTOS" (more than HAL but less than RTOS) for ROV controling and getting sensor data

Hello, I am creating this file to make everyone understand the basis of C++ language which is actually the advanced version of C but better than C because of its OOPs feature.

Turn images into text better than caca/aalib

Fast streaming PNG-QOI converter with some compression-improving extensions

Shamir’s Secret Sharing Algorithm: Shamir’s Secret Sharing is an algorithm in cryptography created by Adi Shamir. The main aim of this algorithm is to divide secret that needs to be encrypted into various unique parts.

Comments

Is it portable?

Try out some other prediction and color filter schemes

Owner

Chris Taylor

Extremely Fast Compression algorithm

Zstandard - Fast real-time compression algorithm

A simple C library implementing the compression algorithm for isosceles triangles.

PNGFuse is a cross-platform application that allows you to embed and extract full zlib-compressed files within PNG metadata.

Converts common image formats (PNG, JPG, etc.) to GPU-native compressed (BCn, ETC, ASTC) in KTX containers.

7Zip Windows Client , re-build with better icons.

Brotli compression format

Multi-format archive and compression library

LZFSE compression library and command line tool

Small strings compression library