data compression library for embedded/real-time systems

Atomic Object

Last update: Jan 7, 2023

Related tags

Compilers heatshrink

Overview

heatshrink

A data compression/decompression library for embedded/real-time systems.

Key Features:

Low memory usage (as low as 50 bytes) It is useful for some cases with less than 50 bytes, and useful for many general cases with < 300 bytes.
Incremental, bounded CPU use You can chew on input data in arbitrarily tiny bites. This is a useful property in hard real-time environments.
Can use either static or dynamic memory allocation The library doesn't impose any constraints on memory management.
ISC license You can use it freely, even for commercial purposes.

Getting Started:

There is a standalone command-line program, heatshrink, but the encoder and decoder can also be used as libraries, independent of each other. To do so, copy heatshrink_common.h, heatshrink_config.h, and either heatshrink_encoder.c or heatshrink_decoder.c (and their respective header) into your project. For projects that use both, static libraries are built that use static and dynamic allocation.

Dynamic allocation is used by default, but in an embedded context, you probably want to statically allocate the encoder/decoder. Set HEATSHRINK_DYNAMIC_ALLOC to 0 in heatshrink_config.h.

Basic Usage

Allocate a heatshrink_encoder or heatshrink_decoder state machine using their alloc function, or statically allocate one and call their reset function to initialize them. (See below for configuration options.)
Use sink to sink an input buffer into the state machine. The input_size pointer argument will be set to indicate how many bytes of the input buffer were actually consumed. (If 0 bytes were conusmed, the buffer is full.)
Use poll to move output from the state machine into an output buffer. The output_size pointer argument will be set to indicate how many bytes were output, and the function return value will indicate whether further output is available. (The state machine may not output any data until it has received enough input.)

Repeat steps 2 and 3 to stream data through the state machine. Since it's doing data compression, the input and output sizes can vary significantly. Looping will be necessary to buffer the input and output as the data is processed.

When the end of the input stream is reached, call finish to notify the state machine that no more input is available. The return value from finish will indicate whether any output remains. if so, call poll to get more.

Continue calling finish and polling to flush remaining output until finish indicates that the output has been exhausted.

Sinking more data after finish has been called will not work without calling reset on the state machine.

Configuration

heatshrink has a couple configuration options, which impact its resource usage and how effectively it can compress data. These are set when dynamically allocating an encoder or decoder, or in heatshrink_config.h if they are statically allocated.

window_sz2, -w in the CLI: Set the window size to 2^W bytes.

The window size determines how far back in the input can be searched for repeated patterns. A window_sz2 of 8 will only use 256 bytes (2^8), while a window_sz2 of 10 will use 1024 bytes (2^10). The latter uses more memory, but may also compress more effectively by detecting more repetition.

The window_sz2 setting currently must be between 4 and 15.

lookahead_sz2, -l in the CLI: Set the lookahead size to 2^L bytes.

The lookahead size determines the max length for repeated patterns that are found. If the lookahead_sz2 is 4, a 50-byte run of 'a' characters will be represented as several repeated 16-byte patterns (2^4 is 16), whereas a larger lookahead_sz2 may be able to represent it all at once. The number of bits used for the lookahead size is fixed, so an overly large lookahead size can reduce compression by adding unused size bits to small patterns.

The lookahead_sz2 setting currently must be between 3 and the window_sz2 - 1.

input_buffer_size - How large an input buffer to use for the decoder. This impacts how much work the decoder can do in a single step, and a larger buffer will use more memory. An extremely small buffer (say, 1 byte) will add overhead due to lots of suspend/resume function calls, but should not change how well data compresses.

Recommended Defaults

For embedded/low memory contexts, a window_sz2 in the 8 to 10 range is probably a good default, depending on how tight memory is. Smaller or larger window sizes may make better trade-offs in specific circumstances, but should be checked with representative data.

The lookahead_sz2 should probably start near the window_sz2/2, e.g. -w 8 -l 4 or -w 10 -l 5. The command-line program can be used to measure how well test data works with different settings.

More Information and Benchmarks:

heatshrink is based on LZSS, since it's particularly suitable for compression in small amounts of memory. It can use an optional, small index to make compression significantly faster, but otherwise can run in under 100 bytes of memory. The index currently adds 2^(window size+1) bytes to memory usage for compression, and temporarily allocates 512 bytes on the stack during index construction (if the index is enabled).

For more information, see the blog post for an overview, and the heatshrink_encoder.h / heatshrink_decoder.h header files for API documentation.

Build Status

Comments

Fail to decompress buffer

This is really strange, I am able to compress but fail to decompress a "specific" set of data buffer. I am using static linking with following configuration:

#define HEATSHRINK_STATIC_INPUT_BUFFER_SIZE 512
#define HEATSHRINK_STATIC_WINDOW_BITS 10
#define HEATSHRINK_STATIC_LOOKAHEAD_BITS 3

uint8_t compressed[] = {
0x80, 0x80, 0x35, 0x45, 0x01, 0x7B, 0xA9, 0x07, 0x01, 0xF4, 0x01, 0xB0, 0x00, 0xE0, 0xA0, 0x0D, 
0x45, 0x80, 0x5E, 0x82, 0x00, 0x35, 0xA8, 0xB7, 0x4F, 0x7E, 0x10, 0x89, 0x10, 0x03, 0x33, 0x3A, 
0x2C, 0xBA, 0x21, 0x7C, 0x04, 0x87, 0xE1, 0x42, 0xB2, 0x7C, 0xE7, 0xFF, 0x88, 0x70, 0x26, 0x20, 
0x00, 0xC2, 0xE0, 0x70, 0x10, 0x16, 0x82, 0x9F, 0x00, 0x1C, 0x00, 0x70
};

/* Actual input */
uint8_t decompressed[] = {
0x01, 0x00, 0x00, 0x00, 0x45, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xD4, 0x07, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x05, 0x00, 0x00, 0x00, 0x16, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x04, 0x00, 0x00, 0x00, 0xA8, 0x6E, 0x3D, 0xF0, 0x08, 0x22, 0x3D, 0xF0, 0x33, 0x45, 0x2E, 0x10, 0x7C, 0x01, 0x00, 0xF0, 0x42, 0x64, 0xF3, 0x3F, 0xF8, 0x0E, 0x00, 0x05, 0x05, 0x05, 0x0B, 0x03, 0x01, 0x64, 0xF3, 0x3F, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00
};

I have also attached my test application. So problem is:

If I try to "only" decompress the compressed buffer, I run into a never ending loop of poll. However if I first compress the "decompressed" buffer and then decompress in the same application everything works. I am not able to understand why?

to check both usecases, make if 0 @line 202 to 1

test_heatshrink.txt

PS: Rename file to .c

opened by ajaybhargav 10

Static and dynamic versions of heatshrink aren't compatible
The static and dynamic versions of heatshrink don't create the same output.

I had to fix some compilation issues c.f. https://github.com/sjaeckel/heatshrink/tree/fix/static

Now both test targets and heatshrink compile fine. Tests pass.

To reproduce that the created files from static and dynamic versions are not the same:

Build heatshrink in default (dynamic) configuration and compress a file with parameters "-i 32 -w 8", the default settings of the static configuration.

Build heatshrink with HEATSHRINK_DYNAMIC_ALLOC 0 and compress the same file.

Resulting files won't be the same! Is this intentional?
opened by sjaeckel 8
Add option to override heatshrink configuration

Currently its impossible to change heatshrink compression configuration wihout touching the library header files. This patch makes it possible for configuration which can be configured at compilation time without having to touch the library. Which makes it easier to keep it sync with github source without modifying the actual source.

Signed-off-by: Ajay Bhargav [email protected]

opened by ajaybhargav 7
Java library

Hi, thanks for your great work! I would like to decompress the data generated by an embedded system on Java. Have you got a ready-to-use solution? Should i use a standard lzss java class?

Thanks
portability beginner java

opened by cagnulein 6
Possible data loss at stream end in -w4 -l2
I got an excellent bug report from @unixdj with detailed, minimal steps to reproduce data loss at the end of the stream:

$ echo -n aaaa | ./heatshrink -e -w4 -l2 | ./heatshrink -d -w4 -l2 a # should be "aaaa"

A regression test has been added.

This is not currently known to occur under any other combinations of settings except -w4 -l2, the absolute minimum.

A change that fixes this without breaking reverse compatibility would be strongly preferred.
bug
opened by silentbicycle 6
Compiling for ESP8266 fails

Hey,

I've just tried to use the library on a ESP8266 platform. Sadly there are several errors displayed in heatshrink.c:

Error GCC invalid conversion from 'void*' to 'io_handle*' [-fpermissive] 121:37 Error GCC 'heatshrink_encoder_alloc' was not declared in this scope 282:86 Error GCC 'heatshrink_encoder_free' was not declared in this scope 305:32 Error GCC 'heatshrink_decoder_alloc' was not declared in this scope 352:39 Error GCC 'heatshrink_decoder_free' was not declared in this scope 382:32

Any help is appreciated :)

opened by dominikfehr 5

Fixed fallthrough compiler warning

Fixed fallthrough warnings thrown in GCC 7.1.1:

heatshrink_encoder.c: In function ‘heatshrink_encoder_poll’:
heatshrink_encoder.c:237:24: warning: this statement may fall through [-Wimplicit-fallthrough=]
             hse->state = st_flush_bit_buffer(hse, &oi);
             ~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
heatshrink_encoder.c:238:9: note: here
         case HSES_DONE:


heatshrink.c: In function ‘proc_args’:
heatshrink.c:411:13: warning: this statement may fall through [-Wimplicit-fallthrough=]
             usage();
             ^~~~~~~
heatshrink.c:412:9: note: here
         case 'e':               /* encode */

opened by tobhe 5

Make heatshrink compile with std=c99 and using getopt().

The current target 'make heatshrink' results in "warning: implicit declaration of function ‘getopt’" and undeclared variable errors. The -std=c99 prevents <unistd.h> from including <getopt.h> which makes the compile fail.

Including getopt.h additionally resolves this issue (as -std=gnu99 or -D_GNU_SOURCE is not a nice option).

opened by turbocode 5
unable to decode the compressed file based on command tool

./test -e -v acc1.txt acc1.z acc1.txt 70.92 % 27301 -> 7940 (-w 11 -l 4)

./test -d -v acc1.z acc1.unz ... -- popping 4 bit(s) -- pulled byte 0x49 -- accumulated 00000004 -- backref count (lsb), got 0x0004 (+1) Segmentation fault: 11

opened by mira67 5

heatshrink binary only decompresses the first block

After patching Heatshrink with the changes in #7 and building with MinGW (gcc 4.7.2), a valid "heatshrink.exe" executable is produced, but this binary fails to decompress data correctly.

$ ./heatshrink.exe -ev LICENSE LICENSE.enc

$ ./heatshrink.exe -dv LICENSE.enc LICENSE.dec

$ md5sum LICENSE*
cd12e61f206f9ee1a6622d0df2f773d1 *LICENSE
e7e84c0c9268751994a1830f9b98d399 *LICENSE.dec
81239e24ddf1a6dde9d5cff95d186911 *LICENSE.enc

$ ls -l LICENSE*
-rw-r--r-- 1 ppemberton Administrators 784 Dec 12 15:11 LICENSE
-rw-r--r-- 1 ppemberton Administrators 387 Dec 22 11:41 LICENSE.dec
-rw-r--r-- 1 ppemberton Administrators 681 Dec 22 11:41 LICENSE.enc

It appears that the decode step is only processing the first block of data in the file.

Interestingly the byte counts in verbose mode are wrong too:

$ ./heatshrink.exe -ev LICENSE LICENSE.enc
LICENSE 14.29 %  784 -> 672 (-w 11 -l 4)

$ ./heatshrink.exe -dv LICENSE.enc LICENSE.dec
LICENSE.enc -3.27 %      367 -> 379 (-w 11 -l 4)

opened by philpem 4

Decoder for JavaScript?

I would like to encode with heatsink a bitmap image data on a micro-controller and display it in the browser.
Therefore my question is: is there a JavaScript implementation of the decoder?

opened by slaff 3
heatshrink_decoder_poll return HSDR_POLL_MORE if out buffer is exactly sized as the decoded result

in this example output buffer is sized exactly as the input buffer: program remain il loop. if i resize the output buffer as sizeof(data)+1 it work, and leftover 1 byte

` const char data[] = "Cappuccetto Rosso, chiamata anche Cappuccetto, e' una bambina che vive con la sua mamma in una casetta vicino al bosco. Un giorno la mamma le consegna un cestino pieno di cose buone da portare alla nonna malata, che vive al di la' della foresta. La mamma "; //256byte data

char compress[1500]; char out[sizeof(data)];

int Encode() { heatshrink_encoder hse; //512byte in stack heatshrink_encoder_reset(&hse); size_t copied; size_t tot_copied = 0; const char* in=data; size_t remaining = sizeof(data); size_t srclen = remaining; size_t readed; do { heatshrink_encoder_sink(&hse, (uint8_t*)in, remaining, &readed); in += readed; remaining -= readed; //printf("readed:%lu\n",readed); heatshrink_encoder_poll(&hse, (uint8_t*)compress + tot_copied, sizeof(compress)- tot_copied,&copied); tot_copied += copied; //printf("copied:%lu tot:%lu\n",copied,tot_copied); } while (remaining); heatshrink_encoder_finish(&hse); HSE_poll_res pres; do { pres = heatshrink_encoder_poll(&hse, (uint8_t*)compress + tot_copied, sizeof(compress)- tot_copied,&copied); tot_copied += copied; } while (pres != HSER_POLL_EMPTY); Uart_Printf(">>>>>>>>>>>>src:%lu dst:%lu\n", srclen, tot_copied); return tot_copied; }

int Decode(size_t remaining) { HSD_sink_res sres; HSD_poll_res pres; HSD_finish_res fres; heatshrink_decoder hsd; heatshrink_decoder_reset(&hsd); size_t tot_copied = 0; size_t readed; size_t copied; const uint8_t* in = (const uint8_t*)compress; do { sres = heatshrink_decoder_sink(&hsd, in, remaining, &readed); in += readed; remaining -= readed; //printf("readed:%lu\n",readed); pres =heatshrink_decoder_poll(&hsd, (uint8_t*)out + tot_copied,sizeof(out)- tot_copied, &copied); tot_copied += copied; //printf("copied:%lu tot:%lu\n",copied,tot_copied); } while (remaining); fres = heatshrink_decoder_finish(&hsd); do //REMAIN ON THIS LOOP FOREVER !!! { pres =heatshrink_decoder_poll(&hsd, (uint8_t*)out + tot_copied, sizeof(out) - tot_copied, &copied); tot_copied += copied; //printf("src:%lu dst:%lu\n",srclen,tot_copied); } while (pres != HSER_POLL_EMPTY); Uart_Printf(">>>>>>>>>>>>decompress:%lu\n", tot_copied); return tot_copied; }

void main() { SysTimer_Init(); Uart_Init(Uart_STDOUT, 0); //console int n=Encode(); Decode(n); Uart_Println(out); Uart_Println("end"); while (1); } `

opened by IvanoBono 4
Fix indexing for platforms where char size is larger than a byte

On platforms where the minimum memory access size is larger than a byte (case in point, μ'nSP where each address refers to one 16-bit word rather than a byte), indexing would never work due to the index buffer being initialized improperly. Instead of each element being initialized to -1 as would typically happen on a system with byte sized access, it gets initialized to 255. This breaks the indexing logic and causes it to go into an infinite loop.

This patch changes the initializer from 0xff to -1, so depending on the platform it should always initialize the entire array with -1. On byte access systems, memset() will truncate the -1 to 0xff as per standard, which results in the same desired behavior.

opened by GMMan 1

Usage simplified if heatshrink_encoder_poll() returns HSER_POLL_MORE when finish flag is set and output buffer is full

heatshrink_encoder_poll() is returning 'HSER_POLL_EMPTY' rather than 'HSER_POLL_MORE' when finishing and the output buffer is full. Is a break missing from the 'HSES_FLUSH_BITS' case? As implemented, it is dropping through to the 'HSES_DONE' case and returning 'HSER_POLL_EMPTY' rather than running through the 'HSER_POLL_MORE' check?

    case HSES_FLUSH_BITS:
        hse->state = st_flush_bit_buffer(hse, &oi);
    case HSES_DONE:
        return HSER_POLL_EMPTY;
    default:
        LOG("-- bad state %s\n", state_names[hse->state]);
        return HSER_POLL_ERROR_MISUSE;
    }

    if (hse->state == in_state) {
        /* Check if output buffer is exhausted. */
        if (*output_size == out_buf_size) return HSER_POLL_MORE;
    }

Logging without break:

heatshrink_encoder_poll()
(12:59:37.602) (>>) -- polling, state 1 (filled), flags 0x01<LF>
...
(13:00:37.174) (>>) -- polling, state 2 (search), flags 0x01<LF>
(13:00:37.174) (>>) ## step_search, scan @ +963 (1927/2048), input size 964<LF>
(13:00:37.199) (>>) -- scanning for match of buf[1987:1988] between buf[963:1987] (max 1 bytes)<LF>
(13:00:37.199) (>>) -- none found<LF>
(13:00:37.230) (>>) ss Match not found<LF>
(13:00:37.230) (>>) -- polling, state 3 (yield_tag_bit), flags 0x01<LF>
(13:00:37.230) (>>) -- adding tag bit: 1<LF>
(13:00:37.230) (>>) ++ push_bits: 1 bits, input of 0x01<LF>
(13:00:37.293) (>>) -- polling, state 4 (yield_literal), flags 0x01<LF>
(13:00:37.293) (>>) -- yielded literal byte 0x44 ('D') from +1987<LF>
(13:00:37.293) (>>) ++ push_bits: 8 bits, input of 0x44<LF>
(13:00:37.293) (>>)  > pushing byte 0x75<LF>
(13:00:37.293) (>>) -- polling, state 2 (search), flags 0x01<LF>
(13:00:37.293) (>>) ## step_search, scan @ +964 (1928/2048), input size 964<LF>
(13:00:37.323) (>>) -- end of search @ 964<LF>
(13:00:37.323) (>>) -- polling, state 8 (flush_bits), flags 0x01<LF>
returns HSER_POLL_EMPTY

I observe the desired behaviour with:

    case HSES_FLUSH_BITS:
        hse->state = st_flush_bit_buffer(hse, &oi);
        break;

Logging with break:

heatshrink_encoder_poll()
(13:15:58.567) (>>) -- polling, state 1 (filled), flags 0x01<LF>
...
(13:17:46.218) (>>) -- polling, state 2 (search), flags 0x01<LF>
(13:17:46.218) (>>) ## step_search, scan @ +963 (1927/2048), input size 964<LF>
(13:17:46.218) (>>) -- scanning for match of buf[1987:1988] between buf[963:1987] (max 1 bytes)<LF>
(13:17:46.227) (>>) -- none found<LF>
(13:17:46.227) (>>) ss Match not found<LF>
(13:17:46.227) (>>) -- polling, state 3 (yield_tag_bit), flags 0x01<LF>
(13:17:46.227) (>>) -- adding tag bit: 1<LF>
(13:17:46.258) (>>) ++ push_bits: 1 bits, input of 0x01<LF>
(13:17:46.258) (>>) -- polling, state 4 (yield_literal), flags 0x01<LF>
(13:17:46.258) (>>) -- yielded literal byte 0x44 ('D') from +1987<LF>
(13:17:46.289) (>>) ++ push_bits: 8 bits, input of 0x44<LF>
(13:17:46.289) (>>)  > pushing byte 0x75<LF>
(13:17:46.289) (>>) -- polling, state 2 (search), flags 0x01<LF>
(13:17:46.289) (>>) ## step_search, scan @ +964 (1928/2048), input size 964<LF>
(13:17:46.362) (>>) -- end of search @ 964<LF>
(13:17:46.362) (>>) -- polling, state 8 (flush_bits), flags 0x01<LF>
returns HSDR_POLL_MORE

heatshrink_encoder_poll()
(13:17:46.362) (>>) -- polling, state 8 (flush_bits), flags 0x01<LF>
(13:17:46.362) (>>) -- flushing remaining byte (bit_index == 0x02)<LF>
(13:17:46.382) (>>) -- done!<LF>
(13:17:46.382) (>>) -- polling, state 9 (done), flags 0x01<LF>
returns HSER_POLL_EMPTY

This behaviour is not seen in the example usage, since heatshrink_decoder_poll is also called until 'poll_sz == 0', rather than just until pres != HSDR_POLL_MORE.

If heatshrink_encoder_poll() is returns 'HSER_POLL_MORE' when finishing and the output buffer is full, the usage can be simplified by only needing to call heatshrink_decoder_finish() once to set the finish flag. If it returns 'HSDR_FINISH_MORE', keep calling heatshrink_encoder_poll() until it returns 'HSER_POLL_EMPTY' i.e. same as when sinking.

Pseudo code without error handling:

do
{
	if (input remaining)
	{
		heatshrink_encoder_sink()
		update input remaining
	}
	
	if (no input remaining)
	{
		fres = heatshrink_encoder_finish()
		if (fres == HSER_FINISH_DONE)
		{
			return
		}
	}
	
	do 
	{
		pres = heatshrink_encoder_poll()
		write output
	} while (pres == HSER_POLL_MORE);
} while (input remaining)

I believe the encoder example in the following pull request has an issue without changing the fall-through to a break in the 'HSES_FLUSH_BITS' case, as this is what I based my implementation on i.e. a single call to heatshrink_encoder_finish(): https://github.com/atomicobject/heatshrink/pull/54/commits

opened by James-NZ 1

heatshrink_encoder_sink doesn't behave as documented

Hi,

for sink the Readme states "If 0 bytes were conusmed, the buffer is full." (with typo). However the function complains about misuse instead https://github.com/atomicobject/heatshrink/blob/7d419e1fa4830d0b919b9b6a91fe2fb786cf3280/heatshrink_encoder.c#L147

opened by notEvil 0
Fallthrough in usage should be `break`
While looking through my local changes I noticed a small difference between the local changes and the upstream develop branch:

https://github.com/atomicobject/heatshrink/blob/ffd95059ff9155fa373b3e6276854db3ad7b4e3c/src/heatshrink.c#L410-L412

In my local version I imported the fix for #46 using a break in that place, which is IMHO the more natural choice, as you normally don't expect the usage() display to cause any side-effects on your program configuration, thus resulting in the following code locally instead:

case 'h': /* help */ usage(); break;

Given that usage() internally calls exit(1); this is even stranger, as this comment causes the expectation of the code continuing after the call to usage(); returns.
bug
opened by BenBE 0

Owner

Atomic Object

GitHub

Superfast compression library

DENSITY Superfast compression library DENSITY is a free C99, open-source, BSD licensed compression library. It is focused on high-speed compression, a

984 Dec 17, 2022

Small strings compression library

SMAZ - compression for very small strings ----------------------------------------- Smaz is a simple compression library suitable for compressing ver

1k Dec 28, 2022

Compression abstraction library and utilities

Squash - Compresion Abstraction Library

375 Dec 22, 2022

Multi-format archive and compression library

Welcome to libarchive! The libarchive project develops a portable, efficient C library that can read and write streaming archives in a variety of form

1.9k Dec 26, 2022

Brotli compression format

SECURITY NOTE Please consider updating brotli to version 1.0.9 (latest). Version 1.0.9 contains a fix to "integer overflow" problem. This happens when

11.8k Jan 5, 2023

Heavily optimized zlib compression algorithm

Optimized version of longest_match for zlib Summary Fast zlib longest_match function. Produces slightly smaller compressed files for significantly fas

124 Dec 12, 2022

Fastest Integer Compression

TurboPFor: Fastest Integer Compression TurboPFor: The new synonym for "integer compression" ?? (2019.11) ALL functions now available for 64 bits ARMv8

647 Dec 26, 2022

zlib replacement with optimizations for "next generation" systems.

zlib-ng zlib data compression library for the next generation systems Maintained by Hans Kristian Rosbach aka Dead2 (zlib-ng àt circlestorm dót org) C

13 Dec 29, 2022

Compile-Time Reflection in C++ for use with Scripting Languages

Introspective Introspective is a header file that brings reflection to any class that wants it, regardless of whether the reflected member is a consta

26 Dec 27, 2022

A simple C library for compressing lists of integers using binary packing

The SIMDComp library A simple C library for compressing lists of integers using binary packing and SIMD instructions. The assumption is either that yo

409 Dec 22, 2022

A portable, simple zip library written in C

A portable (OSX/Linux/Windows), simple zip library written in C This is done by hacking awesome miniz library and layering functions on top of the min

1.1k Dec 29, 2022

is a c++20 compile and runtime Struct Reflections header only library.

is a c++20 compile and runtime Struct Reflections header only library. It allows you to iterate over aggregate type's member variables.

4 Apr 18, 2022

Analysing and implementation of lossless data compression techniques like Huffman encoding and LZW was conducted along with JPEG lossy compression technique based on discrete cosine transform (DCT) for Image compression.

PROJECT FILE COMPRESSION ALGORITHMS - Huffman compression LZW compression DCT Aim of the project - Implement above mentioned compression algorithms an

1 Dec 14, 2021

Lab2: using a physical embedded systems to interact with virtual embedded systems.

Lab2: dotDevice EmSys Autumn 2021 In this lab you will use your TinyPico to interact with a virtual embedded system. Current Virtual Lab URL: [http://

1 Oct 20, 2021

Final Assignment for Embedded Real Time Operating Systems at UCSD Extension.

Final Assignment for Embedded Real Time Operating Systems at UCSD Extension. This program is for a certificate in Embedded Software Engineering at UCSD. We used FreeRTOS running on a STM32L475G Microcontroller.

2 Jun 26, 2022

Overlay Microsoft Flight Simulator (FS2020) aircraft data onto real airport charts in real-time

FLIGHTSIM CHARTS Introduction Overlay Microsoft Flight Simulator (FS2020) aircraft data onto real airport charts in real-time. Instantly teleport to a

3 May 31, 2022

Zstandard - Fast real-time compression algorithm

Zstandard, or zstd as short version, is a fast lossless compression algorithm, targeting real-time compression scenarios at zlib-level and better comp

19.2k Jan 1, 2023

PyFLAC - Real-time lossless audio compression in Python

A simple Pythonic interface for libFLAC. FLAC stands for Free Lossless Audio Codec, an audio format similar to MP3, but lossless, meaning that audio i

97 Dec 9, 2022

A place to collaborate on code for the Embedded.fm book club. Currently reading "STM32 ARM Programming for Embedded Systems".

Welcome to the Book Club Code site! This is a place for the Embedded.fm book club to collaborate and learn together. Repo Structure Guide Top-level fo

11 Jul 21, 2022

Lizard (formerly LZ5) is an efficient compressor with very fast decompression. It achieves compression ratio that is comparable to zip/zlib and zstd/brotli (at low and medium compression levels) at decompression speed of 1000 MB/s and faster.

Lizard - efficient compression with very fast decompression Lizard (formerly LZ5) is a lossless compression algorithm which contains 4 compression met

579 Jan 8, 2023