An embedded-friendly library for decompressing files from zip archives

Related tags

Compression unzipLIB
Overview

unzipLIB

Copyright (c) 2021 BitBank Software, Inc.
Written by Larry Bank
[email protected]

What is it?

An 'embedded-friendly' (aka Arduino) library to extract and decompress files from ZIP archives

Why did you write it?

I wanted to be able to unzip files on MCUs for various projects and I couldn't find any libraries that would do the job. The thing that prevents most Linux open source projects from running on embedded boards is their use of the file system and heap management. There is support for malloc/free on Arduino, but it's better to control how memory is used more directly (e.g. TCM vs FLASH vs DDR).

What's special about it?

I designed the unzip library to not rely on malloc/free, but instead to have a fixed class/structure which could be managed by the user. I also removed the dependency on the file system and instead offer a set of callback hooks to allow zip files to be read from memory, SDCard or any media. These two features give it maximum flexibility to run on low end systems.

Feature summary:

  • Runs on any MCU with at least 41K of free RAM
  • No external dependencies (including malloc/free, file system calls)
  • unzip API implemented in C and C++
  • ZIP files can come from RAM, FLASH, SDCard or any media by using callbacks
  • Decompress files by reading any sized chunk at a time (full sized buffer not needed)
  • Compiles on any target CPU

Documentation:

Detailed information about the API is in the Wiki
See the examples folder for easy starting points

You might also like...
Small strings compression library

SMAZ - compression for very small strings ----------------------------------------- Smaz is a simple compression library suitable for compressing ver

A massively spiffy yet delicately unobtrusive compression library.

ZLIB DATA COMPRESSION LIBRARY zlib 1.2.11 is a general purpose data compression library. All the code is thread safe. The data format used by the z

PhysFS++ is a C++ wrapper for the PhysicsFS library.

PhysFS++ PhysFS++ is a C++ wrapper for the excellent PhysicsFS library by Ryan C. Gordon and others. It is licensed under the zlib license - same as P

Gzip header-only C++ library

Gzip C++ lib for gzip compression and decompression. Extracted from mapnik-vector-tile for light-weight modularity. Usage // Include the specific gzip

Simple data packing library (written in C99)

Features Compressed file pack creation Runtime file pack reading Supported operating systems Ubuntu MacOS Windows Build requirements C99 compiler CMak

A simple C library implementing the compression algorithm for isosceles triangles.

orvaenting Summary A simple C library implementing the compression algorithm for isosceles triangles. License This project's license is GPL 2 (as of J

Advanced DXTc texture compression and transcoding library

crunch/crnlib v1.04 - Advanced DXTn texture compression library Public Domain - Please see license.txt. Portions of this software make use of public d

FUSE file system for ZIP archives

title section header footer date MOUNT-ZIP 1 User Manual mount-zip 1.0 November 2021 NAME mount-zip - Mount a ZIP archive as a FUSE filesystem. SYNOPS

Draco is a library for compressing and decompressing 3D geometric meshes and point clouds.

Draco is a library for compressing and decompressing 3D geometric meshes and point clouds. It is intended to improve the storage and transmission of 3D graphics.

archiver is a compressing/decompressing tool made for educational purposes

archiver 📁 archiver is a compressing/decompressing tool made for educational purposes (specifically, it was a hometask given at a C++ course in the H

FCracker is a command line tool designed to brute force encrypted files like zip, 7z, rar, pdf etc.

FCrack is a command-line tool designed to brute force encrypted files like zip, 7z, rar, pdf, gpg etc.

Runtime Archiver plugin for Unreal Engine. Cross-platform archiving and unarchiving directories and files. Currently supports ZIP format.

Runtime Archiver Archiving and dearchiving directories and files Explore the docs » Marketplace . Releases . Support Chat Features Fast speed Easy arc

Qt 5 addon providing access to numerous types of archives

KArchive Reading, creating, and manipulating file archives Introduction KArchive provides classes for easy reading, creation and manipulation of "arch

Integrate PhysFS with raylib, allowing to load images, audio and fonts from data archives.

raylib-physfs Integrate the virtual file system PhysicsFS with raylib, allowing to load images, audio, and fonts from data archives. Features Load the

cbmconvert: create, extract and convert 8-bit Commodore binary archives

cbmconvert: create, extract and convert 8-bit Commodore binary archives cbmconvert extracts files from most known archive file formats that are used o

An 'embedded-friendly' (aka Arduino) JPEG image encoding library

Starting in the late 80's I wrote my own imaging codecs for the existing standards (CCITT G3/G4 was the first). I soon added GIF, JPEG and not long after that, the PNG specification was ratified. All of this code was "clean room" - written just from the specification. I used my imaging library in many projects and products over the years and recently decided that some of my codecs could get a new lease on life as open source, embedded-friendly libraries for microcontrollers.

A realtime/embedded-friendly C++11 variant type which is never empty and prevents undesirable implicit conversions

strict variant Do you use boost::variant or one of the many open-source C++11 implementations of a "tagged union" or variant type in your C++ projects

A C++ static library offering a clean and simple interface to the 7-zip DLLs.

bit7z A C++ static library offering a clean and simple interface to the 7-zip DLLs Supported Features • Getting Started • Download • Requirements • Bu

Fork of the popular zip manipulation library found in the zlib distribution.

minizip-ng 3.0.0 minizip-ng is a zip manipulation library written in C that is supported on Windows, macOS, and Linux. Developed and maintained by Nat

Comments
  • Chunk by chunk read

    Chunk by chunk read

    Hi,

    I'm still not able to read the unzipped data chunk by chunk. If I would read it at once, It would not fit into RAM. My unzipped data size is 393 KB If I comment out Block 2, it is working, but I would like to read 512 bytes one after another:

    ...
    #define BUFF_SIZE 512
    static uint8_t l_Buff[BUFF_SIZE];
    ...
        // Block 1
        zip.openCurrentFile();
        mySeek(&myfile, 0, SEEK_SET);
        int readSize = zip.readCurrentFile(l_Buff, BUFF_SIZE);
        Serial.print("Bytes read in buffer: "); Serial.println(readSize);
        binFile.write(l_Buff, BUFF_SIZE);
    
        // Block 2
        mySeek(&myfile, 512, SEEK_SET);
        readSize = zip.readCurrentFile(l_Buff, BUFF_SIZE);
        Serial.print("Bytes read in buffer: "); Serial.println(readSize);
        binFile.write(l_Buff, BUFF_SIZE);
    
    opened by samsmith94 7
  • Chunkwise write the uncompressed file.

    Chunkwise write the uncompressed file.

    Hi,

    I could create a simple program to test your library with SdFat library, instead of standard Arduino SD. But I realized, that I still have a problem. I would like to implement FOTA, so I will download the application.zip file. Then I want to unzip it (application.bin). The .bin file is quite large (393216 byte). I don't have this much memory. (I have 56+4 KB of SRAM on STM32F429ZI).

    So my question, how can I use your library, so that I can write this larga application.bin to the SD card in chunkwise manner?

    Here is my code:

    #include <Arduino.h>
    #include <unzipLIB.h>
    #include "SdFat.h"
    #include "sdios.h"
    
    SPIClass SPI_4(PE6, PE5, PE2); // MOSI, MISO. SCLK
    const uint8_t SD_CS_PIN = PE4;
    #define SD_CONFIG SdSpiConfig(SD_CS_PIN, DEDICATED_SPI, SD_SCK_MHZ(4), &SPI_4)
    
    UNZIP zip; // statically allocate the UNZIP structure (41K)
    SdFat sd;
    File root;
    static File myfile;
    static File myfileAnother;
    
    void * myOpen(const char *filename, int32_t *size) {
      root.open("/");
      myfileAnother.open(filename);
      *size = (uint32_t)myfileAnother.fileSize();
      return (void *)&myfileAnother;
    }
    
    void myClose(void *p) {
      ZIPFILE *pzf = (ZIPFILE *)p;
      File *f = (File *)pzf->fHandle;
      if (f) f->close();
    }
    
    int32_t myRead(void *p, uint8_t *buffer, int32_t length) {
      ZIPFILE *pzf = (ZIPFILE *)p;
      File *f = (File *)pzf->fHandle;
      return f->read(buffer, length);
    }
    
    int32_t mySeek(void *p, int32_t position, int iType) {
      ZIPFILE *pzf = (ZIPFILE *)p;
      File *f = (File *)pzf->fHandle;
      if (iType == SEEK_SET)
        return f->seek(position);
      else if (iType == SEEK_END) {
        return f->seek(position + pzf->iSize); 
      } else { // SEEK_CUR
        long l = f->position();
        return f->seek(l + position);
      }
    }
    
    void setup() {
      Serial.begin(115200);
    
      while (!Serial && millis() < 3000);
      Serial.println("Search for ZIP files on the SD card");
      
      if (!sd.begin(SD_CONFIG))
      {
        sd.initErrorHalt(&Serial);
        Serial.println("Unable to access SD Card");
      }
    
      if (!root.open("/"))
      {
        Serial.println("Open root failed");
      }
      //////////////////////////////////////////////////////////////////////////////
      int rc;
      char szComment[256], szName[256];
      unz_file_info fi;
    
      const char *name = "application.zip";
      if (!myfile.open(name, O_RDONLY))
      {
        Serial.println("Opening application.zip failed");
      }
      Serial.println("Openend application.zip");
      rc = rc = zip.openZIP(name, myOpen, myClose, myRead, mySeek);
      if (rc == UNZ_OK) {
        Serial.print("ZIP file found");
    
        // Display the global comment and all of the filenames within
        rc = zip.getGlobalComment(szComment, sizeof(szComment));
        Serial.print("Files in this archive: ");
        zip.gotoFirstFile();
        rc = UNZ_OK;
        rc = zip.getFileInfo(&fi, szName, sizeof(szName), NULL, 0, szComment, sizeof(szComment));
    
        if (rc == UNZ_OK) {
          Serial.println(szName);
          Serial.print("Compressed size: "); Serial.println(fi.compressed_size, DEC);
          Serial.print("Uncompressed size: "); Serial.println(fi.uncompressed_size, DEC);
          
        }
        zip.closeZIP();
      }
    }
    
    void loop() {}
    
    
    opened by samsmith94 4
  • SdFat example

    SdFat example

    Hello, I would have a question. Do you have some example code for SdFat library (https://github.com/greiman/SdFat)? I would like to use your library, but on STM32, I can't use the standard SD library.

    opened by samsmith94 1
Releases(1.0.0)
  • 1.0.0(Jun 10, 2021)

Owner
Larry Bank
Call me Mr. Optimization. I optimize other people's code for a living.
Larry Bank
Runtime Archiver plugin for Unreal Engine. Cross-platform archiving and unarchiving directories and files. Currently supports ZIP format.

Runtime Archiver Archiving and dearchiving directories and files Explore the docs » Marketplace . Releases . Support Chat Features Fast speed Easy arc

Georgy Treshchev 26 Dec 15, 2022
Qt 5 addon providing access to numerous types of archives

KArchive Reading, creating, and manipulating file archives Introduction KArchive provides classes for easy reading, creation and manipulation of "arch

KDE GitHub Mirror 34 Nov 1, 2022
A C++ static library offering a clean and simple interface to the 7-zip DLLs.

bit7z A C++ static library offering a clean and simple interface to the 7-zip DLLs Supported Features • Getting Started • Download • Requirements • Bu

Riccardo 326 Jan 1, 2023
Fork of the popular zip manipulation library found in the zlib distribution.

minizip-ng 3.0.0 minizip-ng is a zip manipulation library written in C that is supported on Windows, macOS, and Linux. Developed and maintained by Nat

zlib-ng 971 Jan 4, 2023
Fork of the popular zip manipulation library found in the zlib distribution.

minizip-ng 3.0.1 minizip-ng is a zip manipulation library written in C that is supported on Windows, macOS, and Linux. Developed and maintained by Nat

zlib-ng 971 Jan 4, 2023
gzip (GNU zip) is a compression utility designed to be a replacement for 'compress'

gzip (GNU zip) is a compression utility designed to be a replacement for 'compress'

ACM at UCLA 8 Nov 6, 2022
PNGFuse is a cross-platform application that allows you to embed and extract full zlib-compressed files within PNG metadata.

PNGFuse PNGFuse is a portable, lightweight, and cross-platform application written in C++ that allows you to embed and extract full zlib-compressed fi

Eta 3 Dec 29, 2021
Multi-format archive and compression library

Welcome to libarchive! The libarchive project develops a portable, efficient C library that can read and write streaming archives in a variety of form

null 1.9k Jan 8, 2023
LZFSE compression library and command line tool

LZFSE This is a reference C implementation of the LZFSE compressor introduced in the Compression library with OS X 10.11 and iOS 9. LZFSE is a Lempel-

null 1.7k Jan 4, 2023
miniz: Single C source file zlib-replacement library, originally from code.google.com/p/miniz

Miniz Miniz is a lossless, high performance data compression library in a single source file that implements the zlib (RFC 1950) and Deflate (RFC 1951

Rich Geldreich 1.6k Jan 5, 2023