CppSerdes is a serialization/deserialization library designed with embedded systems in mind

Overview

cppserdes_icon

CppSerdes is a serialization/deserialization library designed with embedded systems in mind

Build Status license version

Features

  • Bitpacking support - any bit alignment, any bit padding.
  • Portability - uses bit shifting and masking, with no dynamic memory allocation (avoids most std containers).
  • Serializes to/from 8-bit arrays, AND 16-bit, 32-bit, and 64-bit arrays. With no API changes to your code, to work with hardware drivers directly in their native bit widths.
  • Common Embedded Systems types - floating point, unsigned, signed, classes, enums, arrays, custom types, etc. While avoiding pulling in atypically embedded std types such as std::string, std::vector, etc.
  • Arrays - fixed sized arrays, dynamically sized arrays, and delimited arrays.
  • Memory safety - Bound checking for both serial buffers and array fields.
  • Choose from both high abstraction level (object-oriented & streams) and low level (memcpy-like) APIs.
  • Custom formatter types, such as optional validation checks, attached right to a field.
  • Virtual fields, and pure virtual fields, allowing you to easily change formats at runtime.
  • Header only, and works with C++11 and greater.
  • "constexpr" support for C++14 or greater using bitcpy function.
  • Compiles with high warning levels (pedantic, Wall, etc.).

Documentation

darrenlevine.github.io/cppserdes

Quick Start

  1. Check out the examples:
    • Compile and run "examples/01_simple_example.cpp".
    • Read through the examples/ folder for various usage cases.
  2. Use the library in your project:
    • Manual inclusion: Make sure "cppserdes/include/" is added to your project's path and use as desired.
    • With CMake: Add the following lines to your CmakeLists.txt project file:
add_subdirectory(cppserdes/)
target_link_libraries(your_project_name cppserdes)

Object Oriented Example (using serdes::packet_base)

#include "serdes.h"

struct my_packet : serdes::packet_base {
    int8_t x = 6, y = 7, z = 8;

    // define a format (used for BOTH serialization and deserialization)
    void format(serdes::packet &serdes_process) {
        serdes_process + x + y + serdes::pad(1) + serdes::bitpack(z, 7);
    }
};

int main() {
    uint16_t serial_data[] = {0x0102, 0x7B00};

    my_packet obj1;
    obj1.load(serial_data); // loads serial data into obj1 x == 1, y == 2, z == -5

    my_packet obj2;
    obj2.store(serial_data); // stores {0x0607, 0x0800} into serial_data from obj2
}

Functional Example (using serdes::bitcpy)

#include "bitcpy.h"

int main() {
    uint16_t serial_data[] = {0x0102, 0xFB00};
    int8_t x, y, z;

    // deserialization
    serdes::bitcpy(x, serial_data, 0, 8);
    serdes::bitcpy(y, serial_data, 8, 8);
    serdes::bitcpy(z, serial_data, 16, 8);

    // serialization
    serdes::bitcpy(serial_data, x, 0, 8);
    serdes::bitcpy(serial_data, y, 8, 8);
    serdes::bitcpy(serial_data, z, 16, 8);
}

Streams Example (using serdes::packet)

> x >> y >> z; // take same variables and place it into serial data serdes::packet(serial_data) << x << y << z; } ">
#include "serdes.h"

int main() {
    uint16_t serial_data[] = {0x0102, 0x0304, 0x0506};
    int8_t x, y, z;

    // take some serial data and place it into variables
    serdes::packet(serial_data) >> x >> y >> z;

    // take same variables and place it into serial data
    serdes::packet(serial_data) << x << y << z;
}

Known Limitations

  • Little endian "serialization" is not yet supported (Note: little and big endian "platforms" ARE both supported)
  • Cannot bitcpy more than 0.25GB of data in a single function call (due to a memory-usage/speed design decision)
  • The examples are written using C++14 syntax for succinctness and simplicity, so if you're using an older compiler, templated fields might need to have their templates explicitly specified, for example:
serdes::bitpack(123, serdes::bit_length(2)) // works in >= C++14
serdes::bitpack(123, serdes::bit_length(2)) // needed in < C++14

Planned Features

  • Little endian serialization support (compile time only is planned)
  • The ability to avoid serialization/deserialization by setting pointers to point to the appropriate spot within the serial buffer, instead of making a copy into a new buffer, thus saving time and memory if the serial buffer's lifetime persists longer than the reference, and the memory is aligned appropriately to allow this optimization. Similar to how capnproto and flatbuffers work.

Requirements

  • C++11 or greater
Issues
  • Clang compilation issues

    Clang compilation issues

    Hi. I trying to compile the library with clang, but unfortunately, I'm running into errors.

    1. Reference to 'atomic' is ambiguous at bitcpy_common:75 I was able to "fix" it by commenting out forward declared std::atomic at the beginning of the file.

    2. Variable of non-literal type 'const sized_pointer<const uint8_t>' (aka 'const sized_pointer<const unsigned char>') cannot be defined in a constexpr function Not sure what to do here 🤷

    Any ideas on how to properly compile it with Clang? FWIW, I'm using lasted AppleClang with XCode.

    opened by brbrr 2
Owner
Darren V Levine
Darren V Levine
Your binary serialization library

Bitsery Header only C++ binary serialization library. It is designed around the networking requirements for real-time data delivery, especially for ga

Mindaugas Vinkelis 729 Aug 8, 2022
Cap'n Proto serialization/RPC system - core tools and C++ library

Cap'n Proto is an insanely fast data interchange format and capability-based RPC system. Think JSON, except binary. Or think Protocol Buffers, except

Cap'n Proto 9.1k Aug 3, 2022
A C++11 library for serialization

cereal - A C++11 library for serialization cereal is a header-only C++11 serialization library. cereal takes arbitrary data types and reversibly turns

iLab @ USC 3.2k Aug 5, 2022
FlatBuffers: Memory Efficient Serialization Library

FlatBuffers FlatBuffers is a cross platform serialization library architected for maximum memory efficiency. It allows you to directly access serializ

Google 18.5k Aug 12, 2022
Simple C++ 20 Serialization Library that works out of the box with aggregate types!

BinaryLove3 Simple C++ 20 Serialization Library that works out of the box with aggregate types! Requirements BinaryLove3 is a c++20 only library.

RedSkittleFox 13 Jun 29, 2022
Header-only library for automatic (de)serialization of C++ types to/from JSON.

fuser 1-file header-only library for automatic (de)serialization of C++ types to/from JSON. how it works The library has a predefined set of (de)seria

null 49 Jun 4, 2022
C++17 library for all your binary de-/serialization needs

blobify blobify is a header-only C++17 library to handle binary de-/serialization in your project. Given a user-defined C++ struct, blobify can encode

Tony Wasserka 241 Aug 7, 2022
Yet another JSON/YAML/BSON serialization library for C++.

ThorsSerializer Support for Json Yaml Bson NEW Benchmark Results Conformance mac linux Performance max linux For details see: JsonBenchmark Yet anothe

Loki Astari 277 Aug 2, 2022
Cista is a simple, high-performance, zero-copy C++ serialization & reflection library.

Simple C++ Serialization & Reflection. Cista++ is a simple, open source (MIT license) C++17 compatible way of (de-)serializing C++ data structures. Si

Felix Gündling 926 Aug 8, 2022
Microsoft 2.4k Aug 7, 2022
Fast Binary Encoding is ultra fast and universal serialization solution for C++, C#, Go, Java, JavaScript, Kotlin, Python, Ruby, Swift

Fast Binary Encoding (FBE) Fast Binary Encoding allows to describe any domain models, business objects, complex data structures, client/server request

Ivan Shynkarenka 591 Aug 2, 2022
Yet Another Serialization

YAS Yet Another Serialization - YAS is created as a replacement of boost.serialization because of its insufficient speed of serialization (benchmark 1

niXman 558 Aug 4, 2022
Binary Serialization

Binn Binn is a binary data serialization format designed to be compact, fast and easy to use. Performance The elements are stored with their sizes to

null 365 Aug 5, 2022
An implementation of the MessagePack serialization format in C / msgpack.org[C]

CMP CMP is a C implementation of the MessagePack serialization format. It currently implements version 5 of the MessagePack Spec. CMP's goal is to be

Charlie Gunyon 281 Aug 5, 2022
MPack - A C encoder/decoder for the MessagePack serialization format / msgpack.org[C]

Introduction MPack is a C implementation of an encoder and decoder for the MessagePack serialization format. It is: Simple and easy to use Secure agai

Nicholas Fraser 376 Jul 27, 2022
Serialization framework for Unreal Engine Property System that just works!

DataConfig Serialization framework for Unreal Engine Property System that just works! Unreal Engine features a powerful Property System which implemen

null 67 Aug 5, 2022
Yet Another Serialization

YAS Yet Another Serialization - YAS is created as a replacement of boost.serialization because of its insufficient speed of serialization (benchmark 1

niXman 455 Sep 7, 2021
universal serialization engine

A Universal Serialization Engine Based on compile-time Reflection iguana is a modern, universal and easy-to-use serialization engine developed in c++1

qicosmos 625 Aug 3, 2022
A lightweight C++20 serialization framework

zpp::bits A modern C++20 binary serialization library, with just one header file. This library is a successor to zpp::serializer. The library tries to

Eyal Z 223 Aug 9, 2022