Structy is an irresponsibly dumb and simple struct serialization/deserialization library for C, Python, and vanilla JavaScript.

Related tags

Scripting structy
Overview

Structy

Structy is an irresponsibly dumb and simple struct serialization/deserialization library for C, Python, and vanilla JavaScript. You can think of it like protobuf, thrift, flatbuffers, etc. but imagine that instead of a team of engineers maintaining it, it's instead written by a single moron.

Structy was created to exchange data between C-based firmware on embedded devices and Python- and JavaScript-based programming, test, and calibration scripts running on a big-girl computer. As such, it's C implementation is designed specifically for embedded devices: it doesn't do any dynamic allocation and it doesn't have any fancy code for optimizations.

Structy's design goals:

Be small, be simple, be useful

Explicit non-goals:

Be fast, be clever

If you want something far more thought out and comprehensive I'd suggest checking out things like Kaitai Struct.

Using structy

Like protobuf and other complicated data exchange libraries, structy uses a schema. Structy's schemas use Python's syntax (so it can be lazy and re-use Python's parser):

class UserSettings:
    brightness : uint8 = 127
    dark_mode: bool = False
    user_id : uint32

With this nonsense you can run structy's generator to generate C, JavaScript, and Python code for this "struct".

$ python3 -m pip install structy
$ structy user_settings.schema --c generated

For C, you can import and use this struct just like a normal struct:

#import "generated/user_settings.h"

struct UserSettings settings = {
    .brightness = 127,
    .dark_mode = true,
    .user_id = 6,
};

But it also creates the initialization (UserSettings_init), serialization (UserSettings_pack), and deserialization (UserSettings_unpack) functions needed for the struct.

C implementation

Structy's C implementation is written specifically with microcontrollers in mind. Notably:

  • The runtime is tiny, coming in at right at ~250 lines of code and compiles to less than 1kB of thumb code.
  • Never uses the heap
  • Uses very very little stack space
  • Compiles cleanly with -Werror -Wall -Wextra -Wpedantic -std=c17
  • It includes optional support for libfixmath's fix16_t.

Including Structy's runtime

You must include the files in runtimes/c in your project to use structy-generated c code.

Using generated code

The Structy generates the struct definition and four functions for use with the struct:

Initialization

void StructName_init(struct StructName* inst);

Initializes an instance of the struct with the default values specified in the struct's schema.

Pack

struct StructyResult StructName_pack(const struct StructName* inst, uint8_t* buf);

Packs an instance of the struct into the given buffer. The buffer must have at least STRUCTNAME_PACKED_SIZE bytes. Example:

uint8_t output_buffer[STRUCTNAME_PACKED_SIZE];
StructName_pack(&instance, output_buffer);

Unpack

struct StructyResult StructName_unpack(struct StructName* inst, const uint8_t* buf);

Unpacks the buffer into the given instance. The buffer must have at least STRUCTNAME_PACKED_SIZE bytes.

Print

void StructName_print(const struct StructName* inst);

Prints out the struct in a nicely formatted way, for example:

struct UserSettings @ 0x0B00B135
- brightness: 127
- dark_mode: 1
- user_id: 6

Since the runtime is designed for embedded systems, Structy doesn't hardcode printf here. You can override the print function used by creating a structy_config.h file and setting the STRUCTY_PRINTF macro:

#include "super_cool_printf.h"

#define STRUCTY_PRINTF(...) super_cool_printf(__VA_ARGS__)

By default, Structy will try to use 's printf if you're on a big girl computer. If you're on a 32-bit ARM system, Structy will check and see if you have mpland's embedded-friendly printf and use that if you have it. Otherwise, it'll disable printing.

Python implementation

Structy's Python implementation is written to be simple, not fast or "powerful" or whatever. Some notes:

  • The runtime depends on Python 3.7+
  • The runtime has complete type annotations.
  • The runtime supports converting floats between libfixmath's fix16 during packing & unpacking.
  • It's built on top of Python's excellent struct module.

Including Structy's runtime

The runtime can be installed by installing the structy package:

$ python3 -m pip install structy

Using generated code

The Structy generates the struct as a dataclass in its own module.

Initialization

# Default values.
inst = StructName()
# Override default value
inst = StructName(field_name=something)

Since it's a dataclass, an empty constructor gives the default specified in the .structy definition file, but you can pass keyword arguments to override them.

Pack

result: bytes = inst.pack()

Packs an instance and returns a bytes object with the data. len(result) will be StructName.PACKED_SIZE.

Unpack

inst = StructName.unpack(data)

Unpacks the bytes-like buffer into a new instance. The buffer must be at least StructName.PACKED_SIZE long.

Print

Since a Structy Struct is just a dataclass, it uses the dataclass __str__ and __repr__.

JavaScript implementation

Structy's JavaScript implementation, like the others, is intended to be simple. Here's some notes on it:

Including Structy's runtime

The runtime is a single file located at runtimes/js/structy.js. There's no Node/npm package because the last time I used npm my nose suddenly starting bleeding and I passed out and woke up with several mysterious lesions, and the last time I used Node.js I managed to somehow unleash a 10,000 year-old minor demon who's currently causing minor chaos by removing stop signs in low-traffic rural areas.

So just copy it into your project next to wherever you're going to place the generated code, like we did when the web was young. If you're lazy (and you probably are), just run this:

$ wget https://raw.githubusercontent.com/theacodes/structy/runtimes/js/structy.js
$ wget https://raw.githubusercontent.com/theacodes/structy/third_party/struct.js/struct.mjs

Those commands also copy in the one dependency: the excellent and tiny struct.js module- yes, I know it's confusing to have structy.js and stuct.js, but get over it. struct.js implements Python struct module in JavaScript which is great because I can write less code.

Using generated code

The Structy generates the struct as a nice class in its own module. The module's only export is the class, so you can import it like this:

<script type="module">
import StructName from "./structname.js";
script>

Initialization

// Default values.
inst = new StructName();
// Override a default value by passing in an object
inst = new StructName({field_name: something});

It's a JavaScript class so just use new to make an instance. It uses the pattern of passing in an object with field values so you can update fields without needing to specify all of them.

Pack

result = inst.pack();

Packs an instance and returns a Uint8Array object with the data. result.length() will be StructName.packed_size.

Unpack

inst = StructName.unpack(data);

Unpacks the Uint8Array or ArrayBuffer into a new instance. The buffer must be at least StructName.packed_size long.

FAQ

Does Structy support arrays/lists?

No, Struct structs must be a deterministic length when encoded.

How about bitfields?

No yet, but it could. I'm just too lazy.

Little-endian? Mixed-endianess?

No. Supporting anything other than big-endian would complicate the C runtime and I don't want to do that. Idk, maybe I could be talked into accepting a PR for it.

Custom types?

No, but it's possible in the future. There's a little bit of this thought out because Struct supports Q16.16 fixed-point values.

Nested structs?

No, but it could be added. I just haven't needed it yet.

Why does structy use snake_case for properties and methods even in C and JS where it's common to use lowerCamelCase?

Mostly because I want data access to be identical in all languages, but also because I deeply, deeply, hate lowerCamelCase. I find it hard to read.

Support / issues / etc

Unlike some of my other open-source projects, this project was made with one user in mind: me. I made it open source in case someone else finds it useful but I am not in the business of supporting this project. If you run into issues or need help feel free to submit an issue on GitHub, however, please know that I do not feel any obligation to support this project.

License

The Structy generator and runtime components are all published under the MIT license. See LICENSE for the full text.

Issues
  • Fix tty detection

    Fix tty detection

    os.isatty() takes a file descriptor argument which was not being provided here. This raised a TypeError exception on my installation as it expected one argument.

    opened by georgerennie 1
Owner
Stargirl Flowers
Open-source advocate ✨ Synth crafter ✨ PSF Fellow ✨ She/her
Stargirl Flowers
Duktape - embeddable Javascript engine with a focus on portability and compact footprint

Duktape ⚠️ Master branch is undergoing incompatible changes for Duktape 3.x. To track Duktape 2.x, follow the v2-maintenance branch. Introduction Dukt

Sami Vaarala 5.3k Jun 23, 2022
Embedded JavaScript engine for C/C++

V7: Embedded JavaScript engine NOTE: this project is deprecated in favor of https://github.com/cesanta/mjs V7 is the smallest JavaScript engine writte

Cesanta Software 1.3k Jun 21, 2022
ChakraCore is an open source Javascript engine with a C API.

ChakraCore ChakraCore is a Javascript engine with a C API you can use to add support for Javascript to any C or C compatible project. It can be compil

null 8.7k Jun 20, 2022
Seamless operability between C++11 and Python

pybind11 — Seamless operability between C++11 and Python Setuptools example • Scikit-build example • CMake example Warning Combining older versions of

pybind 11.2k Jun 27, 2022
Import C++ files directly from Python!

If you've used cppimport version 0.0.*, some new features for you! Compiler arguments, multiple source files, bug fixes! Read on. Import C or C++ file

Ben Thompson 1k Jun 27, 2022
The most widely used Python to C compiler

Welcome to Cython! Cython is a language that makes writing C extensions for Python as easy as Python itself. Cython is based on Pyrex, but supports mo

null 7.1k Jun 20, 2022
A portable foreign-function interface library.

Status libffi-3.4 was released on TBD. Check the libffi web page for updates: URL:http://sourceware.org/libffi/. What is libffi? Compilers for high le

null 2.5k Jun 20, 2022
A lightweight, dependency-free library for binding Lua to C++

LuaBridge 2.6 LuaBridge is a lightweight and dependency-free library for mapping data, functions, and classes back and forth between C++ and Lua (a po

Vinnie Falco 1.3k Jun 21, 2022
Library to build PHP extensions with C++

PHP-CPP The PHP-CPP library is a C++ library for developing PHP extensions. It offers a collection of well documented and easy-to-use classes that can

Copernica 1.3k Jun 18, 2022
Sol3 (sol2 v3.0) - a C++ <-> Lua API wrapper with advanced features and top notch performance - is here, and it's great! Documentation:

sol2 sol2 is a C++ library binding to Lua. It currently supports all Lua versions 5.1+ (LuaJIT 2.0+ and MoonJIT included). sol2 aims to be easy to use

The Phantom Derpstorm 3k Jun 23, 2022
Libraries and examples to support Pimoroni Pico add-ons in C++ and MicroPython.

Pimoroni Pico Libraries and Examples Welcome to the brave new world of Pico! This repository contains the C/C++ and MicroPython libraries for our rang

Pimoroni Ltd 493 Jun 20, 2022
Tools and libraries to glue C/C++ APIs to high-level languages

CppSharp is a tool and set of libraries which facilitates the usage of native C/C++ code with the .NET ecosystem. It consumes C/C++ header and library

Mono Project 2.4k Jun 24, 2022
A tool for generating cross-language type declarations and interface bindings.

Djinni Djinni is a tool for generating cross-language type declarations and interface bindings. It's designed to connect C++ with either Java or Objec

Dropbox 2.8k Jun 21, 2022
The missing bridge between Java and native C++

JavaCPP Commercial support: Introduction JavaCPP provides efficient access to native C++ inside Java, not unlike the way some C/C++ compilers interact

Bytedeco 3.9k Jun 20, 2022
SWIG is a software development tool that connects programs written in C and C++ with a variety of high-level programming languages.

SWIG (Simplified Wrapper and Interface Generator) Version: 4.1.0 (in progress) Tagline: SWIG is a compiler that integrates C and C++ with languages

SWIG 4.5k Jun 25, 2022
A minimalist and mundane scripting language.

Drift Script A minimalist and mundane scripting language. I like all simple things, simple and beautiful, simple and strong. I know that all developme

Drift 11 Apr 3, 2022
CppSerdes is a serialization/deserialization library designed with embedded systems in mind

A C++ serialization/deserialization library designed with embedded systems in mind

Darren V Levine 79 Jun 3, 2022
Fast Binary Encoding is ultra fast and universal serialization solution for C++, C#, Go, Java, JavaScript, Kotlin, Python, Ruby, Swift

Fast Binary Encoding (FBE) Fast Binary Encoding allows to describe any domain models, business objects, complex data structures, client/server request

Ivan Shynkarenka 567 Jun 16, 2022
matrix-effect This is a dumb matrix effect type thing

matrix-effect This is a dumb matrix effect type thing. It's only like one source file which should compile... Define __POSIX or __WIN though, for posi

null 41 Jun 8, 2022
Python Inference Script is a Python package that enables developers to author machine learning workflows in Python and deploy without Python.

Python Inference Script(PyIS) Python Inference Script is a Python package that enables developers to author machine learning workflows in Python and d

Microsoft 10 Feb 23, 2022
An Arduino library with additions to vanilla Serial.print(). Chainable methods and verbosity levels. Suitable for debug messages.

advancedSerial This library provides some additions to vanilla Serial.print(): 1. Chainable print() and println() methods: // you can chain print() a

Vasily Klenov 15 Dec 8, 2021
Freelancer: HD Edition is a mod that aims to improve every visual, aural, and gameplay aspect of the game Freelancer (2003) while keeping the look and feel as close to vanilla as possible.

Freelancer: HD Edition Freelancer: HD Edition is a mod that aims to improve every visual, aural, and gameplay aspect of the game Freelancer (2003) whi

C.S. Melis 47 Jun 15, 2022
Speed Running and Competition Doom. For strictly vanilla speed runs and competitions - forked from CNDoom

Speed Running and Competition Doom Speed Running and Competition Doom is based on Chocolate Doom and aims to accurately reproduce the original DOS ver

Gibbon 3 May 24, 2022
1.12.1 (5875) Vanilla Emulator (Core & DB)

RaspyWoW - Vanilla Welcome to the new generation of interconnected WoW realms — all hosted on Raspberry PI. Raspy WoW aims to create an efficient, che

Raspy-WoW 8 May 14, 2022
is a c++20 compile and runtime Struct Reflections header only library.

is a c++20 compile and runtime Struct Reflections header only library. It allows you to iterate over aggregate type's member variables.

RedSkittleFox 4 Apr 18, 2022
A miniature library for struct-field reflection in C++

visit_struct A header-only library providing structure visitors for C++11 and C++14. Motivation In C++ there is no built-in way to iterate over the me

null 347 Jun 24, 2022
A miniature library for struct-field reflection in C++

visit_struct A header-only library providing structure visitors for C++11 and C++14. Motivation In C++ there is no built-in way to iterate over the me

null 346 Jun 23, 2022
🎵 Music notation engraving library for MEI with MusicXML and Humdrum support and various toolkits (JavaScript, Python)

Verovio is a fast, portable and lightweight library for engraving Music Encoding Initiative (MEI) digital scores into SVG images. Verovio also contain

RISM Switzerland 484 Jun 23, 2022
Source code and raw content for NEON STRUCT: Desperation Column

NEON STRUCT: Desperation Column copyright © 2021 Minor Key Games, LLC. Source code and raw content for NEON STRUCT: Desperation Column are release

David Pittman 7 Jun 12, 2022