Cross-platform shared memory stream/buffer, header-only library for IPC in C/C++.

Overview

libsharedmemory

libsharedmemory is a small C++11 header-only library for using shared memory on Windows, Linux and macOS. libsharedmemory makes it easy to transfer data between isolated host OS processes. It also helps inter-connecting modules of applications that are implemented in different programming languages. It allows for simple read/write data transfer using single, indexed memory address access and also using array-types like std::string, float*, double*.

Example

std::string dataToTransfer = "{ foo: 'coolest IPC ever! 🧑‍💻' }";

// the name of the shared memory is os-wide announced
// the size is in bytes and must be big enough to handle the data (up to 4GiB)
// if persistency is disabled, the shared memory segment will
// be garbage collected when the process that wrote it is killed
SharedMemoryWriteStream write$ {/*name*/ "jsonPipe", /*size*/ 65535, /*persistent*/ true};
SharedMemoryReadStream read$ {/*name*/ "jsonPipe", /*size*/ 65535, /*persistent*/ true};

// writing the string to the shared memory
write$.write(dataToTransfer);

// reading the string from shared memory
// you can run this in another process, thread,
// even in another app written in another programming language
std::string dataString = read$.read();

std::cout << "UTF8 string written and read" << dataString << std::endl;

Source code package management via npm

In case you want to use this library in your codebase, you could just copy & paste the include/libsharedmemory/libsharedmemory.hpp into your include or deps directory. But then you'd have to manually manage the code base.

However, you could also use npm for a smarter dependency management approach. Therefore, install Node.js which comes bundled with npm, the Node package manager.

Now run npm init in your project root directoy. After initial setup, run npm install cpp_libsharedmemory and add node_modules/cpp_libsharedmemory/include/libsharedmemory to your include path.

Whenever this library updates, you can also update your dependencies via npm upgrade. Futhermore, people who audit the code can announce security reports that are announced when running npm audit. Finally, it's also much easier for you to install all project dependencies by just running npm install in your projects root directory. Managing third party code becomes obsolete at all.

Limits

libsharedmemory does only support the following datatypes (array-like):

  • std::string
  • float*
  • double*

Single value access via .data()[index] API:

  • all scalar datatypes supported in C/C++

  • This library doesn't care for endinanness. This should be naturally fine because shared memory shouldn't be shared between different machine architectures. However, if you plan to copy the shared buffer onto a network layer prototcol, make sure to add an endianess indication bit.

  • Although the binary memory layout should give you no headache when compiling/linking using different compilers, the behavior is undefined.

  • At the time of writing, there is no support for shared memory persistency on Windows. Shared memory is lost after the writing process is killed.

Memory layout

When writing data into a named shared memory segment, libsharedmemory does write 5 bytes of meta information:

  • flags (char) is a bitmask that indicates data change (via an odd/even bit flip) as well as the data type transferred (1 byte)
  • size (int) indicates the buffer size in bytes (4 bytes)

Therefore the binary memory layout is: |flags|size|data|

The following datatype flags are defined:

enum DataType {
  kMemoryChanged = 1,
  kMemoryTypeString = 2,
  kMemoryTypeFloat = 4,
  kMemoryTypeDouble = 8,
};

kMemoryChanged is the change indicator bit. It will flip odd/evenly to indicate data change. Continuous data reader will thus be able to catch every data change.

Build

This project is meant to be built with cmake and clang. However, it should also build with MSVC and GCC.

./build.sh

Test

Test executables are built automatically and can be executed to verify the correct function of the implementation on your machine:

./test.sh

License

libsharedmemory is released under the MIT license, see the LICENSE file.

Roadmap

  1. Windows shared memory persistency support
  2. Multi-threaded non-blocking onChange( lambda fn ) data change handler on the read stream
Owner
Aron Homberg
“the more a mind takes in the more it expands” - Seneca
Aron Homberg
Implementation of System V shared memory (a type of inter process communication) in xv6 operating system.

NOTE: we have stopped maintaining the x86 version of xv6, and switched our efforts to the RISC-V version (https://github.com/mit-pdos/xv6-riscv.git)

Viraj Jadhav 5 Feb 21, 2022
Using shared memory to communicate between two executables or processes, for Windows, Linux and MacOS (posix). Can also be useful for remote visualization/debugging.

shared-memory-example Using shared memory to communicate between two executables or processes, for Windows, Linux and MacOS (posix). Can also be usefu

null 8 Mar 18, 2022
MMCTX (Memory Management ConTeXualizer), is a tiny (< 300 lines), single header C99 library that allows for easier memory management by implementing contexts that remember allocations for you and provide freeall()-like functionality.

MMCTX (Memory Management ConTeXualizer), is a tiny (< 300 lines), single header C99 library that allows for easier memory management by implementing contexts that remember allocations for you and provide freeall()-like functionality.

A.P. Jo. 4 Oct 2, 2021
Public domain cross platform lock free thread caching 16-byte aligned memory allocator implemented in C

rpmalloc - General Purpose Memory Allocator This library provides a public domain cross platform lock free thread caching 16-byte aligned memory alloc

Mattias Jansson 1.5k May 10, 2022
Alloc-test - Cross-platform benchmarking for memory allocators, aiming to be as close to real world as it is practical

Alloc-test - Cross-platform benchmarking for memory allocators, aiming to be as close to real world as it is practical

null 36 Mar 17, 2022
The Hoard Memory Allocator: A Fast, Scalable, and Memory-efficient Malloc for Linux, Windows, and Mac.

The Hoard Memory Allocator Copyright (C) 1998-2020 by Emery Berger The Hoard memory allocator is a fast, scalable, and memory-efficient memory allocat

Emery Berger 882 May 13, 2022
Custom memory allocators in C++ to improve the performance of dynamic memory allocation

Table of Contents Introduction Build instructions What's wrong with Malloc? Custom allocators Linear Allocator Stack Allocator Pool Allocator Free lis

Mariano Trebino 1.2k May 9, 2022
Memory-dumper - A tool for dumping files from processes memory

What is memory-dumper memory-dumper is a tool for dumping files from process's memory. The main purpose is to find patterns inside the process's memor

Alexander Nestorov 29 Feb 5, 2022
Mesh - A memory allocator that automatically reduces the memory footprint of C/C++ applications.

Mesh: Compacting Memory Management for C/C++ Mesh is a drop in replacement for malloc(3) that can transparently recover from memory fragmentation with

PLASMA @ UMass 1.4k May 7, 2022
STL compatible C++ memory allocator library using a new RawAllocator concept that is similar to an Allocator but easier to use and write.

memory The C++ STL allocator model has various flaws. For example, they are fixed to a certain type, because they are almost necessarily required to b

Jonathan Müller 1.1k May 9, 2022
Modern C++ 32bit Windows Process Memory Library.

basil Simple 32-bit Windows Process Memory Library. Written in Modern C++. JavaScript bindings basil (wilL) be available as bindings for JavaScript. C

cristei 3 Sep 23, 2021
STL compatible C++ memory allocator library using a new RawAllocator concept that is similar to an Allocator but easier to use and write.

STL compatible C++ memory allocator library using a new RawAllocator concept that is similar to an Allocator but easier to use and write.

Jonathan Müller 1k Dec 2, 2021
A simple C++ library for creating and managing bitstreams in memory.

ezbitstream (v0.001) A simple C++ library for creating and managing bitstreams in memory. API & Implementation ezbitstream implements bitstreams with

Unsal Ozturk 2 Feb 4, 2022
OpenXenium JTAG and Flash Memory programmer

OpenXenium JTAG and Flash Memory programmer * Read: "Home Brew" on ORIGINAL XBOX - a detailed article on why and how * The tools in this repo will all

Koos du Preez 25 Feb 14, 2022
manually map driver for a signed driver memory space

smap manually map driver for a signed driver memory space credits https://github.com/btbd/umap tested system Windows 10 Education 20H2 UEFI installati

ekknod 71 Apr 9, 2022
Memory instrumentation tool for android app&game developers.

Overview LoliProfiler is a C/C++ memory profiling tool for Android games and applications. LoliProfiler supports profiling debuggable applications out

Tencent 416 May 13, 2022
A single file drop-in memory leak tracking solution for C++ on Windows

MemLeakTracker A single file drop-in memory leak tracking solution for C++ on Windows This small piece of code allows for global memory leak tracking

null 22 Apr 23, 2022
Dump the memory of a PPL with a userland exploit

PPLdump This tool implements a userland exploit that was initially discussed by James Forshaw (a.k.a. @tiraniddo) - in this blog post - for dumping th

Clément Labro 546 May 14, 2022
An In-memory Embedding of CPython

An In-memory Embedding of CPython This repository contains all the build artifacts necessary to build an embedding of CPython 3.8.2 that can be run en

null 100 Apr 26, 2022