Cross-platform C++11 header-only library for memory mapped file IO

Overview

mio

An easy to use header-only cross-platform C++11 memory mapping library with an MIT license.

mio has been created with the goal to be easily includable (i.e. no dependencies) in any C++ project that needs memory mapped file IO without the need to pull in Boost.

Please feel free to open an issue, I'll try to address any concerns as best I can.

Why?

Because memory mapping is the best thing since sliced bread!

More seriously, the primary motivation for writing this library instead of using Boost.Iostreams, was the lack of support for establishing a memory mapping with an already open file handle/descriptor. This is possible with mio.

Furthermore, Boost.Iostreams' solution requires that the user pick offsets exactly at page boundaries, which is cumbersome and error prone. mio, on the other hand, manages this internally, accepting any offset and finding the nearest page boundary.

Albeit a minor nitpick, Boost.Iostreams implements memory mapped file IO with a std::shared_ptr to provide shared semantics, even if not needed, and the overhead of the heap allocation may be unnecessary and/or unwanted. In mio, there are two classes to cover the two use-cases: one that is move-only (basically a zero-cost abstraction over the system specific mmapping functions), and the other that acts just like its Boost.Iostreams counterpart, with shared semantics.

How to create a mapping

NOTE: the file must exist before creating a mapping.

There are three ways to map a file into memory:

  • Using the constructor, which throws a std::system_error on failure:
mio::mmap_source mmap(path, offset, size_to_map);

or you can omit the offset and size_to_map arguments, in which case the entire file is mapped:

mio::mmap_source mmap(path);
  • Using the factory function:
std::error_code error;
mio::mmap_source mmap = mio::make_mmap_source(path, offset, size_to_map, error);

or:

mio::mmap_source mmap = mio::make_mmap_source(path, error);
  • Using the map member function:
std::error_code error;
mio::mmap_source mmap;
mmap.map(path, offset, size_to_map, error);

or:

mmap.map(path, error);

NOTE: The constructors require exceptions to be enabled. If you prefer to build your projects with -fno-exceptions, you can still use the other ways.

Moreover, in each case, you can provide either some string type for the file's path, or you can use an existing, valid file handle.

#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <mio/mmap.hpp>
// #include <mio/mio.hpp> if using single header
#include <algorithm>

int main()
{
    // NOTE: error handling omitted for brevity.
    const int fd = open("file.txt", O_RDONLY);
    mio::mmap_source mmap(fd, 0, mio::map_entire_file);
    // ...
}

However, mio does not check whether the provided file descriptor has the same access permissions as the desired mapping, so the mapping may fail. Such errors are reported via the std::error_code out parameter that is passed to the mapping function.

WINDOWS USERS: This library does support the use of wide character types for functions where character strings are expected (e.g. path parameters).

Example

#include <mio/mmap.hpp>
// #include <mio/mio.hpp> if using single header
#include <system_error> // for std::error_code
#include <cstdio> // for std::printf
#include <cassert>
#include <algorithm>
#include <fstream>

int handle_error(const std::error_code& error);
void allocate_file(const std::string& path, const int size);

int main()
{
    const auto path = "file.txt";

    // NOTE: mio does *not* create the file for you if it doesn't exist! You
    // must ensure that the file exists before establishing a mapping. It
    // must also be non-empty. So for illustrative purposes the file is
    // created now.
    allocate_file(path, 155);

    // Read-write memory map the whole file by using `map_entire_file` where the
    // length of the mapping is otherwise expected, with the factory method.
    std::error_code error;
    mio::mmap_sink rw_mmap = mio::make_mmap_sink(
            path, 0, mio::map_entire_file, error);
    if (error) { return handle_error(error); }

    // You can use any iterator based function.
    std::fill(rw_mmap.begin(), rw_mmap.end(), 'a');

    // Or manually iterate through the mapped region just as if it were any other 
    // container, and change each byte's value (since this is a read-write mapping).
    for (auto& b : rw_mmap) {
        b += 10;
    }

    // Or just change one value with the subscript operator.
    const int answer_index = rw_mmap.size() / 2;
    rw_mmap[answer_index] = 42;

    // Don't forget to flush changes to disk before unmapping. However, if
    // `rw_mmap` were to go out of scope at this point, the destructor would also
    // automatically invoke `sync` before `unmap`.
    rw_mmap.sync(error);
    if (error) { return handle_error(error); }

    // We can then remove the mapping, after which rw_mmap will be in a default
    // constructed state, i.e. this and the above call to `sync` have the same
    // effect as if the destructor had been invoked.
    rw_mmap.unmap();

    // Now create the same mapping, but in read-only mode. Note that calling the
    // overload without the offset and file length parameters maps the entire
    // file.
    mio::mmap_source ro_mmap;
    ro_mmap.map(path, error);
    if (error) { return handle_error(error); }

    const int the_answer_to_everything = ro_mmap[answer_index];
    assert(the_answer_to_everything == 42);
}

int handle_error(const std::error_code& error)
{
    const auto& errmsg = error.message();
    std::printf("error mapping file: %s, exiting...\n", errmsg.c_str());
    return error.value();
}

void allocate_file(const std::string& path, const int size)
{
    std::ofstream file(path);
    std::string s(size, '0');
    file << s;
}

mio::basic_mmap is move-only, but if multiple copies to the same mapping are needed, use mio::basic_shared_mmap which has std::shared_ptr semantics and has the same interface as mio::basic_mmap.

#include <mio/shared_mmap.hpp>

mio::shared_mmap_source shared_mmap1("path", offset, size_to_map);
mio::shared_mmap_source shared_mmap2(std::move(mmap1)); // or use operator=
mio::shared_mmap_source shared_mmap3(std::make_shared<mio::mmap_source>(mmap1)); // or use operator=
mio::shared_mmap_source shared_mmap4;
shared_mmap4.map("path", offset, size_to_map, error);

It's possible to define the type of a byte (which has to be the same width as char), though aliases for the most common ones are provided by default:

using mmap_source = basic_mmap_source<char>;
using ummap_source = basic_mmap_source<unsigned char>;

using mmap_sink = basic_mmap_sink<char>;
using ummap_sink = basic_mmap_sink<unsigned char>;

But it may be useful to define your own types, say when using the new std::byte type in C++17:

using mmap_source = mio::basic_mmap_source<std::byte>;
using mmap_sink = mio::basic_mmap_sink<std::byte>;

Though generally not needed, since mio maps users requested offsets to page boundaries, you can query the underlying system's page allocation granularity by invoking mio::page_size(), which is located in mio/page.hpp.

Single Header File

Mio can be added to your project as a single header file simply by including \single_include\mio\mio.hpp. Single header files can be regenerated at any time by running the amalgamate.py script within \third_party.

python amalgamate.py -c config.json -s ../include

CMake

As a header-only library, mio has no compiled components. Nevertheless, a CMake build system is provided to allow easy testing, installation, and subproject composition on many platforms and operating systems.

Testing

Mio is distributed with a small suite of tests and examples. When mio is configured as the highest level CMake project, this suite of executables is built by default. Mio's test executables are integrated with the CMake test driver program, CTest.

CMake supports a number of backends for compilation and linking.

To use a static configuration build tool, such as GNU Make or Ninja:

cd <mio source directory>
mkdir build
cd build

# Configure the build
cmake -D CMAKE_BUILD_TYPE=<Debug | Release> \
      -G <"Unix Makefiles" | "Ninja"> ..

# build the tests
< make | ninja | cmake --build . >

# run the tests
< make test | ninja test | cmake --build . --target test | ctest >

To use a dynamic configuration build tool, such as Visual Studio or Xcode:

cd <mio source directory>
mkdir build
cd build

# Configure the build
cmake -G <"Visual Studio 14 2015 Win64" | "Xcode"> ..

# build the tests
cmake --build . --config <Debug | Release>

# run the tests via ctest...
ctest --build-config <Debug | Release>

# ... or via CMake build tool mode...
cmake --build . --config <Debug | Release> --target test

Of course the build and test steps can also be executed via the all and test targets, respectively, from within the IDE after opening the project file generated during the configuration step.

Mio's testing is also configured to operate as a client to the CDash software quality dashboard application. Please see the Kitware documentation for more information on this mode of operation.

Installation

Mio's build system provides an installation target and support for downstream consumption via CMake's find_package intrinsic function. CMake allows installation to an arbitrary location, which may be specified by defining CMAKE_INSTALL_PREFIX at configure time. In the absense of a user specification, CMake will install mio to conventional location based on the platform operating system.

To use a static configuration build tool, such as GNU Make or Ninja:

cd <mio source directory>
mkdir build
cd build

# Configure the build
cmake [-D CMAKE_INSTALL_PREFIX="path/to/installation"] \
      [-D BUILD_TESTING=False]                         \
      -D CMAKE_BUILD_TYPE=Release                      \
      -G <"Unix Makefiles" | "Ninja"> ..

# install mio
<make install | ninja install | cmake --build . --target install>

To use a dynamic configuration build tool, such as Visual Studio or Xcode:

cd <mio source directory>
mkdir build
cd build

# Configure the project
cmake [-D CMAKE_INSTALL_PREFIX="path/to/installation"] \
      [-D BUILD_TESTING=False]                         \
      -G <"Visual Studio 14 2015 Win64" | "Xcode"> ..

# install mio
cmake --build . --config Release --target install

Note that the last command of the installation sequence may require administrator privileges (e.g. sudo) if the installation root directory lies outside your home directory.

This installation

  • copies the mio header files to the include/mio subdirectory of the installation root
  • generates and copies several CMake configuration files to the share/cmake/mio subdirectory of the installation root

This latter step allows downstream CMake projects to consume mio via find_package, e.g.

find_package( mio REQUIRED )
target_link_libraries( MyTarget PUBLIC mio::mio )

WINDOWS USERS: The mio::mio target #defines WIN32_LEAN_AND_MEAN and NOMINMAX. The former ensures the imported surface area of the Win API is minimal, and the latter disables Windows' min and max macros so they don't intefere with std::min and std::max. Because mio is a header only library, these defintions will leak into downstream CMake builds. If their presence is causing problems with your build then you can use the alternative mio::mio_full_winapi target, which adds none of these defintions.

If mio was installed to a non-conventional location, it may be necessary for downstream projects to specify the mio installation root directory via either

  • the CMAKE_PREFIX_PATH configuration option,
  • the CMAKE_PREFIX_PATH environment variable, or
  • mio_DIR environment variable.

Please see the Kitware documentation for more information.

In addition, mio supports packaged relocatable installations via CPack. Following configuration, from the build directory, invoke cpack as follows to generate a packaged installation:

cpack -G <generator name> -C Release

The list of supported generators varies from platform to platform. See the output of cpack --help for a complete list of supported generators on your platform.

Subproject Composition

To use mio as a subproject, copy the mio repository to your project's dependencies/externals folder. If your project is version controlled using git, a git submodule or git subtree can be used to syncronize with the updstream repository. The use and relative advantages of these git facilities is beyond the scope of this document, but in brief, each may be established as follows:

# via git submodule
cd <my project's dependencies directory>
git submodule add -b master https://github.com/mandreyel/mio.git

# via git subtree
cd <my project's root directory>
git subtree add --prefix <path/to/dependencies>/mio       \
    https://github.com/mandreyel/mio.git master --squash

Given a mio subdirectory in a project, simply add the following lines to your project's to add mio include directories to your target's include path.

add_subdirectory( path/to/mio/ )
target_link_libraries( MyTarget PUBLIC <mio::mio | mio> )

Note that, as a subproject, mio's tests and examples will not be built and CPack integration is deferred to the host project.

Comments
  • libzippp::ZipArchive seems to be just support utf8 ....

    libzippp::ZipArchive seems to be just support utf8 ....

    Just as the title, when use libzippp to commpress some file with Chinese in path(not just ANSI) failed, through the source code i find it seems to be just support utf8.... So, I think it can wrap a transcode to support tstring... not utf8 image

    opened by aijyo 17
  • Weird file beginning on Windows 10

    Weird file beginning on Windows 10

    Hi all,

    I've just ported my (previously Linux-only) C++ program to Windows and noticed a weird bug: Any file opened through mio has the same 3 bytes in the beginning which cause my program to crash.

    Here's a sample of my usage:

    mio::mmap_source mmap{"path/to/my/file.txt"};
    std::string beginning{mmap.cbegin(), 200};
    std::cout << beginning << std::endl;
    

    This code is supposed to print the first 200 characters of the file (which is assumed to have hundreds of MBs in size). Instead, the first 3 printed characters are just pure gibberish, and then the file goes on as expected.

    Using open() and read(), I've verified that those bytes are not actually present in the read file. I've also tried it with multiple files to make sure this is something systematic. My current conclusion is that this problem must be caused somewhere within mio. The only workaround that works for me now is to simply start reading at mmap.cbegin() + 3.

    Here's my toolchain:

    • Windows 10 x64
    • CMake 3.0
    • Visual Studio 2017 (15)
    • MSVC 19
    • CMAKE_CXX_STANDARD 14

    Note that the same code has worked transparently without workarounds on Linux for the last several months.

    Thank you for investigating this weird behavior!

    opened by petrmanek 14
  • Travis, CDash, and CMake 3.14 warnings

    Travis, CDash, and CMake 3.14 warnings

    Hello @mandreyel !

    I've encountered a few minor bumps regarding mio's CMake build system, and took the opportunity to make some minor improvements and add continuous testing and monitoring.

    This pull request:

    • Modifies the convention for mio::mio and the windows api (in a backwards compatible way) to facilitate decision deferral.
    • Addresses relative path warning in CMake 3.14
    • Added TravisCI support to the repository
    • Adds CDash-integration
    opened by apmccartney 9
  • Search through mapped file line by line

    Search through mapped file line by line

    Hello,

    Many thanks for sharing the mapping header. Is there a way to loop through the mapped text file line by line of the text? I have a huge data file mapped but would like to search through the various lines of text. It was simple to search using getline and a std::vector to store each line but building the vector is initially very slow.

    Many thanks, JW

    opened by JAWilliams123 8
  • Expanded CMake Support

    Expanded CMake Support

    This commit introduces several features for the CMake build system.

    • CTest integration
    • CDash integration
    • CPack integration
    • CMake find_package support

    To accomodate post-installation consumption, the include preprocessor statements in several library header files were updated to use paths relative to a root include directory rather than paths relative to the file itself.

    opened by apmccartney 8
  • example.cpp does not work

    example.cpp does not work

    I have tried the example.cpp and it compiled successfully. However when I try to run it, it does not work.

    $ g++ example.cpp -Wall -std=c++11 -I ../include -o example
    $ ./example
    

    The error message is:

    error mapping file: No such file or directory, exiting...

    then I tried to touch a file named file.txt,another error occured.

    $ touch file.txt
    $ ./example
    

    error mapping file: Invalid argument, exiting...

    Platform Linux Debian 9 .5 gcc version 6.3.0 20170516 (Debian 6.3.0-18+deb9u1)

    Thanks

    opened by xyang619 7
  • Compilation fails on Windows/MSVC if `UNICODE` is defined

    Compilation fails on Windows/MSVC if `UNICODE` is defined

    On MSVC, when compiling with /D UNICODE, the compiler issues the diagnostic:

    ...\include\mio\detail\basic_mmap.ipp(75): error C2664: 'HANDLE CreateFileW(LPCWSTR,D WORD,DWORD,LPSECURITY_ATTRIBUTES,DWORD,DWORD,HANDLE)': cannot convert argument 1 from 'const char *' to 'LPCWSTR

    I suspect this is because somewhere among the windows headers, CreateFile is defined similar to...

    #ifndef UNICODE
    # define CreateFile CreateFileA
    #else 
    # define CreateFile CreateFileW
    #endif
    

    The quick fix would be to just directly call CreateFileA, but I think this will prevent the use of wchar_t const* and std::wstring types being used as the various template<typename String> parameters.

    opened by gmbeard 6
  • Consider making mio single-header

    Consider making mio single-header

    As suggested by https://news.ycombinator.com/item?id=18264141, it seems like it would be nice to make mio, or at least provide the option, to make mio a single-header library. I'd like to avoid changing the actual project structure (that is, I want to keep the current files separate), so this could be done in the build phase.

    opened by mandreyel 5
  • Initial single header support using amalgamate

    Initial single header support using amalgamate

    This PR was to make progress in regards to #23. I've only used amalgamate a handful of times on small personal projects - so feel free to correct or advise on any specifics on it's use in this project.

    Invoking the script is as simple as python amalgamate.py -c config.json -s ../include

    To verify I simply tested the single header file against example.cpp and the test.cpp.

    Thanks!

    Fixes: #23

    opened by ptrks 4
  • Is it threadsafe?

    Is it threadsafe?

    I assume it is, but without knowing impl details I cannot be sure about it. If I create mapping: mio::mmap_source mmap = mio::make_mmap_source(filePath, error); can I then access mmap from multiple threads to read the file without synchronizing access to it?

    opened by pps83 3
  • comparison between signed and unsigned integer

    comparison between signed and unsigned integer

    mio/include/mio/detail/mmap.ipp:327:24: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
         if(offset + length > file_size)
            ~~~~~~~~~~~~~~~~^~~~~~~~~~~
    
    opened by amallia 3
  • Adding or append text to a sink map

    Adding or append text to a sink map

    Hello everyone,

    I have created a map to a file with text only on the first line and I would like some tips for writing text from an STL vector back to the map which will append the text in the vector to my existing file.

    Many thanks, JW

    opened by JAWilliams123 0
  • ambiguity invoking constructor with MSVC 2019

    ambiguity invoking constructor with MSVC 2019

    Hello,

    The following code worked great for g++ 11.2.0

    int fid = ::open(fname.c_str(), O_RDONLY);
    mio::mmap_source data_mmap(fid, size_t(0), size_t(1000000));
    

    which should use the 2nd form of the constructor. However, with visual studio I get this error:

    FAILED: CMakeFiles/read_archive.dir/read_archive.cpp.obj
    C:\PROGRA~2\MICROS~2\2019\PROFES~1\VC\Tools\MSVC\1429~1.301\bin\HostX64\x64\cl.exe  /nologo /TP -DNOMINMAX -DWIN32_LEAN_AND_MEAN -D_USE_MATH_DEFINES -IC:\src\..\..\3rdparty\json\include -IC:\src\..\..\3rdparty\sha2-1.0.1 -IC:\src\..\..\3rdparty\mio-master\single_include -IC:\src\..\..\install\include /DWIN32 /D_WINDOWS /W3 /GR /EHsc /MD /O2 /Ob2 /DNDEBUG -std:c++17 /showIncludes /FoCMakeFiles\read_archive.dir\read_archive.cpp.obj /FdCMakeFiles\read_archive.dir\ /FS -c C:\src\read_archive.cpp
    C:\src\read_archive.cpp(55): warning C4996: 'open': The POSIX name for this item is deprecated. Instead, use the ISO C and C++ conformant name: _open. See online help for details.
    C:\src\..\..\3rdparty\mio-master\single_include\mio/mio.hpp(1045): error C2672: 'mio::detail::empty': no matching overloaded function found
    C:\src\..\..\3rdparty\mio-master\single_include\mio/mio.hpp(199): note: see reference to function template instantiation 'void mio::basic_mmap<mio::access_mode::read,char>::map<String>(const String &,const mio::basic_mmap<mio::access_mode::read,char>::size_type,const mio::basic_mmap<mio::access_mode::read,char>::size_type,std::error_code &)' being compiled
            with
            [
                String=int
            ]
    C:\src\..\..\3rdparty\mio-master\single_include\mio/mio.hpp(199): note: see reference to function template instantiation 'void mio::basic_mmap<mio::access_mode::read,char>::map<String>(const String &,const mio::basic_mmap<mio::access_mode::read,char>::size_type,const mio::basic_mmap<mio::access_mode::read,char>::size_type,std::error_code &)' being compiled
            with
            [
                String=int
            ]
    C:\src\read_archive.cpp(57): note: see reference to function template instantiation 'mio::basic_mmap<mio::access_mode::read,char>::basic_mmap<int>(const String &,const mio::basic_mmap<mio::access_mode::read,char>::size_type,const mio::basic_mmap<mio::access_mode::read,char>::size_type)' being compiled
            with
            [
                String=int
            ]
    C:\src\read_archive.cpp(57): note: see reference to function template instantiation 'mio::basic_mmap<mio::access_mode::read,char>::basic_mmap<int>(const String &,const mio::basic_mmap<mio::access_mode::read,char>::size_type,const mio::basic_mmap<mio::access_mode::read,char>::size_type)' being compiled
            with
            [
                String=int
            ]
    C:\src\..\..\3rdparty\mio-master\single_include\mio/mio.hpp(1045): error C2783: 'bool mio::detail::empty(const String &)': could not deduce template argument for '<unnamed-symbol>'
    C:\src\..\..\3rdparty\mio-master\single_include\mio/mio.hpp(743): note: see declaration of 'mio::detail::empty'
    ninja: build stopped: subcommand failed.
    

    From what I can tell, it's trying to use the first constructor form; it's using String=int as the template specialization!

    This isn't exactly a bug, since I can clearly use the first constructor form directly (passing fname as the first argument). However, I started to think that it's actually a bad idea to have 2 constructor forms that are identical except for one argument, which is templated ... doesn't that just scream ambiguity??

    My suggestion would be to eliminate (well, deprecate) the 2nd constructor form, and instead create a static method with a descriptive name like fromFileHandle(int, size_t, size_t).

    And -- thank you for producing this project.

    opened by gmabey 0
  • Default initialise is_handle_internal_

    Default initialise is_handle_internal_

    gcc12 flags is_handle_internal_ as maybe being uninitialised. This PR adds a default initialisation for is_handle_internal_ to avoid this.

    /usr/local/include/mio/detail/mmap.ipp:419:8: warning: '<anonymous>.mio::basic_mmap<mio::access_mode::write, char>::is_handle_internal_' may be used uninitialized [-Wmaybe-uninitialized]
      419 |     if(is_handle_internal_)
          |        ^~~~~~~~~~~~~~~~~~~
    
    opened by Bidski 0
  • Forward To C++ 2020 and Compatible C++ 2011

    Forward To C++ 2020 and Compatible C++ 2011

    [C++ 2020 template concept support] with repository https://github.com/mandreyel/mio/ Compatible with c++ standard 2011, and add c++ standard 2020 code.

    #82

    opened by Twilight-Dream-Of-Magic 1
  • Linker errors when using latest mio

    Linker errors when using latest mio

    After updating to latest mio I'm getting linker errors all over the place. I didn't have these with older mio.

    These are the linker errors:

    3>common.lib(compressed_reader.obj) : error LNK2005: "class std::basic_string<wchar_t,struct std::char_traits<wchar_t>,class std::allocator<wchar_t> > __cdecl mio::detail::win::s_2_ws(class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > const &)" ([email protected]@[email protected]@@[email protected][email protected][email protected]@@[email protected][email protected]@@[email protected]@[email protected][email protected]@[email protected]@[email protected]@[email protected]@[email protected]@Z) already defined in common.lib(DataSource.obj)
    4>common.lib(compressed_reader.obj) : error LNK2005: "class std::basic_string<wchar_t,struct std::char_traits<wchar_t>,class std::allocator<wchar_t> > __cdecl mio::detail::win::s_2_ws(class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > const &)" ([email protected]@[email protected]@@[email protected][email protected][email protected]@@[email protected][email protected]@@[email protected]@[email protected][email protected]@[email protected]@[email protected]@[email protected]@[email protected]@Z) already defined in common.lib(DataSource.obj)
    5>common.lib(DataSource.obj) : error LNK2005: "class std::basic_string<wchar_t,struct std::char_traits<wchar_t>,class std::allocator<wchar_t> > __cdecl mio::detail::win::s_2_ws(class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > const &)" ([email protected]@[email protected]@@[email protected][email protected][email protected]@@[email protected][email protected]@@[email protected]@[email protected][email protected]@[email protected]@[email protected]@[email protected]@[email protected]@Z) already defined in common.lib(compressed_reader.obj)
    
    opened by pps83 2
Owner
null
A small C library with that portably invokes native file open, folder select and file save dialogs.

Cross platform (Windows, Mac, Linux) native file dialog library with C and C++ bindings, based on mlabbe/nativefiledialog.

Bernard Teo 294 Nov 24, 2022
C++ implementation of a platform independent endian safe binary file stream

EndianSafeBinaryStream A C++ implementation of a platform independent endian safe binary file stream that can save primitive types and is easily exten

Corbinian Gruber 2 Mar 22, 2022
P1031 low level file i/o and filesystem library for the C++ standard

This is the post-peer-review LLFIO v2 rewrite. You can view its documentation at https://ned14.github.io/llfio/ master branch develop branch CMake das

Niall Douglas 533 Nov 26, 2022
Lightweight, portable and easy to integrate C directory and file reader

TinyDir Lightweight, portable and easy to integrate C directory and file reader. TinyDir wraps dirent for POSIX and FindFirstFile for Windows. Windows

Cong 698 Dec 1, 2022
Lightweight, portable and easy to integrate C directory and file reader

TinyDir Lightweight, portable and easy to integrate C directory and file reader. TinyDir wraps dirent for POSIX and FindFirstFile for Windows. Windows

Cong 696 Nov 26, 2022
localfuse - File system implemented on Linux using C and libfuse

localfuse - File system implemented on Linux using C and libfuse Use pure C to have a simple file system of your own on the linux system Install and b

null 4 Sep 23, 2021
FSearch is a fast file search utility for Unix-like systems based on GTK+3

FSearch is a fast file search utility, inspired by Everything Search Engine. It's written in C and based on GTK3.

Christian Boxdörfer 2.2k Nov 25, 2022
The MHS Filesystem- A very simple linked-list based file system designed for recoverability and low data redundancy. Public domain filesystem (Version 1)

MHS Filesystem The MHS filesystem. Features: can be modified to work with any size of disk or sector, even non powers of two! Allocation bitmap stored

DMHSW 8 Sep 17, 2022
PoC of a native Linux file system to mount container images.

composefs PoC of a native Linux file system to mount container images. It is unfinished and just a few days of work. The idea is to pass a binary blob

Giuseppe Scrivano 0 Jun 9, 2022
LoomIO is an object-level coordination system for distributed file systems.

LoomIO is an object-level coordination system for distributed file systems. It adopts wait-free design to enable interfering object requests self-organizing and obtain an optimized scheduling decision. Currently, LoomIO is implemented and integrated in Ceph.

null 2 Jun 19, 2022
Cross-platform C++11 header-only library for memory mapped file IO

mio An easy to use header-only cross-platform C++11 memory mapping library with an MIT license. mio has been created with the goal to be easily includ

null 1.4k Nov 26, 2022
WineFS is a file system for Persistent Memory (PM) which is aimed at maximizing the performance of memory-mapped applications.

WineFS WineFS is a file system for Persistent Memory (PM) which is aimed at maximizing the performance of memory-mapped applications. WineFS uses a no

UT Systems and Storage Lab 23 Oct 2, 2022
C++98 library that encapsulates memory-mapped-files for POSIX or Windows

Memory-Mapped File C++ Library Tutorial and Reference Purpose This is a library, for the C++98 language and its successive versions, to handle files a

Carlo Milanesi 83 Nov 24, 2022
Code Injection via Memory Mapped Files

MMFCodeInjection This technique leverages File Mapping and APC(s) to execute shellcode into another process. By leveraging file mapping we would not h

null 51 Oct 27, 2022
Cross-platform shared memory stream/buffer, header-only library for IPC in C/C++.

libsharedmemory libsharedmemory is a small C++11 header-only library for using shared memory on Windows, Linux and macOS. libsharedmemory makes it eas

Aron Homberg 9 Jul 31, 2022
Memory Process File System (MemProcFS) is an easy and convenient way of viewing physical memory as files in a virtual file system

The Memory Process File System (MemProcFS) is an easy and convenient way of viewing physical memory as files in a virtual file system.

Ulf Frisk 1.7k Dec 2, 2022
A high performance, shared memory, lock free, cross platform, single file, no dependencies, C++11 key-value store

SimDB A high performance, shared memory, lock free, cross platform, single file, no dependencies, C++11 key-value store. SimDB is part of LAVA (Live A

null 446 Nov 23, 2022
A tiny cross-platform webview library for C/C++/Golang to build modern cross-platform GUIs.

webview for golang and c/c++ A tiny cross-platform webview library for C/C++/Golang to build modern cross-platform GUIs. The goal of the project is to

polevpn 20 Nov 1, 2022
External warzone cheat with manual mapped driver (function hook), overlay (nvidia hijack), simple esp, no recoil

external_warzone_cheat External warzone cheat with manual mapped driver (function hook), overlay (nvidia hijack), simple esp, no recoil Offsests are N

NMan 95 Nov 28, 2022
Loads a signed kernel driver which allows you to map any driver to kernel mode without any traces of the signed / mapped driver.

CosMapper Loads a signed kernel driver (signed with leaked cert) which allows you to map any driver to kernel mode without any traces of the signed /

null 154 Nov 25, 2022