Collection of miscellaneous portable C snippets.

Overview

Portable Snippets

This is a collection of public domain (CC0) code snippets written in C for performing various common tasks which are typically OS, architecture, and/or compiler-dependent. Basically, our goal is to move those annoying preprocessor conditionals from your code to ours.

Modules have no fixed target (such as C89) since that would preclude some functionality; instead, we simply try to provide the widest support we can for each module. If you have a platform which isn't supported for a particular feature but could be, please let us know; we'd be happy to try to work out a way to support it.

Everything is tested continuously with various versions of GCC, Clang, and PGI (on Travis CI), MSVC (on AppVeyor), and Emscripten (on Codefresh):

Travis CI status AppVeyor status Codefresh status

Currently ready-to-use modules include:

  • builtin — use compiler built-ins/intrinsics, or fall back on standard C
  • endian — endianness detection and swapping
  • atomic — common atomic synchronization operations
  • safe-math — overflow-safe integer functions
  • exact-int — exact-width integer types (think <stdint.h>)
  • clock — cross-platform wall clock, CPU time, and monotonic time

There are also modules which may not yet be ready for widespread use, but would benefit greatly from testing:

  • unaligned — fast unaligned loads & stores
  • once — one-time initialization
  • random — random number generation (3 flavors: cryptographic, reproducible, and fast)
  • debug-trap — debugging traps and assertions
  • check — assertion checking

Modules can be used in isolation, though some do work better together, so if you can keep the whole repository together instead of just copying a single file we recommend you do so.

Please don't be shy about filing issues about any of these; if you have problems it's likely others will, as well, so we would like to fix them.

If you have something you would like to contribute, please file an issue or pull request. If you'd like to help out by writing something new, take a look at the "enhancement" issues in our issue tracker for ideas.

Some things don't really fit this repository, so here are a few small-ish projects for doing things portably which you may be interested in:

  • Hedley — macros to enable compiler-specific features which make your code easier to use, harder to misuse, safer, faster, and more portable.
  • parg — Argument parsing similar to getopt/getopt_long. Public domain.
  • TinyCThread — Implements the C11 threads API on top of pthreads and the Windows API. zlib license.
  • win-iconv — Windows implementation of the iconv API (character set conversion).
  • mman-win32mman.h (mmap(), etc.) implementation for Windows.

If there is a project you'd like to see added to the list, please file an issue or pull request.

Issues
  • The codefresh badge is showing as

    The codefresh badge is showing as "Build not found"

    At least to me as I see it in README.md.

    It links here which ultimately just bumps me back to the codefresh.io welcome page.

    I don't know squat about codefresh so I can't help too much here...

    opened by travisdowns 9
  • Rotation code is UB for zero shift counts

    Rotation code is UB for zero shift counts

    If I understand it correctly existing portable rotation code will emit a shift count of 32 if you rotate a 4-byte value by 0:

    #define PSNIP_BUILTIN_ROTL_DEFINE_PORTABLE(f_n, T, ST)	\
      PSNIP_BUILTIN__FUNCTION				\
      T psnip_intrin_##f_n(T value, ST shift) {		\
        return						\
          (value >> ((sizeof(T) * 8) - shift)) |		\
          (value << shift);					\
    }
    

    Shifts of amounts equal or greater than the size of the underlying value are UB in C and C++, so a shift of 32 is not legal here.

    Here is a good summary of portable, legal ways to implement this. They are generally just as fast (i.e., recognized) on current compilers, but using it on older compilers may be slow.

    opened by travisdowns 9
  • C11 backend detection for once is broken due to missing underscore

    C11 backend detection for once is broken due to missing underscore

    At line 26 in once.h we auto-detect use of C11 threads:

    #    if __has_include(<threads.h>)
    #      include <threads.h>
    #      define PSNIP_ONCE_BACKEND PSNIP_ONCE_BACKEND_C11
    #    endif
    

    However, PSNIP_ONCE_BACKEND_C11 should be PSNIP_ONCE__BACKEND_C11 (note the extra underscore). Same issue occurs on line 36.

    Originally reported by user jmesmon over here.

    opened by travisdowns 5
  • "gnu" mode is needed for some modules like clock

    Today I got all kinds of weird compile errors while building with clock/clock.h.

    The underlying cause is that I had -std=c11 on the compile command line, which means certainly POSIXY things not in C are not available, including clock_gettime.

    As a simple solution maybe clock could error out in that case, with a message about needing gnu mode, rather than give the compiler errors.

    More comprehensive fixes are possible, but none are necessarily perfect. For example, you could declare the required macros yourself, but this won't to work if time.h was already included by someone else earlier.

    I worked around it by putting a #define _GNU_SOURCE prior to including clock/clock.h.

    opened by travisdowns 3
  • pragma message

    pragma message

    Not sure in which module this would fit into, or if it's even interesting, but it would be nice to have portable ways of emitting messages at compile time, along the lines of what gcc offers with #pragma message since about the 4.4 version. I guess you'd have to emulate it with warnings before that?

    opened by travisdowns 3
  • CC0

    CC0

    According to https://opensource.org/faq#cc-zero, the OSI does not recommend to release OSS as public domain or CC0. Do you see any chance to change the license to BSD style, MIT or ISC?

    opened by tbeu 2
  • Dependencies between components

    Dependencies between components

    @nemequ let's say I want to use a particular component of portable-snippets, say "clock" - is there some way to determine or pull to in all the modules that clock needs? I have tried the brute force approach of adding stuff until it compiles, and that is a bit annoying - but more importantly it doesn't seem reliable because the exact things a module depends on may depend on the platform: compile it somewhere else and it may want atomic when the original platform didn't.

    Just throwing this out there to see if you have any thoughts.

    I face this issue since for a small projects of 3 or 4 files, protable-snippets is still very useful but I don't want to bring in 20 files to make just clock work. It's still better than writing it myself though :).

    opened by travisdowns 2
  • Limit macros for unsigned values

    Limit macros for unsigned values

    In exact-int.h, I believe the limits.h macros for unsigned values are USHRT_MAX, UINT_MAX, ULONG_MAX, and ULLONG_MAX.

    Also, the LLONG versions are only (required to be) available in C99/C++11 and forward.

    I think technically, you are not guaranteed that the signed integer types are two's complement, so they could theoretically not work as base for the fixed-width types. But this is pretty far out on the what-if scale.

    opened by jibsen 2
  • Add `psnip_explicit_bzero` function for securely erasing a buffer

    Add `psnip_explicit_bzero` function for securely erasing a buffer

    Closes #39.

    Changes:

    • Added version checks for Glibc, OpenBSD, NetBSD & FreeBSD.
    • Added code to use memset_s C11 Annex K function, as it is guaranteed to not be optimized away.
    • Added tests for testing psnip_explicit_bzero function.
    • Use trick used by OpenSSL's OPENSSL_cleanse function to access memset using a volatile pointer, so that compiler does not optimize the function call.
    opened by arnavyc 1
  • Check module clarification

    Check module clarification

    Maybe I am being pedantic, but in the documentation for the check module it says (bolding mine):

    Additionally, unless you define PSNIP_CHECK_FAIL_DEFINED, a failing assertion will considered undefined behavior even when NDEBUG is defined.

    The "even" there seems wrong to me - I think it is saying "a failing assertion is UB even when NDEBUG is defined [and obviously also in the case it is not]", the part in brackets being implied by the "even"?

    However a failing assertion should not be UB when NDEBUG is not defined, it should do the well-defined assert thing of aborting the program with a diagnostic.

    If you agree, I think just removing the word even fixes it entirely.

    opened by travisdowns 1
  • Reconsider prioitizing use of CLOCK_MONOTONIC_RAW over non-RAW

    Reconsider prioitizing use of CLOCK_MONOTONIC_RAW over non-RAW

    In clock.h it seems that CLOCK_MONOTONIC_RAW is favored over CLOCK_MONOTONIC (the latter is selected only if the former doesn't exist).

    I think in most cases CLOCK_MONOTONIC is better. Most importantly, it is much faster than CLOCK_MONOTONIC_RAW since it is has been implemented in the vDSO in all recent Linux versions, but CLOCK_MONOTONIC_RAW is a full system call. The difference is something like 10 nanoseconds versus 300ish nanoseconds (after Meltdown and Spectre anyways - the act performance depends a lot of what mitigations you have selected and other features).

    CLOCK_MONOTONIC is still monotonic, but it is subject to NTP "rate" adjustments (i.e., when the local oscillator rate is slewed by some factor in order to cancel out error wrt network time). These are very gentle and preserve monotonicity. It isn't subject to NTP "jumps" so it never goes backwards (or big jumps forwards either, but backwards is "detectable"). Except for very unusual cases, that's what you want from a monotonic clock.

    _RAW doesn't do the rate adjustment.

    opened by travisdowns 1
  • Intel/PGI compiler notes.

    Intel/PGI compiler notes.

    Per Intel Compilers for Linux: Compatibility with GNU Compilers (page 6), Intel compilers prior to OneAPI would attempt to guess the version of of gcc on the system based on what was found in the ${PATH} environment variable and silently degrade its feature set to be compatible what that version of gcc.

    This is almost certainly not a problem unless you happen to work in a certain US Department of Energy nuclear lab where the default version of gcc is still 4.9.3. If that's the case and you have the latest version of icc in your path, __builtin_[add|sub|mul]_overflow will not be defined and anything using math-safe.h fail to compile.

    The workaround is to use -gcc-name=${PATH_TO_MORE_RECENT_GCC} during compilation.

    Based on a much more cursory search, PGI compilers not not appear to implement the necessary builtins.

    Happy to write this up as a PR for the readme if there's interest.

    opened by rountree 0
  • builtin.h does not compile with Arduino IDE g++

    builtin.h does not compile with Arduino IDE g++

    I am compiling psnip using the Arduino IDE for use on a Teensy, but it results in a compilation error when including psnip/builtin/builtin.h in a .cpp file. When I include it in a .c file it works fine, so that it suggests it has something to do specifically with C++. In another issue I've read that you intend to support C++, but that it's just less well-tested.

    In file included from C:\Users\ninjoh\AppData\Local\Temp\arduino_build_148318\sketch\test.cpp:2:0:
    C:\Users\ninjoh\AppData\Local\Temp\arduino_build_148318\sketch\psnip/builtin/builtin.h: In function 'unsigned char psnip_intrin_BitScanForward64(long unsigned int*, uint64_t)':
    C:\Users\ninjoh\AppData\Local\Temp\arduino_build_148318\sketch\psnip/builtin/builtin.h:636:63: error: '_' was not declared in this scope
     #  define psnip_builtin_ctz64(x) PSNIP_BUILTIN__VARIANT_INT64(_,ctz)(x)
                                                                   ^
    C:\Users\ninjoh\AppData\Local\Temp\arduino_build_148318\sketch\psnip/builtin/builtin.h:1253:61: note: in expansion of macro 'psnip_builtin_ctz64'
       return PSNIP_BUILTIN_UNLIKELY(Mask == 0) ? 0 : ((*Index = psnip_builtin_ctz64 (Mask)), 1);
                                                                 ^
    C:\Users\ninjoh\AppData\Local\Temp\arduino_build_148318\sketch\psnip/builtin/builtin.h:636:65: error: 'ctz' was not declared in this scope
     #  define psnip_builtin_ctz64(x) PSNIP_BUILTIN__VARIANT_INT64(_,ctz)(x)
                                                                     ^
    C:\Users\ninjoh\AppData\Local\Temp\arduino_build_148318\sketch\psnip/builtin/builtin.h:1253:61: note: in expansion of macro 'psnip_builtin_ctz64'
       return PSNIP_BUILTIN_UNLIKELY(Mask == 0) ? 0 : ((*Index = psnip_builtin_ctz64 (Mask)), 1);
                                                                 ^
    C:\Users\ninjoh\AppData\Local\Temp\arduino_build_148318\sketch\psnip/builtin/builtin.h:636:68: error: 'PSNIP_BUILTIN__VARIANT_INT64' was not declared in this scope
     #  define psnip_builtin_ctz64(x) PSNIP_BUILTIN__VARIANT_INT64(_,ctz)(x)
                                                                        ^
    C:\Users\ninjoh\AppData\Local\Temp\arduino_build_148318\sketch\psnip/builtin/builtin.h:1253:61: note: in expansion of macro 'psnip_builtin_ctz64'
       return PSNIP_BUILTIN_UNLIKELY(Mask == 0) ? 0 : ((*Index = psnip_builtin_ctz64 (Mask)), 1);
                                                                 ^
    C:\Users\ninjoh\AppData\Local\Temp\arduino_build_148318\sketch\psnip/builtin/builtin.h: In function 'unsigned char psnip_intrin_BitScanReverse64(long unsigned int*, uint64_t)':
    C:\Users\ninjoh\AppData\Local\Temp\arduino_build_148318\sketch\psnip/builtin/builtin.h:481:63: error: '_' was not declared in this scope
     #  define psnip_builtin_clz64(x) PSNIP_BUILTIN__VARIANT_INT64(_,clz)(x)
                                                                   ^
    C:\Users\ninjoh\AppData\Local\Temp\arduino_build_148318\sketch\psnip/builtin/builtin.h:1287:97: note: in expansion of macro 'psnip_builtin_clz64'
       return (PSNIP_BUILTIN_UNLIKELY(Mask == 0)) ? 0 : ((*Index = ((sizeof(Mask) * CHAR_BIT) - 1) - psnip_builtin_clz64 (Mask)), 1);
                                                                                                     ^
    C:\Users\ninjoh\AppData\Local\Temp\arduino_build_148318\sketch\psnip/builtin/builtin.h:481:65: error: 'clz' was not declared in this scope
     #  define psnip_builtin_clz64(x) PSNIP_BUILTIN__VARIANT_INT64(_,clz)(x)
                                                                     ^
    C:\Users\ninjoh\AppData\Local\Temp\arduino_build_148318\sketch\psnip/builtin/builtin.h:1287:97: note: in expansion of macro 'psnip_builtin_clz64'
       return (PSNIP_BUILTIN_UNLIKELY(Mask == 0)) ? 0 : ((*Index = ((sizeof(Mask) * CHAR_BIT) - 1) - psnip_builtin_clz64 (Mask)), 1);
                                                                                                     ^
    C:\Users\ninjoh\AppData\Local\Temp\arduino_build_148318\sketch\psnip/builtin/builtin.h:481:68: error: 'PSNIP_BUILTIN__VARIANT_INT64' was not declared in this scope
     #  define psnip_builtin_clz64(x) PSNIP_BUILTIN__VARIANT_INT64(_,clz)(x)
                                                                        ^
    C:\Users\ninjoh\AppData\Local\Temp\arduino_build_148318\sketch\psnip/builtin/builtin.h:1287:97: note: in expansion of macro 'psnip_builtin_clz64'
       return (PSNIP_BUILTIN_UNLIKELY(Mask == 0)) ? 0 : ((*Index = ((sizeof(Mask) * CHAR_BIT) - 1) - psnip_builtin_clz64 (Mask)), 1);
    
    

    sample code:

    test.cpp
    ---
    #include <stdint.h>
    #include "psnip/builtin/builtin.h"
    #include "psnip/endian/endian.h"
    
    void test_do_nothing(void)
    {
      uint16_t be = psnip_endian_be16(200);
    }
    

    How g++ is actually called by the Arduino IDE (newlines added for clarity):

    "C:\\Users\\ninjoh\\arduino-1.8.13\\hardware\\teensy/../tools/arm/bin/arm-none-eabi-g++" 
    -c 
    -O3 
    -flto 
    -fno-fat-lto-objects 
    -g 
    -Wall 
    -ffunction-sections 
    -fdata-sections 
    -nostdlib
    -MMD 
    -fno-exceptions 
    -fpermissive 
    -felide-constructors 
    -std=gnu++14 
    -Wno-error=narrowing 
    -fno-rtti 
    -mthumb 
    -mcpu=cortex-m4 
    -fsingle-precision-constant 
    -D__MK20DX256__ 
    -DTEENSYDUINO=153 
    -DARDUINO=10813 
    -DARDUINO_TEENSY32 
    -DF_CPU=24000000 
    -DUSB_SERIAL_HID 
    -DLAYOUT_US_ENGLISH
    "-IC:\\Users\\ninjoh\\AppData\\Local\\Temp\\arduino_build_148318/pch" 
    "-IC:\\Users\\ninjoh\\arduino-1.8.13\\hardware\\teensy\\avr\\cores\\teensy3"
    "-IC:\\Users\\ninjoh\\arduino-1.8.13\\portable\\sketchbook\\libraries\\MFRC522\\src" 
    "-IC:\\Users\\ninjoh\\arduino-1.8.13\\hardware\\teensy\\avr\\libraries\\SPI"
    "-IC:\\Users\\ninjoh\\arduino-1.8.13\\portable\\sketchbook\\libraries\\Base64\\src" 
    "-IC:\\Users\\ninjoh\\arduino-1.8.13\\portable\\sketchbook\\libraries\\ArduinoBearSSL\\src" 
    "-IC:\\Users\\ninjoh\\arduino-1.8.13\\hardware\\teensy\\avr\\libraries\\EEPROM" 
    "-IC:\\Users\\ninjoh\\arduino-1.8.13\\portable\\sketchbook\\libraries\\EDB"
    "-IC:\\Users\\ninjoh\\arduino-1.8.13\\hardware\\teensy\\avr\\libraries\\Wire" 
    "-IC:\\Users\\ninjoh\\arduino-1.8.13\\hardware\\teensy\\avr\\libraries\\SD"
    "-IC:\\Users\\ninjoh\\arduino-1.8.13\\portable\\sketchbook\\libraries\\ArduinoJson\\src" 
    "C:\\Users\\ninjoh\\AppData\\Local\\Temp\\arduino_build_148318\\sketch\\test.cpp"
    -o "C:\\Users\\ninjoh\\AppData\\Local\\Temp\\arduino_build_148318\\sketch\\test.cpp.o"
    
    opened by martijn-heil 3
  • Migrate CI away from from Travis CI

    Migrate CI away from from Travis CI

    I have been waiting for CI checks in #40, that resolves #39, to complete for about 15 days. However it always shows Expected — Waiting for status to be reported.

    Screenshot (3)

    opened by arnavyc 2
  • Add a function for securely erasing a buffer

    Add a function for securely erasing a buffer

    After using any sensitive data such as passwords, the buffers which store them must be cleared by the code to prevent it from being misused by a malicious actor by performing core dumps or using other means to access the memory of the application. However, most compilers optimize away the memset call as they interpret the call as dead code because the memory being written to is not being used afterwards.

    I have already added a psnip_memset_explicit function in https://github.com/arnavyc/portable-snippets/tree/arnavyc/memset-explicit/memset-explicit that can erase a buffer without fear of 'dead-store removal'

    opened by arnavyc 3
  • Add versioning

    Add versioning

    Hello,

    Any chance to add versioning for these libraries? I use safemath and it's a bit hard to tell when a new version is "changed" or "released".

    Thanks

    opened by tawmoto 0
Owner
Evan Nemerson
Evan Nemerson
Miscellaneous data mangling utilities for Path of Exile

poe-utils Miscellaneous data mangling utilities for Path of Exile. process-image Replacement tool for ImageMagick to convert from many DDS pixel forma

Lars Viklund 2 Nov 24, 2021
Not related to software bugs and exploits; this repo contains snippets of code that demonstrate some interesting functionality or a handy trick.

Proof-of-Concept Not related to software bugs and exploits; this repo contains snippets of code that demonstrate some interesting functionality or a h

Alisa Esage 31 May 29, 2022
Library of useful C++ snippets and reusable classes I've created as I build out Arduino Uno and ESP32 projects.

Arduino Snippets Library of useful C++ snippets and reusable classes I've created as I build out Arduino Uno and ESP32 projects. Button A simple butto

Max Lynch 7 Feb 5, 2022
CONTEST 7 : 🥳 #snipathon-21🥳 contribute your favour code snippets in different languages

Your Favorite Code Snippet Collection of code snippets that can be anything from hello world to Complex Algo & DS (In any programming language). How t

CodeMacrocosm 8 Oct 31, 2021
A One Stop Place to find many useful Snippets for competitive programming

CP_Snippets A One Stop Place to find many useful Snippets for competetive programming Contribution Procedure Raise an issue about the snippet you want

Rasesh Shetty 5 Dec 6, 2021
Write snippets of C code in your txt files for notes and skip the hassle of compiling and running

Write snippets of C code in your txt files for notes and skip the hassle of compiling and running. Greatly helps organization and note-taking to make sure you do not miss anything.

Seamus Walden 4 Jun 13, 2022
WIP Plan 9 port of Simon Tatham's Portable Puzzle Collection

This is the README accompanying the source code to Simon Tatham's puzzle collection. The collection's web site is at <https://www.chiark.greenend.org.

Sean Hinchee 4 Sep 16, 2021
sbase is a collection of unix tools that are inherently portable across UNIX and UNIX-like systems.

sbase is a collection of unix tools that are inherently portable across UNIX and UNIX-like systems.

Anton Samokhvalov 1 Nov 1, 2021
A personal collection of Windows CVE I have turned in to exploit source, as well as a collection of payloads I've written to be used in conjunction with these exploits.

This repository contains a personal collection of Windows CVE I have turned in to exploit source, as well as a collection of payloads I've written to

null 76 Aug 10, 2022
C.impl is a small portable C interpreter integrated with a line text editor

C.impl C.impl is a small portable C interpreter integrated with a line text editor, originally developed for the ELLO 1A computer: http://ello.cc The

KnivD 19 Jun 9, 2022
A portable full system emulator of Z280 based boards

z280emu A portable full system emulator of Z280 based boards Motivation There is no working Z280 emulator that I'm aware of so I decided to write one.

Michal Tomek 14 Jul 21, 2022
ASMotor is a portable and generic assembler engine and development system written in ANSI C99

ASMotor is a portable and generic assembler engine and development system written in ANSI C99 and licensed under the GNU Public License v3. The package consists of the assembler, the librarian and the linker. It can be used as either a cross or native development system.

null 40 Jul 28, 2022
F Graphics Library (FGL) is a small graphics C++ portable library for LCD displays on embedded systems

F Graphics Library (FGL) Full documentation: fgl.docsforge.com (By Filipe Chagas) F Graphics Library is a C++ library that I created for use in embedd

Filipe Chagas 8 Dec 14, 2021
Portable, Noto-powered font-rendering abstraction based on FreeType and Raqm

Mechascribe Mechascribe is still under construction and nothing is functional yet. Mechascribe is a text rendering abstraction trying to support as mu

null 4 Jan 27, 2022
Tiny and portable usb host and device stack for mcu with usb ip

Tiny and portable usb host and device stack for mcu with usb ip

sakumisu 414 Aug 12, 2022
Flexible, portable, high-performance bit fields C++ library. unsigned a:13 becomes F<13> a;

C-plus-plus-library-bit-fields Flexible, portible, high-performance bit fields C++ library. The bit fields are specified with a dummy structure where

Walt Karas 25 Feb 27, 2022
BMI_automatics-faceshielding-device - this is a project utilizing STM32, MATLAB,Softmax classifier to achieve a portable face shielding(mask) device

BMI_automatics-faceshielding-device this is a project utilizing STM32, MATLAB,Softmax classifier to achieve a portable face shielding(mask) device in

Lyu Tian 2 Dec 23, 2021
Mirror of Apache Portable Runtime

Apache Portable Runtime Library (APR) ===================================== The Apache Portable Runtime Library provides a predictable and cons

The Apache Software Foundation 370 Aug 9, 2022
OpenVi is an enhanced and portable implementation of the Berkeley vi / ex text editor, originally developed by Bill Joy.

OpenVi is an enhanced and portable implementation of the Berkeley vi / ex text editor, originally developed by Bill Joy.

Jeffrey H. Johnson 91 Aug 9, 2022