C++11 header-only message digest library

Overview

digestpp

Experimental C++11 header-only message digest library.

Derived from cppcrypto in an attempt to devise a more modern yet flexible and universal C++ API for cryptographic hash functions.

Tested with g++ 6.4.0, clang 4.0.1 and Visual C++ 2017.

Examples

Calculate BLAKE2b digest from a double quoted string and output it in hex format:

cout << blake2b().absorb("The quick brown fox jumps over the lazy dog").hexdigest();

Calculate BLAKE2b-256 digest from an std::string and output it in hex format:

string str = "The quick brown fox jumps over the lazy dog";
cout << blake2b(256).absorb(str).hexdigest();

Calculate SHA-512 digest of a vector and output it in hex format:

vector<unsigned char> v;
// ...fill the vector
cout << sha512().absorb(v.begin(), v.end()).hexdigest();

Calculate SHA-512/256 digest of a C array and output it in hex format:

unsigned char c[32];
// ...fill the array
cout << sha512(256).absorb(c, sizeof(c)).hexdigest();

Calculate SHA-256 digest of a file and output it in hex format:

ifstream file("filename", ios_base::in|ios_base::binary);
cout << sha256().absorb(file).hexdigest();

Generate SHA3-224 digest using multiple calls to absorb():

cout << sha3(224).absorb("The quick brown fox ").absorb("jumps over the lazy dog").hexdigest();

Output binary digest to a vector:

vector<unsigned char> v;
sha3(256).absorb("The quick brown fox jumps over the lazy dog").digest(back_inserter(v));

Output binary digest to a raw C array:

unsigned char buf[32];
sha3(256).absorb("The quick brown fox jumps over the lazy dog").digest(buf, sizeof(buf));

Output binary digest to a stream:

string str = "The quick brown fox jumps over the lazy dog";
string output;
ostringstream os(output);
sha3(256).absorb(str).digest(ostream_iterator<char>(os, ""));

Generate long output using SHAKE-256 extendable output function using multiple calls to squeeze():

vector<unsigned char> v;
shake256 xof;
xof.absorb("The quick brown fox jumps over the lazy dog");
xof.squeeze(1000, back_inserter(v));
xof.squeeze(1000, back_inserter(v));
xof.squeeze(1000, back_inserter(v));
cout << "Squeezed " << v.size() << " bytes." << endl;

Generate 64-byte digest using customizable cSHAKE-256 algorithm and print it in hex format:

cshake256 xof;
xof.set_customization("Customization");
cout << xof.absorb("The quick brown fox jumps over the lazy dog").hexsqueeze(64);

Hasher class

Hasher is a main class template implementing the public API for hashing.

It has two template parameters:

  • HashProvider is a class implementing the algorithm via traditional init/update/final interface. We provide our own implementations of hash functions listed in the next section, but using the traditional interface allows anyone to trivially implement the providers as wrappers over popular libraries, such as OpenSSL, Crypto++, Botan.
  • Mixin is a class template which can be used to inject additional functions to the public API of the hasher, for example for setting the customization string for cSHAKE, the salt for BLAKE, etc.
template<class HashProvider, template <class> class Mixin = detail::null_mixin>
class hasher : public Mixin<HashProvider>
{
public:
    // Default constructor
    // Used for hash functions with fixed output size, for hash functions with sensible
    // default output size and for exendable output functions (XOFs).
    template<typename H=HashProvider,
        typename std::enable_if<std::is_default_constructible<H>::value>::type* = nullptr>
    hasher();

    // Constructor with hashsize parameter
    // Used with hash functions which can produce hashes of different lengths.
    // If the requested output size is not supported by the algorithm, std::runtime_error will be thrown.
    template<typename H=HashProvider, typename std::enable_if<!detail::is_xof<H>::value>::type* = nullptr>
    hasher(size_t hashsize);

     // Absorbs bytes from a C-style pointer to character buffer
    template<typename T, typename std::enable_if<detail::is_byte<T>::value>::type* = nullptr>
    inline hasher& absorb(const T* data, size_t len);

    // Absorbs bytes from std::basic_string
    template<typename T,
        typename std::enable_if<detail::is_byte<T>::value
            && !std::is_same<T, std::string::value_type>::value>::type* = nullptr>
    inline hasher& absorb(const std::basic_string<T>& str);

    // Absorbs bytes from std::string
    inline hasher& absorb(const std::string& str);

    // Absorbs bytes from std::istream
    template<typename T, typename std::enable_if<detail::is_byte<T>::value>::type* = nullptr>;
    inline hasher& absorb(std::basic_istream<T>& istr);

    // Absorbs bytes from an iterator sequence
    template<typename IT>
    inline hasher& absorb(IT begin, IT end);

    // In case HashProvider is an extendable output function, squeeze <len> bytes from absorbed data
    // into user-provided preallocated buffer.
    template<typename T, typename H=HashProvider,
        typename std::enable_if<detail::is_byte<T>::value && detail::is_xof<H>::value>::type* = nullptr>
    inline void squeeze(T* buf, size_t len);

    // In case HashProvider is an extendable output function, squeeze <len> bytes from absorbed data
    // and write them into the output iterator.
    template<typename OI, typename H=HashProvider,
        typename std::enable_if<detail::is_xof<H>::value>::type* = nullptr>
    inline void squeeze(size_t len, OI it);

    // In case HashProvider is an extendable output function, squeeze <len> bytes from absorbed data
    // and return them as a hex string.
    template<typename H=HashProvider, typename std::enable_if<detail::is_xof<H>::value>::type* = nullptr>
    inline std::string hexsqueeze(size_t len);

    // In case HashProvider is a hash function, output binary digest to user-provided preallocated buffer.
    template<typename T, typename H=HashProvider,
        typename std::enable_if<detail::is_byte<T>::value && !detail::is_xof<H>::value>::type* = nullptr>
    inline void digest(T* buf, size_t len) const;

    // In case HashProvider is a hash function, generates binary digest from absorbed data
    // and write it via output iterator.
    template<typename OI, typename H=HashProvider,
        typename std::enable_if<!detail::is_xof<H>::value>::type* = nullptr>
    inline void digest(OI it) const;

    // In case HashProvider is a hash function, returns hex digest of absorbed data.
    template<typename H=HashProvider, typename std::enable_if<!detail::is_xof<H>::value>::type* = nullptr>
    inline std::string hexdigest() const;

    // Resets the state to start new digest computation.
    // If resetParameters is true, all customization parameters such as salt will also be cleared.
    inline void reset(bool resetParameters = false);
};

Individual hash algorithms are defined by typedefs, e.g.

    typedef hasher<detail::sha3_provider> sha3;

    typedef hasher<detail::blake_provider, detail::blake_mixin> blake;

    // ...

Supported algorithms

Hash functions

Typedef Description Supported output sizes Optional parameters
blake Original BLAKE algorithm 224, 256, 384, 512 salt
blake2b BLAKE2b 8-512 salt, personalization, key
blake2s BLAKE2s 8-256 salt, personalization, key
blake2xb BLAKE2xb arbitrary salt, personalization, key
blake2xs BLAKE2xs arbitrary salt, personalization, key
groestl Grøstl 8-512 -
jh JH 8-512 -
kmac128 KMAC128 arbitrary key, customization
kmac256 KMAC256 arbitrary key, customization
kupyna Kupyna 256, 512 -
md5 MD5 128 -
sha1 SHA-1 160 -
sha224 SHA-224 224 -
sha256 SHA-256 256 -
sha384 SHA-384 384 -
sha512 SHA-512 8-512 -
sha3 SHA-3 224, 256, 384, 512 -
skein256 Skein256 arbitrary personalization, key, nonce
skein512 Skein512 arbitrary personalization, key, nonce
skein1024 Skein1024 arbitrary personalization, key, nonce
sm3 SM3 256 -
streebog Streebog 256, 512 -
whirlpool Whirlpool 512 -

Extendable output functions

Typedef Description Optional parameters
blake2xb_xof BLAKE2xb in XOF mode salt, personalization, key
blake2xs_xof BLAKE2xs in XOF mode salt, personalization, key
k12 KangarooTwelve customization
m14 MarsupilamiFourteen customization
shake128 SHAKE-128 -
shake256 SHAKE-256 -
cshake128 cSHAKE-128 function name, customization
cshake256 cSHAKE-256 function name, customization
kmac128_xof KMAC128 in XOF mode key, customization
kmac256_xof KMAC256 in XOF mode key, customization
skein256_xof Skein256 in XOF mode personalization, key, nonce
skein512_xof Skein512 in XOF mode personalization, key, nonce
skein1024_xof Skein1024 in XOF mode personalization, key, nonce

Design rationale in questions and answers

Q: What is the difference between a hash function with variable output size and an extendable output function (XOF)?

A: Hash functions require the digest size to be known at the moment of initialization and normally produce unrelated outputs for different digest sizes. For example, blake2b(256) and blake2b(512) produce completely different digests. XOFs are functions that do not need to know the output size in advance and can produce outputs of unrestricted size. Bytes generated by XOFs depend only on the input data, but not on the digest size. It is generally recommended to use hash functions instead of XOFs when the output size is known in advance.

Q: What is the difference between digest() and squeeze()?

A. digest() is used with hash functions; it retrieves a digest of a certain length (defined by the algorithm or specified in the constructor). Calling digest() or hexdigest() does not change the internal state, so that these functions can be called more than once and will produce the same output. squeeze() is used with XOF functions; it can be called multiple times to squeeze an arbitrary number of output bytes. After each invocation of squeeze() the internal state changes so that the next call to squeeze() will generate different (additional) output bytes.

Q: For hash functions with variable output size, why the output size is not a template parameter, e.g. sha3<256>?

A: While it may seem cool to make the output size a template parameter, in some usage scenarios the required digest size is not known at compile time. One simple example is Argon2 password hashing algorithm, which requires us to hash its state using BLAKE2b with dynamically calculated digest size. We can't just use the largest digest size and truncate the result, because most hash functions (unlike XOFs) produce completely different digests depending on the requested output size. Using a template parameter for the digest size would encumber implementation of such algorithms. Additionally, some hash functions support arbitrary output sizes which are not limited by the security level (examples of such functions are Skein, BLAKE2x, ParallelHash). Some functions are specifically designed to be usable both in hashing and in XOF modes, where the required output size is not known in advance even at runtime. Taking all this factors in consideration, specifying the output size at compile time does not seem like a good design.

Q: Why hasher does not support hashing non-byte types?

A: Cryptographic hash functions are always defined for a sequence of bytes. We support only those data types that can be unambiguosly converted to bytes (sequences of char, signed char, or unsigned char). Other data types should be converted to a sequence of bytes in non-ambiguous way before they can be hashed (eg wide strings could be encoded using UTF-8 or another encoding), which is beyond the scope of the library.

Q: Since the output size has to be provided to the constructor, why there are separate typedefs for sha256 and sha512 instead of one hasher with output size parameter: sha2(256) / sha2(512)?

A: SHA-2 family of hash functions is special because SHA-512 can produce output of any size up to 512 bits (SHA-512/t), e.g. sha512(256) will calculate SHA-512/256. The resulting hash is different from SHA-256, but has the same length. Thus SHA-512 is an independent hash function supporting variable output sizes. On the other hand, the 32-bit version of SHA-2 is only defined for 224-bit and 256-bit outputs, and they are widely known as SHA-224 and SHA-256. We decided to use different typedefs for SHA-224 and SHA-256 because requiring users to use sha256(224) for getting SHA-224 digests would be confusing. Internally all SHA-2 functions are implemented using one template class.

Q: Why there are separate typedefs for skein256, skein512 and skein1024 instead of one hasher with output size parameter: skein(256) / skein(512) / skein(1024)?

A: Skein256, Skein512 and Skein1024 are different algorithms. Each of them can produce digests of any size. The outputs are unrelated, e.g. skein256(256) != skein512(256) != skein1024(256). Internally all Skein variants are implemented using one template class.

Q: Why there are so many typedefs for BLAKE2 hash function?

A: BLAKE2 has many variants that produce incompatible digests for the same output sizes. We support different variants via different typedef. For the 512-bit version, blake2b is the oldest algorithm which can produce digests of any size up to 512 bits. blake2xb can be used to produce larger digests but requires the output size to be known in advance; it can't be merged with blake2b because their output are different for the same digest sizes. blake2xb_xof can be used in XOF mode when the output size is not known in advance. Then there is a 256-bit version blake2s which supports all these variants as well. Internally all BLAKE2 variants are implemented using one template class.

Known limitations

  • Included providers are written in standard C++ and may be slower than SIMD optimized implementations.
  • Only complete bytes are supported for input and output.
  • Big endian systems are not supported.
  • No attempts were made to make implementation of every algorithm constant time.

Reference documentation

Reference documentation is here: https://kerukuro.github.io/digestpp/

Issues
  • Error: call to 'byteswap' is ambiguous (groestl_provider.hpp line 124)

    Error: call to 'byteswap' is ambiguous (groestl_provider.hpp line 124)

    Hi @kerukuro,

    I'm getting a single error when trying to use digestpp:

    algorithm/detail/groestl_provider.hpp:124:26: error:
          call to 'byteswap' is ambiguous
                    h[hs > 256 ? 15 : 7] = byteswap(hash_size());
                                           ^~~~~~~~
    

    It seems to be getting confused between uint(16/32/64)_t types.

    If I comment out the #include for groestl_provider.hpp, everything works great.

    Any ideas? Thanks much.

    opened by brev 5
  • Groestl/* test passed with errors with 32bit compilation

    Groestl/* test passed with errors with 32bit compilation

    Platform and compiler:


    Windows 7 x64 [Version 6.1.7601] Microsoft Visual Studio Community 2017 Version 15.9.26 VisualStudio.15.Release/15.9.26+28307.1234 Visual C++ 2017 00369-60000-00001-AA395


    Compiling test.cc with VS2017 using x86 debug/release:


    Groestl/256 error: expected 8c7ad62eb26a21297bc39c2d7293b4bd4d3399fa8afab29e970471739e28b301, actual b70d81f0215c295a43193b0439d688d852f417a186f173bededd5a5d71c5f17b Groestl/512 error: expected badc1f70ccd69e0cf3760c3f93884289da84ec13c70b3d12a53a7a8a4a513f99715d46288f55e1dbf926e6d084a0538e4eebfc91cf2b21452921ccde9131718d, actual 545f354922267ece6b14fa129074058acd34b3314d65b30be3d29a64d92355ee2dee2086dfc9028e0c2aa455fd05e18b3544697b522533a5640ac40c9766d258 Self-test completed with 2 errors.


    Compiling test.cc with VS2017 using x64 debug/release, execution passes witout any error.

    g++ x64 compilation and execution also passed without any error.

    opened by MicSm 1
  • Conflict with asio library or termios.h

    Conflict with asio library or termios.h

    When i try to compile digestpp with asio (header only), the compiler fails with this message:

    inc/digestpp/algorithm/detail/sha3_provider.hpp:40:13:
    error: expected unqualified-id before numeric constant
         uint64_t B0 = A[0 * 5 + 0] ^ D[0];
                  ^
    

    The issue is because asio includes termios.h and termios.h define B0 as 00000000. Like this: #define B0 0000000 /* hang up */

    I think the solution may can be replace the variable B0 with another name or is there a better solution?

    opened by c4explosive 0
  • Fix #1

    Fix #1

    Hi!

    This PR fixes issue #1. I found out that this is a macOS-only error, so it is possible most of the users of this library haven't found out about this. All the changes here fix the issue on macOS while maintaining all functionality. I've tested the changes on macOS (Clang), Windows (MSVC), and Linux (Clang/GCC) and it works perfectly on all of them.

    I've also removed all trailing whitespaces from both the library and tests.

    opened by marcizhu 0
  • Reduced round skein?

    Reduced round skein?

    As per https://github.com/BLAKE3-team/BLAKE3/issues/19#issuecomment-574061588 It might be a good idea to include reduced round skein for possible speedup, for both standard and tree versions of skein.

    opened by DonaldTsang 0
  • Known issue on Windows platform

    Known issue on Windows platform

    #define NOMINMAX should before including any of Windows / Platform SDK header. Otherwise min/max macroses from there break std::min and std::max giving syntax errors during compilation

    Reference: https://stackoverflow.com/questions/5004858/why-is-stdmin-failing-when-windows-h-is-included

    opened by alfishe 1
Header-only VMWare Backdoor API Implementation & Effortless VMX Patcher for Custom Guest-to-Host RPCs

VmxHijack Header-only VMWare Backdoor API Implementation & Effortless VMX Patcher for Custom Guest-to-Host RPCs Sample // --- RPC Server Code (VmxHija

Can Bölük 84 Jun 12, 2022
a header-file-only, SHA256 hash generator in C++

PicoSHA2 - a C++ SHA256 hash generator Copyright © 2017 okdshin Introduction PicoSHA2 is a tiny SHA256 hash generator for C++ with following propertie

Shintarou Okada 503 Jun 17, 2022
A small HOTP/TOTP SHA1 client written in C, depending only on libcrypto (OpenSSL)

A small HOTP/TOTP SHA1 client written in C, depending only on libcrypto (OpenSSL)

null 3 Jan 21, 2022
MIRACL Cryptographic SDK: Multiprecision Integer and Rational Arithmetic Cryptographic Library is a C software library that is widely regarded by developers as the gold standard open source SDK for elliptic curve cryptography (ECC).

MIRACL What is MIRACL? Multiprecision Integer and Rational Arithmetic Cryptographic Library – the MIRACL Crypto SDK – is a C software library that is

MIRACL 483 Jun 27, 2022
free C++ class library of cryptographic schemes

Crypto++: free C++ Class Library of Cryptographic Schemes Version 8.4 - TBD Crypto++ Library is a free C++ class library of cryptographic schemes. Cu

null 3.5k Jun 24, 2022
A modern, portable, easy to use crypto library.

Sodium is a new, easy-to-use software library for encryption, decryption, signatures, password hashing and more. It is a portable, cross-compilable, i

Frank Denis 10.2k Jun 25, 2022
A lightweight, secure, easy-to-use crypto library suitable for constrained environments.

The Hydrogen library is a small, easy-to-use, hard-to-misuse cryptographic library. Features: Consistent high-level API, inspired by libsodium. Instea

Frank Denis 388 Jun 18, 2022
An open source, portable, easy to use, readable and flexible SSL library

README for Mbed TLS Mbed TLS is a C library that implements cryptographic primitives, X.509 certificate manipulation and the SSL/TLS and DTLS protocol

Arm Mbed 3.6k Jun 20, 2022
TLS/SSL and crypto library

Welcome to the OpenSSL Project OpenSSL is a robust, commercial-grade, full-featured Open Source Toolkit for the Transport Layer Security (TLS) protoco

OpenSSL 18.7k Jun 28, 2022
Library and command line tool to detect SHA-1 collision in a file

sha1collisiondetection Library and command line tool to detect SHA-1 collisions in files Copyright 2017 Marc Stevens [email protected] Distributed

Marc Stevens 1.2k Jun 24, 2022
Tink is a multi-language, cross-platform, open source library that provides cryptographic APIs that are secure, easy to use correctly, and hard(er) to misuse.

Tink A multi-language, cross-platform library that provides cryptographic APIs that are secure, easy to use correctly, and hard(er) to misuse. Ubuntu

Google 12.4k Jun 23, 2022
LibSWIFFT - A fast C/C++ library for the SWIFFT secure homomorphic hash function

LibSWIFFT - A fast C/C++ library for the SWIFFT secure homomorphic hash function Official Repository LibSWIFFT is a production-ready C/C++ library pro

Gvili Tech Ltd 21 Jun 19, 2021
Intel:registered: Homomorphic Encryption Acceleration Library accelerates modular arithmetic operations used in homomorphic encryption

Intel Homomorphic Encryption Acceleration Library (HEXL) Intel ®️ HEXL is an open-source library which provides efficient implementations of integer a

Intel Corporation 143 Jun 27, 2022
PTHash is a C++ library implementing fast and compact minimal perfect hash functions

Fast and compact minimal perfect hash functions in C++.

Giulio Ermanno Pibiri 65 Jun 18, 2022
StrCrypt Compile-time string crypter library for C++

StrCrypt Compile-time string crypter library for C++ Having plain strings stored in the binary file or in memory can help reversering attempts to be m

null 58 Jun 26, 2022
x509cert is a tool and library for generating X.509 certificates and certificate requests.

x509cert is a tool and library for generating X.509 certificates and certificate requests. It is written in C99 and uses BearSSL to decode keys and compute signatures.

Michael Forney 9 Nov 25, 2021
NanoSwift: A Swift Library for the Nano cryptocurrency

Nano is an instant, feeless and eco-friendly cryptocurrency that is also super easy to use. This library lets you create wallets, accounts and blocks as well as manage Nano amounts, interact with a node and more.

Christian 23 Mar 1, 2022
HashLibPlus is a recommended C++11 hashing library that provides a fluent interface for computing hashes and checksums of strings, files, streams, bytearrays and untyped data to mention but a few.

HashLibPlus HashLibPlus is a recommended C++11 hashing library that provides a fluent interface for computing hashes and checksums of strings, files,

Telepati 7 Apr 11, 2022
vsomeip Library for Windows Msys2 MinGW64

vsomeip-msys2-mingw64 vsomeip Library for Windows Msys2 MinGW64 vsomeip Copyright Copyright (C) 2015-2017, Bayerische Motoren Werke Aktiengesellschaft

null 1 Oct 27, 2021