Cryptography Toolkit

Overview

Botan: Crypto and TLS for Modern C++

Botan (Japanese for peony flower) is a C++ cryptography library released under the permissive Simplified BSD license.

Botan's goal is to be the best option for cryptography in C++ by offering the tools necessary to implement a range of practical systems, such as TLS protocol, X.509 certificates, modern AEAD ciphers, PKCS#11 and TPM hardware support, password hashing, and post quantum crypto schemes. A Python binding is included, and several other language bindings are available. It is used in many open source and commercial products. The library is accompanied by a featureful command line interface.

See the documentation for more information about included features.

Development is coordinated on GitHub and contributions are welcome. If you need help, please open an issue on GitHub or email the botan-devel mailing list. New releases are announced on the botan-announce mailing list. If you think you have found a security issue, see the security page for contact information.

The latest release is 2.18.2 (sig), released on 2021-10-25. All releases are signed with a PGP key. See the release notes for what is new. Botan is also available through most distributions such as Fedora, Debian, Arch and Homebrew.

Travis CI status AppVeyor CI status Code coverage report LGTM alerts OSS-Fuzz status Coverity results Packaging status CII Best Practices statement

Find Enclosed

Transport Layer Security (TLS) Protocol

  • TLS v1.2 and DTLS v1.2
  • Supported extensions include session tickets, SNI, ALPN, OCSP stapling, encrypt-then-mac CBC, and extended master secret.
  • Supports authentication using certificates or preshared keys (PSK)
  • Supports record encryption with ChaCha20Poly1305, AES/OCB, AES/GCM, AES/CCM, Camellia/GCM as well as legacy CBC ciphersuites.
  • Key exchange using CECPQ1, ECDH, FFDHE, or RSA

Public Key Infrastructure

  • X.509v3 certificates and CRL creation and handling
  • PKIX certificate path validation, including name constraints.
  • OCSP request creation and response handling
  • PKCS #10 certificate request generation and processing
  • Access to Windows, macOS and Unix system certificate stores
  • SQL database backed certificate store

Public Key Cryptography

  • RSA signatures and encryption
  • DH and ECDH key agreement
  • Signature schemes ECDSA, DSA, Ed25519, ECGDSA, ECKCDSA, SM2, GOST 34.10
  • Post-quantum signature scheme XMSS
  • Post-quantum key agreement schemes McEliece and NewHope
  • ElGamal encryption
  • Padding schemes OAEP, PSS, PKCS #1 v1.5, X9.31

Ciphers, hashes, MACs, and checksums

  • Authenticated cipher modes EAX, OCB, GCM, SIV, CCM, (X)ChaCha20Poly1305
  • Cipher modes CTR, CBC, XTS, CFB, OFB
  • Block ciphers AES, ARIA, Blowfish, Camellia, CAST-128, DES/3DES, IDEA, Lion, SEED, Serpent, SHACAL2, SM4, Threefish-512, Twofish
  • Stream ciphers (X)ChaCha20, (X)Salsa20, SHAKE-128, RC4
  • Hash functions SHA-1, SHA-2, SHA-3, MD5, RIPEMD-160, BLAKE2b, Skein-512, SM3, Streebog, Whirlpool
  • Authentication codes HMAC, CMAC, Poly1305, SipHash, GMAC, X9.19 DES-MAC
  • Non-cryptographic checksums Adler32, CRC24, CRC32

Other Useful Things

  • Full C++ PKCS #11 API wrapper
  • Interfaces for TPM v1.2 device access
  • Simple compression API wrapping zlib, bzip2, and lzma libraries
  • RNG wrappers for system RNG and hardware RNGs
  • HMAC_DRBG and entropy collection system for userspace RNGs
  • Password hashing schemes PBKDF2, Argon2, Scrypt, bcrypt
  • SRP-6a password authenticated key exchange
  • Key derivation functions including HKDF, KDF2, SP 800-108, SP 800-56A, SP 800-56C
  • HOTP and TOTP algorithms
  • Format preserving encryption scheme FE1
  • Threshold secret sharing
  • NIST key wrapping
  • Boost.Asio compatible TLS client stream
Comments
  • Using PKCS#11 with TLS

    Using PKCS#11 with TLS

    This is a follow-up of #725, where @mouse07410 asked about using PKCS#11 objects with TLS. Correct me if I'm wrong, but from my understanding it should already be possible to have a TLS server with the server's long-term key stored in a PKCS#11 device, now as the PK ops are tied to the Private_Key and Public_Key objects (which was introduced in #625). All you have to do is to create a PKCS11 key, e.g., a PKCS11_RSA_PrivateKey, and return it from Credentials_Manager::private_key_for(). TLS_Server will forward it to PK_Signer, which will select and sign with the PK signature op returned by PKCS11_RSA_PrivateKey::create_signature_op().

    If you want more control over the handshake, e.g., you want do the ECDH key exchange ops in a PKCS#11 device, there is currently no way to do it. We are currently thinking about a new interface, let it be named TLS Adapter for now, for doing these operations in hardware. We will have to do these operations in a non-PKCS#11 compliant HSM, but we want to make it generic enough that PKCS#11 devices can be supported easily.

    We will come up with a proposal for this use case probably in the next few weeks.

    Update: Removed the chain validation part, of course it's already possible by overriding Credentials_Manager::verify_certificate_chain().

    enhancement 
    opened by securitykernel 36
  • Roughtime client

    Roughtime client

    This pull request implements a Roughtime client API and CLI.

    Roughtime is a authenticated time protocol developed by Google.

    The protocol is under standardization, but there initial version is available one multiple public servers, including from Google and Cloulflare.

    This PR implements this initial version since it's the one actually being used on the wild and the standard one is not finished yet.

    • [x] UDP client
    • [x] Roughtime client
    • [x] CLI
    • [x] Complete test suite
    opened by nunojpg 31
  • MacOS: botan built with Xcode-13 fails SHA-3 tests

    MacOS: botan built with Xcode-13 fails SHA-3 tests

    Apple released Xcode-13. When Botan is built with it, it fails tests. Both master and release-2, in the same way. On all my Macs.

    Branch release-2

    Configuration

    ./configure.py --prefix=/opt/local --with-os-features=security_framework,apple_keychain,commoncrypto,getentropy --with-openmp --with-commoncrypto --with-openssl --with-boost --with-lzma --with-bzip2 --with-zlib --with-sqlite3 --with-python-version=2.7 --with-sphinx --with-pdf --cc-abi-flags='-march=native -O3 -I/opt/local/include' 2>&1 | tee conf-out.txt
    

    Configuration output:

    $ cat conf-out.txt 
       INFO: ./configure.py invoked with options "--prefix=/opt/local --with-os-features=security_framework,apple_keychain,commoncrypto,getentropy --with-openmp --with-commoncrypto --with-openssl --with-boost --with-lzma --with-bzip2 --with-zlib --with-sqlite3 --with-python-version=2.7 --with-sphinx --with-pdf --cc-abi-flags=-march=native -O3 -I/opt/local/include"
       INFO: Configuring to build Botan 2.18.1 (revision git:b420a4545b0f9219a88c209e4c8c2474d519dfac)
       INFO: Running under 3.9.7 (default, Sep  1 2021, 12:35:15) [Clang 12.0.5 (clang-1205.0.22.9)]
       INFO: Implicit --cc-bin=clang++ due to environment variable CXX
       INFO: Implicit --cxxflags=-std=gnu++17 -O3 -march=native -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk due to environment variable CXXFLAGS
       INFO: Autodetected platform information: OS="Darwin" machine="x86_64" proc="i386"
       INFO: Guessing target OS is darwin (use --os to set)
       INFO: Guessing target processor is a x86_64 (use --cpu to set)
       INFO: Using /etc/ssl/cert.pem as system certificate store
       INFO: Auto-detected compiler version 4.0
       INFO: Auto-detected compiler arch x86_64
       INFO: Target is clang:4.0-macos-x86_64
       INFO: Assuming target x86_64 is little endian
       INFO: Skipping (incompatible CPU): aes_armv8 aes_power8 sha1_armv8 sha2_32_armv8 sm4_armv8
       INFO: Skipping (incompatible OS): certstor_system_windows proc_walk win32_stats
       INFO: Skipping (requires external dependency): tpm
       INFO: Enabling use of external dependency boost
       INFO: Enabling use of external dependency bzip2
       INFO: Enabling use of external dependency commoncrypto
       INFO: Enabling use of external dependency lzma
       INFO: Enabling use of external dependency openssl
       INFO: Enabling use of external dependency sqlite3
       INFO: Enabling use of external dependency zlib
       INFO: Loading modules: adler32 aead aes aes_ni aes_vperm aont argon2 aria asio asn1 auto_rng base base32 base58 base64 bcrypt bcrypt_pbkdf bigint blake2 block blowfish boost bzip2 camellia cascade cast128 cast256 cbc cbc_mac ccm cecpq1 certstor_flatfile certstor_sql certstor_sqlite3 certstor_system certstor_system_macos cfb chacha chacha20poly1305 chacha_avx2 chacha_rng chacha_simd32 checksum cmac comb4p commoncrypto compression cpuid crc24 crc32 cryptobox ctr curve25519 des dev_random dh dl_algo dl_group dlies dsa dyn_load eax ec_group ecc_key ecdh ecdsa ecgdsa ecies eckcdsa ed25519 elgamal eme_oaep eme_pkcs1 eme_raw emsa1 emsa_pkcs1 emsa_pssr emsa_raw emsa_x931 entropy fd_unix ffi filters fpe_fe1 gcm getentropy ghash ghash_cpu ghash_vperm gmac gost_28147 gost_3410 gost_3411 hash hash_id hex hkdf hmac hmac_drbg hotp http_util idea idea_sse2 iso9796 kasumi kdf kdf1 kdf1_iso18033 kdf2 keccak keypair lion locking_allocator lzma mac mce mceies md4 md5 mdx_hash mem_pool mgf1 misty1 mode_pad modes mp newhope nist_keywrap noekeon noekeon_simd numbertheory ocb ofb openssl par_hash passhash9 pbes2 pbkdf pbkdf1 pbkdf2 pem pgp_s2k pk_pad pkcs11 poly1305 poly_dbl prf_tls prf_x942 processor_rng psk_db pubkey rc4 rdrand_rng rdseed rfc3394 rfc6979 rmd160 rng roughtime rsa salsa20 scrypt seed serpent serpent_avx2 serpent_simd sessions_sql sessions_sqlite3 sha1 sha1_sse2 sha1_x86 sha2_32 sha2_32_bmi2 sha2_32_x86 sha2_64 sha2_64_bmi2 sha3 sha3_bmi2 shacal2 shacal2_avx2 shacal2_simd shacal2_x86 shake shake_cipher simd simd_avx2 siphash siv skein sm2 sm3 sm4 socket sodium sp800_108 sp800_56a sp800_56c sqlite3 srp6 stateful_rng stream streebog system_rng thread_utils threefish_512 threefish_512_avx2 tiger tls tls_10 tls_cbc tss twofish utils uuid whirlpool x509 x919_mac xmss xtea xts zlib
       INFO: Using symlink to link files into build dir (use --link-method to change)
       INFO: Botan 2.18.1 (revision git:b420a4545b0f9219a88c209e4c8c2474d519dfac) (unreleased undated) build setup is complete
    

    Build output: make-out.txt.gz

    Tests output

    .  .  .  .  .
    1416a6f128a2567fdf10079d74d2f64aaa8e2834216c698118f69109580b0f61c6fc53fdd578276e4f6b1e8fb1e5cd04a2450620c1dca97c517dc81ecfbd3776fbb75b2f211ddef474304929e0a2ef57121ba873a145e7cec15d3af0605f6e9cbc84ff70e4072f9e694557c302e2c2bb3db14bd52707b47890731e0cf6181d297d012967c3fd561f905b8a4ba23487]
    xmss_verify_invalid:
    XMSS/SHA2_10_256 verify invalid signature ran 28 tests in 28.65 msec all ok
    XMSS/SHA2_10_512 verify invalid signature ran 28 tests in 74.90 msec all ok
    XMSS/SHA2_16_256 verify invalid signature ran 28 tests in 37.11 msec all ok
    XMSS/SHA2_16_512 verify invalid signature ran 28 tests in 94.26 msec all ok
    XMSS/SHA2_20_256 verify invalid signature ran 28 tests in 41.33 msec all ok
    XMSS/SHA2_20_512 verify invalid signature ran 28 tests in 88.67 msec all ok
    XMSS/SHAKE_10_256 verify invalid signature ran 28 tests in 25.33 msec all ok
    XMSS/SHAKE_10_512 verify invalid signature ran 28 tests in 95.61 msec all ok
    XMSS/SHAKE_16_256 verify invalid signature ran 28 tests in 53.11 msec all ok
    XMSS/SHAKE_16_512 verify invalid signature ran 28 tests in 118.49 msec all ok
    XMSS/SHAKE_20_256 verify invalid signature ran 28 tests in 31.91 msec all ok
    XMSS/SHAKE_20_512 verify invalid signature ran 28 tests in 129.48 msec all ok
    Tests complete ran 2858301 tests in 18.89 sec 17713 tests failed
    

    Full output: make-out.txt.gz

    Branch master

    Configuration

    ./configure.py --prefix=/opt/local --with-os-features=security_framework,apple_keychain,commoncrypto,getentropy --with-commoncrypto --with-openssl --with-boost --with-lzma --with-bzip2 --with-zlib --with-sqlite3 --with-python-version=3.9 --with-sphinx --with-pdf --system-cert-bundle=/opt/local/share/curl/curl-ca-bundle.crt --cc-abi-flags='-march=native -O3 -I/opt/local/include' 2>&1 | tee conf-out.txt
    

    Output:

    $ cat conf-out.txt 
       INFO: ./configure.py invoked with options "--prefix=/opt/local --with-os-features=security_framework,apple_keychain,commoncrypto,getentropy --with-commoncrypto --with-openssl --with-boost --with-lzma --with-bzip2 --with-zlib --with-sqlite3 --with-python-version=3.9 --with-sphinx --with-pdf --system-cert-bundle=/opt/local/share/curl/curl-ca-bundle.crt --cc-abi-flags=-march=native -O3 -I/opt/local/include"
       INFO: Configuring to build Botan 3.0.0-alpha0 (revision git:20e87b077c113744600510c431af1396663260a0)
       INFO: Running under 3.9.7 (default, Sep  1 2021, 12:35:15) [Clang 12.0.5 (clang-1205.0.22.9)]
       INFO: Implicit --cc-bin=clang++ due to environment variable CXX
       INFO: Implicit --cxxflags=-std=gnu++17 -O3 -march=native -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk due to environment variable CXXFLAGS
       INFO: Autodetected platform information: OS="Darwin" machine="x86_64" proc="i386"
       INFO: Guessing target OS is darwin (use --os to set)
       INFO: Guessing target processor is a x86_64 (use --cpu to set)
       INFO: Auto-detected compiler version 4.0
       INFO: Auto-detected compiler arch x86_64
       INFO: Target is clang:4.0-macos-x86_64
       INFO: Assuming target x86_64 is little endian
       INFO: Skipping (incompatible CPU): aes_armv8 aes_power8 sha1_armv8 sha2_32_armv8 shacal2_armv8 sm4_armv8
       INFO: Skipping (incompatible OS): certstor_system_windows win32_stats
       INFO: Skipping (requires external dependency): tpm
       INFO: Enabling use of external dependency boost
       INFO: Enabling use of external dependency bzip2
       INFO: Enabling use of external dependency commoncrypto
       INFO: Enabling use of external dependency lzma
       INFO: Enabling use of external dependency openssl
       INFO: Enabling use of external dependency sqlite3
       INFO: Enabling use of external dependency zlib
       INFO: Loading modules: adler32 aead aes aes_ni aes_vperm argon2 argon2fmt aria asio asn1 auto_rng base base32 base58 base64 bcrypt bcrypt_pbkdf bigint blake2 blake2mac block blowfish boost bzip2 camellia cascade cast128 cbc ccm cecpq1 certstor_flatfile certstor_sql certstor_sqlite3 certstor_system certstor_system_macos cfb chacha chacha20poly1305 chacha_avx2 chacha_rng chacha_simd32 checksum cmac comb4p commoncrypto compression cpuid crc24 crc32 cryptobox ctr curve25519 des dh dl_algo dl_group dlies dsa dyn_load eax ec_group ec_h2c ecc_key ecdh ecdsa ecgdsa ecies eckcdsa ed25519 elgamal eme_oaep eme_pkcs1 eme_raw emsa1 emsa_pkcs1 emsa_pssr emsa_raw emsa_x931 entropy fd_unix ffi filters fpe_fe1 gcm getentropy ghash ghash_cpu ghash_vperm gmac gost_28147 gost_3410 gost_3411 hash hash_id hex hkdf hmac hmac_drbg hotp http_util idea idea_sse2 iso9796 kdf kdf1 kdf1_iso18033 kdf2 keccak keypair lion locking_allocator lzma mac mce md4 md5 mdx_hash mem_pool mgf1 mode_pad modes mp newhope nist_keywrap noekeon noekeon_simd numbertheory ocb ofb openssl par_hash passhash9 pbes2 pbkdf pbkdf2 pem pgp_s2k pk_pad pkcs11 poly1305 poly_dbl prf_tls prf_x942 processor_rng psk_db pubkey rc4 rdseed rfc3394 rfc6979 rmd160 rng roughtime rsa salsa20 scrypt seed serpent serpent_avx2 serpent_simd sessions_sql sessions_sqlite3 sha1 sha1_sse2 sha1_x86 sha2_32 sha2_32_bmi2 sha2_32_x86 sha2_64 sha2_64_bmi2 sha3 sha3_bmi2 shacal2 shacal2_avx2 shacal2_simd shacal2_x86 shake shake_cipher simd simd_avx2 siphash siv skein sm2 sm3 sm4 socket sodium sp800_108 sp800_56a sp800_56c sqlite3 srp6 stateful_rng stream streebog system_rng thread_utils threefish_512 threefish_512_avx2 tls tls_cbc tss twofish utils uuid whirlpool x509 x919_mac xmss xts zlib
       INFO: Using symlink to link files into build dir (use --link-method to change)
       INFO: Botan 3.0.0-alpha0 (revision git:20e87b077c113744600510c431af1396663260a0) (unreleased undated) build setup is complete
    

    Build output: make-out.txt.gz

    Tests

    .  .  .  .  .
    c81ecfbd3776fbb75b2f211ddef474304929e0a2ef57121ba873a145e7cec15d3af0605f6e9cbc84ff70e4072f9e694557c302e2c2bb3db14bd52707b47890731e0cf6181d297d012967c3fd561f905b8a4ba23487 
    xmss_verify_invalid:
    XMSS/SHA2_10_256 verify invalid signature ran 28 tests in 34.19 msec all ok
    XMSS/SHA2_10_512 verify invalid signature ran 28 tests in 69.04 msec all ok
    XMSS/SHA2_16_256 verify invalid signature ran 28 tests in 34.38 msec all ok
    XMSS/SHA2_16_512 verify invalid signature ran 28 tests in 94.06 msec all ok
    XMSS/SHA2_20_256 verify invalid signature ran 28 tests in 40.87 msec all ok
    XMSS/SHA2_20_512 verify invalid signature ran 28 tests in 79.71 msec all ok
    XMSS/SHAKE_10_256 verify invalid signature ran 28 tests in 30.20 msec all ok
    XMSS/SHAKE_10_512 verify invalid signature ran 28 tests in 83.43 msec all ok
    XMSS/SHAKE_16_256 verify invalid signature ran 28 tests in 46.80 msec all ok
    XMSS/SHAKE_16_512 verify invalid signature ran 28 tests in 130.14 msec all ok
    XMSS/SHAKE_20_256 verify invalid signature ran 28 tests in 29.96 msec all ok
    XMSS/SHAKE_20_512 verify invalid signature ran 28 tests in 121.47 msec all ok
    Tests complete ran 2850756 tests in 18.28 sec 17713 tests failed (in entropy hash hash_rep mac newhope stream xmss_sign xmss_verify)
    

    Full output: test-out.txt.gz

    opened by mouse07410 30
  • Support for the TLS 1.3 - development

    Support for the TLS 1.3 - development

    We'd like to contribute to Botan by starting the implementation of the TLS 1.3 protocol support. The motivation is that our use-cases require the TLS 1.3 from the client side and therefore we could support the development in that topic. The plan is to start in May (or even earlier with rough design and development plan).

    The rough plan:

    • design draft, to agree the direction of the development with the community,
    • refactoring of the existing TLS code, i.e. splitting the common code and the protocol version specific, introducing pimpl idiom to have generic interfaces and version specific implementation, etc.,
    • verification of refactored code to keep the quality on high level and avoid any regression,
    • start the development of TLS 1.3 protocol,
    • next steps...

    The important thing would be to select proper start point (version) for the development. It seems that the release 2 (with the latest 2.18) is already mature and development on master branch is probably meant for upcoming 3.x release. Is there any rough plan, when first 3.x version will be released, as we would like to pick up some 'official tag' for the development?

    Please, comment whether this topic sounds interesting and implementing the TLS 1.3 support would be valuable here?

    opened by pist-eb 29
  • Cannot build Botan: undefined reference to 'Botan::CPUID::state()'

    Cannot build Botan: undefined reference to 'Botan::CPUID::state()'

    Has anybody experienced this? I am trying to build Botan (version 2.10.0) in Windows 10, with mingw32, don't know if I use the correct flags. This is what I use:

    configure.py --prefix=$BUILD_DIR --with-external- includedir=$OPENSSL_PREFIX/include --with-external-libdir=$OPENSSL_PREFIX/lib --os=mingw --cpu=i386 --minimized-build --enable- modules=rsa,dsa,ecdsa,ed25519,hmac,hmac_drbg,mode_pad,bigint,filters,block,auto_rng,x509,cbc,dh --with-openssl

    Very thankful for any help with this

    build problem 
    opened by PerraJ60 28
  • Compiling botan 2.5.0 on Windowsx64

    Compiling botan 2.5.0 on Windowsx64

    Hello!

    Looking at guide, doing things as said: python configure.py --cc=msvc --os=windows --link-method=copy --cpu=i386 --disable-modules=pmull

    then "nmake". And then accepting hoard of errors, like: build\include\botan/types.h(89) : error C2143: синтаксическая ошибка: отсутствие ";" перед "=" and so on.

    When doing so: python configure.py --cc=gcc --os=mingw --link-method=copy --cpu=i386 --disable-modules=pmull and then while doing make accepting this: fatal error: botan/alg_id.h: No such file or directory #include <botan/alg_id.h>

    Can anybody please give some hints to make things going? Totally newbie in cpp, help is badly needed.

    opened by ghost 27
  • [TLS 1.3] Limited TLS 1.3 Client Implementation

    [TLS 1.3] Limited TLS 1.3 Client Implementation

    Limited TLS 1.3 Client

    Pull Request Dependencies

    • The RFC 8448 based test uses the Test::Result consolidation constructor from this pull request. For simplicity the required commit is duplicated here and should be removed after the consolidation constructor is finalized and merged.

    Follow-up Pull Requests

    The number of open pull requests that are based on this are getting somewhat out of hand. Below is a list to keep track of the dependencies:

    • THIS ONE: https://github.com/randombit/botan/pull/2922
      • https://github.com/randombit/botan/pull/2957
      • https://github.com/randombit/botan/pull/2974
      • https://github.com/randombit/botan/pull/2977
      • https://github.com/randombit/botan/pull/2983
      • https://github.com/randombit/botan/pull/2988
      • https://github.com/randombit/botan/pull/2989
      • INFORMATIVE: https://github.com/randombit/botan/pull/2981
      • INFORMATIVE: https://github.com/randombit/botan/discussions/2954

    Potential Future Work

    This is a non-exhaustive list of potential improvements or further development. Note that those points are not tackled by the follow-up PRs. I'll leave this here for future reference.

    • [ ] Overhaul of TLS_Data_Reader
      • Some of its methods leave room for performance improvements
      • The interface could be easier to use
      • More invariant checks (e.g. assert !has_remaining() on destruction).
      • Also a "sub-reader" concept: i.e. a new ::get_tls_length_value_as_reader() could return a sub-reader that can then be passed into a sub-parser (e.g. Client Hello parsing its extensions)
      • Use it consistently for parsing
    • [ ] Allow disabling TLS 1.2 at compile time
    • [ ] std::span to avoid copying
      • C++17 doesn't actually provide it, but a custom implementation should be straight forward
    • [ ] Separate TLS 1.2 and 1.3 cipher suites
      • TLS 1.3 suites are more focussed (some protocol parameters are negotiated differently)
      • the Ciphersuite class provides methods that are strictly not defined for TLS 1.3 (potential cause for bugs)
      • cipher suite handling code becomes more convoluted that necessary (e.g. Policy::ciphersuite_list(), generation via tls_suite_info.py)

    This PR adds limited TLS 1.3 support. Available functionality:

    • functional TLS 1.3 client
      • automatically downgrades to TLS 1.2 if allowed and required by the server
      • usable via Botan-CLI tls_client
    • additional extensions as required per RFC 8446:
      • Key_Share
      • Supported_Versions
      • Cookie
      • Signature Algorithms
      • Signature Algorithms Certificate
      • Negotiated Groups
    • additional extensions (not strictly required by RFC 8446)
      • Certificate Status Request (OCSP stapling)
      • Record_Size_Limit (RFC 8449)
      • Application Layer Protocol Notification (RFC 7301)
      • Server Name Indication
    • Utility and Misc
      • ./botan tls_client --debug to print raw TLS traffic
      • Fixed_Output_RNG can optionally fall back to another RNG when its pool is empty
    • Tests integrated or performed
      • New infrastructure has unit tests
      • Integration tests using the test vectors from RFC 8448
      • BoGo tests that do not depend on not-yet-implemented protocol features (see here for a list of issues found by BoGo)
      • Handshake and data exchange works using ./botan tls_client with the big players (google.com, cloudflare.com, ...)

    Limitations:

    • TLS 1.3 support cannot be compiled without enabling TLS 1.2 (TLS 1.2 can be disabled in the TLS policy though)
    • no TLS 1.3 compatible server
    • no client authentication (added in: https://github.com/randombit/botan/pull/2957)
    • no session resumption via Tickets (added in: https://github.com/randombit/botan/pull/2974)
    • no Pre-Shared-Keys (technical foundation added in: https://github.com/randombit/botan/pull/2974)
    • no 0-RTT support (early data) (technical foundation added in: https://github.com/randombit/botan/pull/2974)
    • no DTLS
    • no fuzzing of the TLS 1.3 implementation (basic fuzz target for parser: https://github.com/randombit/botan/pull/2977)

    Implementation Overview

    To make this large PR a bit more approachable, we'll provide an overview to the newly introduced components and their responsibilities. Note that this work-in-progress pull request strives towards a usable implementation of "Minimal Viable Product" as outlined in the ToDo-List of this issue comment.

    TODO

    • [x] server certificate validation
      • [x] OCSP stapling
    • [x] Hello Retry Request
    • [x] handle CLOSE_NOTIFY properly (difference between TLS 1.2 and 1.3)
    • [x] protocol version downgrade
    • [x] Key Update
    • [x] Key Material Export (RFC 8446 7.5 / RFC 5705)
    • [x] review TLS callbacks invocations
    • [x] bogo test suite integration
    • [x] check occurrences of extensions as indicated in the table in RFC 8446 4.2
    • [x] configure ClientHello settings (e.g. ALPN)
    • [x] honor the record size limit extension
    • ~~Session Resumption (PSK w/o 0-RTT)~~ not part of this PR

    Intro

    The descriptions mostly refer to the components in the tls13 module, as the TLS 1.2 implementation remains unchanged as far as possible. As initiated in the refactoring by Elektrobit Automotive GmbH, the user-facing interface of TLS::Client and TLS::Server also remains unchanged by means of a Pimpl-pattern.

    We nevertheless anticipate some API changes in the public API. Please refer to this issue comment for further details.

    Components

    Screenshot 2022-03-04 at 12 18 24 https://excalidraw.com/#json=uJjsg_me6QrHsybCbkgxs,Ql21IMba1ZERgxfIiFElKg

    Channel

    The Channel acts as "composition root" of the entire TLS 1.3 implementation. It implements the library's public APIs via a Pimpl-construction as detailed in this issue comment. This class implements the client/server agnostic parts of the protocol. Hence, orchestrating the Record_Layer and Handshake_Layer to transform received bytes from the network into handshake messages to be interpreted by the Client (and later Server) implementations or application data to be passed to the using code. Furthermore, it takes care of handling TLS alerts.

    Record_Layer

    This layer implements the record layer level of the protocol. It parses raw data from the wire to individual records and decrypts protected records using Cipher_State. This corresponds to tasks performed by the Channel (Channel_Impl_12) and the Record_Header class in the 1.2 implementation. When sending data it add record headers and encryption to the records.

    Handshake_Layer

    This class takes care of parsing and marshalling of Handshake_Messages as well as equipping them with the appropriate handshake protocol headers. As this class handles the byte-representation of all TLS handshake messages (i.e. before parsing or after serialization), it is responsible for updating the Transcript_Hash_State appropriately.

    In TLS 1.2 the task of the Handshake_Layer were mostly implemented in the Handshake_IO class. Note that Handshake_Layer does not perform any handshake state validations. It simply parses/serializes messages as they come in from the wire or the downstream protocol implementation.

    Cipher_State

    This class implments the Key Schedule mechanism. Most importantly, it derives and holds traffic secrets and provides interfaces for the Record_Layer protect and deprotect records. The Client advances the Cipher_State through the Key Schedule "state machine".

    Transcript_Hash_State

    This class keeps track of the transcript hash while sending/receiving messages in the Handshake_Layer. Client (and later Server) consult the Transcript_Hash_State for relevant hash-data when updating the Cipher_State appropriately.

    Client

    The Client's main responsibility continues to lie in process_handshake_message, implemented via individual handle-methods for each specific message type. Hence, the client's TLS state machine is implemented here. During message processing, it updates the state of other components:

    • advancing the Key Schedule in Cipher_State
    • consult the Transcript_Hash_State
    • manage expected next messages in Handshake_Transitions

    Handshake_Transitions

    This class aids the Client implementation in validating state transitions. It is merely an extraction of the confirm_transition_to and set_expected_next functionality from TLS 1.2's Handshake_State.

    Handshake_State

    HandshakeState manages the incoming and outgoing messages and keeps a record of them for later reference. Before storing the messages it filters them regarding the Outbound_Message and Inbound_Message variants. Hence, a misuse of Handshake_State will fail to compile (e.g. when a client implementation tries to send a Server_Hello_13). Similarly, the compiler can check that Client implements handle() methods for exactly the relevant handshake messages.

    Handshake_Messages

    The class hierarchy of Handshake_Messages implements the specifics of individual handshake messages. In the TLS 1.3 implementation we don't rely on the actual polymorphic nature of Handshake_Message but replace it with std::variant constructions to specifically define which handshake message types are expected where.

    Each handshake message takes care of "local" protocol semantic checks. I.e. all validations that can be performed without contextual information is happening right in the parsing code. The same holds true for handshake message extensions (e.g. Key_Share, Supported_Groups, ...). Protocol logic that is tied to specific messages or extensions is implemented locally in those classes. For instance, Key_Share implements Diffie-Hellman in its exchange method and Client_Hello_13::retry() implements necessary updates and validations for handling "hello retry requests".

    This is a different paradigm compared to the messages in TLS 1.2, where many messages where mere "Data Transfer Objects" without much logic. Hence, messages derive a _12 and _13 sub-class and share the parsing code in the parent class where possible. This is also useful in situations where the interface and usage of the same message type differs between TLS 1.2 and TLS 1.3, but the wire representation is unchanged.

    opened by hrantzsch 25
  • ECIES_Encryptor::encrypt() throws

    ECIES_Encryptor::encrypt() throws

    Just trying out ECIES:

    #include <botan/auto_rng.h>
    #include <botan/ecies.h>
    #include <botan/ecdh.h>
    #include <iostream>
    
    int main()
    {
        try
        {
            Botan::AutoSeeded_RNG rng;
            Botan::ECDH_PrivateKey private_key(rng, Botan::EC_Group("secp521r1"));
            Botan::ECDH_PrivateKey private_key2(rng, Botan::EC_Group("secp521r1"));
            Botan::ECIES_System_Params params(private_key.domain(), "KDF1-18033(SHA-512)",
                                              "AES-256/CBC", 32, "HMAC(SHA-512)", 20,
                                              Botan::PointGFp::Compression_Type::COMPRESSED, Botan::ECIES_Flags::CHECK_MODE);
            Botan::ECIES_Encryptor enc(private_key, params, rng);
            enc.set_other_key(private_key2.public_point());
            std::string plain("some random data");
            std::vector<uint8_t> data(plain.data(), plain.data() + plain.size());
            auto ret = enc.encrypt(data, rng);
        }
        catch (const Botan::Invalid_State &e)
        {
            std::cout << e.what() << std::endl;
        }
        catch (const std::exception &e)
        {
            std::cout << e.what() << std::endl;
        }
    }
    

    output: Invalid state: state().empty() == false was false in finish:src/lib/modes/cbc/cbc.cpp

    tried with 2.11.0 and 2.12.0 releases, results are the same. not sure if i did something wrong? i dont find ECIES example in the docs, so i tried to grab code from test_ecies.cpp

    it still throws even if i change dem_algo_spec to other unauthenticated cipher modes, so i think its an issue in the ECIES_Encryptor implementation?

    bug 
    opened by DisableAsync 25
  • Turn --destdir option into an environment variable

    Turn --destdir option into an environment variable

    Only install.py is using this. It is not used during configure and build time. This allows for a clean separation of destdir and prefix in the style of autotools.

    Fixes #996

    opened by danimo 23
  • VS2013::Botan app hangs forever, maybe related to AutoSeeded_RNG

    VS2013::Botan app hangs forever, maybe related to AutoSeeded_RNG

    I've tried the latest src from github, compiled fine on VS2013 with cpu=x64. But when trying to run a simple program that creates an RSA key, program hangs forever. Same thing happens trying to run botan.exe and botan_test.exe. Not sure if its an issue with AutoSeeded_RNG or what.

    Code is pretty simple:

    For 1.11.16

    #include <fstream>
    #include <iostream>
    #include <string>
    #include <vector>
    #include <cstring>
    #include <memory>
    
    #include <botan/botan.h>
    #include <botan/rsa.h>
    #include <botan/x509cert.h>
    #include <botan/x509self.h>
    #include <botan/pem.h>
    #include <botan/oids.h>
    
    using namespace std;
    
    int main()
    {
        try
        {
            cout << "Hello Botan!" << endl;
        cout << "Generate key.." << endl;
        AutoSeeded_RNG rng;
            RSA_PrivateKey rsaPrivate(rng, 1024);
    
            // just testing some stuff
            vector<Botan::byte> publicBytes  = X509::BER_encode(rsaPrivate);
        vector<Botan::byte> privateBytes = PKCS8::BER_encode(rsaPrivate, rng, "passphrase");
    
        ofstream pub("public.pem");
        pub << X509::PEM_encode(rsaPrivate);
    
        getchar();
    
         }
         catch(std::exception& e)
         {
            cerr << "Error: " << e.what() << "\n";
         }
    
         return 0;
    }
    
    bug 
    opened by sebastiandev 22
  • Add PKCS#11 support

    Add PKCS#11 support

    The implementation is divided into two parts: a low level and a high level API. Both interfaces are implemented against the PKCS#11 standard in version 2.4.

    The implemented low level API provides access to all functions described in the PKCS#11 standard. The advantage of using the low level API instead of the C-style PKCS#11 API directly is a C++ interface that provides features like RAII, exceptions and automatic memory management. The low level API is implemented in src/lib/prov/pkcs11/p11.h and src/lib/prov/pkcs11/p11.cpp.

    The high level API provides access to the most commonly used PKCS#11 functionality in an object oriented manner. Functionality of the high level API includes:

    • Loading/unloading of PKCS#11 modules
    • Initialization of token
    • Change of PIN/PUK
    • Session management
    • Random number generation
    • Enumeration of objects on the token (certificates, public keys, private keys)
    • Import/export/deletion of certificates
    • Generation/import/export/deletion of RSA and EC public and private keys
    • Encryption/decryption using RSA with support for OAEP and PKCS1-v1_5 (and raw)
    • Signature generation/verification using RSA with support for PSS and PKCS1-v1_5
    • Signature generation/verification using ECDSA
    • Key derivation using ECDH

    Tests were written for the low level API in test_pkcs11_low_level.cpp and for the high level API in test_pkcs11_high_level.cpp. These tests need to know against which PKCS#11 module the tests should be executed. For this we've extended the test command line tool with a parameter --pkcs11-lib= where the user can specify the path to a PKCS#11 module. The tests assume that the PIN is set to “123456” and that the SO_PIN is set to “12345678”. Maybe we can drop the low level tests because 99% of the tests are testing only the PKCS#11 module and token. These tests can be executed manually for example by executing botan-test pkcs11-manage or all together by executing botan-test pkcs11. The PKCS#11 tests are not part of the normal test suite because of the dependency on the PKCS#11 module and potentially on a token reader and token. The implementation is tested against SoftHSMv2. Testing against SoftHSMv2 has the advantage that it is a software implementation and therefore to execute the tests no token reader and token is required. Furthermore, because it's open source, we can inspect the code in case there are errors or unexpected behavior. All tests execute successfully against SoftHSMv2 2.1.0.

    Furthermore configure.py is extended with a new command --with-external-includedir. This must point to a directory containing the PKCS#11 header files. The header files can be obtained from here

    opened by neusdan 20
  • Ideas for solving problems with the incorrect output of BigInts (for Issue #3073)

    Ideas for solving problems with the incorrect output of BigInts (for Issue #3073)

    Since BigInts are only output in the cli and in tests I have changed the behaviour so that by default no hexadecimal prefix is output when using operator<< (std::ostream&, const BigInt&). The function to_hex_string (line 88 in src/lib/math/bigint/big_code.cpp) can now optionally not output the prefix. Therefore, other places in the code that use this function (line 328 in src/lib/asn1/asn1_print.cpp and line 95, 107 in src/lib/ffi/ffi_mp.cpp) are not affected.

    opened by felix-mk 0
  • Handle GitHub Actions deprecations

    Handle GitHub Actions deprecations

    GitHub Actions will remove support for the 'save-state' and 'set-output' commands in May 2023: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/

    ubuntu-latest will soon be Ubuntu 22.04 same as macos-latest becoming macOS 12.

    Actions based on Node.js 12 are being phased out until summer 2023: https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/ For us this seems to affect the actions/[email protected] action.

    Support for the Ubuntu 18.04 image will end in April 2023: https://github.com/actions/runner-images/issues/6002

    opened by reneme 0
  • BigInt: Problems with the output in hexadecimal representation and negative numbers

    BigInt: Problems with the output in hexadecimal representation and negative numbers

    Output of BigInts in hexadecimal representation

    If you output BigInts with std::hex via operator<<(std::ostream&, const BigInt&) (in src/lib/math/bigint/big_io.cpp) the prefix 0x is automatically added:

    std::cout << std::hex << Botan::BigInt{0x123} << '\n';
    // Output: 0x123
    

    However, this does not happen with the standard library:

    std::cout << std::hex << 0x123 << '\n';
    // Output: 123 and not 0x123
    
    std::cout << "0x" << std::hex << 0x123 << '\n';
    // Output: 0x123
    

    This is probably the reason why it was not noticed that this code from the cli outputs the prefix twice.

    // src/cli/math.cpp, line: 66
    const Botan::BigInt p = Botan::random_prime(rng(), bits);
    
    if(hex)
        output() << "0x" << std::hex << p << "\n";
        //          ^^^^^^^^^^^^^^^^^^^^^
    else
        output() << p << "\n";
    
    $ botan gen_prime --hex 2
    # Output: 0x0x02 and not 0x02
    

    Nevertheless, the current implementation of BigInt makes sense in that it allows a negative sign to be placed in front. (Which of course does not happen with normal integers in the two's complement). However, there is another problem.

    Output of negative BigInts

    When displaying negative BigInts, the sign is output twice:

    std::cout << -Botan::BigInt{10} << '\n';
    // Output: --10 and not -10
    

    This happens because both operator<<(std::ostream&, const BigInt&) and the functions called from it to_dec_string/to_hex_string (in src/lib/math/bigint/big_code.cpp) prepend a negative sign.

    This leads to the output being far from the expected/normal:

    std::cout << "0x" << std::hex << -Botan::BigInt{10} << '\n';
    // Output: 0x--0x0A and not 0x-0A or -0x0A
    

    My ideas for the solution

    1. No more prefix 0x in operator<<(std::ostream&, const BigInt&).
    2. No more sign in operator<<(std::ostream&, const BigInt&), but still in to_dec_string/to_hex_string, as some functions call to_dec_string/to_hex_string directly.

    I don't know if these changes are realistic, because I haven't found a way to find all the places in the code that depend on the overloaded operator <<.


    This is my first issue on GitHub. I hope I haven't missed anything obvious.

    bug 
    opened by felix-mk 0
  • Handle LGTM shutdown

    Handle LGTM shutdown

    Just learned LGTM is shutting down December 16th, replaced by some Github integration https://github.blog/2022-08-15-the-next-step-for-lgtm-com-github-code-scanning/

    opened by randombit 0
Owner
Jack Lloyd
Cryptography and goats
Jack Lloyd
Cryptography Toolkit

Botan: Crypto and TLS for Modern C++ Botan (Japanese for peony flower) is a C++ cryptography library released under the permissive Simplified BSD lice

Jack Lloyd 2k Nov 26, 2022
MIRACL Cryptographic SDK: Multiprecision Integer and Rational Arithmetic Cryptographic Library is a C software library that is widely regarded by developers as the gold standard open source SDK for elliptic curve cryptography (ECC).

MIRACL What is MIRACL? Multiprecision Integer and Rational Arithmetic Cryptographic Library – the MIRACL Crypto SDK – is a C software library that is

MIRACL 517 Nov 22, 2022
Finalists to the NIST lightweight cryptography competition

LWC Finalists This repository contains implementations of the 10 finalists in the NIST lightweight cryptography competition: ASCON, Elephant, GIFT-COF

null 32 Sep 4, 2022
In cryptography, a cipher is an algorithm for performing encryption or decryption.

Cipher 1.0 In cryptography, a cipher is an algorithm for performing encryption or decryption. What can be done using this program? This program can us

null 1 Apr 21, 2022
Mbedcrypto - a portable, small, easy to use and fast c++14 library for cryptography.

mbedcrypto mbedcrypto is a portable, small, easy to use, feature rich and fast c++14 library for cryptography based on fantastic and clean mbedtlsnote

amir zamani 38 Nov 22, 2022
Retter - A collection of hash functions, ciphers, tools, libraries, and materials related to cryptography & security

Retter - A collection of hash functions, ciphers, tools, libraries, and materials related to cryptography & security.

Maciej A. Czyzewski 79 Nov 26, 2022
A collection of public domain/unlicense single-file cryptography

simple-crypto A collection of single-file public domain/unlicense cryptographic functions in different programming languages. Feel free to copy-paste

null 4 Sep 24, 2022
Cryptography Toolkit

Botan: Crypto and TLS for Modern C++ Botan (Japanese for peony flower) is a C++ cryptography library released under the permissive Simplified BSD lice

Jack Lloyd 2k Nov 26, 2022
Microsoft Cognitive Toolkit (CNTK), an open source deep-learning toolkit

CNTK Chat Windows build status Linux build status The Microsoft Cognitive Toolkit (https://cntk.ai) is a unified deep learning toolkit that describes

Microsoft 17.3k Nov 22, 2022
Lite.AI.ToolKit 🚀🚀🌟: A lite C++ toolkit of awesome AI models such as RobustVideoMatting🔥, YOLOX🔥, YOLOP🔥 etc.

Lite.AI.ToolKit ?? ?? ?? : A lite C++ toolkit of awesome AI models which contains 70+ models now. It's a collection of personal interests. Such as RVM, YOLOX, YOLOP, YOLOR, YoloV5, DeepLabV3, ArcFace, etc.

DefTruth 2.3k Nov 29, 2022
Zenotech 6 Oct 21, 2022
Insight Toolkit (ITK) is an open-source, cross-platform toolkit for N-dimensional scientific image processing, segmentation, and registration

ITK: The Insight Toolkit C++ Python Linux macOS Windows Linux (Code coverage) Links Homepage Download Discussion Software Guide Help Examples Issue tr

Insight Software Consortium 1.1k Nov 24, 2022
Microsoft Cognitive Toolkit (CNTK), an open source deep-learning toolkit

The Microsoft Cognitive Toolkit is a unified deep learning toolkit that describes neural networks as a series of computational steps via a directed graph.

Microsoft 17.3k Nov 24, 2022
MIRACL Cryptographic SDK: Multiprecision Integer and Rational Arithmetic Cryptographic Library is a C software library that is widely regarded by developers as the gold standard open source SDK for elliptic curve cryptography (ECC).

MIRACL What is MIRACL? Multiprecision Integer and Rational Arithmetic Cryptographic Library – the MIRACL Crypto SDK – is a C software library that is

MIRACL 517 Nov 22, 2022
MIRACL Cryptographic SDK: Multiprecision Integer and Rational Arithmetic Cryptographic Library is a C software library that is widely regarded by developers as the gold standard open source SDK for elliptic curve cryptography (ECC).

MIRACL What is MIRACL? Multiprecision Integer and Rational Arithmetic Cryptographic Library – the MIRACL Crypto SDK – is a C software library that is

MIRACL 517 Nov 22, 2022
Finalists to the NIST lightweight cryptography competition

LWC Finalists This repository contains implementations of the 10 finalists in the NIST lightweight cryptography competition: ASCON, Elephant, GIFT-COF

null 32 Sep 4, 2022
In cryptography, a cipher is an algorithm for performing encryption or decryption.

Cipher 1.0 In cryptography, a cipher is an algorithm for performing encryption or decryption. What can be done using this program? This program can us

null 1 Apr 21, 2022