TLS 1.3 implementation in C (master supports RFC8446 as well as draft-26, -27, -28)

Overview

picotls

CI

Picotls is a TLS 1.3 (RFC 8446) protocol stack written in C, with the following features:

Primary goal of the project is to create a fast, tiny, low-latency TLS 1.3 implementation that can be used with the HTTP/2 protocol stack and the upcoming QUIC stack of the H2O HTTP/2 server.

The TLS protocol implementation of picotls is licensed under the MIT license.

License and the cryptographic algorithms supported by the crypto bindings are as follows:

Binding License Key Exchange Certificate AEAD cipher
minicrypto CC0 / 2-clause BSD secp256r1, x25519 ECDSA (secp256r1)1 AES-128-GCM, chacha20-poly1305
OpenSSL OpenSSL secp256r1, secp384r1, secp521r1, x25519 RSA, ECDSA (secp256r1, secp384r1, secp521r1), ed25519 AES-128-GCM, AES-256-GCM, chacha20-poly1305

Note 1: Minicrypto binding is capable of signing a handshake using the certificate's key, but cannot verify a signature sent by the peer.

Building picotls

If you have cloned picotls from git then ensure that you have initialised the submodules:

% git submodule init
% git submodule update

Build using cmake:

% cmake .
% make
% make check

A dedicated documentation for using picotls with Visual Studio can be found in WindowsPort.md.

Developer documentation

Developer documentation should be available on the wiki.

Using the cli command

Run the test server (at 127.0.0.1:8443):

% ./cli -c /path/to/certificate.pem -k /path/to/private-key.pem  127.0.0.1 8443

Connect to the test server:

% ./cli 127.0.0.1 8443

Using resumption:

% ./cli -s session-file 127.0.0.1 8443

The session-file is read-write. The cli server implements a single-entry session cache. The cli server sends NewSessionTicket when it first sends application data after receiving ClientFinished.

Using early-data:

% ./cli -s session-file -e 127.0.0.1 8443

When -e option is used, client first waits for user input, and then sends CLIENT_HELLO along with the early-data.

License

The software is provided under the MIT license. Note that additional licences apply if you use the minicrypto binding (see above).

Issues
  • Implements `CertificateRequest` for client authentication

    Implements `CertificateRequest` for client authentication

    The context has a new parameter client_authentication to enable/disable the feature.

    Should fix: https://github.com/h2o/picotls/issues/116

    I create this pull request to get some feedback. The implementation should be finished.

    I still want to add some testing to ensure that everything works as expected :)

    opened by bkchr 16
  • improve (or provide hook to customize) 0-RTT ticket replay detection

    improve (or provide hook to customize) 0-RTT ticket replay detection

    The rule we have now is very lax. We accept 0-RTT unless we see an unexpectedly old ticket (i.e., issued more than 10 seconds ago).

                int64_t delta = (now - issue_at) - (identity->obfuscated_ticket_age - age_add);
                if (delta <= PTLS_EARLY_DATA_MAX_DELAY)
                    *accept_early_data = 1;
    

    This is fine when the max_early_data_size is small (compared to the size of the TCP receive buffer) and if the application protocol has a way to delay the processing of 0-RTT data (e.g., HTTP 425).

    However, considering the fact that picotls might be used for application protocols other than HTTP, it's worth considering tightening the limits (we do not check the lower bound at all!), or providing a hook to customize reuse detection.

    opened by kazuho 15
  • wireshark with key logging for ID 19

    wireshark with key logging for ID 19

    wireshark: Wireshark 2.3.0-3008-g71905a3 picotls: the latest of the kazuho/draft19 branch TLS 1.3 ID 19 server: www.mew.org:13443

    I got packet dump between cli and www.mew.org:

    % sudo tcpdump -i en0 -s 0 -x -w dump.pcap port 13443
    
    % ./cli -l ~/key.log www.mew.org 13443
    

    This TLS 1.3 ID 19 communication has no problem.

    Then I tried to visualize dump.pcap with Wireshark. Plain parts, CH and SH, are displayed well. (Thank you!) But after letting key.log to Wireshark, nothing happens. That is, encrypted record are not decrypted.

    Here is key.log:

    % cat key.log 
    SERVER_HANDSHAKE_TRAFFIC_SECRET bd4e5e9078115ce037cf5280b5e1b343ae778a045b79cdce78b48451f192e140 a4ed5073e772aab348036d0070ea7836ec9c2766b776e5e3299cfd5f883151ac
    CLIENT_HANDSHAKE_TRAFFIC_SECRET bd4e5e9078115ce037cf5280b5e1b343ae778a045b79cdce78b48451f192e140 28a138be355b1d431971be591d53658030fb79d51f65271bce6f97b6a0d2c541
    SERVER_TRAFFIC_SECRET_0 bd4e5e9078115ce037cf5280b5e1b343ae778a045b79cdce78b48451f192e140 5d9c27a2129f3cbd5f1e9192f2d689bf602ca23a556959a7ddc11e1e6a686dbc
    CLIENT_TRAFFIC_SECRET_0 bd4e5e9078115ce037cf5280b5e1b343ae778a045b79cdce78b48451f192e140 eb633ab1cf4abb4a5e9023f4e41cd31b56e48dc5715bd48925e4e2d5aa50b163
    
    opened by kazu-yamamoto 15
  • Implement premaster/SSLKEYLOGFILE

    Implement premaster/SSLKEYLOGFILE

    Can you add support of new ke for SSLKEYLOFILE (premaster)

    https://boringssl-review.googlesource.com/#/c/8841/

    I will be help to add decrypt on #wireshark

    opened by alagoutte 14
  • Documentation for HRR

    Documentation for HRR

    I am trying to find how to implement HRR in QUIC, and I am not sure how to do such things as:

    1. Tell PicoTLS to not accept the Client Hello and generate a Cookie instead.

    2. Tell the client to consume the cookie: is it sufficient to just send the TLS message through the normal handshake API?

    3. Tell the server to verify the Cookie. Again, is it sufficient to pipe the messages through the handshake API?

    If PicoTLS does not in fact support HRR, I suppose that the server application could generate the HRR and the cookie, and that the client could parse it and extract the Cookie extension. Then, client and server could use the handshake extension mechanism to send and verify the cookie. But that's quite a bit of work, and I would rather not have to do that...

    opened by huitema 13
  • Use X509_VERIFY_PARAM for certificate verification

    Use X509_VERIFY_PARAM for certificate verification

    when using a custom certificate verification callback in picoquic_enable_custom_verify_certificate_callback currently no algo values are installed on the callback object, so we get segfaulted in push_signature_algorithms during the handshake.

    the algo values are hidden inside openssl.c, and ptls_openssl_init_verify_certificate allocates an X509_STORE if you pass in NULL so i figured this is the best solution?

    also SNI was broken for me. but adopting this new way corrects it.

    and openssl says...

    Applications are strongly advised to use this interface in preference to explicitly calling X509_check_host(3), hostname checks may be out of scope with the DANE-EE(3) certificate usage, and the internal check will be suppressed as appropriate when DANE verification is enabled.

    https://www.openssl.org/docs/man1.1.0/man3/X509_VERIFY_PARAM_set1_host.html

    this approach also unifies the verification and makes other vectors of verification simply appendable.

    my server certificate looks like this:

    Certificate:
        Data:
            Version: 3 (0x2)
            Serial Number: 4096 (0x1000)
            Signature Algorithm: ED25519
            Issuer: C = US, ST = NY, O = Test Inc., OU = Test Intermediary CA, CN = 1nt3r, emailAddress = [email protected]
            Validity
                Not Before: Mar 23 16:16:10 2021 GMT
                Not After : Jun  9 16:16:10 2029 GMT
            Subject: C = US, ST = NY, L = NYC, O = Test Inc., OU = Test Intermediary CA, CN = server, emailAddress = [email protected]
            Subject Public Key Info:
                Public Key Algorithm: ED25519
                    ED25519 Public-Key:
                    pub:
                        c1:ce:ec:2e:94:85:e1:40:88:dc:be:82:73:71:ca:
                        96:12:6c:f0:ff:82:43:76:bf:10:2b:ea:f0:be:34:
                        45:85
            X509v3 extensions:
                X509v3 Basic Constraints: 
                    CA:FALSE
                X509v3 Subject Key Identifier: 
                    7E:2B:3F:64:2F:C6:90:F1:C9:94:7B:A5:06:1F:51:F7:D1:90:90:8B
                X509v3 Authority Key Identifier: 
                    keyid:31:BC:26:9B:7D:39:22:0F:3C:D0:58:F3:DC:1E:B9:33:A3:24:96:9D
                    DirName:/C=US/ST=NY/L=NYC/O=Test Inc./OU=Testing Root CA/CN=r00t/[email protected]
                    serial:10:00
    
                X509v3 Key Usage: critical
                    Digital Signature, Key Encipherment
                X509v3 Extended Key Usage: 
                    TLS Web Client Authentication
                X509v3 Subject Alternative Name: 
                    IP Address:0.0.0.0, IP Address:127.0.0.1, IP Address:0:0:0:0:0:0:0:1, DNS:localhost, DNS:0.0.0.0, DNS:127.0.0.1, DNS:::1
        Signature Algorithm: ED25519
             20:e7:35:10:ee:ef:c6:4e:57:a5:da:b0:c0:ea:34:45:99:30:
             e3:4d:75:6e:d4:4f:61:e7:92:85:58:96:7d:6b:fa:a8:0a:94:
             d0:e5:b7:43:f3:09:77:2a:2f:85:07:3d:21:43:7b:1b:45:50:
             b3:dd:71:6d:8a:cb:4c:8e:44:0c
    
    opened by victorstewart 11
  • fix `make -jN` failure when dtrace is enabled

    fix `make -jN` failure when dtrace is enabled

    cf72eb3 is to suppress warnings like this: > CMake Warning (dev) at deps/picotls/cmake/dtrace-utils.cmake:37 (ADD_DEPENDENCIES): Policy CMP0046 is not set: Error on non-existent dependency in add_dependencies. Run "cmake --help-policy CMP0046" for policy details. Use the cmake_policy command to set the policy and suppress this warning.



    3fd9529 is the fix by keeping (-k) statically-named temporary files used by dtrace, which leads to a build failure like:

     Traceback (most recent call last):
      File "/usr/bin/dtrace", line 445, in <module>
        sys.exit(main())
      File "/usr/bin/dtrace", line 432, in main
        os.remove(fname)
    FileNotFoundError: [Errno 2] No such file or directory: '/home/goro/ghq/github.com/h2o/picotls/b/picotls-probes.o.dtrace-temp.c'
    

    where picotls-probes.o.dtrace-temp.c is the temp file used by dtrace and has been removed by another sibling processes.

    opened by gfx 11
  • Assertion `server_name != NULL &&

    Assertion `server_name != NULL && "ptls_set_server_name MUST be called"

    Picotls has a default verifier for server certificate. That default verifier (verify_cert_chain) verifies that the server name is documented before calling openssl to check the chain. There is an assert for that at line 1008 in lib/openssl.c.

    It is very clear that if the server name is not specified, verifying the cert chain is pointless. But then, we have lots of picoquic users who just want to test the transport features, and start a test with something like "picoquicdemo 127.0.0.1 4433", and then trigger the assert.

    Is it possible to change the behavior and simply "return 0" if the server name is not specified? Or, do I need to just create my own version of the verifier that will bypass the tests in these cases?

    opened by huitema 8
  • Client certificate

    Client certificate

    Does picotls already supports sending client certificates and the verification of these certificates?

    After looking into the code, I'm not 100% sure, but assume that this is currently not supported.

    opened by bkchr 8
  • Adding test for the Visual Studio compile, plus some fixes

    Adding test for the Visual Studio compile, plus some fixes

    The Visual Studio project includes a basic test: a console application that performs a handshake "in memory", thus exercising both server and client functions.

    opened by huitema 8
  • vectorized encrypt API

    vectorized encrypt API

    At the moment, the encrypt API provided by picotls only receives one ptl_iovec_t, which in turn produces an AEAD record. This leads to inefficiency when multiple fragments of data has to be sent concatenated.

    To give an example, HTTP/2 implementations typically split HTTP response into small chunks, prepending HTTP/2 frame headers to each of those chunks, before they pass them to the TLS stack. At the moment, HTTP/2 implementations have to either:

    • a) build a flat image that has the HTTP/2 frame headers and chunks of response being concatenated, or
    • b) call ptls_send for encrypting the HTTP/2 frame header, then make a separate call for encrypting the response chunk.

    a is inefficient when the implementation already has the response buffered in memory, because an extra copy would be required to concatenate the HTTP/2 frame header and the chunk of the response. b is inefficient due to generating a tiny TLS record that contains only the HTTP/2 frame header, not to mention the privacy aspect of leaking information through how things are chunked.

    To address this problem, this pull request attempts to introduce a variant of ptls_send function that accepts a vector of buffers as input.

    At the moment, this PR also replaces the init -> update -> final API at the AEAD layer with a new vectorized API as well. The rationale is that vectorized APIs are expected to provide better performance (as they can be integrated tighter to the encrypting code, see #384). But we can probably keep the existing ones deprecated.

    opened by kazuho 7
  • Loading picotls using FetchContent_MakeAvailable() fails because of fusion

    Loading picotls using FetchContent_MakeAvailable() fails because of fusion

    I am trying to integrate the build of picotls in the build of picoquic. The first attempt was to add lines in the CMakeFile.txt of Picoquic, such as:

      include(FetchContent)
      FetchContent_Declare(   picotls
          GIT_REPOSITORY      https://github.com/h2o/picotls.git)
      FetchContent_MakeAvailable(picotls)
      set(PTLS_INCLUDE_DIRS ${picotls_SOURCE_DIR}/include)
      set(PTLS_LIBRARIES picotls-core picotls-openssl picotls-fusion)
    

    This works on some platforms, as long as picotls-fusion is supported, but it fails on others, such as MacOS M1, that have no support for fusion. In that case, the build will later fail.

    In the current state of picoquic, without using "FetchContent", things work because picotls is entirely build, before calling cmake for picoquic. The cmake script of Picoquic will test whether the fusion library was built, and set compile flags accordingly. But I cannot use that trick here -- there is a single call for cmake of picoquic and picotls, before make itself is run. I only have a bad solution: never use "fusion" in picoquic, remove all dependencies. Works, but then, fusion is never used. Any idea on how to do that better?

    opened by huitema 1
  • non-temporal aes-gcm engine

    non-temporal aes-gcm engine

    This PR implements "non-temporal aes-gcm" engine, that uses non-temporal store instructions when emitting encrypted bytes. When doing zero-copy to NIC, use of ordinary store instructions requires 2x the main memory bandwidth compared to what is actually being sent, because when an ordinary store instruction is issued against an uncached memory location, the CPU has to read that cache line into cache before updating part of that line (note: L3 cache is so small compared to the sum of the send buffers for all the connections inflight). By using non-temporal store instructions, the new engine avoids this read ("read-for-ownership" in terms of MOESI).

    ToDo:

    • [ ] Add more tests (like unaligned output).
    • [x] Use 256-bit AESNI / CLMUL instructions when available (Zen 3).
    • [x] Suppress false positives found by ASAN. The engine works on 64-byte lines. If either edge of the supplied output buffer is not aligned to 64-byte, the engine reads the 64-byte line covering the edge, modifies the necessary bytes, then write back the entire line.
    • [x] Implement decrypt side. It's a bit slower than OpenSSL with GCC, bit faster with clang.

    Builds on top of #385.

    non-temporal aes-gcm engine (encrypt; commit 9f8e12ae)

    Older result: as of e0caecc, aes128gcm (16KB block) throughput, compared to openssl (ubuntu 20.04): ||gcc (nt on)|gcc (nt off1)|clang (nt on)|clang (nt off)| |:--:|--:|--:|--:|--:| |Core i5 9400|84.1%|95.8%|91.1%|92.2%| |Ryzen 7 4750G|97.9%|101.8%|97.3%|101.2%|

    *: "nt off" indicates that non-temporal store instructions (_mm_stream_si256) were replaced by ordinary stores (_mm_store_si256).

    opened by kazuho 1
  • Valgrind complains about ptls_fusion_is_supported_by_cpu

    Valgrind complains about ptls_fusion_is_supported_by_cpu

    The code of the function is:

    int ptls_fusion_is_supported_by_cpu(void)
    {
        unsigned leaf1_ecx, leaf7_ebx;
    
        { /* GCC-specific code to obtain CPU features */
            unsigned leaf_cnt;
            __asm__("cpuid" : "=a"(leaf_cnt) : "a"(0) : "ebx", "ecx", "edx");
            if (leaf_cnt < 7)
                return 0;
            __asm__("cpuid" : "=c"(leaf1_ecx) : "a"(1) : "ebx", "edx");
            __asm__("cpuid" : "=b"(leaf7_ebx) : "a"(7), "c"(0) : "edx");
        }
    
        /* AVX2 */
        if ((leaf7_ebx & (1 << 5)) == 0)
            return 0;
        /* AES */
        if ((leaf1_ecx & (1 << 25)) == 0)
            return 0;
        /* PCLMUL */
        if ((leaf1_ecx & (1 << 1)) == 0)
            return 0;
    
        return 1;
    }
    

    Valgrind complains that the "Conditional jump or move depends on uninitialized value". This is very annoying, because the message will fill the valgrind logs when testing code that uses picotls.

    I think this could be fixed by changing the declaration to unsigned leaf_cnt = 0;.

    opened by huitema 0
  • Knowing handshake state

    Knowing handshake state

    I am trying to follow cli example available in t directory. I want to send application data. I see two problems

    1. In handle connection I do not know when the handshake is done and it is time to send app data.
    2. In the example I see a method send_fully. But I get a linker error.

    Your thoughts will be appreciated.

    opened by gmusjha 1
  • Why does the client hello change?

    Why does the client hello change?

    As part of the picoquic test suite, we have tests of the qlog implementation. The test is simple: run a QUIC connection on a simulator, capture a log, verify that the captured log matches a reference file. For the test to work, we need to make sure that execution remains the same from run to run, so the test detects possible changes in the logging code, not unrelated changes in the environment. For example, in order to minimize variability in the size of the "client hello", the code:

    • Sets the list of supported cipher suites to just one value, AES128GCM
    • Sets the list of supported key exchanges to just one value, secp256r1

    I understand that the value of the client hello will change based on different random numbers, keys shares, etc. But by forcing the code to use just on ciphersuite and one key exchange, I would expect the size of the ClientHello to remain constant. But we observe variations based on versions of picotls, or versions of the underlying openssl library. Why?

    opened by huitema 4
Owner
H2O
The H2O project
H2O
Single C file TLS 1.2/1.3 implementation, using tomcrypt as crypto library

TLSe Single C file TLS 1.3, 1.2, 1.1 and 1.0(without the weak ciphers) implementation, using libtomcrypt as crypto library. It also supports DTLS 1.2

Eduard Suica 442 Jun 22, 2022
Winpcap-based network packet capture tool, support TLS (part), UDP, ICMP, TCP, ARP, DNS and other protocol analysis, interface reference wireshark.

Winpcap-based network packet capture tool, support TLS (part), UDP, ICMP, TCP, ARP, DNS and other protocol analysis, interface reference wireshark.

null 34 Jun 19, 2022
Extract TLS session keys from running programs

Tlskeydump Tlskeydump extracts TLS key material from processes at runtime so that packet captures containing TLS-encrypted data can be decrypted and a

Konstantinos Tsanaktsidis 1 Feb 18, 2022
SNIF ~ e2e TLS trust for IoT

/************************************************************************** * _________ * /````````_\ S N I F ~ e2e TLS trus

VESvault 11 Apr 24, 2022
GnuTLS implements the TLS/SSL (Transport Layer Security aka Secure Sockets Layer) protocol

GnuTLS implements the TLS/SSL (Transport Layer Security aka Secure Sockets Layer) protocol

Jonathan Bastien-Filiatrault 3 Jun 3, 2021
Perf-ninja - This is an online course where you can learn and master the skill of low-level performance analysis and tuning.

Performance Ninja Class This is an online course where you can learn to find and fix low-level performance issues, for example CPU cache misses and br

Denis Bakhvalov 838 Jun 27, 2022
BingBing 53 Jun 10, 2022
A simple tcp tunnel on c using sockets Right now it only supports linux systems

A simple tcp tunnel on c using sockets Right now it only supports linux systems build BY MAKE mkdir build make cd build ./tunnel.o <localport> <rem

notaweeb 8 Sep 20, 2021
a lightweight and performant multicast DNS (mDNS) reflector with modern design, supports zone based reflection and IPv6

mDNS Reflector mDNS Reflector (mdns-reflector) is a lightweight and performant multicast DNS (mDNS) reflector with a modern design. It reflects mDNS q

Yuxiang Zhu 61 Jun 14, 2022
A GlobalProtect VPN client (GUI) for Linux based on OpenConnect and built with Qt5, supports SAML auth mode.

A GlobalProtect VPN client (GUI) for Linux based on OpenConnect and built with Qt5, supports SAML auth mode.

Kevin Yue 466 Jun 24, 2022
It is a wireless temperature, pressure and humidity sensor, supports working in Zigbee networks.

It is a wireless temperature, pressure and humidity sensor, supports working in Zigbee networks. (Zigbee2mqtt open source project). Built on CC2530 chip (Zigbee), two modications: Ebyte E18-MS1PA2-PCB radio module with amplifier and Ebyte E18-MS1-PCB. Powered by the most common AAA batteries.

Andrew Lamchenko 23 Jun 10, 2022
Built a peer-to-peer group based file sharing system where users could share or download files from the groups they belonged to. Supports parallel downloading with multiple file chunks from multiple peers.

Mini-Torrent Built a peer-to-peer group based file sharing system where users could share or download files from the groups they belonged to. Supports

null 1 Nov 15, 2021
Graphical small-internet client for windows, linux, MacOS X and BSDs. Supports gemini, http, https, gopher, finger.

Graphical small-internet client for windows, linux, MacOS X and BSDs. Supports gemini, http, https, gopher, finger.

Felix Queißner 536 Jun 24, 2022
RPC based on C++ Workflow. Supports Baidu bRPC, Tencent tRPC, thrift protocols.

中文版入口 SRPC Introduction SRPC is an RPC system developed by Sogou. Its main features include: Base on Sogou C++ Workflow, with the following features:

C++ Workflow Project and Ecosystem 1.3k Jun 20, 2022
Dependency manager library that supports decentralization

Arbiter Arbiter is a cross-platform C library1 which implements the baseline functionality that should be expected of any dependency manager or packag

Arbiter 114 Dec 11, 2021
We use Clash as the backend proxy, which supports Shadowsocks(R), V2Ray, and Trojan protocols.

We use Clash as the backend proxy, which supports Shadowsocks(R), V2Ray, and Trojan protocols.

Dr. Incognito 1.2k Jun 25, 2022
C++11 implementation of Socket.IO client

By virtue of being written in C++, this client works in several different platforms. The examples folder contains an iPhone, QT and Console example chat client! It depends on websocket++ and is inspired by socket.io-clientpp.

Socket.IO 1.9k Jun 27, 2022
hessian2-codec it is a complete C++ implementation of hessian2 spec

hessian2-codec is a C++ library from Alibaba for hessian2 codec. It is a complete C++ implementation of hessian2 spec. Because it was originally intended to implement the Dubbo Filter of Envoy, it did not provide good support for serialization of user-defined types (there is only one way to implement user-defined types using ADL, but it is not very complete and does not support nested types well). At the moment it is simply deserializing content into some C++ intermediate types.

Alibaba 15 Dec 27, 2021
FreeModbus is a Modbus ASCII/RTU and Modbus TCP implementation for embedded systems

FreeModbus is a Modbus ASCII/RTU and Modbus TCP implementation for embedded systems. It provides an implementation of the Modbus Application Protocol

Mahmood Hosseini 17 Apr 7, 2022