A stable nginx module for SSL/TLS ja3 fingerprint, with high performance.

Related tags

Cryptography nginx tls ssl fingerprint ja3
Overview

nginx-ssl-fingerprint

A stable nginx module for SSL/TLS ja3 fingerprint, with high performance.

Description

This module adds new nginx variables for the SSL/TLS ja3 fingerprint. For more information, please see the salesforce ja3

Configuration

Variables

Name Default Value Comments
http_ssl_greased 0 Chrome grease flag
http_ssl_ja3 NULL The ja3 fingerprint for a SSL connection for a HTTP server.
http_ssl_ja3_hash NULL ja3 md5 hash

Example

http {
    server {
        listen                 127.0.0.1:8443 ssl;
        ssl_certificate        cert.pem;
        ssl_certificate_key    priv.key;
        error_log              /dev/stderr debug;
        return                 200 "$http_ssl_ja3";
    }
}

Build

Patches

Compilation and test

Build as a common nginx module.

# Clone

$ git clone -b OpenSSL_1_1_1-stable https://github.com/openssl/openssl
$ git clone -b branches/stable-1.18 https://github.com/nginx/nginx
$ git clone https://github.com/phuslu/nginx-ssl-fingerprint

# Patch

$ patch -p1 -d openssl < nginx-ssl-fingerprint/patches/openssl.1_1_1.patch
$ patch -p1 -d nginx < nginx-ssl-fingerprint/patches/nginx.patch

# Configure & Build

$ cd nginx
$ ASAN_OPTIONS=symbolize=1 ./auto/configure --with-openssl=$(pwd)/../openssl --add-module=$(pwd)/../nginx-ssl-fingerprint --with-http_ssl_module --with-stream_ssl_module --with-debug --with-stream --with-cc-opt="-fsanitize=address -O -fno-omit-frame-pointer" --with-ld-opt="-L/usr/local/lib -Wl,-E -lasan"
$ make

# Test

$ sudo objs/nginx -c $(pwd)/../nginx-ssl-fingerprint/conf/nginx.conf
$ curl -k https://127.0.0.1:8444

Performance Testing

Version

git clone https://github.com/nginx/nginx -b release-1.17.8
git clone https://github.com/openssl/openssl -b OpenSSL_1_1_1g
git clone https://github.com/phuslu/nginx-ssl-fingerprint -b v0.1.0

Server

Type Service Cores Memeory(G)
Server nginx with 5 worker 8 8
Client wrk 8 8

Performance Results

for i in $(seq 1 10); do
    wrk https://localhost/  --latency -t48 -d15 -c2000  >/tmp/wrk.log.$i
done
  • QPS: Average Req/Second in 10 times
  • Latency: Average 50% latency (ms) in 10 times
WRK Connection QPS Cost Origin Req/Sec Origin Latency Req/Sec with fingerprint Latency with fingerprint
50 4.3% 75896.9 571.4us 72599.5 597.9us
100 3.2% 80044.3 1.105 77492.3 1.125
200 5.2% 87101.5 2.063 82601.1 2.144
500 4.6% 93582.7 5.048 89311.6 5.282
1000 6.6% 96417.9 9.802 90020.6 10.519
1500 6.8% 95786.3 12.688 89246 13.868
2000 5.1% 94399.1 14.38 89553.4 91030.35
Comments
  • groups and formats are lost during tls session ticket

    groups and formats are lost during tls session ticket

    Per the original JA3 project README:

    "We also needed to introduce some code to account for Google’s GREASE (Generate Random Extensions And Sustain Extensibility) as described here. Google uses this as a mechanism to prevent extensibility failures in the TLS ecosystem. JA3 ignores these values completely to ensure that programs utilizing GREASE can still be identified with a single JA3 hash."

    Original implementation: GREASE_TABLE = {0x0a0a: True, 0x1a1a: True, 0x2a2a: True, 0x3a3a: True, 0x4a4a: True, 0x5a5a: True, 0x6a6a: True, 0x7a7a: True, 0x8a8a: True, 0x9a9a: True, 0xaaaa: True, 0xbaba: True, 0xcaca: True, 0xdada: True, 0xeaea: True, 0xfafa: True}

    GREASE_TABLE Ref: https://tools.ietf.org/html/draft-davidben-tls-grease-00

    Here are some sample different JA3 for Chrome using the example server in the README:

    user_agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36 ja3: 772,4865-4866-4867-49195-49199-49196-49200-52393-52392-49171-49172-156-157-47-53,0-23-65281-10-11-35-16-5-13-18-51-45-43-27-17513,29-23-24,0 ja3_hash: 59686f806cae30344b525e99af5b655d ssl_greased: 1

    user_agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36 ja3: 772,4865-4866-4867-49195-49199-49196-49200-52393-52392-49171-49172-156-157-47-53,0-23-65281-10-11-35-16-5-13-18-51-45-43-27-17513 ja3_hash: 217b989439002f3b88747b7b038532d9 ssl_greased: 1

    opened by fkasler 25
  • Use OpenSSL ClientHello callback

    Use OpenSSL ClientHello callback

    Would it be possible to implement this using https://www.openssl.org/docs/manmaster/man3/SSL_CTX_set_client_hello_cb.html from recently released version of OpenSSL instead of patching it?

    That should simplify maintenance quite a bit.

    opened by zabbal 2
  • [wip] fix padding,group,format null issue

    [wip] fix padding,group,format null issue

    change:

      1. left comma if group or format is null, keep ja3 has 5 fields.
      1. ignore padding extension due to padding will null when resuing session ticket
    opened by deancn 0
  • fix padding,group,format null issue

    fix padding,group,format null issue

    change:

    1. left comma if group or format is null, keep ja3 has 5 fields.
    2. ignore padding extension due to padding will null when resuing session ticket
    opened by deancn 0
  • [WIP] Add tests

    [WIP] Add tests

    Add tests. Related: https://github.com/phuslu/nginx-ssl-fingerprint/issues/25

    Progress:

    • [x] Add simple smoke tests
    • [ ] Add more ja3transport.Browser's (currently any browser except ja3transport.ChromeVersion103 trigger error, so need research)
    • [ ] Add badges with nginx/openssl version compatibility in readme to solve https://github.com/phuslu/nginx-ssl-fingerprint/issues/27
    • [ ] Speed up github actions:
      • [ ] need to reuse docker's image layer. i think cache at dockerhub can help)
      • [ ] use github actions matrix to run tests in parallel for different versions
    opened by paragor 0
  • Nginx/openssl version compatibility for releases

    Nginx/openssl version compatibility for releases

    Stumbled on this while doing testing, was pulling master while testing a nginx 1.21.4 / openssl 1.1.1q deployment and was getting segfaults. Traced this to the 0.4.0 and subsequent patches, reverting to 0.3.0 tag worked fine. Would be nice to understand what version compatibility there is for the releases.

    opened by sp3nx0r 1
  • Add tests and pipelines

    Add tests and pipelines

    I think project need some tests in pipeline for production grade.

    For example:

    1. some popular ja3 hashes
    2. http and https smoke tests
    3. invalid ssl handshake
    4. something else?

    I can write some.

    But what language choose? python (with tlsfuzzer library) and go (with ja3transport) is pretty good candidate.

    I would rather choose go. What do you think @phuslu @deancn ?

    p.s. golang transport cant use "tls heartbeat", but i think it is ok.

    opened by paragor 2
Releases(v0.4.2)
Owner
phuslu
Geek
phuslu
GitHub
s2n : an implementation of the TLS/SSL protocols

s2n is a C99 implementation of the TLS/SSL protocols that is designed to be simple, small, fast, and with security as a priority. It is released and l

Amazon Web Services 4.2k Dec 31, 2022
s2n : an implementation of the TLS/SSL protocols

s2n-tls is a C99 implementation of the TLS/SSL protocols that is designed to be simple, small, fast, and with security as a priority. It is released a

Amazon Web Services 4.2k Jan 4, 2023
An open source, portable, easy to use, readable and flexible SSL library

README for Mbed TLS Mbed TLS is a C library that implements cryptographic primitives, X.509 certificate manipulation and the SSL/TLS and DTLS protocol

Arm Mbed 3.9k Jan 7, 2023
Obtain and parse SSL certificates

CheckCert A small utility to request the SSL certificate from a public or private web application. CheckCert helps operators in the following ways: It

Sanjiv Kawa 81 Dec 19, 2022
inetd-style TLS encryption wrapper

Description The tlswrapper is an TLS encryption wrapper between remote client and local program prog. Is executed from systemd.socket/inetd/tcpserver/

Jan Mojžíš 2 Dec 22, 2022
A high-performance distributed Bitcoin mining pool server.

Viabtc Mining Server ViaBTC Mining Server is a high-performance distributed Bitcoin mining pool server. We have made a lot of optimizations for Bitcoi

ViaBTC 96 Nov 22, 2022
XMRig is a high performance, open source, cross platform RandomX, KawPow, CryptoNight and AstroBWT unified CPU/GPU miner

XMRig is a high performance, open source, cross platform RandomX, KawPow, CryptoNight and AstroBWT unified CPU/GPU miner and RandomX benchmark. Official binaries are available for Windows, Linux, macOS and FreeBSD.

null 7.3k Jan 9, 2023
High-level build system for distributed, multi-platform C/C++ projects.

fips fips is a highlevel build system wrapper written in Python for C/C++ projects. (this project has nothing to do with the Federal Information Proce

Andre Weissflog 427 Dec 25, 2022
A Nginx module which tries to implement proxy wasm ABI in Nginx.

Status This library is under construction. Description A Nginx module which tries to implement proxy wasm ABI in Nginx. Install dependencies Download

API7 104 Dec 29, 2022
Fingerprint doorlock Arduino project make from Fingerprint sensor and Solenoid door lock

Fingerprint-doorlock Fingerprint doorlock Arduino project make from Fingerprint sensor and Solenoid door lock Main Idea Create Doorlock that uses Fing

Narongkorn 1 Dec 15, 2021
TLS/SSL and crypto library

Welcome to the OpenSSL Project OpenSSL is a robust, commercial-grade, full-featured Open Source Toolkit for the Transport Layer Security (TLS) protoco

OpenSSL 20.5k Jan 6, 2023
s2n : an implementation of the TLS/SSL protocols

s2n is a C99 implementation of the TLS/SSL protocols that is designed to be simple, small, fast, and with security as a priority. It is released and l

Amazon Web Services 4.2k Dec 31, 2022
GnuTLS implements the TLS/SSL (Transport Layer Security aka Secure Sockets Layer) protocol

GnuTLS implements the TLS/SSL (Transport Layer Security aka Secure Sockets Layer) protocol

Jonathan Bastien-Filiatrault 3 Jun 3, 2021
s2n : an implementation of the TLS/SSL protocols

s2n-tls is a C99 implementation of the TLS/SSL protocols that is designed to be simple, small, fast, and with security as a priority. It is released a

Amazon Web Services 4.2k Jan 4, 2023
TLS handshake utilities for in-kernel TLS consumers

Release Notes for ktls-utils 0.5 Note well: This is experimental prototype software. It's purpose is purely as a demonstration and proof-of-concept. U

Oracle 7 Nov 28, 2022
upstream module that allows nginx to communicate directly with PostgreSQL database.

About ngx_postgres is an upstream module that allows nginx to communicate directly with PostgreSQL database. Configuration directives postgres_server

RekGRpth 1 Apr 29, 2022
Pipy is a tiny, high performance, highly stable, programmable proxy.

Pipy Pipy is a tiny, high performance, highly stable, programmable proxy. Written in C++, built on top of Asio asynchronous I/O library, Pipy is extre

null 539 Dec 28, 2022
Cetus is a high performance, stable, protocol aware proxy for MySQL Group Replication.

Introduction Cetus is a high performance, stable, protocol aware proxy for MySQL Group Replication. Getting started 1. Prerequisites cmake gcc glib2-d

null 37 Sep 19, 2022
Pipy is a tiny, high performance, highly stable, programmable proxy written in C++

Pipy is a tiny, high performance, highly stable, programmable proxy. Written in C++, built on top of Asio asynchronous I/O library, Pipy is extremely lightweight and fast, making it one of the best choices for service mesh sidecars.

null 538 Dec 23, 2022
docker-nginx-with-image-filter

docker-nginx-with-image-filter 因为官方镜像默认没有启用该模块，故自己做了一个Dockerfile,并编译，docker镜像地址： docker pull smally84/nginx-with-image-filter:1.20.1 附上Dockerfile ###

null 2 Dec 11, 2021