Brynet - Header Only Cross platform high performance TCP network library using C++ 11.

Overview

Brynet

Header Only Cross platform high performance TCP network library using C++ 11.

996.icu LICENSE SonarCloud Platform

Build status

Windows : Build status Linux/MacOS : Build Status

Features

  • Header only
  • Cross platform (Linux | Windows | MacOS)
  • High performance and safety use.
  • None depend
  • Multi-threaded
  • SSL support
  • Support HTTP、HTTPS、WebSocket
  • IPv6 support

Documentation

Compatibility

  • Visual C++ 2013+ (32/64-bit)
  • GCC 4.8+ (32/64-bit)
  • Clang (Supported C++ 11)

Macro

  • BRYNET_VERSION
  • BRYNET_USE_OPENSSL

Build Example

  1. cmake . -Dbrynet_BUILD_EXAMPLES=ON -Dbrynet_BUILD_TESTS=ON
  2. If you use Windows, please open brynet.sln then build. If on Linux or MacOS, only enter make.

Only Install

  1. cmake .
  2. sudo make install

Usages

Benchmark

Under localhost, use CentOS 6.5 virtual mahcine(host machine is Win10 i5)

  • PingPong

    Benchamrk's server and client both only use one thread, and packet size is 4k

    PingPong

  • Broadcast

    Server use two network threads and one logic thread, client use one network(also process logic) thread. every packet size is 46 bytes. every packet contain client's id. server broadcast packet to all client when recv one packet from any client. client send one packet when recv packet from server and packet's id equal self.

    Broadcast

  • Ab HTTP(1 network thread)

    Server Hostname:        127.0.0.1
    Server Port:            9999
  
    Document Path:          /abc/de?a=1
    Document Length:        25 bytes
    
    Concurrency Level:      100
    Time taken for tests:   17.734 seconds
    Complete requests:      500000
    Failed requests:        0
    Total transferred:      41000000 bytes
    HTML transferred:       12500000 bytes
    Requests per second:    28194.36 [#/sec] (mean)
    Time per request:       3.547 [ms] (mean)
    Time per request:       0.035 [ms] (mean, across all concurrent requests)
    Transfer rate:          2257.75 [Kbytes/sec] received
    
    Connection Times (ms)
    min  mean[+/-sd] median   max
    Connect:        0    2   0.2      2       3
    Processing:     1    2   0.3      2       7
    Waiting:        0    1   0.4      1       6
    Total:          2    4   0.2      4       7
    
    Percentage of the requests served within a certain time (ms)
    50%      4
    66%      4
    75%      4
    80%      4
    90%      4
    95%      4
    98%      4
    99%      4
    100%      7 (longest request)

Examples

Users

Issues
  • Buffer filling up quickly / how to end connections

    Buffer filling up quickly / how to end connections

    Hey,

    this is kind of a follow up to my last request #70 as I went further with my implementation. As I got a stable sample running I integrated the code into another project to have the image transfer inside an application. Currently I have a class managing the image transmission in a thread where other modules of the software deliver an image defined as in the sample https://gist.github.com/irieger/74808b0db67c39e33d37641e210e26bc. The image transmission currently is done in by first creating a connectionBuilder and then sending the data if an image is in the queue:

        brynet::net::wrapper::ConnectionBuilder connectionBuilder;
        connectionBuilder.configureService(service)
            .configureConnector(connector)
            .configureConnectionOptions({
                brynet::net::AddSocketOption::AddEnterCallback(enterCallback),
                brynet::net::AddSocketOption::WithMaxRecvBufferSize(m_network_buffer_megabytes * 1024 * 1024)
            });
    
        bool has_data = false;
        while (m_running)
        {
            {
                std::lock_guard<std::mutex> lock(m_queue_mtx);
                has_data = (m_queue.size() > 0);
            }
            if (has_data)
            {
                try
                {
                    connectionBuilder.configureConnectOptions({
                            brynet::net::ConnectOption::WithAddr(m_target_ip, m_target_port),
                            brynet::net::ConnectOption::WithTimeout(std::chrono::seconds(1)),
                            brynet::net::ConnectOption::WithFailedCallback(failedCallback),
                            brynet::net::ConnectOption::AddProcessTcpSocketCallback([](brynet::net::TcpSocket& socket) {
                                socket.setNodelay();
                            })
                        })
                        .asyncConnect();
                }
                catch (std::runtime_error& e)
                {
                    std::cerr << "[ImageTransmitter]  error:" << e.what() << std::endl;
                }
            }
    
            std::this_thread::sleep_for(std::chrono::milliseconds(3));
        }
    

    This might not be the ideal solution but worked in my first tests. Today I did some longer experiments and realized my memory usage on the receiver side increased steadily until I ran into errors. As I found out the memory was only freed after closing the sender, thus it seems something isn't freed unless the client connection is stopped. Is this correct? When is the receiver freeing the memory? I expected that after returning the size of processed bytes in the DataCallback the memory would be reused.

    The other point is, I now have a number of open connections blocking resources besides the memory usage. Can I automatically close after the send is done? I send with code like this to ensure the data is removed from RAM after the sent (this is the enter callback):

        auto enterCallback = [this](const brynet::net::TcpConnection::Ptr& session)
        {
            session->setDataCallback([session, this](const char* buffer, size_t len)
            {
                return len;  // No answer is currently expected so should be irrelevant.
            });
            // while (m_running)  // uncommented in alternative route (see below)
            {
                // ... code pulling the next image data/size pair from an std::queue
                // ... fills the following variables:
                char* data;
                size_t data_size;
    
                // sending the actual data
                session->send(data, data_size, [data]()
                    {
                        std::free(data);
                    });
                // std::this_thread::sleep_for(std::chrono::milliseconds(10));  // sleep in the alternative case to slow queue/lock checking
            }
    

    After the free I'd also like to close the session now but there seems to be no close/disconnect whatever method in this class.

    Btw. the brynet::net::ConnectOption::WithTimeout(std::chrono::seconds(1)) only means a timeout in case the connection can't be established, not an overall timeout, right?

    Alternative route

    I tried an alternative route creating one or two sender connections waiting for new data in the queue and sending it. Besides a while(m_running) around the block pulling data from the queue and sending it nothing is changed but I don't reach the send callback anymore. This is with or without an additional mutex blocking as long as send is executed (mutex.lock() before the send and mutex.unlock() in the send callback after the free.

    With the std::cout debug output of the size send data (after the call) I see the sender passed the send call, but the send-Callback isn't reached and neither is the debugging output on the server side showing the DataCallback being called. Any hint on this?

    I could take the time to rewrite the example code to fully show the data flow if needed for understanding what I did.

    opened by irieger 14
  • Feauture Request: Add REUSEPORT and REUSEADDR

    Feauture Request: Add REUSEPORT and REUSEADDR

    This feature allows us to run multiple instance of the same App running on a single port. I have this I'm using in my code in /brynet/net/SocketLibFunction.hpp

      static int  SocketSetReusePort(BrynetSocketFD fd)
      {
         int enable = 1;
         return ::setsockopt(fd, SOL_SOCKET, SO_REUSEADDR | SO_REUSEPORT, &enable, sizeof(enable));
      }
    

    The issue now is where to put it. I currently call this function in brynet/net/SocketLibFunction.hpp in ``

            const auto socketfd = isIPV6 ?
                socket(AF_INET6, SOCK_STREAM, 0) :
                socket(AF_INET, SOCK_STREAM, 0);
            if (socketfd == BRYNET_INVALID_SOCKET)
            {
                return BRYNET_INVALID_SOCKET;
            }
    
            SocketSetReusePort(socketfd);
    ...
    

    I tried adding:

            void             setReusePort(bool yes = true)
            {
                if (yes)
                {
                   brynet::net::base::SocketSetReusePort(mFD);
                }
            }
    

    to Socket.hpp, it seems nothing in

    .configureSocketOptions({
                    [&](TcpSocket& socket) {
                        socket.setReuseAddr(true);
                        socket.setReusePort(true);
                        socket.setNodelay();
                },
            })
    

    ever gets called.

    opened by ibroheem 8
  • Crashing when session send response with post close

    Crashing when session send response with post close

    First of all, thank you for your efforts.

    I think your library is great and working as properly in Win10 and as I have tested it, It never crashed to now.

    But My application is a cross-platform application and it needs to be run on Macintosh and Linux distros.

    I have tried my code when I did not any changes in my codes, your library was crashed. I investigated to find what was going to be wrong. I found that if I send response with "post-close" your library will be crashed.

    If you want me to get any information such as core dump, debug info I can send it.

    My compiler is AppleClang 13.0.0.13000029. My code is here that how to send my response:

    session->send(httpResponse.getResult(), [session]() { session->postClose(); });

    opened by thirtyninekilovolts 7
  • 请教个问题,详见内文

    请教个问题,详见内文

    我使用httpclient携带一个参数,发送http请求到后端,处理正常,携带3个参数,发生异常,新加的两个参数格式为“年 月-日 时-分-秒”,打印出来requestStr看结果,符合预期,能够抓到发出的http请求包,和响应包,响应包为400 badRequest,但是我在setHttpCallback设置的打印未生效,但是使用打印出的requestStr码流,使用postman构造后发出,却能正确处理,请教两个问题: 1、我使用了asyncConnect启动发送请求操作,也使用了while循环保证退出,但是抓包看到400响应,但是为什么设置的回调未成功? 2、发出的http请求码流编码有什么特殊处理吗?我根据打印的码流借助postman发出去,却能正确处理?

    opened by lizengwu 7
  • Does this work with Android/iOS?

    Does this work with Android/iOS?

    Hi, I'm planning to build a cross-platform program which must work in windows, linux, macOS, android and iOS.

    Does brynet works with all of these platforms? (of course, in android, for example, using the NDK + JNI).

    If it isn't, which libraries are using brynet that are incompatible?

    Regards, Knife.

    opened by soknifedev 6
  • How to get a list of opened sessions ?

    How to get a list of opened sessions ?

    Hi, I'd like to send messages from the server to every client connected (using websocket). I don't see how to do that. I was expecting to find a list with all the sessions opened.

    there is a way to send messages to each client from the server (using websocket) ?

    thx

    opened by jagernicolas 5
  • Problems sending larger data junks - data splitted on receive

    Problems sending larger data junks - data splitted on receive

    Hey, I stumbled past brynet a few times and thought that I should give it a try. I'm doing a small proof of concept simple binary data send/receive in the local network (image transfer for displaying on another machine). As I want to keep dependencies low and prefer not to depend on a big codebase as boost, brynet catched my eye. The base was easy to understand with the sample PingPongClient/Server and my adaptation looked good. However there is one showstopper for me:

    Data fragmentation

    (Modified sample code used here: https://gist.github.com/irieger/74808b0db67c39e33d37641e210e26bc)

    Without finding a documentation I expected the send/receive code to abstract the fragmentation and split/combine data into the fragments needed for network transfer. For example sending an image with 1920x1080 pixels with RGBA (4 channels) of float32 data the image has already 33177600 bytes. Or in the sample code 1920*1080 with 3 channels of uint16_t 74649600 bytes. Therefore I set WithMaxRecvBufferSize(1535 * 1024 * 1024); and just send the data in the client. The data consist of a simple header (format[int32_t], height[uint32_t], width[uint32_t]) + the image data, thus in my sample 33177612 bytes. (Sample code above adapted as the actual code is inside another project, but the sample code shows the effect)

    This data is now received via three calls of my callback (set with setDataCallback). My test code (interpreting the first 12 bytes as the header so any data there is expected to represent my format, height and width) results in a correct expected size for the first junk and invalid expected sizes afterwards instead of one received package. Combined the size is correct:

    Received image smaller than described by header
         Expected: 33177612, got: 65482
    Received image smaller than described by header
         Expected: 3044704916, got: 65482
    Received image smaller than described by header
         Expected: 4185915404, got: 33046648
    

    Three calls to the callback seem to be reproducible when only one transfer happens. When sending two images in parallel the callback is executed a varying number of times, some samples are 387 or 403 times. However using a different buffer size like 512 MB the number of packets differs.

    Is there an integrated way to send/receive larger binary junks reliable at once or would some boilerplate be needed to split and receive in junks below a certain limit? Where is the limit or how would one need to modify the examples for larger datasets?

    opened by irieger 4
  • multipart/form-data support

    multipart/form-data support

    I am planning to provide mjpeg stream over HTTP in my project. Briefly, I need to send each frame as jpeg over http but did not see anything about setting body to multipart or there is not a setBody function or its overloads that take byte(unsigned char) as the argument. All of them want std::string as the argument.

    opened by thirtyninekilovolts 23
  • Question to API  “service->startWorkerThread(xxx)”

    Question to API “service->startWorkerThread(xxx)”

    Is this API to set the capacity of internal thread pool? Furthermore, how does the brynet schedule the multiple tcp clients to these threads? Such as num of clients greater than num of threads.

    opened by eeyrw 2
  • 循环get请求 间隔1毫秒 就会出这个问题

    循环get请求 间隔1毫秒 就会出这个问题

    /brynet/net/http/http_parser.h:1604: size_t http_parser_execute(http_parser*, const http_parser_settings*, const char*, size_t): assertion "HTTP_PARSER_ERRNO(parser) == HPE_OK" failed Segmentation fault

    opened by jxwdsb 1
  • void brynet::net::EventLoop::loop(int64_t): Assertion `isInLoopThread()' failed. How do we make LoopThread without getting this error?

    void brynet::net::EventLoop::loop(int64_t): Assertion `isInLoopThread()' failed. How do we make LoopThread without getting this error?

    When I put an EventLoop in an std::thread like:

    iothrds.emplace_back([&]
    {
      while (1) { ioloops[i]->loop(1); }
    });
    

    I get: void brynet::net::EventLoop::loop(int64_t): Assertion isInLoopThread()' failed.

    How do we make LoopThread the library support ? Or the Library Just handle any EventLoop created by user?

    opened by ibroheem 4
Releases(v1.12.0)
Owner
IronsDu
IronsDu
A cross-platform network learning demos. Like high-performance http server

Network-Learn A cross-platform network learning demos (toys). And I try not to use 3rd-party libraries. Welcome to try it out and leave your comments.

Ho 229 23 Jul 6, 2022
Header-only C++14 library for getting network addresses associated with network interface without name lookups on Windows, macOS, Linux, and FreeBSD

NetIF Get addresses associated with network interfaces on a system without using name lookups. Header-only, requires C++14. Usage Add the header file

GMLC-TDC 9 Feb 4, 2022
Netif - Header-only C++14 library for getting network addresses associated with network interface without name lookups on Windows, macOS, Linux, and FreeBSD

NetIF Get addresses associated with network interfaces on a system without using name lookups. Header-only, requires C++14. Usage Add the header file

GMLC-TDC 9 Feb 4, 2022
A Tcp/Ip stack implementation on top of Solarflare ef_vi, and a C++ headers only framework for tcp multiplexing client/server.

Efvitcp Efvitcp is a tcp library using Solarflare ef_vi interface on linux, and also a tcp multiplexing framework for both C++ client and server progr

Meng Rao 17 Jun 9, 2022
Asynchronous, Header-only C++ HTTP-over-(TCP|UNIX Socket|STDIO) Library

CXXHTTP A C++ library implementing an asynchronous HTTP server and client. To clone this library, make sure you also clone the submodules. The --recur

null 25 Mar 19, 2021
BingBing 53 Jun 29, 2022
A high-performance and easy-to-use C++ network library.

pine A high-performance and easy-to-use C++ network library. Now this is just a toy library for education purpose, do not use in production. example A

Baroquer 40 Jun 25, 2022
an easy implementation of a multi-process tcp server and a multi-thread tcp client

一个TCP多进程服务器-多线程客户端的简单实现。 客户端类似Apache ab的测试功能,能够通过向某一个ip端口发送指定并发量和总数量的tcp短连接;服务端处理tcp短连接,每来一条消息就打印一条log。 使用cmake编译,建议在vscode里编译,或者命令行 # 终端进入目录 mkdir bu

adin 1 Nov 28, 2021
KBMS is a C++11 high performance network framework based on libevent.

KBMS is a C++11 high performance network framework based on libevent. It is light and easy to deploy. At present, it mainly supports HTTP protocol.

Ke Technologies 28 Apr 12, 2022
The C++ Network Library Project -- cross-platform, standards compliant networking library.

C++ Network Library Modern C++ network programming libraries. Join us on Slack: http://slack.cpp-netlib.org/ Subscribe to the mailing list: https://gr

C++ Network Library 1.9k Jun 10, 2022
modern c++(c++17), cross-platform, header-only, easy to use http framework

cinatra--一个高效易用的c++ http框架 English | 中文 目录 使用cinatra常见问题汇总(FAQ) cinatra简介 如何使用 快速示例 性能测试 注意事项 roadmap 联系方式 cinatra简介 cinatra是一个高性能易用的http框架,它是用modern

qicosmos 1.3k Jul 4, 2022
A simple tcp tunnel on c using sockets Right now it only supports linux systems

A simple tcp tunnel on c using sockets Right now it only supports linux systems build BY MAKE mkdir build make cd build ./tunnel.o <localport> <rem

notaweeb 8 Sep 20, 2021
Winpcap-based network packet capture tool, support TLS (part), UDP, ICMP, TCP, ARP, DNS and other protocol analysis, interface reference wireshark.

Winpcap-based network packet capture tool, support TLS (part), UDP, ICMP, TCP, ARP, DNS and other protocol analysis, interface reference wireshark.

null 35 Jun 25, 2022
Cross-platform, efficient, customizable, and robust asynchronous HTTP/WebSocket server C++14 library with the right balance between performance and ease of use

What Is RESTinio? RESTinio is a header-only C++14 library that gives you an embedded HTTP/Websocket server. It is based on standalone version of ASIO

Stiffstream 863 Jun 23, 2022
A headers only high performance C++ middleware framework/lib. See README for details.

# README # hmbdc is an open source headers only C++ framework and library built on top of various real-world tested lockfree algorithms that facilit

null 12 Jun 29, 2022
Mars is a cross-platform network component developed by WeChat.

Mars is a cross-platform infrastructure component developed by WeChat Mobile Team

Tencent 16.3k Jul 5, 2022
LANDrop is a cross-platform tool that you can use to conveniently transfer photos, videos, and other types of files to other devices on the same local network.

LANDrop is a cross-platform tool that you can use to conveniently transfer photos, videos, and other types of files to other devices on the same local network.

LANDrop 2.8k Jul 1, 2022
High performant TCP server for rtl-sdr

About Key features Share available RF bandwidth between several independent clients: Total bandwidth can be 2016000 samples/sec at 436,600,000 hz One

dernasherbrezon 110 Jun 17, 2022
An extensible, cross-platform, single-header C/C++ OpenGL loader library.

Simple OpenGL Loader An extensible, cross-platform, single-header C/C++ OpenGL loader library. Usage For Windows Win32 or Linux X11 applications, the

Tarek Sherif 78 May 30, 2022