A golang-style C++ coroutine library and more.

Overview

Basic (中文)

CO is an elegant and efficient C++ base library that supports Linux, Windows and Mac platforms. It pursues minimalism and efficiency, and does not rely on third-party library such as boost.

CO includes coroutine library (golang-style), network library (tcp/http/rpc), log library, command line and configuration file parsing library, unit testing framework, json library and other basic components.

Documents

Highlights

  • co

    co is a golang-style C++ coroutine library with the following features:

    • Support multi-thread scheduling, the default number of threads is the number of system CPU cores.

    • Coroutines share the thread stack (the default size is 1MB), and the memory footprint is extremely low, a single machine can easily create millions of coroutines.

    • Support system api hook (Linux & Mac).

    • Support coroutine lock co::Mutex.

    • Support coroutine synchronization event co::Event.

    • Support coroutine pool co::Pool.

    • create coroutine with go():

    void fun() {
        std::cout << "hello world" << std::endl;
    }
    
    go(fun);
  • so

    so is a C++ network library based on coroutines. You can easily write network programs that support both ipv4 and ipv6 with this library. It includes the following modules:

    • tcp module, supports general tcp programming.

    • http module, supports basic http programming.

    • rpc module, implements a rpc framework based on json, single-threaded qps can reach 120k+.

    • Write a static web server:

    #include "co/flag.h"
    #include "co/log.h"
    #include "co/so.h"
    
    DEF_string(d, ".", "root dir"); // Specify the root directory of the web server
    
    int main(int argc, char** argv) {
        flag::init(argc, argv);
        log::init();
    
        so::easy(FLG_d.c_str()); // mum never have to worry again
    
        return 0;
    }
    • Write a general http server
    http::Server serv("0.0.0.0", 80);
    
    serv.on_req(
        [](const http::Req& req, http::Res& res) {
            if (req.is_method_get()) {
                if (req.url() == "/hello") {
                    res.set_status(200);
                    res.set_body("hello world");
                } else {
                    res.set_status(404);
                }
            } else {
                res.set_status(501);
            }
        }
    );
    
    serv.start();
  • log

    log is a super fast local logging system, printing logs is safer than printf:

    LOG << "hello "<< 23;   // info
    ELOG << "hello again";  // error

    Let's see how fast it is below:

    log vs glog google glog co/log
    win2012 HHD 1.6MB/s 180MB/s
    win10 SSD 3.7MB/s 560MB/s
    mac SSD 17MB/s 450MB/s
    linux SSD 54MB/s 1023MB/s

    The above table is the test result of one million info logs (about 50 bytes each) continuously printed by a single thread. The co/log is almost two orders of magnitude faster than glog.

    Why is it so fast? The first is that it is based on fastream that is 8-25 times faster than sprintf. The second is that it has almost no memory allocation operations.

  • flag

    flag is a command line and configuration file parsing library that supports automatic generation of configuration files.

    #include "co/flag.h"
    
    DEF_int32(i, 32, "comments");
    DEF_string(s, "xxx", "string type");
    
    int main(int argc, char** argv) {
        flag::init(argc, argv);
        std::cout << "i: "<< FLG_i << std::endl;
        std::cout << "s: "<< FLG_s << std::endl;
        return 0;
    }

    Build and run:

    ./xx                         # start with default parameters
    ./xx -i=4k -s="hello world"  # integers can take unit k,m,g,t,p (case insensitive)
    ./xx -i 4k -s "hello world"  # equivalent to above
    ./xx --mkconf                # automatically generate configuration file xx.conf
    ./xx -config=xx.conf         # start from configuration file
  • json

    json is a json library comparable to rapidjson, if you use jemalloc, the performance of parse and stringify will be further improved. This library's support for the json standard is not as comprehensive as rapidjson, but it can meet the basic needs of programmers and is easier to use.

Components

  • co/include
    Header files of libco.

  • co/src
    Source files of libco.

  • co/test
    Some test code, each .cc file will be compiled into a separate test program.

  • co/unitest
    Some unit test code, each .cc file corresponds to a different test unit, and all code is compiled into a single test program.

  • co/gen
    A code generation tool automatically generates rpc framework code according to the proto file.

Compiling

xmake

Xmake is recommended for compiling the CO project.

  • Compiler

  • Install xmake

    For windows, mac and debian/ubuntu, you can directly go to the xmake release page to download the installation package. For other systems, please refer to xmake's installation instructions.

    Xmake disables compilation as root by default on linux. ruki says it is not safe. You can add the following line to ~/.bashrc to enable root compilation:

    export XMAKE_ROOT=y
  • Quick start

    # All commands are executed in the root directory of co (the same below)
    xmake       # build libco and gen by default
    xmake -a    # build all projects (libco, gen, co/test, co/unitest)
  • Build libco

    xmake build libco      # build libco only
    xmake -b libco         # the same as above
  • Build and run unitest code

    co/unitest is unit test code that verifies the correctness of the functionality of the base library.

    xmake build unitest    # build can be abbreviated as -b
    xmake run unitest -a   # run all unit tests
    xmake r unitest -a     # the same as above
    xmake r unitest -os    # run unit test os
    xmake r unitest -json  # run unit test json
  • Build and run test code

    co/test contains some test code. You can easily add a xxx.cc source file in the co/test directory, and then execute xmake build xxx to build it.

    xmake build flag             # flag.cc
    xmake build log              # log.cc
    xmake build json             # json.cc
    xmake build rapidjson        # rapidjson.cc
    xmake build rpc              # rpc.cc
    xmake build easy             # so/easy.cc
    xmake build pingpong         # so/pingpong.cc
    
    xmake r flag -xz             # test flag
    xmake r log                  # test log
    xmake r log -cout            # also log to terminal
    xmake r log -perf            # performance test
    xmake r json                 # test json
    xmake r rapidjson            # test rapidjson
    xmake r rpc                  # start rpc server
    xmake r rpc -c               # start rpc client
    xmake r easy -d xxx          # start web server
    xmake r pingpong             # pingpong server:   127.0.0.1:9988
    xmake r pingpong ip=::       # pingpong server:   :::9988  (ipv6)
    xmake r pingpong -c ip=::1   # pingpong client -> ::1:9988
  • Build gen

    # It is recommended to put gen in the system directory (e.g. /usr/local/bin/).
    xmake build gen
    gen hello_world.proto

    Proto file format can refer to hello_world.proto.

  • Installation

    # Install header files, libco, gen by default.
    xmake install -o pkg         # package related files to the pkg directory
    xmake i -o pkg               # the same as above
    xmake install -o /usr/local  # install to the /usr/local directory

cmake

izhengfan has helped to provide cmake support:

  • Build libco and gen by default.
  • The library files are in the build/lib directory, and the executable files are in the build/bin directory.
  • You can use BUILD_ALL to compile all projects.
  • You can use CMAKE_INSTALL_PREFIX to specify the installation directory.
mkdir build && cd build
cmake ..
cmake .. -DBUILD_ALL=ON -DCMAKE_INSTALL_PREFIX=pkg
make -j8
make install

License

CO is licensed under the MIT License. It includes code from some other projects, which have their own licenses, see details in LICENSE.md.

Special thanks

  • The code of co/context is from tbox by ruki, special thanks!
  • The English reference documents of CO are translated by Leedehai (1-10), daidai21 (11-15) and google, special thanks!
  • ruki has helped to improve the xmake compilation scripts, thanks in particular!
  • izhengfan provided cmake compilation scripts, thank you very much!

Donate

Goto the Donate page.

Comments
  • It's possible to coroutinize blocking sockets (?)

    It's possible to coroutinize blocking sockets (?)

    co_recv() is taken from co's doc and simplified for system sockets. It requires the socket to be set in non-blocking mode. co_recv_r() is basically co_recv() but with read/wait ordering switched. And it works on both blocking and non-blocking sockets?

    // doc's approach
    // https://idealvin.github.io/en/co/coroutine/#coroutineization
    int co_recv(int socket, void* buf, int n, int ms = 10000)
    {
        do
        {
            /* ============ READ FIRST ============ */
            if (const auto r = co::recv(socket, buf, n); r > 0)
                return r;
            else if (r == 0)
                return 0;
    
            /* ============ WAIT AFTER ============ */
            co::IoEvent ev(socket, co::ev_read);
            if (!ev.wait(ms))
                return -1;
        }
        while (true);
    }
    // Supports both blocking and non-blocking sockets, but are there pitfalls?
    int co_recv_r(int socket, void* buf, int n, int ms = 10000)
    {
        do
        {
            /* ============ WAIT FIRST ============ */
            co::IoEvent ev(socket, co::ev_read);
            if (!ev.wait(ms))
                return -1;
    
            /* ============ READ AFTER ============ */
            if (const auto r = co::recv(socket, buf, n); r > 0)
                return r;
            else if (r == 0)
                return 0;
        }
        while (true);
    }
    

    Full code for a simple test:

    #include "co/all.h"
    
    // doc's approach
    // https://idealvin.github.io/en/co/coroutine/#coroutineization
    int co_recv(int socket, void* buf, int n, int ms = 10000)
    {
        do
        {
            /* ============ READ FIRST ============ */
            if (const auto r = co::recv(socket, buf, n); r > 0)
                return r;
            else if (r == 0)
                return 0;
    
            /* ============ WAIT AFTER ============ */
            co::IoEvent ev(socket, co::ev_read);
            if (!ev.wait(ms))
                return -1;
        }
        while (true);
    }
    // Supports both blocking and non-blocking sockets, but are there pitfalls?
    int co_recv_r(int socket, void* buf, int n, int ms = 10000)
    {
        do
        {
            /* ============ WAIT FIRST ============ */
            co::IoEvent ev(socket, co::ev_read);
            if (!ev.wait(ms))
                return -1;
    
            /* ============ READ AFTER ============ */
            if (const auto r = co::recv(socket, buf, n); r > 0)
                return r;
            else if (r == 0)
                return 0;
        }
        while (true);
    }
    void handle1(tcp::Connection* conn)
    {
        std::cout << std::time(nullptr) << ' ' << __PRETTY_FUNCTION__ << " Start " << std::endl;
        uint8_t buffer[256]{};
        // while (const auto bytes = co_recv(conn->socket(), buffer, 256))
        while (const auto bytes = co_recv_r(conn->socket(), buffer, 256))
            conn->send(buffer, bytes);
        delete conn;
        std::cout << std::time(nullptr) << ' ' << __PRETTY_FUNCTION__ << " End " << std::endl;
    }
    void handle2(tcp::Connection*)
    {
        std::cout << std::time(nullptr) << ' ' << __PRETTY_FUNCTION__ << std::endl;
    }
    void on_connection(tcp::Connection* conn)
    {
        fcntl(conn->socket(), F_SETFL, fcntl(conn->socket(), F_GETFL) & ~O_NONBLOCK);
        if (fcntl(conn->socket(), F_GETFL) & O_NONBLOCK)
            throw std::runtime_error(std::string(__PRETTY_FUNCTION__) + ": the socket is still in non-blocking mode. ");
        auto* s = co::next_scheduler();
        s->go(handle1, conn);
        s->go(handle2, conn);
    }
    void client()
    {
        tcp::Client c("127.1", 8899);
        if (!c.connect(3000))
            throw std::runtime_error(std::string(__PRETTY_FUNCTION__) + ": failed to connect. ");
        for (int i = 0; i < 3; ++i)
        {
            c.send("hello", 5);
            char buffer[6]{};
            if (c.recv(buffer, 5))
            {
                //std::cout << __PRETTY_FUNCTION__ << ": " << buffer << std::endl;
            }
            sleep::sec(1);
        }
    }
    int main(int, char**)
    {
        tcp::Server serv;
        serv.on_connection(on_connection);
        serv.start("0.0.0.0", 8899);
    
        sleep::ms(200);
        go(client);
    
        while (true)
            sleep::sec(1024);
    }
    
    

    With co_recv_r() handle2() will be executed between handle1()'s beginning and ending, no matter whether the socket's blocking flag is set or not. But with co_recv() it only happens if the socket is set in non-blocking.

    I wonder if there is any pitfall in this code? Or is there serious performance penalty using co::IoEvent? Why isn't this the example in documents?

    PS: I have zero knowledge about SSL sockets, not clue if we have to do SSL_read() before SSL_get_error(). I only tested with POCO's SecureStreamSocket and it worked as well.

    question 
    opened by DisableAsync 42
  • cannot build with msvc x86

    cannot build with msvc x86

    Under co/base/:

    $ xmake f -a x86
    checking for the Microsoft Visual Studio (x86) version ... 2019
    
    $ xmake
    [ 23%]: compiling.release co\impl\epoll.cc
    [ 26%]: compiling.release co\impl\co_win.cpp
    [ 29%]: compiling.release stack_trace\stack_trace_win.cpp
    [ 33%]: compiling.release win\time.cpp
    [ 39%]: compiling.release fast.cc
    [  6%]: compiling.release fastring.cc
    [  9%]: compiling.release hash\base64.cc
    [ 16%]: compiling.release co\impl\co_unix.cc
    [ 19%]: compiling.release json.cc
    [ 42%]: compiling.release co\impl\scheduler.cc
    [ 46%]: compiling.release unix\time.cc
    [ 49%]: compiling.release win\os.cpp
    [ 52%]: compiling.release str.cc
    error: time.cpp
    C:\Users\fzheng\source\repos\co\base\win/atomic.h(46): error C3861: '_InterlockedIncrement64': identifier not found
    C:\Users\fzheng\source\repos\co\base\win/atomic.h(62): error C3861: '_InterlockedDecrement64': identifier not found
    C:\Users\fzheng\source\repos\co\base\win/atomic.h(78): error C3861: '_InterlockedExchangeAdd64': identifier not found
    C:\Users\fzheng\source\repos\co\base\win/atomic.h(94): error C3861: '_InterlockedExchangeAdd64': identifier not found
    C:\Users\fzheng\source\repos\co\base\win/atomic.h(110): error C3861: '_InterlockedExchangeAdd64': identifier not found
    C:\Users\fzheng\source\repos\co\base\win/atomic.h(126): error C3861: '_InterlockedExchangeAdd64': identifier not found
    C:\Users\fzheng\source\repos\co\base\win/atomic.h(142): error C3861: '_InterlockedExchangeAdd64': identifier not found
    C:\Users\fzheng\source\repos\co\base\win/atomic.h(158): error C3861: '_InterlockedExchangeAdd64': identifier not found
    C:\Users\fzheng\source\repos\co\base\win/atomic.h(174): error C3861: '_InterlockedOr64': identifier not found
    C:\Users\fzheng\source\repos\co\base\win/atomic.h(190): error C3861: '_InterlockedAnd64': identifier not found
    C:\Users\fzheng\source\repos\co\base\win/atomic.h(206): error C3861: '_InterlockedXor64': identifier not found
    C:\Users\fzheng\source\repos\co\base\win/atomic.h(222): error C3861: '_InterlockedOr64': identifier not found
    C:\Users\fzheng\source\repos\co\base\win/atomic.h(238): error C3861: '_InterlockedAnd64': identifier not found
    C:\Users\fzheng\source\repos\co\base\win/atomic.h(254): error C3861: '_InterlockedXor64': identifier not found
    C:\Users\fzheng\source\repos\co\base\win/atomic.h(270): error C3861: '_InterlockedExchange64': identifier not found
    
    opened by izhengfan 25
  • 是否有意向将install后的库名以及目录名由base改为co?

    是否有意向将install后的库名以及目录名由base改为co?

    目前执行xmake install后,会将头文件放在base目录下,静态库被命名为libbase.abase这个名字太普通了,不够将库与程序内部的名字区分开。既然项目名叫co,何不将头文件放在include/co目录下,且库名为libco.alibco.so

    这个动静有点大,但从长远来看,我觉得有必要来执行这个改动。

    opened by tigerlee 23
  • macOS 平台 动态库hook不生效

    macOS 平台 动态库hook不生效

    目前的http::Client使用curl_easy_perform,这是一个阻塞接口。 使用单个协程调度器时,http请求将阻塞其他协程。

    http::Client应该设计成非阻塞协程的,这才能体现协程的优势。推广到其他API,作为协程库里的实现,应当都设计为非阻塞的~

    PS:

    1. 非阻塞可通过curl_multi_perform+辅助线程/协程(也许),或者有更好的办法(基于curl)?
    2. 很多测试貌似应该用单个协程调度器,比较容易发现阻塞的问题。
    bug improvement 
    opened by shuai132 21
  • vcpkg conan 都找不到coost这个包,xmake编译出来也用不了

    vcpkg conan 都找不到coost这个包,xmake编译出来也用不了

    如题,我看readme里面写了这几种使用方式

    vcpkg search coost
    conan search coost -r all
    

    都找不到这个包

    另外,xmake.lua 直接引入coost后,

    add_requires("fmt", "coost", "conan::pugixml/1.12.1")
    add_packages("fmt", "coost", "conan::pugixml/1.12.1")
    

    在 ubuntu 编译示例代码也会报错

    #include "co/all.h"
    
    DEF_string(s, "nice", "");
    
    int main(int argc, char** argv) {
        flag::init(argc, argv);
        LOG << FLG_s;
        return 0;
    }
    

    会产生下面的错误

    [email protected] /m/c/m/c/cpp_practical_util> x
    [ 88%]: linking.debug test
    error: /usr/bin/ld: /home/cat/.xmake/packages/c/coost/v3.0.0/e7a64508b634445f93e6b7f099ab5e52/lib/libco.a(stack_trace.cc.o): in function `___::log::StackTraceImpl::dump_stack(void*, int)':
    stack_trace.cc:(.text+0xb8): undefined reference to `backtrace_create_state'
    /usr/bin/ld: stack_trace.cc:(.text+0xcf): undefined reference to `backtrace_full'
    collect2: error: ld returned 1 exit status
    
    opened by heheda123123 19
  • [help] co::go(): any easy and efficient way to send notices to all schedulers and/or all coroutines?

    [help] co::go(): any easy and efficient way to send notices to all schedulers and/or all coroutines?

    i have a program like this

    server.listen();
    while (server.poll())
        go([socket = socket.acceptConnection()]() mutable {
            char bytes[1024];
            while (int size = socket.receive(bytes, 1024))
            {
                // do something and write to socket
            }
        });
    

    this works perfectly for request-response pattern. but in situations where server push is also desired (like in subscribe-publish pattern, but work with request-response at the same time. ), what can i do if i dont wanna open up another connection and do different things? i dont think frequently sending "check" commands to server is a good idea.

    ps: im not using so for networking becuz it doesnt have built-in ssl support (yet).

    opened by DisableAsync 18
  • unitest xmake错误:error: unknown target(base) for unitest.deps!

    unitest xmake错误:error: unknown target(base) for unitest.deps!

    我根据文档上的xmake编译步骤,编译co/unitest时报错 checking for the architecture ... x64 checking for the Microsoft Visual Studio (x64) version ... 2017 error: unknown target(base) for unitest.deps! 该如何解决

    opened by ylsislove 17
  • libcurl 版本冲突导致 http get 没有发送请求

    libcurl 版本冲突导致 http get 没有发送请求

        string URL = "http://github.com";
        auto req_lambda3 = [](shared_ptr<LambdaParams2> p)
        {
            http::Client c("http://github.com"); // https client to github.com:443
            std::cout << "1--nothing---------" << std::endl;
            c.get("/");
            std::cout << c.body() << c.strerror() << "2--nothing---------" << std::endl;
            std::cout << "3--nothing---------" << std::endl;
        };
        co::go(&req_lambda3, make_shared<LambdaParams2>(make_shared<http::Client>(URL.c_str()), make_shared<string>("asdk")));
        std::cout << "4--nothing---------" << std::endl;
        std::this_thread::sleep_for(10000ms);
    

    输出 4--nothing--------- │ 1--nothing---------

    抓包没有发送请求

    预期 应当 get 成功发送请求

    环境

    ubuntu18 amd64. g++ 5.5. c++14. libcurl 7.73.0.

    opened by meet-ai 16
  • arm7协程中opencv resize触发Bus error

    arm7协程中opencv resize触发Bus error

    #include <opencv2/imgproc.hpp>
    #include <opencv2/imgcodecs.hpp>
    #include <co/co.h>
    #include <co/log.h>
    
    DEC_bool(cout);
    
    int main(int argc, char** argv)
    {
        co::init();
        log::init();
    
        FLG_cout = true;
    
        co::WaitGroup wg;
    
        auto mat = cv::imread("./bar.jpeg", cv::IMREAD_UNCHANGED);
        cv::Mat tmp;
        cv::resize(mat, tmp, cv::Size(100, 100));
    
        LOG << "resize " << tmp.cols << " " << tmp.rows;
    
        go([wg, mat]{
            wg.add();
            cv::Mat tmp;
            cv::resize(mat, tmp, cv::Size(200, 200));
    
            LOG << "resize2 " << tmp.cols << " " << tmp.rows;
    
            wg.done();
        });
    
        wg.wait();
    
        return 0;
    }
    

    堆栈

    #0  cv::hal_baseline::v_int16x8::v_int16x8 (this=0x9fafd01c, v=...)
        at /Volumes/Docs/git/cppframe/build-armeabihf-v7a-Debug/_deps/opencv-src/modules/core/include/opencv2/core/hal/intrin_neon.hpp:206
    #1  0xa4b53fd8 in cv::hal_baseline::v_load (ptr=0x9fb1e780)
        at /Volumes/Docs/git/cppframe/build-armeabihf-v7a-Debug/_deps/opencv-src/modules/core/include/opencv2/core/hal/intrin_neon.hpp:1277
    #2  0xa4e1bcf0 in cv::HResizeLinearVecU8_X4::operator() (this=0x9fafd688, src=0x9fafd78c, 
        dst=0x9fafd74c, count=2, xofs=0x9fb1db00, alpha=0x9fb1e780, dmax=600, cn=3, xmax=600)
        at /Volumes/Docs/git/cppframe/build-armeabihf-v7a-Debug/_deps/opencv-src/modules/imgproc/src/resize.cpp:1744
    #3  0xa4e4bd78 in cv::HResizeLinear<unsigned char, int, short, 2048, cv::HResizeLinearVecU8_X4>::operator() (this=0x9fafdbf8, src=0x9fafd78c, dst=0x9fafd74c, count=2, xofs=0x9fb1db00, 
        alpha=0x9fb1e780, swidth=3240, dwidth=600, cn=3, xmin=0, xmax=600)
        at /Volumes/Docs/git/cppframe/build-armeabihf-v7a-Debug/_deps/opencv-src/modules/imgproc/src/resize.cpp:1830
    #4  0xa4e4357c in cv::resizeGeneric_Invoker<cv::HResizeLinear<unsigned char, int, short, 2048, cv::HResizeLinearVecU8_X4>, cv::VResizeLinear<unsigned char, int, short, cv::FixedPtCast<int, unsigned char, 22>, cv::VResizeLinearVec_32s8u> >::operator() (this=0x9fafdcfc, range=...)
        at /Volumes/Docs/git/cppframe/build-armeabihf-v7a-Debug/_deps/opencv-src/modules/imgproc/src/resize.cpp:2167
    #5  0xa490fc04 in cv::parallel_for_impl (range=..., body=..., nstripes=0.6103515625)
        at /Volumes/Docs/git/cppframe/build-armeabihf-v7a-Debug/_deps/opencv-src/modules/core/src/parallel.cpp:540
    #6  0xa490f9a8 in cv::parallel_for_ (range=..., body=..., nstripes=0.6103515625)
        at /Volumes/Docs/git/cppframe/build-armeabihf-v7a-Debug/_deps/opencv-src/modules/core/src/parallel.cpp:506
    #7  0xa4e21438 in cv::resizeGeneric_<cv::HResizeLinear<unsigned char, int, short, 2048, cv::HResizeLinearVecU8_X4>, cv::VResizeLinear<unsigned char, int, short, cv::FixedPtCast<int, unsigned char, 22>, cv::VResizeLinearVec_32s8u> > (src=..., dst=..., xofs=0x9fb1db00, _alpha=0x9fb1e780, 
        yofs=0x9fb1e460, _beta=0x9fb1f0e0, xmin=0, xmax=600, ksize=2)
        at /Volumes/Docs/git/cppframe/build-armeabihf-v7a-Debug/_deps/opencv-src/modules/imgproc/src/resize.cpp:2204
    #8  0xa4e20268 in cv::hal::resize (src_type=16, 
    --Type <RET> for more, q to quit, c to continue without paging--c
        src_data=0x9ff25040 "4MQ-FJ)BF*CG0FK1GL3IN5KP6MO7OO2JJ1JF3NJ5QK5RI:VP9TQ0JJ7NP;RT0FK)?D.DJ2HN4GN5HO5HP2EM.AI.AI4GO9NV1HP+EL+CI.FL-EK)AG(@F,DJ.GK/HL/HL-FJ+DH+DH-FJ0IM/IO3MS5MS4LR6KS1FN0CK4GO0CK(;C/BI5HO,@E,@E2HM2IK3GH/DB,A"..., src_step=3240, src_width=1080, src_height=720, dst_data=0x9fb005c0 "", dst_step=600, dst_width=200, dst_height=200, inv_scale_x=0.18518518518518517, inv_scale_y=0.27777777777777779, interpolation=1) at /Volumes/Docs/git/cppframe/build-armeabihf-v7a-Debug/_deps/opencv-src/modules/imgproc/src/resize.cpp:4036
    #9  0xa4e209d8 in cv::resize (_src=..., _dst=..., dsize=..., inv_scale_x=0.18518518518518517, inv_scale_y=0.27777777777777779, interpolation=1) at /Volumes/Docs/git/cppframe/build-armeabihf-v7a-Debug/_deps/opencv-src/modules/imgproc/src/resize.cpp:4089
    #10 0x000126c0 in <lambda()>::operator()(void) const (__closure=0x340d4) at /Volumes/Docs/git/cppframe/frameworks/wuyi_core/main.cpp:26
    #11 0x00014004 in co::xx::Function0<main(int, char**)::<lambda()> >::run(void) (this=0x340d0) at /Volumes/Docs/git/cppframe/build-armeabihf-v7a-Debug/install/wuyi_util/include/co/closure.h:24
    #12 0xa6477230 in co::SchedulerImpl::main_func (from=...) at /Volumes/Docs/git/cppframe/build-armeabihf-v7a-Debug/_deps/libco-src/src/co/scheduler.cc:38
    #13 0xa64b5b44 in tb_context_make () at /Volumes/Docs/git/cppframe/build-armeabihf-v7a-Debug/_deps/libco-src/src/co/context/context_arm.S:131
    
    need more details 
    opened by aijle 13
  • udpsocket  co::close无法让co::recvfrom跳出等待

    udpsocket co::close无法让co::recvfrom跳出等待

    co::close 触发了del_event(ev) 但是co::recvfrom等待的EV_read 消息订阅被删除,但是recvfrom仍然在等待。

    解决方法: 1、或许可以加入EV_close/EV_ignore之类的事件。 2、IoEvent加入多事件绑定,同时绑定EV_read和EV_close/EV_ignore。

    opened by mosxuqian 13
  • 关于Json::Value::operator[]的疑问

    关于Json::Value::operator[]的疑问

    https://github.com/idealvin/co/blob/110eadde2c89dffa259a161c100cd4161253441f/base/json.cc#L15

    正常访问一个objectmember时,应该先调用has_member()来进行判断。但看上面函数的代码,如果直接使用[]来访问相关member,这里的逻辑似乎是打算支持这种行为?

    如果不支持的话,21&22行的意义何在呢? 如果支持的话,接下来访问其它member或者调用str()就会出错了。如以下代码:

    #include "co/json.h"
    
    int main() {
      auto v = json::parse("{\"name\": \"tiger\"}");
      v["n"];
      printf("%s\n", v["name"].str().c_str());
      return 0;
    }
    

    输出结果是null

    opened by tigerlee 12
  • SylixOS嵌入式实时操作系统上移植coost

    SylixOS嵌入式实时操作系统上移植coost

    编译器采用gcc-arm-none-eabi

    在编译context_arm.S时,出现如下问题

    make all arm-sylixos-eabi-gcc -mcpu=cortex-a9 -mno-unaligned-access -O0 -g3 -gdwarf-2 -Wall -fmessage-length=0 -fsigned-char -fno-short-enums -fno-strict-aliasing -x assembler-with-cpp -DSYLIXOS -D__linux__ -D__ARM_ARCH_7A__ -DSYLIXOS_LIB -I"D:\SylixOS_WS\workspace\coost/include" -I"D:/SylixOS_WS/workspace/ArmLite/libsylixos/SylixOS" -I"D:/SylixOS_WS/workspace/ArmLite/libsylixos/SylixOS/include" -I"D:/SylixOS_WS/workspace/ArmLite/libsylixos/SylixOS/include/network" -MMD -MP -MF ./Debug/dep/libcoost.so/src/co/context/context_arm.d -c src/co/context/context_arm.S -o Debug/obj/libcoost.so/src/co/context/context_arm.o src/co/context/context_arm.S: Assembler messages: src/co/context/context_arm.S:102: Error: bad instructionfunction tb_context_make,export=1' src/co/context/context_arm.S:142: Error: bad instruction endfunc' src/co/context/context_arm.S:152: Error: bad instructionfunction tb_context_jump,export=1' src/co/context/context_arm.S:204: Error: bad instruction endfunc' make: *** [Debug/obj/libcoost.so/src/co/context/context_arm.o] Error 1

    无法识别汇编代码中的 function 以及 endfunc

    opened by hyena04 1
  • 有支持格式化字符串吗?

    有支持格式化字符串吗?

    支持格式化字符串,而不再需要用sprintf,例如folly库中的这种用法:

    using folly::format;
    using folly::sformat;
    
    // Objects produced by format() can be streamed without creating
    // an intermediary string; {} yields the next argument using default
    // formatting.
    std::cout << format("The answers are {} and {}", 23, 42);
    // => "The answers are 23 and 42"
    
    // If you just want the string, though, you're covered.
    std::string result = sformat("The answers are {} and {}", 23, 42);
    // => "The answers are 23 and 42"
    
    // To insert a literal '{' or '}', just double it.
    std::cout << format("{} {{}} {{{}}}", 23, 42);
    // => "23 {} {42}"
    
    // Arguments can be referenced out of order, even multiple times
    std::cout << format("The answers are {1}, {0}, and {1} again", 23, 42);
    // => "The answers are 42, 23, and 42 again"
    
    // It's perfectly fine to not reference all arguments
    std::cout << format("The only answer is {1}", 23, 42);
    // => "The only answer is 42"
    
    // Values can be extracted from indexable containers
    // (random-access sequences and integral-keyed maps), and also from
    // string-keyed maps
    std::vector<int> v {23, 42};
    std::map<std::string, std::string> m { {"what", "answer"} };
    std::cout << format("The only {1[what]} is {0[1]}", v, m);
    // => "The only answer is 42"
    
    // format works with pairs and tuples
    std::tuple<int, std::string, int> t {42, "hello", 23};
    std::cout << format("{0} {2} {1}", t);
    // => "42 23 hello"
    
    // Format supports width, alignment, arbitrary fill, and various
    // format specifiers, with meanings similar to printf
    // "X<10": fill with 'X', left-align ('<'), width 10
    std::cout << format("{:X<10} {}", "hello", "world");
    // => "helloXXXXX world"
    
    // Field width may be a runtime value rather than part of the format string
    int x = 6;
    std::cout << format("{:-^*}", x, "hi");
    // => "--hi--"
    
    // Explicit arguments work with dynamic field width, as long as indexes are
    // given for both the value and the field width.
    std::cout << format("{2:+^*0}",
    9, "unused", 456); // => "+++456+++"
    
    // Format supports printf-style format specifiers
    std::cout << format("{0:05d} decimal = {0:04x} hex", 42);
    // => "00042 decimal = 002a hex"
    
    // Formatter objects may be written to a string using folly::to or
    // folly::toAppend (see folly/Conv.h), or by calling their appendTo(),
    // and str() methods
    std::string s = format("The only answer is {}", 42).str();
    std::cout << s;
    // => "The only answer is 42"
    
    // Decimal precision usage
    std::cout << format("Only 2 decimals is {:.2f}", 23.34134534535);
    // => "Only 2 decimals is 23.34"
    
    opened by xingrui94 1
  • add HTTP long-conn-session support

    add HTTP long-conn-session support

    添加对http long-connection session模式的支持。即一直保持http连接,并且response不断追加发送数据的工作模式。 此模式常用于需要保持连接完成大量数据发送(如发送超长文件)或者数据不定时发送(如:视频实时推流,x-mixed-replace,部分web-socket应用场景等)。 主要改动(API接口部分): 1、在Http Response类(Res)中,新增一个send_body方法。此方法可在设置完header(add_header)后使用。用户每次调用send_body将即时发送一批body数据(因此,第一次调用send_body会附加Http response header的发送)。用户可以在保证当前协程函数(on_req)不退出的情况下任意多次调用send_body,直至业务数据发送完毕或者出错(send_body返回非0)。无论出错还是正常完成,退出当前协程函数即完成本http session资源的清理。 2、因为send_body(长连接)模式与set_body(短链接)模式互斥,因此修改set_body函数返回值为bool。用户调用set_body以后将无法调用send_body,同理用户调用send_body以后也将无法调用set_body(返回false)。 实现部分改动: 在Response-ctx类实现中,增加3个成员:

    • tcp::Connection pc_; //存放本http Session socket connection的指针。短连接时用不到,仅在长连接模式下使用
    • header_send; //header是否发送的状态码。初始化为0,表示header未处理。为-1表示header已经pack但未发送(短链接模式),为1表示header已经发送(长连接模式)。
    • s_err:仅用于长连接模式,表示是否有socket_error。
    opened by ltmit 0
Releases(v3.0.0)
  • v3.0.0(Sep 2, 2022)

    New features & improvement

    • Improve cmake scripts and support Conan. Improve export of symbols in shared library. Thanks to SpaceIm.

    • A better JSON library. Support precision control for float point numbers in Json.

      // {"a":23,"b":false,"s":"xx","v":[1,2,3],"o":{"xx":0}}
      Json x = {
          { "a", 23 },
          { "b", false },
          { "s", "xx" },
          { "v", {1,2,3} },
          { "o", {
              {"xx", 0}
          }},
      };
      
      // equal to x
      Json y = Json()
          .add_member("a", 23)
          .add_member("b", false)
          .add_member("s", "xx")
          .add_member("v", Json().push_back(1).push_back(2).push_back(3))
          .add_member("o", Json().add_member("xx", 0));
      
      x.get("a").as_int();       // 23
      x.get("s").as_string();    // "xx"
      x.get("v", 0).as_int();    // 1
      x.get("v", 2).as_int();    // 3
      x.get("o", "xx").as_int(); // 0
      
    • Add TLOG to print logs which are grouped by topics.

      TLOG("rpc") << "hello " << 23;
      TLOG("xxx") << "hello " << 23;
      
    • Improve exception handling in co/log on Windows. Thanks to 909254.

    • Remove path from __FILE__ in co/log.

    • Support alias for flag. Add api flag::alias().

      DEF_bool(debug, false, "");         // no alias
      DEF_bool(debug, false, "", d);      // d is an alias of debug
      DEF_bool(debug, false, "", d, dbg); // 2 aliases
      
    • Ignore non-existing flags in config file.

    • Add --version for co/flag.

    • --help shows only user-defined flags in co/flag by default, and limit flag level to [0-9].

    • Add api flag::set_value() for setting value of a flag.

    • Add log::set_write_cb() to set a callback for writting logs.

    • Add macros __arch64, __arch32, __fname__ and __fnlen__ in co/def.h.

    • Add a fast memory allocator. Provide co::stl_allocator for STL.

    • Provide containers based on co::stl_allocator. See co/stl.h for details.

    • Improve atomic operations, support memory order. Add atomic_cas as an alias of atomic_compare_swap and add atomic_bool_cas.

    • Add nanoid(), md5digest(), sha256() in co/hash.h.

    • Support HTTP protocol for JSON-based RPC framework.

    • Support graceful exit for tcp::Server, http::Server, rpc::Server.

    • Add fs::dir for readding directory.

    • Support + mode for both read and write for fs::file.

    • Support precision control for float point numbers.

      fastream s;
      s << 3.14159; // "3.14159"
      
      s.clear();
      s.maxdp(3) << 3.14159; // "3.141"
      
      s.clear();
      s << co::maxdp(2) << 3.14159; // "3.14"
      
      LOG << co::maxdp(2) << 1.234e-7; // "1.23e-7"
      
    • Improve hook on macos with fishhook. Thanks to shuai132.

    Changed

    • Remove flag a from co/unitest, run all tests by default.

    • Remove log::init(), log::close(), log::set_single_write_cb().

    • Remove co::init(), co::exit().

    • Rename co::all_schedulers() to co::schedulers().

    • Rename co::Table to co::table.

    • Remove atomic_get, use atomic_load instead.

    • Remove atomic_set, use atomic_store instead.

    • Remove atomic_reset, use atomic_store(&x, 0) instead.

    • Remove err::get(), err::set(), use co::error() instead.

    Bugs fixed

    Source code(tar.gz)
    Source code(zip)
  • v2.0.3(Nov 13, 2021)

    New features

    • Add str::cat(...), fastream::cat(...), fastring::cat(...)
    fastring s("hello");
    s.cat(' ', 23, "xx", false);          // s -> "hello 23xxfalse"
    s = str::cat("hello", ' ', 23, true); // s -> "hello 23true"
    
    • Support writing logs to customed destinations by setting a writing callback in co/log.

    • Support daily rotation and log compression for co/log. Thanks to kuyoonjo. Compression is experimental at present.

    • Add method reset() for fastring & fastream.

    • Add method exit() for http::Server & rpc::Server.

    • Add co::maybe.

    • Add os::system().

    • Add some god-oriented programming features in co/god.h.

    Changed

    • Improve exit() method for tcp::Server.

    • Memory optimization for coroutines.

    • Improve operator<< for fastream & fastring, optimization for string literal.

    Bugs fixed

    Source code(tar.gz)
    Source code(zip)
  • v2.0.2(Oct 3, 2021)

    New features

    • Support mingw on windows.
    xmake f -p mingw
    xmake -v
    
    • Support shared library on windows.
    xmake f -k shared
    xmake -v
    
    • Support syslog header for co/log.
    xmake b log
    xmake r log -syslog -cout
    
    • Add god mode. When you feel uneasy about writing a function of several hundred lines, you may call the following api to increase your confidence:
    #include "co/god.h"
    
    god::bless_no_bugs();
    

    Changed

    • Add co/cout.h, make COUT and CLOG thread-safe.

    • Add co::thread_id(), which may replace current_thread_id() in the future.

    • co/log: add time in name of old log files.

    • co/log: improve stack trace, use libbacktrace instead of fork()+backtrace().

    • Add os::env(name, value) for setting value of environment variables.

    • Always use / as the path separator, and convert \ to / in results of os::cwd(), os::homedir(), os::exepath().

    • Improve the path library, support windows path like d:/xx.

    • Improve stability of coroutine hook.

    • tcp::Server::on_connection(tcp::Connection*) to tcp::Server::on_connection(tcp::Connection), no need to delete the connection object any more.

    • Refactor http::Client, use curl easy handle only.

    • Refactor http::Server, reduce memory copy to improve the performance.

    Bugs fixed

    • Fix RemoveVectoredExceptionHandler bug.

    • Fix crash of http::Client in #168.

    • Fix macro comment directive bug.

    • Fix bug in select hook in #176.

    • Fix a errno bug in co/flag, set errno to 0 before set_value.

    • Fix errno bug in str::to_xxx().

    • Fix hang-at-exit bug when using dll on windows.

    • Fix iterator bug in std::multimap on windows.

    • Fix: failed to hook fcntl with a different _FILE_OFFSET_BITS.

    Source code(tar.gz)
    Source code(zip)
  • v2.0.1(Aug 12, 2021)

    Documents

    New features

    • xrepo
    xrepo install -f "openssl=true,libcurl=true" co
    
    • vcpkg
    vcpkg install co:x64-windows
    
    # http & ssl support
    vcpkg install co[libcurl,openssl]:x64-windows
    
    • defer (similar to defer in golang)
    #include "co/defer.h"
    Timer t;
    defer(LOG << "time elapse: " << t.us() << "us");
    
    • channel for coroutine (similar to channel in golang)
    #include "co/co.h"
    
    DEF_main(argc, argv) {
        co::Chan<int> ch;
        go([ch]() {
            ch << 7;
        });
    
        int v = 0;
        ch >> v;
        LOG << "v: "<< v;
    
        return 0;
    }
    
    • waitgroup (similar to sync.WaitGroup in golang)
    #include "co/co.h"
    
    DEF_main(argc, argv) {
        FLG_cout = true;
    
        co::WaitGroup wg;
        wg.add(8);
    
        for (int i = 0; i <8; ++i) {
            go([wg]() {
                LOG << "co: "<< co::coroutine_id();
                wg.done();
            });
        }
    
        wg.wait();
        return 0;
    }
    
    • Coroutine hook for windows.

    • Create coroutines in specified scheduler(thread).

    auto s = co::next_scheduler();
    s->go(f1);
    s->go(f2);
    
    • Create coroutine in all schedulers.
    auto& s = co::all_schedulers();
    for (size_t i = 0; i <s.size(); ++i) {
        s[i]->go(f);
    }
    
    • Add void flag::init(const fastring& path);

    Changed

    • Closure to co::Closure.

    • Improve co::Event, can be used anywhere, and support copy constructor and capture by value in lambda.

    • Improve co::Mutex, co::Pool, support copy constructor and capture by value in lambda.

    • co::close() now can be called anywhere, not necessary to call it in the thread that performed the IO operations.

    • Partial support for mingw. Coroutine and coroutine-based features do not work for mingw at present.

    Bugs fixed

    • fix bug in fs::file when read/write more than 4G bytes.

    • fix connect timeout error for http::Client.

    • fix link problem in #165.

    Source code(tar.gz)
    Source code(zip)
  • v2.0.0(May 28, 2021)

    Detailed reference documents


    First of all, it is particularly emphasized that more detailed documents are provided this time:


    New features

    SSL


    Co 2.0 finally supports SSL. Users need to install openssl 1.1 or above. It has been tested on openssl and other SSL libraries have not been tested yet.

    It is recommended to use xmake as the build tool, it will prompt the user whether to install openssl, libcurl, zlib and other third-party libraries. To enable the SSL feature, you need to predefine the CO_SSL macro, which is automatically defined by xmake when openssl is detected.

    co/so/ssl.h provides a coroutine-based openssl interface, but users may not use it directly, because co has embedded the SSL feature into the TCP module, and users can use tcp::Server and tcp::Client instead.


    Important improvements

    Coroutine

    • go
      go() supports any function or class method with 0 or 1 parameter, as well as function objects or pointers of type std::function<void()>.

      void f();
      void g(int);
      void h(int, int);
      struct T {
      	void f();
      	void g(int);
      };
      
      T o;
      std::function<void()> k(std::bind(h, 3, 7));
      
      go(f);
      go(g, 7);
      go(&T::f, &o);
      go(&T::g, &o, 3);
      go(k);
      go(&k); // The user must ensure that k is alive when the coroutine is running.
      
    • Coroutine API

      • Adding the co::timeout() function, users can use it to determine whether the last I/O function like co::recv or co::send has timed out.
      • The co::coroutine_id() function returns a globally unique id. In version 1.x, coroutines in different scheduling threads may have the same id.
    • co::Event
      The signaled state is added internally to solve the problem that the synchronization signal will be lost when there is no waiting coroutines.

    • co::IoEvent
      In the 1.x version, it is only used internally, and this class is public in 2.0, so that users can coroutineize third-party network libraries by themselves.

      int recv(SSL* s, void* buf, int n, int ms) {
          CHECK(co::scheduler()) << "must be called in coroutine..";
          int r, e;
          int fd = SSL_get_fd(s);
          if (fd <0) return -1;
      
          do {
              ERR_clear_error();
              r = SSL_read(s, buf, n);
              if (r> 0) return r; // success
              if (r == 0) {
                  DLOG << "SSL_read return 0, error: "<< SSL_get_error(s, 0);
                  return 0;
              }
      
              e = SSL_get_error(s, r);
              if (e == SSL_ERROR_WANT_READ) {
                  co::IoEvent ev(fd, co::EV_read);
                  if (!ev.wait(ms)) return -1;
              } else if (e == SSL_ERROR_WANT_WRITE) {
                  co::IoEvent ev(fd, co::EV_write);
                  if (!ev.wait(ms)) return -1;
              } else {
                  DLOG << "SSL_read return "<< r << ", error:" << e;
                  return r;
              }
          } while (true);
      }
      


      The above is an example of coroutineizing the SSL_read() function in openssl. You only need to use a non-blocking socket. When openssl generates an SSL_ERROR_WANT_READ or SSL_ERROR_WANT_WRITE error, call the wait() method of co::IoEvent and wait for the corresponding I/O Event.


    TCP

    • Added the tcp::Connection class for the implementing TCP server. This class provides recv(), send() and other methods. Users can directly use this class to receive and send data without worrying about whether the underlying SSL is enabled.

    • tcp::Server

      void on_connection(tcp::Connection* conn);
      
      tcp::Server s;
      s.on_connection(on_connection);
      s.start("0.0.0.0", 7788); // no ssl
      s.start("0.0.0.0", 7788, "privkey.pem", "certificate.pem"); // use ssl
      

      Users can specify the SSL private key and certificate file in the start() method to enable SSL.


    • tcp::Client

      bool use_ssl = false;
      tcp::Client c("127.0.0.1", 7788, use_ssl);
      c.connect(1000);
      c.send(...);
      


      tcp::Client can enable SSL through the third parameter of the constructor.


    HTTP

    • http::Client
      In co 2.0, http::Client is implemented based on libcurl. To enable this feature, you must install libcurl and predefine the HAS_LIBCURL macro. Again, it is recommended to use xmake to build, it will automatically handle these third-party dependencies.

      http::Client c("https://github.com");
      http::Client c("http://127.0.0.1:7777");
      c.get("/");
      c.get("/idealvin/co");
      LOG << c.response_code();
      
    • http::Server

      http::Server s
      s.on_req(...);
      s.start("0.0.0.0", 7777); // http
      s.start("0.0.0.0", 7777, "privkey.pem", "certificate.pem"); // https
      

    RPC


    In co 2.0, some new features have been added to the RPC framework:

    • Support SSL.
    • Support username and password authentication, and multiple usernames and passwords can be set for rpc::Server.
    • Multiple services can be added into rpc::Server.
    • The code generator can also generate code for RPC client.

    JSON


    In the 1.x version, the JSON library uses only one json::Value class to represent a JSON. The elements in the JSON object are also json::Value. When constructing a JSON object, you need to allocate memory for each element. When the JSON object is destructed, All internal elements must call the destructor of json::Value. Implementation based on this method will cause frequent memory allocation and release, which greatly affects program performance.

    In co 2.0, the JSON object is built into a continuous memory. After the program runs stably, the parsing of JSON requires almost no memory allocation, which greatly improves the parsing speed.

    In addition, co 2.0 uses the Json class to represent a JSON, and the json::Value class to represent the elements in the JSON. json::Value is just a trivial class, only including a index position in the JSON memory block. When a JSON is destructed, the destructor of the Json class is called only once, and the destructor of json::Value will never be called.


    Others

    • Fix the nested log bug in co/log.
    • Fix some bugs caused by dependence of global static variables.
    • Add milliseconds in log time of co/log.
    • The TaskSched class is renamed to Tasked.
    • co/time.h. Add epoch::ms() and epoch::us() to obtain the time since the EPOCH.
    • co/os.h Add os::signal() method to set the signal handler function.
    • Added safe_clear() method in fastring.
    • Added safe_clear() method in Json.
    Source code(tar.gz)
    Source code(zip)
  • v2.0.0-preview(May 23, 2021)

  • v1.2.0(Jul 16, 2020)

    English

    Changes

    Bug fix

    • fix #77
    • fix #83
    • Fix bug due to Coroutine in Copool not cleared inside the coroutine library.
    • Fix assert bug in disconnect() of tcp::Client.

    coroutine

    • Refactor coroutine library to simplify internal logic.
    • Add unit test unitest/co for testing internal logic of coroutine schedulers.

    support http

    • Implement the http::Server class.
    • Implement the http::Client class.
    • Implement the so::easy(...) interface for quickly creating a static web server.

    hash

    • Add size_t murmur_hash(...) interface.

    fastring

    • Support std::hash<fastring>, now std::unordered_map can take fastring as its key.

      std::unordered_map<fastring, fastring> um;
      LruMap<fastring, fastring> lm;
      
    • Added lshift() interface for left shifting the string.

      fastring s = "hello";
      s.lshift(2); // s -> "llo";
      s.lshift(8); // s -> "";
      
    • Added shrink() interface, similar to shrink_to_fit() of std::string, for reducing the capacity of fastring.

      fastring s(4096); // cap -> 4096
      s.shrink();       // cap <4096
      

    test/unitest

    • Remove the _test suffix in the file name of the test/unitest code.

    中文

    Changes

    Bug 修复

    • fix #77
    • fix #83
    • 修复协程库内部 Copool 未清空 Coroutine 中旧数据引起的 bug.
    • 修复 tcp::Client::disconnect() 中的 assert bug.

    coroutine

    • 重构协程库,简化内部逻辑
    • 增加单元测试 unitest/co,用于测试 Scheduler 内部逻辑.

    新增 http 模块

    • 实现 http::Server 类.
    • 实现 http::Client 类.
    • 实现 so::easy(...) 接口,用于快速创建静态 web server.

    hash

    • 新增 size_t murmur_hash(...) 接口.

    fastring

    • 支持 std::hash<fastring>std::unordered_map 可以使用 fastring 作为 key.

      std::unordered_map<fastring, fastring> um;
      LruMap<fastring, fastring> lm;
      
    • 新增 lshift() 接口,将字符串左移若干字节.

      fastring s = "hello";
      s.lshift(2);  // s -> "llo";
      s.lshift(8);  // s -> "";
      
    • 新增 shrink() 接口,与 std::stringshrink_to_fit() 类似,用于缩减 fastring 的容量.

      fastring s(4096); // cap -> 4096
      s.shrink();       // cap < 4096
      

    test/unitest

    • 去掉 test/unitest 代码文件名中的 _test 后缀.
    Source code(tar.gz)
    Source code(zip)
  • v1.2.3(Oct 10, 2020)

  • v1.2.2(Jul 30, 2020)

  • v1.2.1(Jul 19, 2020)

  • v1.1(Jul 2, 2020)

    English

    github

    Changes

    Code structure adjustment

    • Put the header file in the include directory.
    • The source files are placed in the src directory.
    • rpcgen was renamed to gen.
    • Remove the unitest/base directory and put the unit test code directly under the unitest directory.
    • Support subdirectories under test directory.

    fast

    • The definition of static variables in fast.cc is put into functions, and initialization is safer.
    • fast::dtoa deprecated the implementation based on LruMap and replaced with the implementation of Milo Yip.
    • Added fast::stream class to provide basic streaming output operations.

    fastream

    • Inherited from fast::stream class.
    • Support move constructor.
    • Added empty status.
      fastream fs; // Define an empty fastream object without allocating memory
      
    • Support append itself.
      fastream fs;
      fs << "hello "<< 23;
      fs.append(fs);
      fs << fs; // <==> fs.append(fs)
      

    fastring

    • Like fastream, it inherits from fast::stream class, so it also supports streaming output operations.
      fastring s;
      s << "hello "<< 23;
      
    • The memory structure of fastring and fastream is the same, the two can be converted seamlessly.
      fastring s;
      fastream& fs = *(fastream*)&s
      
    • Removed reference counting to make fastring's copying behavior similar to std::string, which is not easy to make mistakes.
      fastring s("hello");
      fastring t(s); // Create a new string through memory copy
      
    • append operation adds inside check to fix logic bug on memory overlaps.
      fastring s("123");
      s.append(s.c_str() + 1); // s -> "12323"
      
    • Remove the clone(), it is no more needed as the reference count was removed.

    str

    • str::dbg() supports std::unordered_map and std::unordered_set.

    flag

    • Optimize the parsing order of command line parameters and configuration files, first parse the configuration file, and then parse other command line parameters.
      # First parse the configuration file xx.conf, then parse other command line parameters
      # The values ​​of x and s in the command line will override the values ​​in xx.conf for easy debugging
      ./xx -x -s="hello" -config=xx.conf
      
    • Added built-in bool flag daemon on Linux platform to support background running programs
      # add -daemon in the command line parameters
      # or set in the configuration file: daemon = true
      ./xx -daemon
      
    • Command line parameters support multiple styles, -x=y can be written as -x y or x=y
      ./xx -i=8 u=88 -s="hello world"
      ./xx -i 8 -u 88 -s "hello world"
      
    • Optimize the way to specify the configuration file when the program starts.
      ./xx config=xx.conf  # Use flag config to display the specified
      ./xx xx.conf         # The configuration file name ends with .conf or config 
                           # and is the first non-flag parameter, then config= can be omitted
      ./xx -x xx.conf      # -x is the flag, xx.conf is the first non-flag parameter
      
    • When defining the flag, you can specify the level in the comment to control the order of the flag in the configuration file.
      // Use #3 at the beginning of the comment to specify level 3
      // The supported level is 0-99, the default is 10
      // When using --mkconf to automatically generate a configuration file, the flags are sorted by level, file name, and number of lines of code
      DEF_bool(b, false, "#3 xxx");
      

    log

    • Some functions in the signal handler are changed to async-signal-safe version functions, which is safer.

    coroutine

    • Fixed bugs caused by io events registered in epoll(kevent, iocp) that were not removed in time.
    • Fix the bug that the internal iterator is not updated correctly when Scheduler::add_timer() is called.
    • Improve the implementation of co::connect, co::accept, etc. to support ipv6.
    • Added co::max_sched_num() interface to get the maximum number of scheduling threads supported, which is currently the number of CPU cores in the system.
    • Added co::sched_id() interface to get current scheduling thread id.
    • Added coroutine_id() interface to get the id of current coroutine.
    • Refactored Scheduler, the internal logic structure is clearer, and the code is more readable.
    • Modify the parameters of the co::tcp_socket(), co::udp_socket() interface to address family, deprecating the earlier way of specifying ipv4 and ipv6 with 4 and 6.
      sock_t tcp_socket(int af=AF_INET); // @af: address family, AF_INET, AF_INET6, etc.
      sock_t udp_socket(int af=AF_INET); // @af: address family, AF_INET, AF_INET6, etc.
      
    • Added co::socket() interface, which is consistent with the native API.
    • Fixed the initialization problem of some static global variables in the hook implementation.
    • Optimized the internal implementation of co::Event.
    • Refactored co::Pool:
      • Users can specify callback ccb and dcb, which are used to create and destroy an element respectively.
      • Users can specify the maximum capacity of the pool (only valid when dcb is set).
      • Register cleanup callback with Scheduler in the internal implementation to ensure the cleanup of co::Pool at the end of the scheduling thread.
    • co::Kakalot was renamed to co::PoolGuard.

    json

    • Internal reference counting, using atomic operations, copying Json objects is safe in multi-thread environment.
    • Reconstruct the internal memory model of Json, and fix the bugs caused by the internal memory changes.
    • A simple memory allocator Jalloc is added to improve the performance of Json.
    • json::parse() supports parsing of array objects.
      Json v = json::parse("[1, 2, 3]");
      
    • Added Json::dbg() interface to convert Json object to debug string (longer strings in Json objects may be truncated).
    • The log library calls Json::dbg() to output Json objects, making the output log more streamlined.

    rpc

    • Simplify, remove some unnecessary configuration items.
    • Optimize connection management, you can specify the timeout period of idle connections and the maximum number of idle connections through rpc_conn_idle_sec and rpc_max_idle_conn.

    hash

    • Modify the implementation of hash32(), the 32-bit system uses the 32-bit version of murmur 2, the 64-bit system directly takes the lower 32 bits of hash64.

    Compile

    • Removed scons compilation script.
    • Support xmake compilation.
    • Support cmake compilation (contributed by izhengfan).
    • Windows supports compilation with VS project files (automatically generated by xmake).

    中文

    github

    Changes

    代码结构调整

    • 头文件放到 include 目录.
    • 源文件放到 src 目录.
    • rpcgen 更名为 gen.
    • 移除 unitest/base 目录,单元测试代码直接放到 unitest 目录下.
    • test 目录下支持子目录.

    fast

    • fast.cc 中静态变量的定义放到函数中,初始化更安全.
    • fast::dtoa 弃用基于 LruMap 的实现,换用 Milo Yip 的实现(miloyip/dtoa-benchmark).
    • 新增 fast::stream 类,提供基本的流式输出操作.

    fastream

    • 继承于 fast::stream 类.
    • 支持 move 构造函数.
    • 增加空状态.
      fastream fs; // 定义一个空的 fastream 对象,不分配内存
      
    • 支持 append 自己.
      fastream fs;
      fs << "hello " << 23;
      fs.append(fs);
      fs << fs;     // <==> fs.append(fs)
      

    fastring

    • fastream 一样,继承于 fast::stream 类,因此也支持流式输出操作.
      fastring s;
      s << "hello " << 23;
      
    • fastring 与 fastream 的内存结构相同,二者可以无缝转换.
      fastring s;
      fastream& fs = *(fastream*)&s
      
    • 移除引用计数,使 fastring 的复制行为与 std::string 类似,使用起来不容易出错.
      fastring s("hello");
      fastring t(s);  // 通过内存拷贝创建一个新的字符串
      
    • append 操作增加 inside 检查,修复内存重叠时的逻辑漏洞.
      fastring s("123");
      s.append(s.c_str() + 1); // s -> "12323"
      
    • 删除 clone() 方法,移除引用计数后,此方法多余.

    str

    • str::dbg() 支持 std::unordered_mapstd::unordered_set.

    flag

    • 优化命令行参数与配置文件的解析顺序,先解析配置文件,再解析其他命令行参数.
      # 先解析配置文件 xx.conf,再解析其他命令行参数
      # 命令行中 x, s 的值会覆盖 xx.conf 中的值,方便调试
      ./xx -x -s="hello" -config=xx.conf
      
    • Linux 平台增加内置 bool flag daemon,以支持后台运行程序
      # 可在命令行参数中带上 -daemon
      # 也可在配置文件中设置: daemon = true
      ./xx -daemon
      
    • 命令行参数支持多种格式,-x=y 可以写成 -x y 或者 x=y
      ./xx -i=8 u=88 -s="hello world"
      ./xx -i 8 -u 88 -s "hello world"
      
    • 优化程序启动时指定配置文件的方式.
      ./xx config=xx.conf  # 用 flag config 显示指定
      ./xx xx.conf         # 配置文件名以 .conf 或 config 结尾,且是程序命令行的第一个非 flag 参数,则可省略 config=
      ./xx -x xx.conf      # -x 是 flag,xx.conf 是第一个非 flag 参数
      
    • 定义 flag 时,可以在注释中指定级别,以控制 flag 在配置文件中的顺序.
      // 在注释开头用 #3 指定级别为 3
      // 支持的级别为 0-99,默认为 10
      // 使用 --mkconf 自动生成配置文件时,flag 按级别、文件名、代码行数排序
      DEF_bool(b, false, "#3 xxx");
      

    log

    • signal handler 中部分函数修改为 async-signal-safe 版本的函数,更安全.

    协程库

    • 修复未及时移除 epoll(kevent, iocp) 中注册的 io 事件引起的 bug.
    • 修复 Scheduler 在 add_timer() 时,内部 iterator 未正确更新的 bug.
    • 改进 co::connect, co::accept 等的实现,以支持 ipv6.
    • 新增 co::max_sched_num() 接口,获取支持的最大调度线程数,目前为系统 cpu 核数.
    • 新增 co::sched_id() 接口,获取当前的调度线程 id.
    • 新增 coroutine_id() 接口,获取当前协程的 id.
    • 重构 Scheduler,内部逻辑结构更清晰,同时提高代码的可读性.
    • 修改 co::tcp_socket(), co::udp_socket() 接口的参数为 address family,弃用早期用 4 与 6 指定 ipv4 与 ipv6 的方式.
      sock_t tcp_socket(int af=AF_INET); // @af: address family, AF_INET, AF_INET6, etc.
      sock_t udp_socket(int af=AF_INET); // @af: address family, AF_INET, AF_INET6, etc.
      
    • 新增 co::socket() 接口,与原生 api 保持一致.
    • 修复 hook 实现中一些静态全局变量的初始化问题.
    • 优化 co::Event 的内部实现.
    • 重构 co::Pool:
      • 用户可以指定 callback ccbdcb,分别用于创建、销毁元素.
      • 用户可以指定 pool 的最大容量(仅在 dcb 设置时有效).
      • 内部实现中向 Scheduler 注册 cleanup callback,保证在调度线程结束时进行 co::Pool 的清理工作.
    • co::Kakalot 重命名为 co::PoolGuard.

    json

    • 内部引用计数,使用原子操作,复制 Json 对象更安全.
    • 重构 Json 内部的内存模型,修复之前因内部内存变化引起的 bug.
    • 内部增加简单的内存分配器 Jalloc,提升 Json 的性能.
    • json::parse() 支持数组对象的解析.
      Json v = json::parse("[1, 2, 3]");
      
    • 新增 Json::dbg() 接口,将 Json 对象转换为 debug string (Json 对象中较长的字符串可能被截断).
    • log 库调用 Json::dbg() 输出 Json 对象,使得输出的日志更精简.

    rpc

    • 简化,移除一些不必要的配置项.
    • 优化连接管理,可以通过 rpc_conn_idle_secrpc_max_idle_conn 指定空闲连接的超时时间、最大的空闲连接数.

    hash

    • 修改 hash32() 的实现,32 位系统使用 murmur 2 的 32 位版本,64 位系统直接取 hash64 的低 32 位.

    编译

    • 移除 scons 编译脚本.
    • 支持 xmake 编译.
    • 支持 cmake 编译 (由 izhengfan 贡献).
    • windows 支持用 vs 工程文件编译 (由 xmake 自动生成).
    Source code(tar.gz)
    Source code(zip)
Owner
Alvin
Keep simple.
Alvin
Coroutine - C++11 single .h asymmetric coroutine implementation via ucontext / fiber

C++11 single .h asymmetric coroutine implementation API in namespace coroutine: routine_t create(std::function<void()> f); void destroy(routine_t id);

null 386 Nov 16, 2022
Cpp-concurrency - cpp implementation of golang style concurrency

cpp-concurrency C++ implementation of golang style concurrency Usage Use existing single header concurrency.hpp or run script to merge multiple header

YoungJoong Kim 14 Aug 11, 2022
:copyright: Concurrent Programming Library (Coroutine) for C11

libconcurrent tiny asymmetric-coroutine library. Description asymmetric-coroutine bidirectional communication by yield_value/resume_value native conte

sharow 350 Sep 2, 2022
Single header asymmetric stackful cross-platform coroutine library in pure C.

minicoro Minicoro is single-file library for using asymmetric coroutines in C. The API is inspired by Lua coroutines but with C use in mind. The proje

Eduardo Bart 394 Nov 30, 2022
A C++20 coroutine library based off asyncio

kuro A C++20 coroutine library, somewhat modelled on Python's asyncio Requirements Kuro requires a C++20 compliant compiler and a Linux OS. Tested on

null 19 Nov 9, 2022
C++20 Coroutine-Based Synchronous Parser Combinator Library

This library contains a monadic parser type and associated combinators that can be composed to create parsers using C++20 Coroutines.

null 47 Oct 13, 2022
Cppcoro - A library of C++ coroutine abstractions for the coroutines TS

CppCoro - A coroutine library for C++ The 'cppcoro' library provides a large set of general-purpose primitives for making use of the coroutines TS pro

Lewis Baker 2.6k Nov 29, 2022
C++14 coroutine-based task library for games

SquidTasks Squid::Tasks is a header-only C++14 coroutine-based task library for games. Full project and source code available at https://github.com/we

Tim Ambrogi Saxon 72 Nov 17, 2022
Powerful multi-threaded coroutine dispatcher and parallel execution engine

Quantum Library : A scalable C++ coroutine framework Quantum is a full-featured and powerful C++ framework build on top of the Boost coroutine library

Bloomberg 480 Nov 28, 2022
Async GRPC with C++20 coroutine support

agrpc Build an elegant GRPC async interface with C++20 coroutine and libunifex (target for C++23 executor). Get started mkdir build && cd build conan

Yuchao Zhang 63 Nov 15, 2022
Mx - C++ coroutine await, yield, channels, i/o events (single header + link to boost)

mx C++11 coroutine await, yield, channels, i/o events (single header + link to boost). This was originally part of my c++ util library kit, but I'm se

Grady O'Connell 4 Sep 21, 2019
Elle - The Elle coroutine-based asynchronous C++ development framework.

Elle, the coroutine-based asynchronous C++ development framework Elle is a collection of libraries, written in modern C++ (C++14). It contains a rich

Infinit 466 Nov 19, 2022
Go-style concurrency in C

LIBMILL Libmill is a library that introduces Go-style concurrency to C. Documentation For the documentation check the project website: http://libmill.

Martin Sustrik 2.6k Nov 23, 2022
Libgo - Go-style concurrency in C++11

libgo libgo -- a coroutine library and a parallel Programming Library Libgo is a stackful coroutine library for collaborative scheduling written in C+

null 2.8k Nov 28, 2022
Bolt is a C++ template library optimized for GPUs. Bolt provides high-performance library implementations for common algorithms such as scan, reduce, transform, and sort.

Bolt is a C++ template library optimized for heterogeneous computing. Bolt is designed to provide high-performance library implementations for common

null 358 Nov 14, 2022
oneAPI DPC++ Library (oneDPL) https://software.intel.com/content/www/us/en/develop/tools/oneapi/components/dpc-library.html

oneAPI DPC++ Library (oneDPL) The oneAPI DPC++ Library (oneDPL) aims to work with the oneAPI DPC++ Compiler to provide high-productivity APIs to devel

oneAPI-SRC 641 Nov 16, 2022
Simple and fast C library implementing a thread-safe API to manage hash-tables, linked lists, lock-free ring buffers and queues

libhl C library implementing a set of APIs to efficiently manage some basic data structures such as : hashtables, linked lists, queues, trees, ringbuf

Andrea Guzzo 391 Nov 19, 2022
OOX: Out-of-Order Executor library. Yet another approach to efficient and scalable tasking API and task scheduling.

OOX Out-of-Order Executor library. Yet another approach to efficient and scalable tasking API and task scheduling. Try it Requirements: Install cmake,

Intel Corporation 18 Oct 25, 2022