一个基于C++11的轻量级网络框架,基于线程池技术可以实现大并发网络IO

Overview

一个基于C++11简单易用的轻量级网络编程框架

Build Status

项目特点

  • 基于C++11开发,避免使用裸指针,代码稳定可靠;同时跨平台移植简单方便,代码清晰简洁。
  • 使用epoll+线程池+异步网络IO模式开发,并发性能优越。
  • 代码经过大量的稳定性、性能测试,可满足商用服务器项目。
  • 支持linux、macos、ios、android、windows平台
  • 了解更多:ZLMediaKit

特性

  • 网络库
    • tcp/udp客户端,接口简单易用并且是线程安全的,用户不必关心具体的socket api操作。
    • tcp服务器,使用非常简单,只要实现具体的tcp会话(TcpSession类)逻辑,使用模板的方式可以快速的构建高性能的服务器。
    • 对套接字多种操作的封装。
  • 线程库
    • 使用线程实现的简单易用的定时器。
    • 信号量。
    • 线程组。
    • 简单易用的线程池,可以异步或同步执行任务,支持functional 和 lambad表达式。
  • 工具库
    • 文件操作。
    • std::cout风格的日志库,支持颜色高亮、代码定位、异步打印。
    • INI配置文件的读写。
    • 监听者模式的消息广播器。
    • 基于智能指针的循环池,不需要显式手动释放。
    • 环形缓冲,支持主动读取和读取事件两种模式。
    • mysql链接池,使用占位符(?)方式生成sql语句,支持同步异步操作。
    • 简单易用的ssl加解密黑盒,支持多线程。
    • 其他一些有用的工具。
    • 命令行解析工具,可以很便捷的实现可配置应用程序

编译(Linux)

  • 我的编译环境

    • Ubuntu16.04 64 bit + gcc5.4(最低gcc4.7)
    • cmake 3.5.1
  • 编译

    cd ZLToolKit
    ./build_for_linux.sh
    

编译(macOS)

  • 我的编译环境

    • macOS Sierra(10.12.1) + xcode8.3.1
    • Homebrew 1.1.3
    • cmake 3.8.0
  • 编译

    cd ZLToolKit
    ./build_for_mac.sh
    

编译(iOS)

  • 编译环境:请参考macOS的编译指导。

  • 编译

    cd ZLToolKit
    ./build_for_ios.sh
    
  • 你也可以生成Xcode工程再编译:

    cd ZLToolKit
    mkdir -p build
    cd build
    # 生成Xcode工程,工程文件在build目录下
    cmake .. -DCMAKE_TOOLCHAIN_FILE=../cmake/iOS.cmake -DIOS_PLATFORM=SIMULATOR64 -G "Xcode"
    

编译(Android)

  • 我的编译环境

    • macOS Sierra(10.12.1) + xcode8.3.1
    • Homebrew 1.1.3
    • cmake 3.8.0
    • android-ndk-r14b
  • 编译

    cd ZLToolKit
    export ANDROID_NDK_ROOT=/path/to/ndk
    ./build_for_android.sh
    

编译(Windows)

   1 使用cmake-gui打开工程并生成vs工程文件.
   2 找到工程文件(ZLToolKit.sln),双击用vs2017打开.
   3 选择编译Release 版本.
   4 依次编译 ZLToolKit_static、ZLToolKit_shared、ALL_BUILD、INSTALL.
   5 找到目标文件并运行测试用例.
   6 找到安装的头文件及库文件(在源码所在分区根目录).

授权协议

本项目自有代码使用宽松的MIT协议,在保留版权信息的情况下可以自由应用于各自商用、非商业的项目。 但是本项目也零碎的使用了一些其他的开源代码,在商用的情况下请自行替代或剔除; 由于使用本项目而产生的商业纠纷或侵权行为一概与本项项目及开发者无关,请自行承担法律风险。

QA

  • 该库性能怎么样?

基于ZLToolKit,我实现了一个流媒体服务器ZLMediaKit;作者已经对其进行了性能测试,可以查看benchmark.md了解详情。

  • 该库稳定性怎么样?

该库经过作者严格的valgrind测试,长时间大负荷的测试;作者也使用该库进行了多个线上项目的开发。实践证明该库稳定性很好;可以无看门狗脚本的方式连续运行几个月。

  • 在windows下编译很多错误?

由于本项目主体代码在macOS/linux下开发,部分源码采用的是无bom头的UTF-8编码;由于windows对于utf-8支持不甚友好,所以如果发现编译错误请先尝试添加bom头再编译。

联系方式

  • 邮箱:[email protected](本项目相关或网络编程相关问题请走issue流程,否则恕不邮件答复)
  • QQ群:542509000
Issues
  • 这年头怎么是个人就敢写自旋锁了?

    这年头怎么是个人就敢写自旋锁了?

    https://github.com/xiongziliang/ZLToolKit/blob/7353a131b46ab6af46d9de23078076d1932f33fb/src/Thread/spin_mutex.h#L44

    且不说用户态根本无法屏蔽 context switch, 就你这渣渣实现,和 其它实现 比较过吗? 有做过 性能测试 吗?

    opened by lhmouse 14
  • 关于ThreadLoadCounter统计问题

    关于ThreadLoadCounter统计问题

    ThreadPool(int num = 1, Priority priority = PRIORITY_HIGHEST, bool autoRun = true) ;

    ThreadPool允许创建多条线程. 在ThreadPool::run()中调用startSleep()与sleepWakeUp()进行load统计,当线程数大于1时,load统计是否不准确?

    opened by lam2003 11
  • fixed win32 std namespace bug

    fixed win32 std namespace bug

    用vs2017编译最新的toolkit报这错: 1>e:\zlmediakit\3rdpart\zltoolkit\src\util\List.h(188): error C2039: “list”: 不是“toolkit::std”的成员 1>C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\include\cctype(33): note: 参见“toolkit::std”的声明 1>E:\ZLMediaKit\3rdpart\ZLToolKit\src\Util\logger.cpp(214): note: 参见对正在编译的函数 模板 实例化“toolkit::List<std::pair<toolkit::LogContextPtr,toolkit::Logger *>>::List<>(void)”的引用 1>E:\ZLMediaKit\3rdpart\ZLToolKit\src\Util\logger.cpp(214): note: 参见对正在编译的函数 模板 实例化“toolkit::List<std::pair<toolkit::LogContextPtr,toolkit::Logger *>>::List<>(void)”的引用 1>e:\zlmediakit\3rdpart\zltoolkit\src\util\List.h(188): error C2059: 语法错误:“<” 1>e:\zlmediakit\3rdpart\zltoolkit\src\util\NoticeCenter.h(102): error C2039: “forward”: 不是“toolkit::std”的成员

    经检查代码应该是这样导致的,做个小修复,应尽力避免在名字空间里面包含头文件 namespace toolkit{ #include std_namespace header }

    opened by mtdxc 9
  • 持续读会撑爆内存

    持续读会撑爆内存

    当一个连接来的速度很快,多到超过实际处理速度。比如一个server接受上传文件。客户端以1G的速度上传文件,但是该server只能以每秒100M的速度写文件。这种情况下,server会持续的通过epoll读,最终撑爆内存。 当server发现自己缓存的内存已经超过一定阈值,它不能持续read直到EAGAIN。如果是EDGE触发,它得在自己消费完了缓存的内存之后再read,如果是LEVEL触发,它得调用epoll_ctl删除这个fd,并且在自己消费完了缓存的内存之后再加入这个fd。 当server不再read时,tcp协议栈会降低tcp wnd,从而抑制发送方的发送速度。

    你这个库里似乎并没有这个逻辑。

    opened by huyuguang 8
  • 请教TCP拆包和粘包的处理解决办法

    请教TCP拆包和粘包的处理解决办法

    我仔细看了您流媒体服务器中HTTP中关于TCP合包的代码,但是无论才疏学浅,没有很明白,希望作者可以大概讲解下TCP处理拆包粘包的思路,现在基于ZLToolKit要实现一个大文件收发,出现拆包粘包问题. 另外我现在需要实现http上传文件,您的http里面有实现好的吗?或者告诉我一个解决思路,非常感谢

    opened by lixinhan2019 7
  • Windows下关于 g_defaultLogger 的编译问题

    Windows下关于 g_defaultLogger 的编译问题

    VS2017,到编译 ALL_BUILD 时,出现如下问题:

    test_logger.obj : error LNK2019: 无法解析的外部符号 "class toolkit::Logger * toolkit::g_defaultLogger" ([email protected]@@[email protected]@A),该符号在函数 _main 中被引用

    test_delayTask.obj : error LNK2019: 无法解析的外部符号 "class toolkit::Logger * toolkit::g_defaultLogger" ([email protected]@@[email protected]@A),该符号在函数 "private: virtual void __thiscall std::_Func_impl_no_alloc<class <lambda_1f860e808c4732d9497d62aaa8d648d8>,void>::_Do_call(void)" ([email protected][email protected]<lambda_1f860e808c4732d9497d62aaa8d648d8>@@[email protected]@@EAEXXZ) 中被引用

    ...

    请问是test项目的引用有问题吗

    opened by yantanglife 5
  • 如何使用ZLToolKit构建高性能的媒体转发服务器

    如何使用ZLToolKit构建高性能的媒体转发服务器

    背景需求: 应用场景类似互动聊天直播室,可能有1-8个同时说话(视频)的嘉宾,还有n个观众。服务器将嘉宾上传的udp媒体数据包转发给房间其余人。 还有一点简单信令,比如进房间,离开房间,上嘉宾位,离开嘉宾位(变成观众)等,这部分打算用TCP做。 房间之间是相互独立的,一个服务器需要同时支持很多个房间,所以对UDP的性能要求会高,tcp只是信令以及长连接维持。UDP媒体数据包头会包含房间号和用户uid。 问题如下: 1、UDP的使用参考test_udpSock.cpp即可吗,对于我们这种场景需求,zltoolkit的使用上有没什么其他的注意,比如避免内存拷贝等事项? 2、将一个包转发给房间其他人,是自己维护一个房间所有人的udp地址列表,然后遍历发送吗,zltoolkit有没什么其他更高端高效率的办法? 3、关于房间数据结构的查找问题,目前是打算用一个udp端口进行监听,udp数据包头带上房间号,解析包头得到房间号,然后通过map查找到这个房间的数据结构;另外一种方式是,一个房间使用一个udp端口进行监听,这样这个udp端口收到的数据天然就是这个房间的,避免了我们使用map查找房间。请问这两种方式,哪种房间数量多了性能会好些呢?

    opened by airx 5
  • Patch 12

    Patch 12

    简单封装了redis的自有实现的无锁的localtime

    localtime_r是线程安全的,但是,对如下两种情况并不安全,甚至会引发死锁。

    (1)信号处理函数调用localtime: 假如进程调用localtime,已经获取全局锁,且并没有释放。此时,如果进程接收到信号,在信号处理函数也调用了localtime,就会造成死锁

    (2)多线程下fork: 在多线程下,若线程A调用localtime,已经获取全局锁,尚未释放锁。此时,假如线程B调用了fork,并且在子进程也调用localtime,也会造成死锁,导致子进程一直被hang住

    opened by alexliyu7352 4
  • udp  是否有类似 TcpServer 的性能优化

    udp 是否有类似 TcpServer 的性能优化

    目前看 udp server 主要看主要是在 rtsp 及 rtpproxy 部分用到, 对性能要求可能不高.

    我这边的情况是需要一个及主要通过 udp 接入的服务器, 这边是否有类似 TcpServer 这样的优化方案呢?

    这里只有一个 tcp 的 connect 接口 https://github.com/xiongziliang/ZLToolKit/blob/ed47015f92cc79dfe3344b3666aafb54f1bbc2f4/src/Network/sockutil.h#L62

    udp 也可以 connect

       If the socket sockfd is of type SOCK_DGRAM, then addr is the address to
       which datagrams are sent by default, and the only  address  from  which
       datagrams  are  received.   If  the  socket  is  of type SOCK_STREAM or
       SOCK_SEQPACKET, this call attempts to make a connection to  the  socket
       that is bound to the address specified by addr.
    

    这样如果双向 connect 一下, 是否可以实现类似 TcpServer 中的效果? 这部分相关的改动如果有的话, 是否会考虑合到项目中?

    opened by wasphin 4
  • 编译失败

    编译失败

    环境是 ubuntu 17.10 64bit gcc 版本 gcc version 7.2.0 (Ubuntu 7.2.0-8ubuntu3.2)

    错误记录如下:

    ./build_for_linux.sh
    -- 找到openssl库:"/usr/include",ENABLE_OPENSSL宏已打开
    -- Found MySQL: /usr/include, /usr/lib/x86_64-linux-gnu/libmysqlclient.so
    -- MySQL Embedded not found.
    -- 找到mysqlclient库:"/usr/include",ENABLE_MYSQL宏已打开
    -- 将链接依赖库:/usr/lib/x86_64-linux-gnu/libssl.so;/usr/lib/x86_64-linux-gnu/libcrypto.so;/usr/lib/x86_64-linux-gnu/libmysqlclient.so
    -- 添加测试程序:test_asyncTaskThrad
    -- 添加测试程序:test_eventPoller
    -- 添加测试程序:test_logger
    -- 添加测试程序:test_noticeCenter
    -- 添加测试程序:test_pipe
    -- 添加测试程序:test_resourcePool
    -- 添加测试程序:test_ringBuffer
    -- 添加测试程序:test_rwmutex
    -- 添加测试程序:test_semaphore
    -- 添加测试程序:test_shell
    -- 添加测试程序:test_spin_mutex
    -- 添加测试程序:test_sql
    -- 添加测试程序:test_ssl
    -- 添加测试程序:test_tcpClient
    -- 添加测试程序:test_tcpEchoServer
    -- 添加测试程序:test_threadPool
    -- 添加测试程序:test_threadPoolBenchmark
    -- 添加测试程序:test_udpSock
    -- Configuring done
    -- Generating done
    -- Build files have been written to: /home/zhang/src/cpp/ZLToolKit/linux_build
    [  1%] Building CXX object CMakeFiles/ZLToolKit_shared.dir/src/Thread/AsyncTaskThread.cpp.o
    [  1%] Building CXX object CMakeFiles/ZLToolKit_shared.dir/src/Util/SqlConnection.cpp.o
    [  1%] Building CXX object CMakeFiles/ZLToolKit_static.dir/src/Thread/AsyncTaskThread.cpp.o
    [  1%] Building CXX object CMakeFiles/ZLToolKit_static.dir/src/Util/File.cpp.o
    In file included from /home/zhang/src/cpp/ZLToolKit/src/Thread/AsyncTaskThread.cpp:27:0:
    /home/zhang/src/cpp/ZLToolKit/src/Thread/AsyncTaskThread.h:50:2: error: ‘function’ does not name a type; did you mean ‘union’?
      function<bool()> task;
      ^~~~~~~~
      union
    /home/zhang/src/cpp/ZLToolKit/src/Thread/AsyncTaskThread.h:59:62: error: ‘function’ does not name a type; did you mean ‘union’?
      void DoTaskDelay(uint64_t type, uint64_t millisecond, const function<bool()> &func);
                                                                  ^~~~~~~~
                                                                  union
    /home/zhang/src/cpp/ZLToolKit/src/Thread/AsyncTaskThread.h:59:70: error: expected ‘,’ or ‘...’ before ‘<’ token
      void DoTaskDelay(uint64_t type, uint64_t millisecond, const function<bool()> &func);
                                                                          ^
    In file included from /home/zhang/src/cpp/ZLToolKit/src/Thread/AsyncTaskThread.cpp:27:0:
    /home/zhang/src/cpp/ZLToolKit/src/Thread/AsyncTaskThread.h:50:2: error: ‘function’ does not name a type; did you mean ‘union’?
      function<bool()> task;
      ^~~~~~~~
      union
    /home/zhang/src/cpp/ZLToolKit/src/Thread/AsyncTaskThread.h:59:62: error: ‘function’ does not name a type; did you mean ‘union’?
      void DoTaskDelay(uint64_t type, uint64_t millisecond, const function<bool()> &func);
                                                                  ^~~~~~~~
                                                                  union
    /home/zhang/src/cpp/ZLToolKit/src/Thread/AsyncTaskThread.h:59:70: error: expected ‘,’ or ‘...’ before ‘<’ token
      void DoTaskDelay(uint64_t type, uint64_t millisecond, const function<bool()> &func);
                                                                          ^
    /home/zhang/src/cpp/ZLToolKit/src/Thread/AsyncTaskThread.h:83:45: error: ‘function’ does not name a type; did you mean ‘union’?
      AsyncTaskHelper(uint64_t millisecond,const function<bool()> &task){
                                                 ^~~~~~~~
                                                 union
    /home/zhang/src/cpp/ZLToolKit/src/Thread/AsyncTaskThread.h:83:53: error: expected ‘,’ or ‘...’ before ‘<’ token
      AsyncTaskHelper(uint64_t millisecond,const function<bool()> &task){
                                                         ^
    /home/zhang/src/cpp/ZLToolKit/src/Thread/AsyncTaskThread.h: In constructor ‘ZL::Thread::AsyncTaskHelper::AsyncTaskHelper(uint64_t, int)’:
    /home/zhang/src/cpp/ZLToolKit/src/Thread/AsyncTaskThread.h:84:88: error: ‘task’ was not declared in this scope
       AsyncTaskThread::Instance().DoTaskDelay(reinterpret_cast<uint64_t>(this),millisecond,task);
                                                                                            ^~~~
    /home/zhang/src/cpp/ZLToolKit/src/Thread/AsyncTaskThread.cpp: At global scope:
    /home/zhang/src/cpp/ZLToolKit/src/Thread/AsyncTaskThread.cpp:50:77: error: ‘function’ does not name a type; did you mean ‘union’?
     void AsyncTaskThread::DoTaskDelay(uint64_t type, uint64_t millisecond,const function<bool()> &func) {
                                                                                 ^~~~~~~~
                                                                                 union
    /home/zhang/src/cpp/ZLToolKit/src/Thread/AsyncTaskThread.cpp:50:85: error: expected ‘,’ or ‘...’ before ‘<’ token
     void AsyncTaskThread::DoTaskDelay(uint64_t type, uint64_t millisecond,const function<bool()> &func) {
                                                                                         ^
    /home/zhang/src/cpp/ZLToolKit/src/Thread/AsyncTaskThread.cpp: In member function ‘void ZL::Thread::AsyncTaskThread::DoTaskDelay(uint64_t, uint64_t, int)’:
    /home/zhang/src/cpp/ZLToolKit/src/Thread/AsyncTaskThread.cpp:55:8: error: ‘using element_type = struct ZL::Thread::TaskInfo {aka struct ZL::Thread::TaskInfo}’ has no member named ‘task’
      info->task = func;
            ^~~~
    /home/zhang/src/cpp/ZLToolKit/src/Thread/AsyncTaskThread.cpp:55:15: error: ‘func’ was not declared in this scope
      info->task = func;
                   ^~~~
    /home/zhang/src/cpp/ZLToolKit/src/Thread/AsyncTaskThread.cpp:55:15: note: suggested alternative: ‘sync’
      info->task = func;
                   ^~~~
                   sync
    /home/zhang/src/cpp/ZLToolKit/src/Thread/AsyncTaskThread.h:83:45: error: ‘function’ does not name a type; did you mean ‘union’?
      AsyncTaskHelper(uint64_t millisecond,const function<bool()> &task){
                                                 ^~~~~~~~
                                                 union
    /home/zhang/src/cpp/ZLToolKit/src/Thread/AsyncTaskThread.h:83:53: error: expected ‘,’ or ‘...’ before ‘<’ token
      AsyncTaskHelper(uint64_t millisecond,const function<bool()> &task){
                                                         ^
    /home/zhang/src/cpp/ZLToolKit/src/Thread/AsyncTaskThread.h: In constructor ‘ZL::Thread::AsyncTaskHelper::AsyncTaskHelper(uint64_t, int)’:
    /home/zhang/src/cpp/ZLToolKit/src/Thread/AsyncTaskThread.h:84:88: error: ‘task’ was not declared in this scope
       AsyncTaskThread::Instance().DoTaskDelay(reinterpret_cast<uint64_t>(this),millisecond,task);
                                                                                            ^~~~
    /home/zhang/src/cpp/ZLToolKit/src/Thread/AsyncTaskThread.cpp: In member function ‘void ZL::Thread::AsyncTaskThread::DoTask()’:
    /home/zhang/src/cpp/ZLToolKit/src/Thread/AsyncTaskThread.cpp:100:18: error: ‘using element_type = struct ZL::Thread::TaskInfo {aka struct ZL::Thread::TaskInfo}’ has no member named ‘task’
         flag = info->task();
                      ^~~~
    /home/zhang/src/cpp/ZLToolKit/src/Thread/AsyncTaskThread.cpp: At global scope:
    /home/zhang/src/cpp/ZLToolKit/src/Thread/AsyncTaskThread.cpp:50:77: error: ‘function’ does not name a type; did you mean ‘union’?
     void AsyncTaskThread::DoTaskDelay(uint64_t type, uint64_t millisecond,const function<bool()> &func) {
                                                                                 ^~~~~~~~
                                                                                 union
    /home/zhang/src/cpp/ZLToolKit/src/Thread/AsyncTaskThread.cpp:50:85: error: expected ‘,’ or ‘...’ before ‘<’ token
     void AsyncTaskThread::DoTaskDelay(uint64_t type, uint64_t millisecond,const function<bool()> &func) {
                                                                                         ^
    /home/zhang/src/cpp/ZLToolKit/src/Thread/AsyncTaskThread.cpp: In member function ‘void ZL::Thread::AsyncTaskThread::DoTaskDelay(uint64_t, uint64_t, int)’:
    /home/zhang/src/cpp/ZLToolKit/src/Thread/AsyncTaskThread.cpp:55:8: error: ‘using element_type = struct ZL::Thread::TaskInfo {aka struct ZL::Thread::TaskInfo}’ has no member named ‘task’
      info->task = func;
            ^~~~
    /home/zhang/src/cpp/ZLToolKit/src/Thread/AsyncTaskThread.cpp:55:15: error: ‘func’ was not declared in this scope
      info->task = func;
                   ^~~~
    /home/zhang/src/cpp/ZLToolKit/src/Thread/AsyncTaskThread.cpp:55:15: note: suggested alternative: ‘sync’
      info->task = func;
                   ^~~~
                   sync
    /home/zhang/src/cpp/ZLToolKit/src/Thread/AsyncTaskThread.cpp: In member function ‘void ZL::Thread::AsyncTaskThread::DoTask()’:
    /home/zhang/src/cpp/ZLToolKit/src/Thread/AsyncTaskThread.cpp:100:18: error: ‘using element_type = struct ZL::Thread::TaskInfo {aka struct ZL::Thread::TaskInfo}’ has no member named ‘task’
         flag = info->task();
                      ^~~~
    [  1%] Building CXX object CMakeFiles/ZLToolKit_shared.dir/src/Util/util.cpp.o
    [  1%] Building CXX object CMakeFiles/ZLToolKit_static.dir/src/Util/MD5.cpp.o
    At global scope:
    cc1plus: warning: unrecognized command line option ‘-Wno-predefined-identifier-outside-function’
    At global scope:
    cc1plus: warning: unrecognized command line option ‘-Wno-predefined-identifier-outside-function’
    CMakeFiles/ZLToolKit_shared.dir/build.make:254: recipe for target 'CMakeFiles/ZLToolKit_shared.dir/src/Thread/AsyncTaskThread.cpp.o' failed
    make[2]: *** [CMakeFiles/ZLToolKit_shared.dir/src/Thread/AsyncTaskThread.cpp.o] Error 1
    make[2]: *** Waiting for unfinished jobs....
    CMakeFiles/ZLToolKit_static.dir/build.make:254: recipe for target 'CMakeFiles/ZLToolKit_static.dir/src/Thread/AsyncTaskThread.cpp.o' failed
    make[2]: *** [CMakeFiles/ZLToolKit_static.dir/src/Thread/AsyncTaskThread.cpp.o] Error 1
    make[2]: *** Waiting for unfinished jobs....
    [  1%] Building CXX object CMakeFiles/ZLToolKit_static.dir/src/Util/SSLBox.cpp.o
    CMakeFiles/Makefile2:67: recipe for target 'CMakeFiles/ZLToolKit_shared.dir/all' failed
    make[1]: *** [CMakeFiles/ZLToolKit_shared.dir/all] Error 2
    make[1]: *** Waiting for unfinished jobs....
    CMakeFiles/Makefile2:104: recipe for target 'CMakeFiles/ZLToolKit_static.dir/all' failed
    make[1]: *** [CMakeFiles/ZLToolKit_static.dir/all] Error 2
    Makefile:129: recipe for target 'all' failed
    make: *** [all] Error 2
    [  1%] Building CXX object CMakeFiles/ZLToolKit_shared.dir/src/Thread/AsyncTaskThread.cpp.o
    In file included from /home/zhang/src/cpp/ZLToolKit/src/Thread/AsyncTaskThread.cpp:27:0:
    /home/zhang/src/cpp/ZLToolKit/src/Thread/AsyncTaskThread.h:50:2: error: ‘function’ does not name a type; did you mean ‘union’?
      function<bool()> task;
      ^~~~~~~~
      union
    /home/zhang/src/cpp/ZLToolKit/src/Thread/AsyncTaskThread.h:59:62: error: ‘function’ does not name a type; did you mean ‘union’?
      void DoTaskDelay(uint64_t type, uint64_t millisecond, const function<bool()> &func);
                                                                  ^~~~~~~~
                                                                  union
    /home/zhang/src/cpp/ZLToolKit/src/Thread/AsyncTaskThread.h:59:70: error: expected ‘,’ or ‘...’ before ‘<’ token
      void DoTaskDelay(uint64_t type, uint64_t millisecond, const function<bool()> &func);
                                                                          ^
    /home/zhang/src/cpp/ZLToolKit/src/Thread/AsyncTaskThread.h:83:45: error: ‘function’ does not name a type; did you mean ‘union’?
      AsyncTaskHelper(uint64_t millisecond,const function<bool()> &task){
                                                 ^~~~~~~~
                                                 union
    /home/zhang/src/cpp/ZLToolKit/src/Thread/AsyncTaskThread.h:83:53: error: expected ‘,’ or ‘...’ before ‘<’ token
      AsyncTaskHelper(uint64_t millisecond,const function<bool()> &task){
                                                         ^
    /home/zhang/src/cpp/ZLToolKit/src/Thread/AsyncTaskThread.h: In constructor ‘ZL::Thread::AsyncTaskHelper::AsyncTaskHelper(uint64_t, int)’:
    /home/zhang/src/cpp/ZLToolKit/src/Thread/AsyncTaskThread.h:84:88: error: ‘task’ was not declared in this scope
       AsyncTaskThread::Instance().DoTaskDelay(reinterpret_cast<uint64_t>(this),millisecond,task);
                                                                                            ^~~~
    /home/zhang/src/cpp/ZLToolKit/src/Thread/AsyncTaskThread.cpp: At global scope:
    /home/zhang/src/cpp/ZLToolKit/src/Thread/AsyncTaskThread.cpp:50:77: error: ‘function’ does not name a type; did you mean ‘union’?
     void AsyncTaskThread::DoTaskDelay(uint64_t type, uint64_t millisecond,const function<bool()> &func) {
                                                                                 ^~~~~~~~
                                                                                 union
    /home/zhang/src/cpp/ZLToolKit/src/Thread/AsyncTaskThread.cpp:50:85: error: expected ‘,’ or ‘...’ before ‘<’ token
     void AsyncTaskThread::DoTaskDelay(uint64_t type, uint64_t millisecond,const function<bool()> &func) {
                                                                                         ^
    /home/zhang/src/cpp/ZLToolKit/src/Thread/AsyncTaskThread.cpp: In member function ‘void ZL::Thread::AsyncTaskThread::DoTaskDelay(uint64_t, uint64_t, int)’:
    /home/zhang/src/cpp/ZLToolKit/src/Thread/AsyncTaskThread.cpp:55:8: error: ‘using element_type = struct ZL::Thread::TaskInfo {aka struct ZL::Thread::TaskInfo}’ has no member named ‘task’
      info->task = func;
            ^~~~
    /home/zhang/src/cpp/ZLToolKit/src/Thread/AsyncTaskThread.cpp:55:15: error: ‘func’ was not declared in this scope
      info->task = func;
                   ^~~~
    /home/zhang/src/cpp/ZLToolKit/src/Thread/AsyncTaskThread.cpp:55:15: note: suggested alternative: ‘sync’
      info->task = func;
                   ^~~~
                   sync
    /home/zhang/src/cpp/ZLToolKit/src/Thread/AsyncTaskThread.cpp: In member function ‘void ZL::Thread::AsyncTaskThread::DoTask()’:
    /home/zhang/src/cpp/ZLToolKit/src/Thread/AsyncTaskThread.cpp:100:18: error: ‘using element_type = struct ZL::Thread::TaskInfo {aka struct ZL::Thread::TaskInfo}’ has no member named ‘task’
         flag = info->task();
                      ^~~~
    At global scope:
    cc1plus: warning: unrecognized command line option ‘-Wno-predefined-identifier-outside-function’
    CMakeFiles/ZLToolKit_shared.dir/build.make:254: recipe for target 'CMakeFiles/ZLToolKit_shared.dir/src/Thread/AsyncTaskThread.cpp.o' failed
    make[2]: *** [CMakeFiles/ZLToolKit_shared.dir/src/Thread/AsyncTaskThread.cpp.o] Error 1
    CMakeFiles/Makefile2:67: recipe for target 'CMakeFiles/ZLToolKit_shared.dir/all' failed
    make[1]: *** [CMakeFiles/ZLToolKit_shared.dir/all] Error 2
    Makefile:129: recipe for target 'all' failed
    make: *** [all] Error 2
    

    看上去像是function模板不支持,应该怎么办?

    opened by oska874 4
  • 替换localtime_r为no block版本

    替换localtime_r为no block版本

    简单封装了redis的自有实现的无锁的localtime

    localtime_r是线程安全的,但是,对如下两种情况并不安全,甚至会引发死锁。

    (1)信号处理函数调用localtime: 假如进程调用localtime,已经获取全局锁,且并没有释放。此时,如果进程接收到信号,在信号处理函数也调用了localtime,就会造成死锁

    (2)多线程下fork: 在多线程下,若线程A调用localtime,已经获取全局锁,尚未释放锁。此时,假如线程B调用了fork,并且在子进程也调用localtime,也会造成死锁,导致子进程一直被hang住

    opened by alexliyu7352 3
  • 建议套接字创建和连接分两步进行

    建议套接字创建和连接分两步进行

    目前SockUtil::connect函数内完成套接字创建和端口绑定,然后立即连接到远程目的ip和端口,但有时候这两件事需要分开进行,比如在gb28181播放协商的时候,下级在收到上级invite请求,返回200ok的sdp报文中附带发流源ip和端口,有些上级加了安全边界设备,需要核对下级的源ip端口才允许下级连接并发送码流数据,这种情况就需要做三个步骤:1、创建并绑定本地端口;2、向上级发送发流ip和端口并等待上级ack;3、连接远程目的ip和端口。具体实现代码:

    bool Socket::createAndBindSock(bool udpOrTcp, uint16_t port, const string &local_ip) {
        closeSock();
        int fd = SockUtil::createAndBindSock(udpOrTcp, port, local_ip.data());
        if (fd == -1)
            return false;
    
        SockFD::Ptr sock;
        if (udpOrTcp) {
            sock = makeSock(fd, SockNum::Sock_UDP);
            if (!attachEvent(sock, true))
                return false;
        }
        else
        {
            // 注意tcp在连接目标成功后才会监听事件
            sock = makeSock(fd, SockNum::Sock_TCP);
        }
        LOCK_GUARD(_mtx_sock_fd);
        _sock_fd = sock;
        return true;
    }
    void Socket::connect(const string &url, uint16_t port, onErrCB con_cb_in,
        float timeout_sec, const string &local_ip, uint16_t local_port, bool newSock) {
        if (newSock) {
            // 重置当前socket
            closeSock();
        }
        weak_ptr<Socket> weak_self = shared_from_this();
        auto con_cb = [con_cb_in, weak_self](const SockException &err) {
            auto strong_self = weak_self.lock();
            if (!strong_self) {
                return;
            }
            strong_self->_async_con_cb = nullptr;
            strong_self->_con_timer = nullptr;
            if (err) {
                LOCK_GUARD(strong_self->_mtx_sock_fd);
                strong_self->_sock_fd = nullptr;
            }
            con_cb_in(err);
        };
    
        auto async_con_cb = std::make_shared<function<void(int)> >([weak_self, con_cb, newSock](int sock) {
            auto strong_self = weak_self.lock();
            if (sock == -1 || !strong_self) {
                if (!strong_self) {
                    CLOSE_SOCK(sock);
                } else {
                    con_cb(SockException(Err_dns, get_uv_errmsg(true)));
                }
                return;
            }
            bool test = false;
            SockFD::Ptr sock_fd = strong_self->getSock();
            if (newSock || !sock_fd) {
                test = true;
                sock_fd = strong_self->makeSock(sock, SockNum::Sock_TCP);
            }
            weak_ptr<SockFD> weak_sock_fd = sock_fd;
            // 监听该socket是否可写,可写表明已经连接服务器成功
            int result = strong_self->_poller->addEvent(sock, Event_Write, [weak_self, weak_sock_fd, con_cb](int event) {
                auto strong_sock_fd = weak_sock_fd.lock();
                auto strong_self = weak_self.lock();
                if (strong_sock_fd && strong_self) {
                    //socket可写事件,说明已经连接服务器成功
                    strong_self->onConnected(strong_sock_fd, con_cb);
                }
            });
    
            if (result == -1) {
                con_cb(SockException(Err_other, "add event to poller failed when start connect"));
                return;
            }
            if (test) {
                // 保存fd
                strong_self->setSock(sock_fd);
            }
        });
    
        auto poller = _poller;
        weak_ptr<function<void(int)> > weak_task = async_con_cb;
        int sockfd_in = -1;
        if (!newSock) {
            SockFD::Ptr sock_fd = getSock();
            if (sock_fd)
                sockfd_in = sock_fd->rawFd();
        }
        WorkThreadPool::Instance().getExecutor()->async([url, port, local_ip, local_port, sockfd_in, weak_task, poller]() {
            // 阻塞式dns解析放在后台线程执行
            int sock = SockUtil::connect(url.data(), port, true, local_ip.data(), local_port, sockfd_in);
            poller->async([sock, weak_task]() {
                auto strong_task = weak_task.lock();
                if (strong_task) {
                    (*strong_task)(sock);
                } else {
                    CLOSE_SOCK(sock);
                }
            });
        });
    
        //连接超时定时器
        _con_timer = std::make_shared<Timer>(timeout_sec, [weak_self, con_cb]() {
            con_cb(SockException(Err_timeout, uv_strerror(UV_ETIMEDOUT)));
            return false;
        }, _poller);
    
        _async_con_cb = async_con_cb;
    }
    int SockUtil::createAndBindSock(bool udpOrTcp, const uint16_t port, const char* localIp, int af, bool bAsync, bool reusePort) {
        int sockfd = -1;
        if (udpOrTcp)
            sockfd = (int)socket(af, SOCK_DGRAM, IPPROTO_UDP);
        else
            sockfd = (int)socket(af, SOCK_STREAM, IPPROTO_TCP);
        if (sockfd == -1) {
            WarnL << "创建套接字失败:" << get_uv_errmsg(true);
            return -1;
        }
        setReuseable(sockfd, reusePort);
        setNoSigpipe(sockfd);
        setNoBlocked(sockfd, bAsync);
        if (!udpOrTcp)
            setNoDelay(sockfd);
        setSendBuf(sockfd);
        setRecvBuf(sockfd);
        setCloseWait(sockfd);
        setCloExec(sockfd);
        if (bindSock(sockfd, localIp, port) == -1) {
            close(sockfd);
            return -1;
        }
        return sockfd;
    }
    int SockUtil::connect(const char *host, uint16_t port, bool bAsync, const char* localIp, uint16_t localPort, int sockfd_in) {
        sockaddr addr;
        if(!DnsCache::Instance().getDomainIP(host,addr)){
            //dns解析失败
            return -1;
        }
        //设置端口号
        ((sockaddr_in *)&addr)->sin_port = htons(port);
    
        int sockfd = sockfd_in;
        if (sockfd < 0)
            sockfd = createAndBindSock(false, localPort, localIp, addr.sa_family, bAsync);
        if (sockfd < 0) {
            WarnL << "创建套接字失败:" << host;
            return -1;
        }
        if (::connect(sockfd, &addr, sizeof(struct sockaddr)) == 0) {
            //同步连接成功
            return sockfd;
        }
        if (bAsync &&  get_uv_error(true) == UV_EAGAIN) {
            //异步连接成功
            return sockfd;
        }
        WarnL << "连接主机失败:" << host << " " << port << " " << get_uv_errmsg(true);
        close(sockfd);
        return -1;
    }
    
    opened by wujianGit123 1
  • makeRandStr获取的随机值问题

    makeRandStr获取的随机值问题

    平台:Linux, ARMv7 问题: 连续执行makeRandStr获取的随机字符串不随机。

    能否修改为以下实现,谢谢!

    string makeRandStr(int sz, bool printable) {
        char *tmp = new char[sz + 1];
        static const char CCH[] =
            "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ";
        int i;
        std::mt19937 rng(std::random_device{}());
        for (i = 0; i < sz; i++) {
            if (printable) {
                uint32_t x = rng() % (sizeof(CCH) - 1);
                tmp[i] = CCH[x];
            }
            else {
                tmp[i] = rng() % 0xFF;
            }
        }
        tmp[i] = 0;
        string ret = tmp;
        delete[] tmp;
        return ret;
    }
    
    opened by bzd2132 4
  • getCurrentMicrosecondOrigin 获取时间可能有随机的错误

    getCurrentMicrosecondOrigin 获取时间可能有随机的错误

    运行平台:Linux, ARMv7 现象: 运行过程中会打印下面的日志。

    2021-11-24 14:45:35.000 W test[1279-stamp thread] util.cpp:362 operator() | Stamp expired is not abnormal:-6631 2021-11-24 14:46:49.000 W test[1279-stamp thread] util.cpp:362 operator() | Stamp expired is not abnormal:-319974 2021-11-24 14:47:26.000 W test[1279-stamp thread] util.cpp:362 operator() | Stamp expired is not abnormal:-233909 2021-11-24 14:48:03.000 W test[1279-stamp thread] util.cpp:362 operator() | Stamp expired is not abnormal:-137712 原因: 在util.cpp中的getCurrentMicrosecondOrigin函数,获取的时间有可能会出现小于前面的获取的情况。

    static inline uint64_t getCurrentMicrosecondOrigin() {
    #if !defined(_WIN32)
        struct timeval tv;
        gettimeofday(&tv, NULL);
        return tv.tv_sec * 1000000LL + tv.tv_usec;
    #else
        return  std::chrono::duration_cast<std::chrono::microseconds>(std::chrono::system_clock::now().time_since_epoch()).count();
    #endif
    }
    

    修改成下面的实现后,就不会出现了

    static inline uint64_t getCurrentMicrosecondOrigin() {
    #if !defined(_WIN32)
        struct timespec ts;
        clock_gettime(CLOCK_MONOTONIC, &ts);
        return ts.tv_sec * 1000000LL + ts.tv_nsec/1000;
    #else
        return  std::chrono::duration_cast<std::chrono::microseconds>(std::chrono::system_clock::now().time_since_epoch()).count();
    #endif
    }
    

    请帮忙看看修改是否有问题。谢谢!

    opened by bzd2132 2
Owner
zlmediakit开源
zlmediakit开源