nanomsg-next-generation -- light-weight brokerless messaging

Overview

nng - nanomsg-next-gen

Linux Status Windows Status macOS Status Coverage Codacy LGTM Discord Manual MIT License Latest version

ℹ️
If you are looking for the legacy version of nanomsg, please see the nanomsg repository.

This project is a rewrite of the Scalability Protocols library known as libnanomsg, and adds significant new capabilities, while retaining compatibility with the original.

It may help to think of this as "nanomsg-next-generation".

NNG: Lightweight Messaging Library

NNG, like its predecessors nanomsg (and to some extent ZeroMQ), is a lightweight, broker-less library, offering a simple API to solve common recurring messaging problems, such as publish/subscribe, RPC-style request/reply, or service discovery. The API frees the programmer from worrying about details like connection management, retries, and other common considerations, so that they can focus on the application instead of the plumbing.

NNG is implemented in C, requiring only C99 and CMake to build. It can be built as a shared or a static library, and is readily embeddable. It is also designed to be easy to port to new platforms if your platform is not already supported.

License

NNG is licensed under a liberal, and commercial friendly, MIT license. The goal to the license is to minimize friction in adoption, use, and contribution.

Enhancements (Relative to nanomsg)

Here are areas where this project improves on "nanomsg":

Reliability

NNG is designed for production use from the beginning. Every error case is considered, and it is designed to avoid crashing except in cases of gross developer error. (Hopefully we don’t have any of these in our own code.)

Scalability

NNG scales out to engage multiple cores using a bespoke asynchronous I/O framework, using thread pools to spread load without exceeding typical system limits.

Maintainability

NNG’s architecture is designed to be modular and easily grasped by developers unfamiliar with the code base. The code is also well documented.

Extensibility

Because it avoids ties to file descriptors, and avoids confusing interlocking state machines, it is easier to add new protocols and transports to NNG. This was demonstrated by the addition of the TLS and ZeroTier transports.

Security

NNG provides TLS 1.2 and ZeroTier transports, offering support for robust and industry standard authentication and encryption. In addition, it is hardened to be resilient against malicious attackers, with special consideration given to use in a hostile Internet.

Usability

NNG eschews slavish adherence parts of the more complex and less well understood POSIX APIs, while adopting the semantics that are familiar and useful. New APIs are intuitive, and the optional support for separating protocol context and state from sockets makes creating concurrent applications vastly simpler than previously possible.

Compatibility

This project offers both wire compatibility and API compatibility, so most nanomsg users can begin using NNG right away.

Existing nanomsg and mangos applications can inter-operate with NNG applications automatically.

That said, there are some areas where legacy nanomsg still offers capabilities NNG lacks — specifically enhanced observability with statistics, and tunable prioritization of different destinations are missing, but will be added in a future release.

Additionally, some API capabilities that are useful for foreign language bindings are not implemented yet.

Some simple single threaded, synchronous applications may perform better under legacy nanomsg than under NNG. (We believe that these applications are the least commonly deployed, and least interesting from a performance perspective. NNG’s internal design is slightly less efficient in such scenarios, but it greatly benefits when concurrency or when multiple sockets or network peers are involved.)

Supported Platforms

NNG supports Linux, macOS, Windows (Vista or better), illumos, Solaris, FreeBSD, Android, and iOS. Most other POSIX platforms should work out of the box but have not been tested. Very old versions of otherwise supported platforms might not work.

Requirements

To build this project, you will need a C99 compatible compiler and CMake version 3.13 or newer.

We recommend using the Ninja build system (pass "-G Ninja" to CMake) when you can. (And not just because Ninja sounds like "NNG" — it’s also blindingly fast and has made our lives as developers measurably better.)

If you want to build with TLS support you will also need mbedTLS. See docs/BUILD_TLS.adoc for details.

Quick Start

With a Linux or UNIX environment:

  $ mkdir build
  $ cd build
  $ cmake -G Ninja ..
  $ ninja
  $ ninja test
  $ ninja install

API Documentation

The API documentation is provided in Asciidoc format in the docs/man subdirectory, and also online. The libnng(3) page is a good starting point.

You can also purchase a copy of the NNG Reference Manual. (It is published in both electronic and printed formats.) Purchases of the book help fund continued development of NNG.

Example Programs

Some demonstration programs have been created to help serve as examples. These are located in the demo directory.

Legacy Compatibility

A legacy libnanomsg compatible API is available, and while it offers less capability than the modern NNG API, it may serve as a transition aid. Please see nng_compat(3) for details.

Commercial Support

Commercial support for NNG is available.

Please contact Staysail Systems to inquire further.

Commercial Sponsors

The development of NNG has been made possible through the generous sponsorship of Capitar IT Group BV and Staysail Systems, Inc..

Comments
  • Reproducing IPC transport unit tests causing INFINITE lock wait

    Reproducing IPC transport unit tests causing INFINITE lock wait

    Couched in my understanding that the API isn't quite ready yet. This may be one of those areas, I don't know.

    Whether following the C-style calls or the C++ wrapper, it does not seem to matter. Something in the way the testing is initialized "under the hood" is causing the tests to hang INFINITE-ly.

    When there is a valid listen/dial pair in effect, the dial operation hangs INFINITE-ly. Debugging gets as far as nni_ep_dial(...) and the subsequent nni_aoi_wait(...). Which then waits with INFINITE options on the call to SleepConditionVariableSRW.

    // Gets as far as here...
    int nni_ep_dial(nni_ep *ep, int flags)
    
    // Then blocks indefinitely on the (synchronous?) AOI to return.
    nni_aio_wait(aio);
    
    // Ultimately waiting on a condition variable that is never cleared, apparently.
    // And with "INFINITE" timeout? not otherwise queued on the socket options? i.e. timeouts?
    void
    nni_plat_cv_wait(nni_plat_cv *cv)
    {
      (void) SleepConditionVariableSRW(&cv->cv, cv->srl, INFINITE, 0);
    }
    

    TCP works great. So does INPROC. IPC, on the other hand, is problematic under Windows (i.e. Windows 7 x64).

    The only difference I can determine at this point is in how the NNG IPC transport tests are initialized.

    void
    trantest_init(trantest *tt, const char *addr)
    {
      trantest_next_address(tt->addr, addr);
      So(nng_req_open(&tt->reqsock) == 0); // These look innocent enough.
      So(nng_rep_open(&tt->repsock) == 0);
    
      tt->tran = nni_tran_find(addr); // This however I find suspicious.
      So(tt->tran != NULL); // And/or perhaps something in the way that the project is build, flags, etc.
    }
    
    opened by mwpowellhtx 61
  • SIGSEGV in RepReq's rep0 recv - use after free

    SIGSEGV in RepReq's rep0 recv - use after free

    As briefly discussed in #1240 there appears to be a bug in the current implementation that results in a SIGSEGV when a socket is closed but the resend timeout fired at least once. Or that is my current theory on it at least.

    SIGSEGV from C implementation:

    https://gist.github.com/AlexKornitzer/371eef55e558e89d10850d53986dfb35
    
    ./reqrep server ipc:///tmp/abcd 
    ./reqrep client ipc:///tmp/abcd
    
    panic: pthread_mutex_lock: Invalid argument
    This message is indicative of a BUG.
    Report this at https://github.com/nanomsg/nng/issues
    /home/developer/Developer/scratch/nng/reqrep(+0x15bb9) [0x555555569bb9]
    /home/developer/Developer/scratch/nng/reqrep(+0x1f7af) [0x5555555737af]
    /home/developer/Developer/scratch/nng/reqrep(+0x1f958) [0x555555573958]
    /home/developer/Developer/scratch/nng/reqrep(+0x1c826) [0x555555570826]
    /home/developer/Developer/scratch/nng/reqrep(+0x1c562) [0x555555570562]
    /home/developer/Developer/scratch/nng/reqrep(+0xf21f) [0x55555556321f]
    /home/developer/Developer/scratch/nng/reqrep(+0x24617) [0x555555578617]
    /home/developer/Developer/scratch/nng/reqrep(+0x15f4e) [0x555555569f4e]
    /home/developer/Developer/scratch/nng/reqrep(+0xe60d) [0x55555556260d]
    /home/developer/Developer/scratch/nng/reqrep(+0xec35) [0x555555562c35]
    /home/developer/Developer/scratch/nng/reqrep(+0x18a00) [0x55555556ca00]
    /home/developer/Developer/scratch/nng/reqrep(+0x739f) [0x55555555b39f]
    /home/developer/Developer/scratch/nng/reqrep(+0x710d) [0x55555555b10d]
    /home/developer/Developer/scratch/nng/reqrep(+0x6f29) [0x55555555af29]
    /home/developer/Developer/scratch/nng/reqrep(+0x690f) [0x55555555a90f]
    /home/developer/Developer/scratch/nng/reqrep(+0x6dac) [0x55555555adac]
    /usr/lib/libc.so.6(__libc_start_main+0xf3) [0x7ffff7ded023]
    /home/developer/Developer/scratch/nng/reqrep(+0x672e) [0x55555555a72e]
    
    (gdb) bt
    #0  0x00007ffff7e01ce5 in raise () from /usr/lib/libc.so.6
    #1  0x00007ffff7deb857 in abort () from /usr/lib/libc.so.6
    #2  0x00005555555733bf in nni_plat_abort ()
    #3  0x0000555555569bbe in nni_panic ()
    #4  0x00005555555737af in nni_pthread_mutex_lock ()
    #5  0x0000555555573958 in nni_plat_mtx_lock ()
    #6  0x0000555555570826 in nni_mtx_lock ()
    #7  0x0000555555570562 in nni_task_dispatch ()
    #8  0x000055555556321f in nni_aio_begin ()
    #9  0x0000555555578617 in ipctran_pipe_recv ()
    #10 0x0000555555569f4e in nni_pipe_recv ()
    #11 0x000055555556260d in rep0_ctx_recv ()
    #12 0x0000555555562c35 in rep0_sock_recv ()
    #13 0x000055555556ca00 in nni_sock_recv ()
    #14 0x000055555555b39f in nng_recv_aio ()
    #15 0x000055555555b10d in nng_recvmsg ()
    #16 0x000055555555af29 in nng_recv ()
    #17 0x000055555555a90f in server ()
    #18 0x000055555555adac in main ()
    

    SIGSEGV from Rust version:

    Thread 1 "scratch" received signal SIGSEGV, Segmentation fault.
    0x0000555555577ce4 in nni_pipe_recv (p=0x0, aio=0x7fffe8001430)
        at /home/developer/.cargo/registry/src/github.com-1ecc6299db9ec823/nng-sys-1.2.4-rc.1/nng/src/core/pipe.c:141
    141             p->p_tran_ops.p_recv(p->p_tran_data, aio);
    (gdb) bt
    #0  0x0000555555577ce4 in nni_pipe_recv (p=0x0, aio=0x7fffe8001430)
        at /home/developer/.cargo/registry/src/github.com-1ecc6299db9ec823/nng-sys-1.2.4-rc.1/nng/src/core/pipe.c:141
    #1  0x000055555556dddf in rep0_ctx_recv (arg=0x5555555ead20, aio=0x5555555edcb0)
        at /home/developer/.cargo/registry/src/github.com-1ecc6299db9ec823/nng-sys-1.2.4-rc.1/nng/src/protocol/reqrep0/rep.c:504
    #2  0x000055555556e361 in rep0_sock_recv (arg=0x5555555eac10, aio=0x5555555edcb0)
        at /home/developer/.cargo/registry/src/github.com-1ecc6299db9ec823/nng-sys-1.2.4-rc.1/nng/src/protocol/reqrep0/rep.c:670
    #3  0x000055555557ade3 in nni_sock_recv (sock=0x5555555ea220, aio=0x5555555edcb0)
        at /home/developer/.cargo/registry/src/github.com-1ecc6299db9ec823/nng-sys-1.2.4-rc.1/nng/src/core/socket.c:847
    #4  0x00005555555666e6 in nng_recv_aio (s=..., aio=0x5555555edcb0)
        at /home/developer/.cargo/registry/src/github.com-1ecc6299db9ec823/nng-sys-1.2.4-rc.1/nng/src/nng.c:214
    #5  0x000055555556651d in nng_recvmsg (s=..., msgp=0x7fffffffcf50, flags=0)
        at /home/developer/.cargo/registry/src/github.com-1ecc6299db9ec823/nng-sys-1.2.4-rc.1/nng/src/nng.c:136
    #6  0x0000555555564a84 in nng::socket::Socket::recv (self=0x7fffffffd018)
        at /home/developer/.cargo/git/checkouts/nng-rs-585a5382ec72d3dc/fc301c2/src/socket.rs:245
    #7  0x00005555555600f1 in scratch::server () at src/main.rs:76
    #8  0x0000555555560339 in scratch::main () at src/main.rs:87
    
    bug 
    opened by alexkornitzer 39
  • nng_recv() sometimes acts on null `msg` pointer

    nng_recv() sometimes acts on null `msg` pointer

    NNG & Platform details.

    NNG commit 6401d100e8b616c65014ae7fd62ac9a575466bef Windows 10 (RS3, 10.0.16299) using from C++

    Expected Behavior

    No segfault in nng_recv on calling nng_msg_len.

    Actual Behavior

    Segfault, nng_recvmsg rv = 0, msg = NULL.

    Steps to Reproduce

    Haven't found a minimal repro yet, however some details on our usage:

    • 2 threads, let's name them 'main' and 'net'
    • each thread owns a client req (our understanding is this allows resending?) inproc:// socket to the thread's rep socket (inproc://net and inproc://main)
    • 4-byte messages get sent to signal wakeup, no other data is sent at this time
    • recv call:
    					while (nng_recv(netSocket, &msgBuffer, &msgLen, NNG_FLAG_NONBLOCK | NNG_FLAG_ALLOC) == 0)
    					{
    						nng_free(msgBuffer, msgLen);
    
    						int ok = 0;
    						nng_send(netSocket, &ok, 4, NNG_FLAG_NONBLOCK);
    
    						m_netThreadCallbacks.Run();
    					}
    
    • send call:
    		nng_send(sockets[i], &idxList[0], idxList.size() * sizeof(int), NNG_FLAG_NONBLOCK);
    

    We're also having some other issues with aio lifetime on both Linux and Windows - possibly some kind of use-after-free or double frees, but no actual coredump at hand for this. 😕

    bug 
    opened by blattersturm 39
  • IPC error during Bus tests

    IPC error during Bus tests

    2017.11.04 17:55:10.830   ERROR Process C:\Users\Michael\AppData\Local\JetBrains\Installations\ReSharperPlatformVs14\JetBrains.ReSharper.TaskRunner.CLR45.x64.exe:14588 exited with code '3':
    panic: G:\Source\Spikes\nanomsg\csnng\prototype\repos\nng\src\platform\windows\win_ipc.c: 65: assert err: aio->a_iov[0].iov_len
    This message is indicative of a BUG.
    Report this at http://github.com/nanomsg/nanomsg
    

    Test log:

    Running protocol tests for address family 'InterProcess'.
    Testing using address 'ipc://pipe/325656c3-4e60-4c89-81d4-097e4a20dfa2'.
    Given three Nanomsg2.Sharp.Protocols.Bus.LatestBusSocket instances.
      And messages can be delivered.
        And receive times out.
          And Bus1 delivers message to both Bus2 and Bus3.
    

    Based on IPC specialized Bus tests:

    namespace Nanomsg2.Sharp.Protocols.Bus
    {
        using Xunit.Abstractions;
    
        public class InterProcessBusTests : BusTests
        {
            protected override SocketAddressFamily Family { get; } = SocketAddressFamily.InterProcess;
    
            public InterProcessBusTests(ITestOutputHelper @out)
                : base(@out)
            {
            }
        }
    }
    

    Running under Visual Studio 2015, C# 6.0, xUnit, Windows 7 Professional (x64).

    opened by mwpowellhtx 33
  • [Question] req/rep inverted async demo

    [Question] req/rep inverted async demo

    Hello,

    I'm filing this issue to have an answer for a question that I have: I have a very similar need than the one described in the async demo. The only slight difference is that my server is actually a REQ and my clients are REPs.

    I've first prototyped this in my project and realized that the library was behaving weird so I figured I'd try to make something smaller. I modified the async demo to do just that: make the server a REQ instead of a REP; instead of receiving then sending, it sends, then receives. Also modified the client to accommodate that.

    To give a few more details on what I'm seeing (I haven't really debugged nor tried to find where / why this is happening yet): if I have X aio waiting on a recv (to be able to handle X simultaneous clients) and a client comes, the response of the server gets sent more than once; once with the expected nng_ctx but I can also see the same data getting sent with other nng_ctxs as well (I simply display the context identifier).

    Before I write a reproducer, is the above expected to work, or is there some sort of limitation / something I misunderstood along the way regarding the APIs/REQ/REP? FWIW I am doing my tests with the latest release which is v1.3.2; and I've verified that the same things happen on both Linux / Windows with the tcp transport.

    Cheers

    question feedback 
    opened by 0vercl0k 30
  • kqueue: add kqueue-based pollq implementation

    kqueue: add kqueue-based pollq implementation

    Initial cut at kqueue support for #35

    This passes all tests on my machine, will see how it does on CI. Happy to rename/reorg as needed.

    A couple thoughts:

    • since some of the poller API (add, remove, enable, disable) may involve syscalls, it would nice to return errors from those routines and handle them accordingly, possibly in a future PR
    • i wasn't totally clear on the expected handling of event flags - it looks like the existing poll implementation treats event flag manipulation as a cheap operation (it just manipulates userspace memory in the poll implementation), but since epoll and kqueue implementations might prefer to use the provided syscall interfaces to enable/disable events as distinct from adding/removing nodes, it could be nice to take that into consideration and minimize changes to enable/disable state.
    opened by liamstask 29
  • PUB/SUB Multi-Message Handling

    PUB/SUB Multi-Message Handling

    When porting a script from zmq to pynng (much nicer python coding experience) to broadcast data (bursts of 4 direct follow up messages per loop, approx. 60 loops/s, approx. 1 KB/s TCP traffic) from a PUB to two SUBs, I experienced an unmissable data/message loss that was not experienced with zmq.

    In followup investigations on this topic I found some issues in the nanomsg repository describing this problem (nanomsg/nanomsg#283, nanomsg/nanomsg#390).

    I was wondering, if this "behavior" is still expected to happen with nng. Probably yes, as of #762 and the Conceptual Overview section of the nng manual:

    nng sockets are message oriented, so that messages are either delivered wholly, or not at all. Partial delivery is not possible. Furthermore, nng does not provide any other delivery or ordering guarantees; messages may be dropped or reordered. (Some protocols, such as nng_req(7) may offer stronger guarantees by performing their own retry and validation schemes.)

    NNG & Platform details.

    NNG: Using pynng (codypiersall/pynng@61c9f11) and the nng version shipped with it. OS: Windows 10 10.0.17134 (64-Bit) PL: Python 3.7.0

    Expected Behavior

    Not drop 0.5 % (or even 75 %) of messages.

    Actual Behavior

    Drops 0.5 % (or even 75 %) of messages.

    Steps to Reproduce

    The following python code compares the percentage of dropped messages between nng and zmq

    Single Message Example

    import pynng  # From commit 61c9f11
    import zmq
    import time
    
    pub = pynng.Pub0(listen='tcp://*:5555')
    
    sub = pynng.Sub0(dial='tcp://localhost:5555')
    sub.subscribe(b'0')
    
    time.sleep(1)
    
    i_send = 0
    i_recv = 0
    try:
        while True:
            i_send += 1
            
            pub.send(b'0')
            # time.sleep(0.000001)  # Cures the loss
            
            try:
                msg = sub.recv(block=False)
                i_recv += 1
            except pynng.exceptions.TryAgain:
                pass
            
            print(f'pynng: Lost {i_send - i_recv} of {i_send} msgs ' + '({0:3.3f} % loss)'.format((i_send - i_recv) / i_send * 100) + '  [Exit with Ctrl+C]', end='\r')
            if i_send >= 10**6:
                break
    except KeyboardInterrupt:
        pass
    finally:
        print()
        sub.close()
        pub.close()
        time.sleep(1)
    
    
    ctx = zmq.Context()
    
    pub = ctx.socket(zmq.PUB)
    pub.bind('tcp://*:5555')
    
    sub = ctx.socket(zmq.SUB)
    sub.connect('tcp://localhost:5555')
    sub.setsockopt(zmq.SUBSCRIBE, b'0')
    
    time.sleep(1)
    
    i_send = 0
    i_recv = 0
    try:
        while True:
            i_send += 1
            
            pub.send(b'0')
            # time.sleep(0.000001)  # Cures the loss
    
            try:
                msg = sub.recv(zmq.NOBLOCK)
                i_recv += 1
            except zmq.error.Again:
                pass
            
            print(f'zmq:   Lost {i_send - i_recv} of {i_send} msgs ' + '({0:3.3f} % loss)'.format((i_send - i_recv) / i_send * 100) + '  [Exit with Ctrl+C]', end='\r')
            if i_send >= 10**6:
                break
    except KeyboardInterrupt:
        pass
    finally:
        sub.close()
        pub.close()
        time.sleep(1)
    
    exit()
    

    Yielding quite reproducibly the following values:

     pynng: Lost 5258 of 1000000 messages (0.526 % loss)  [Exit with Ctrl+C]
     zmq:   Lost 45 of 1000000 messages (0.005 % loss)  [Exit with Ctrl+C]
    

    Quite interestingly, zmq manages to handle this fast PUB/SUB by 2 orders of magnitude better than nng. A short time.sleep() after send cures the loss in both cases, however, dramatically slows down the script.

    Message Burst Example

    The following script simulates the beforehand mentioned 4 message data bursts:

    import pynng  # From commit 61c9f11
    import zmq
    import time
    
    burstsize = 4
    
    pub = pynng.Pub0(listen='tcp://*:5555')
    
    sub = pynng.Sub0(dial='tcp://localhost:5555')
    sub.subscribe(b'')
    
    time.sleep(1)
    
    i_send = 0
    i_recv = 0
    try:
        while True:
            msgs = []
            for i in range(burstsize):
                i_send += 1
                pub.send(bytes(str(i), encoding='utf-8'))
                # time.sleep(0.000001)  #Leads to 75 % loss - only 1 msg is received!
            
            while True:
                try:
                    msgs += [sub.recv(block=False)]
                    i_recv += 1
                    # time.sleep(0.000001)  # Has no effect
                except pynng.exceptions.TryAgain:
                    break
            
            print(f'pynng: Lost {i_send - i_recv} of {i_send} msgs ' + '({0:3.3f} % loss)  '.format((i_send - i_recv) / i_send * 100) + f'Burst: Recvd {len(msgs)} of {burstsize} msgs ' + '({0:3.3f} % loss)  '.format((burstsize - len(msgs)) / burstsize * 100) + '[Exit with Ctrl+C]', end='\r')
            if i_send >= 10**6:
                break
    except KeyboardInterrupt:
        pass
    finally:
        sub.close()
        pub.close()
        time.sleep(1)
        print()
    
    ctx = zmq.Context()
    
    pub = ctx.socket(zmq.PUB)
    pub.bind('tcp://*:5555')
    
    sub = ctx.socket(zmq.SUB)
    sub.connect('tcp://localhost:5555')
    sub.setsockopt(zmq.SUBSCRIBE, b'')
    
    time.sleep(1)
    
    i_send = 0
    i_recv = 0
    try:
        while True:
            msgs = []
            for i in range(burstsize):
                i_send += 1
                pub.send(bytes(str(i), encoding='utf-8'))
                # time.sleep(0.000001)  # Leads to 75 % loss - only 1 msg is received!
            
            while True:
                try:
                    msgs += [sub.recv(zmq.NOBLOCK)]
                    i_recv += 1
                    # time.sleep(0.000001)  # Has no effect
                except zmq.error.Again:
                    break
            
            print(f'zmq  : Lost {i_send - i_recv} of {i_send} msgs ' + '({0:3.3f} % loss)  '.format((i_send - i_recv) / i_send * 100) + f'Burst: Recvd {len(msgs)} of {burstsize} msgs ' + '({0:3.3f} % loss)  '.format((burstsize - len(msgs)) / burstsize * 100) + '[Exit with Ctrl+C]', end='\r')
            if i_send >= 10**6:
                break
    except KeyboardInterrupt:
        pass
    finally:
        sub.close()
        pub.close()
        time.sleep(1)
    
    exit()
    

    Delivering the following results:

    pynng: Lost 620351 of 1000000 messages (62.035 % loss)  [Exit with Ctrl+C]
    zmq:   Lost 4 of 1000000 messages (0.000 % loss)  [Exit with Ctrl+C]
    

    Again, zmq is the definite winner. In the code example it seems quite random which messages are lost (each of the for different messages are received over time). However, as stated in the code, putting time.sleep() after send, leads to 75 % data loss for nng and the sole message that received is the 1st of the 4 (1 out of 4 = 75 % loss).

    Am I missing some setting here to define a buffer size? Is there any option to make the PUB wait/block until the message was really sent (just sent, not received)?

    As stated by @gdamore in https://github.com/nanomsg/nanomsg/issues/390#issuecomment-98404170

    I'm closing this as the problem is not with nanomsg, but rather with your usage and understanding the limitations of the design.

    a PUB/SUB socket was not the right choice to multicast "a lot of" data in nanomsg, which seemingly still is the case for nng.

    What I thus wonder, and clearly do not understand: If nng's PUB/SUB is not the right choice, what else is?

    As I have to send this data fast and lossless, this issue is the only reason why I have to stick with zmq for my application. (Ok, except for this https://github.com/codypiersall/pynng/issues/9#issuecomment-437969587 - even with some C code from @codypiersall)

    (Sorry for this extremely long issue :) )


    Edit: Code example was 2 × the same. Changed the second to the real burst example.

    feedback 
    opened by lcnittl 28
  • nng_sockaddr_in6 should support 32-bit and 16-bit unionized fields

    nng_sockaddr_in6 should support 32-bit and 16-bit unionized fields

    If I understand my blogs reading according to the spec and the responses I've read, that's the layout, which does appear to care about network byte order. So while 16-byte array is interesting, it is virtually useless to work with.

    See this SO for more detail.

    So, this:

    struct nng_sockaddr_in6 {
        uint16_t sa_family;
        uint16_t sa_port;
        uint8_t  sa_addr[16];
    };
    

    Should probably be something more like this:

    struct nng_sockaddr_in6 {
        uint16_t sa_family;
        uint16_t sa_port;
        union {
            uint8_t sa_addr8[16];
            uint16_t sa_addr16[8];
            uint32_t sa_addr32[4];
        } sa_addr;
    };
    

    Which, actually the 16-bit representation is probably most accurate according to the statement: "An IPv6 address is represented as eight groups of four hexadecimal digits, each group representing 16 bits ".

    Oh, for fun with C/C++ unions.

    Could be wrong.

    opened by mwpowellhtx 28
  • always see 1% message drop on listener

    always see 1% message drop on listener

    NNG Version : 1.1.0

    I did some experiments about how much msg/sec i can pump on a Pair socket with one listener and one dialer. What i found out is irrespective of message size and number of messages per iteration i always see a message drop of around 1%

    iteration | total messages received | total time | msg/sec | drop rate -- | -- | -- | -- | -- 0 | 14927 | 1105 | 13508.60 | 0.49 1 | 14866 | 1072 | 13867.50 | 0.89 2 | 14931 | 1255 | 11897.20 | 0.46 3 | 14965 | 1252 | 11952.90 | 0.23 4 | 14983 | 1101 | 13608.50 | 0.11 5 | 14913 | 1082 | 13782.80 | 0.58 6 | 14919 | 1197 | 12463.70 | 0.54 7 | 14861 | 1067 | 13927.80 | 0.93 8 | 14931 | 1093 | 13660.60 | 0.46 9 | 14900 | 1130 | 13185.80 | 0.67

    these stats are collected on listener and sent back to dialer upon request. Each iteration is attempting to send 15k messages. So if I 4k messages per iteration then receive side would give number in 1% less range. same for any number of messages i send. i thought m

    opened by vk-coder 26
  •  memory leak ( I am not sure if it was by nng)

    memory leak ( I am not sure if it was by nng)

    pattern: pair v1 poly mode. transport: tcp server: pair v1 listen .VS2013 X64 compile. (win server 2016) count:1 clients: pair v1 dial (win 7) count: about 15K+ clients keep getting online and offline all the time , and the memory increases...

    /////////////////////////////////// extra information: every client keep sending a msg (length is 1) to server every 10 minutes(HeartBeat). I doubt if this result in the memory leak of server.

    ///////////////////////// extra information-2: server p-code: for (;;) { nng_recvmsg; nng_pipe pi =nng_msg_get_pipe(msg); nng_pipe_get_addr (opt remote addr); } you see, I didn't close the pi. I doubt if this leaks. I will read nng docs continusly. /////end -add -extra2

    I have found out one of the leak code with windbg:

    static int tcptran_pipe_alloc(tcptran_pipe **pipep) { tcptran_pipe *p; int rv; if ((p = NNI_ALLOC_STRUCT(p)) == NULL) { return (NNG_ENOMEM); } nni_mtx_init(&p->mtx); if (((rv = nni_aio_alloc(&p->txaio, tcptran_pipe_send_cb, p)) != 0) || ((rv = nni_aio_alloc(&p->rxaio, tcptran_pipe_recv_cb, p)) != 0) || ((rv = nni_aio_alloc(&p->negoaio, tcptran_pipe_nego_cb, p)) != 0)) { tcptran_pipe_fini(p); return (rv); }

    the "p->rxaio" didn't free all the time, I don't know why.

    nng x64 leak callstack -1 : (leak block size:1d8)===="tcptran_pipe_alloc" leak

    00000285784a0cb0 0020 0000 [00] 00000285784a0ce0 001d8 - (busy) 7ffce6c20543 ntdll!RtlpCallInterceptRoutine+0x000000000000003f 7ffce6bbeee2 ntdll!RtlpAllocateHeapInternal+0x0000000000001142 *** WARNING: Unable to verify checksum for C:\test_Svr.exe 7ff7b83e40d5 test_Svr!_calloc_impl+0x000000000000005d 7ff7b840cca9 test_Svr!calloc+0x0000000000000015 7ff7b83ec777 test_Svr!nni_aio_init+0x0000000000000027 7ff7b83fc3d0 test_Svr!nng_tcp_register+0x0000000000000460 (in fact, it's nni_aio_alloc +xx) 7ff7b83fbff3 test_Svr!nng_tcp_register+0x0000000000000083 ( in fact ,it's tcptran_pipe_alloc+xx) 7ff7b83f69a9 test_Svr!nni_taskq_sys_init+0x00000000000000a9 7ff7b83f2ef4 test_Svr!nni_thr_run+0x0000000000000094 7ff7b83ebfc8 test_Svr!nni_plat_thr_is_self+0x0000000000000038 7ff7b840caeb test_Svr!_callthreadstartex+0x0000000000000017 7ff7b840cc92 test_Svr!_threadstartex+0x0000000000000102 7ffce67f84d4 KERNEL32!BaseThreadInitThunk+0x0000000000000014 7ffce6bfe851 ntdll!RtlUserThreadStart+0x0000000000000021

    nng x64 leak callstack -2 : (leak block size:4d8) (this callstack is corrent, no analysis mistake)

    0:031> !heap -p -a 000002856b618970
    address 000002856b618970 found in _HEAP @ 28556500000 HEAP_ENTRY Size Prev Flags UserPtr UserSize - state 000002856b618940 0051 0000 [00] 000002856b618970 004d8 - (busy) 7ffce6c20543 ntdll!RtlpCallInterceptRoutine+0x000000000000003f 7ffce6bbeee2 ntdll!RtlpAllocateHeapInternal+0x0000000000001142 7ff7b83e40d5 test_Svr!_calloc_impl+0x000000000000005d 7ff7b840cca9 test_Svr!calloc+0x0000000000000015 7ff7b83efd5e test_Svr!nni_pipe_sys_init+0x000000000000008e 7ff7b83efaad test_Svr!nni_pipe_create_listener+0x000000000000003d 7ff7b83f05be test_Svr!nni_listener_add_pipe+0x000000000000004e 7ff7b83eeef4 test_Svr!nni_dialer_sys_init+0x00000000000000b4 7ff7b83f69a9 test_Svr!nni_taskq_sys_init+0x00000000000000a9 7ff7b83f2ef4 test_Svr!nni_thr_run+0x0000000000000094 7ff7b83ebfc8 test_Svr!nni_plat_thr_is_self+0x0000000000000038 7ff7b840caeb test_Svr!_callthreadstartex+0x0000000000000017 7ff7b840cc92 test_Svr!_threadstartex+0x0000000000000102 7ffce67f84d4 KERNEL32!BaseThreadInitThunk+0x0000000000000014 7ffce6bfe851 ntdll!RtlUserThreadStart+0x0000000000000021`

    leak size 470:

    0:031> !heap -p -a 0000028565028690
    address 0000028565028690 found in _HEAP @ 28556500000 HEAP_ENTRY Size Prev Flags UserPtr UserSize - state 0000028565028660 004d 0000 [00] 0000028565028690 00470 - (busy) ? test_Svr!nni_win_tcp_init+220 7ffce6c20543 ntdll!RtlpCallInterceptRoutine+0x000000000000003f 7ffce6bbeee2 ntdll!RtlpAllocateHeapInternal+0x0000000000001142 7ff7b83e40d5 test_Svr!_calloc_impl+0x000000000000005d 7ff7b840cca9 test_Svr!calloc+0x0000000000000015 7ff7b840b589 test_Svr!nni_win_tcp_init+0x0000000000000029 7ff7b840a342 test_Svr!nni_tcp_listener_accept+0x00000000000000e2 7ff7b83fc02d test_Svr!nng_tcp_register+0x00000000000000bd 7ff7b83f69a9 test_Svr!nni_taskq_sys_init+0x00000000000000a9 7ff7b83f2ef4 test_Svr!nni_thr_run+0x0000000000000094 7ff7b83ebfc8 test_Svr!nni_plat_thr_is_self+0x0000000000000038 7ff7b840caeb test_Svr!_callthreadstartex+0x0000000000000017 7ff7b840cc92 test_Svr!_threadstartex+0x0000000000000102 7ffce67f84d4 KERNEL32!BaseThreadInitThunk+0x0000000000000014 7ffce6bfe851 ntdll!RtlUserThreadStart+0x0000000000000021

    Okay, in fact , the top 3 size block leaks most of my memory. the windbg heap stats is below:

    0:031> !heap -stat -h 000001a363440000 heap @ 000001a363440000 group-by: TOTSIZE max-display: 20 size #blocks total ( %) (percent of total busy bytes) 1d8 36aeab - 64d20b48 (43.40) 4d8 7cfce - 25d68dd0 (16.29) 470 7cfd4 - 22aa3cc0 (14.92) 58 36aec5 - 12cc13b8 (8.09) 138 7cfd8 - 9854f40 (4.10) 100 7cfd9 - 7cfd900 (3.36) 78 7cfcf - 3a96908 (1.58) 1800000 2 - 3000000 (1.29) ed4100 3 - 2c7c300 (1.20) 48 7cfc3 - 2326ed8 (0.95) 219dbe3 1 - 219dbe3 (0.90) 68 5106f - 20ead18 (0.89) 80010 34 - 1a00340 (0.70) 30 7cff2 - 176fd60 (0.63) 40 59075 - 1641d40 (0.60) 46 5106c - 1627d88 (0.60) 20 7cfd5 - f9faa0 (0.42) 3ff70 c - 2ff940 (0.08) 4059f 1 - 4059f (0.01) 40000 1 - 40000 (0.01)

    opened by jimmybutler1201 24
  • Running multiple Bus tests causes error code 3

    Running multiple Bus tests causes error code 3

    Taken individually, InProc, IPv4, or IPv6, the tests all run just fine. But run in parallel causes problems.

    2017.11.04 18:36:17.392   ERROR Process C:\Users\Michael\AppData\Local\JetBrains\Installations\ReSharperPlatformVs14\JetBrains.ReSharper.TaskRunner.CLR45.x64.exe:13608 exited with code '3'.
    

    However, by contrast, the pipeline tests are running just fine, individually, or altogether in parallel (same set as above).

    Generally, I am following the same approach as the C code, closer to the C++ code, probably even a bit "simpler" due no smart pointer gymnastics going on. I am also not bothering to vet any of the protocol/peer stuff, just focusing on messaging, the socket protocols, etc.

    I'll have my repo committed this weekend so you can take a look.

    opened by mwpowellhtx 24
  • enhancement: hub protocol

    enhancement: hub protocol

    Cogent Embedded created new protocol (hub) and would like to share it.

    Hub protocol looks like bus, but it is implemented with guarantees for send / receive messages. For our purposes we made it compatible with pair0

    opened by apokriff 0
  • src: sp: tcp: Finish receive aio on close

    src: sp: tcp: Finish receive aio on close

    Issue: I use bus protocol on tcp. Programm some times hangs on closing socket.

    This change fixes hangs on socket close. I saw hang with bus protocol on close from time to time. It hangs in bus0_pipe_stop() on nni_aio_stop(&p->aio_recv); As I found protocol's aio calls on stopping tcptran_pipe_recv_cancel() that should remove it from tcp->recvq, but tcp->rxaio is stopped at that time by tcp close function. As I understand tcptran_pipe_recv_cb() some times is called in between of setting up closed flag and real stopping of tcp->rxaio. This change should fix handling of this scenario

    opened by apokriff 0
  • cmake find_package fails due to too restrictive permissions of /usr/local/lib64/cmake directory

    cmake find_package fails due to too restrictive permissions of /usr/local/lib64/cmake directory

    Describe the bug Trying to build my code with nng. When I put this line to my CMakeLists.txt:

    find_package(nng CONFIG REQUIRED)
    

    then cmake fails, it cannot find the necessary cmake files:

      Could not find a package configuration file provided by "nng" with any of
      the following names:
    
        nngConfig.cmake
        nng-config.cmake
    
      Add the installation prefix of "nng" to CMAKE_PREFIX_PATH or set "nng_DIR"
      to a directory containing one of the above files.  If "nng" provides a
      separate development package or SDK, be sure it has been installed.
    

    Expected behavior find_package should run successfully.

    Actual Behavior find_package fails.

    To Reproduce Create a CMakeLists.txt which calls find_package for nng

    ** Environment Details **

    cat /etc/os-release
    NAME="SLES"
    VERSION="15-SP4"
    VERSION_ID="15.4"
    PRETTY_NAME="SUSE Linux Enterprise Server 15 SP4"
    ID="sles"
    ID_LIKE="suse"
    ANSI_COLOR="0;32"
    CPE_NAME="cpe:/o:suse:sles:15:sp4"
    DOCUMENTATION_URL="https://documentation.suse.com/"
    

    nng build with master 5385b788d28f42078b7fd342ab241e2043e158f5

    g++ --version
    g++ (SUSE Linux) 11.3.0
    Copyright (C) 2021 Free Software Foundation, Inc.
    This is free software; see the source for copying conditions.  There is NO
    warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
    

    Additional context

    I found that nng cmake files are installed, but the cmake directory is not accessible.

    /usr/local/lib64/cmake> ls -l
    total 0
    drwxr-x--- 1 root root 166 Dec 19 08:53 nng
    

    I had to do chmod 755 to the directory and then it was OK.

    opened by szevzol 1
  • NNG Architecture and Constraints

    NNG Architecture and Constraints

    Hello there:

    Modularity and code oraganization is one of the most important design criteria which should be considered by early beginning.

    In short, how is it possible to design and implement a DNS service in NNG?

    At first glance, it may seems not required. But, when considering scenarios which demand such capability, it may seems required.

    The boundary between hard coding and degree of flexibility is not easily determined but is something which should be paid sufficient attention.

    Thank you for all efforts and contribution. Omid

    question 
    opened by chip5441 1
  • NNG Socket Concept and Endpoints

    NNG Socket Concept and Endpoints

    After going through "NNG Manual" and a quick look at source code, I think there are few shortcomings which should be taken into consideration for better understanding, for people using this lovely library to develop efficient and elegant products on top of it.

    In "NNG Manual / Conceptual Overview" mentioned that, "Each socket can have zero, one or many endpoints, which are either listeners or dialers. (A given socket may freely choose whether it uses listeners, dialers or both.)"

    Correspondingly, we could have a socket with different endpoints, listeners and dialers, bound to it on different addresses. Right? If correct, what if it is required to send different messages from that single socket with different dialers to different peers, selectively? Or how to filter incoming messages from different peers received through listeners, selectively?

    According to interface, a dialer could be queried by its "id" and then associated socket could be retrieved through another function call. Is this socket different from the original socket by which dialer has been created? And same for listeners. This whole story is not about pipes but using the interface, its design and the perception.

    Another thing I would like to discuss at this moment is about "NNG" aio. "NNG" is getting its way and many developers are using it in variety of projects. One thing which is very common among them, I think, is using only scaling protocols without using "aio" for asynchronous operations. I think one of the major reasons for such decision making is about execution mechanism. I think it would be a game changer if execution mechanism of "NNG" could be configurable and make it possible to integrate an execution mechanism of choice.

    I know "NNG" is a journey and still in progress. But key decision makings and priorities could put it in front of queue as users have a basket with limited resources and different utility functions.

    Thank you for all your efforts and contribution. Omid

    question 
    opened by chip5441 1
  • [question] How to run the server normally when running in REQ mode?

    [question] How to run the server normally when running in REQ mode?

    I want to design a reliable delivery configuration function. Select the REQ/REP model.

    Scene conditions: 1, The client connects to the server, and clinet runs in the REP model 2, Server listens to the port, the server runs in REQ mode, and will actively deliver the configuration to the client at a specific time

    I have a question, in the server, how can I save the context corresponding to a specific client for subsequent sending messages?

    question 
    opened by waxwax0099 3
Releases(v1.5.2)
  • v1.5.2(Aug 11, 2021)

    This release addresses a number of issues with 1.5.1 and 1.5.0, and users of those versions are encouraged to upgrade.

    • MbedTLS 3.0 is now supported
    • Several bugs in the aio subsystem leading to hangs or performance issues are addressed
    • Possible crash due to mismatched nni_strdup/free usage fixed
    • Fix for building on iOS, tvOS, iPadOS, watchOS
    • Incorrect version number macros in CMake configuration fixed
    • Several other minor cleanups (remove dead code, simplify some things)
    Source code(tar.gz)
    Source code(zip)
  • v1.5.1(Jul 11, 2021)

  • v1.5.0(Jul 10, 2021)

    This release provides a two new convenience APIs, nng_msg_reserve() and nng_msg_capacity(), which can help with avoiding preallocations.

    Additionally this release fixes a bug introduced in v1.4.0 where setting IPC socket permissions on Linux did not work.

    Source code(tar.gz)
    Source code(zip)
  • v1.4.0(Feb 8, 2021)

    This is principally a performance release, as we have introduced a bunch of new features that should improve performance, especially for higher end systems (those with multiple cores will see substantially improved scalability, and lower latencies.)

    Other features:

    • TCP ports may now be specified as service names.
    • wss4, wss6, ws4, and ws6 can be used to force IPv6 or IPv4 binding for websocket URLs.
    • REQ will fail fast if no retry timer is present, and the peer disconnects
    • abstract sockets can be used on Linux (see nng_ipc.7 for details)
    • websocket stream mode now supports TEXT mode streams
    • thread names can be set, and NNG will set names for its own (see nng_thr_setname.3)
    • IPv6 scoped addresses are supported
    • nngcat grew --file and --data options to supply data to send

    Thanks.

    Source code(tar.gz)
    Source code(zip)
  • v1.3.2(Aug 2, 2020)

    This release is just a set of improvements to fix some documentation bugs. These fixes are necessary for some of the automatic tooling we use for publication of documentation.

    If already running v1.3.1, there is no urgency to update.

    Source code(tar.gz)
    Source code(zip)
  • v1.3.1(Jul 28, 2020)

    This is a bug fix release that rolls up a bunch of bug fixes.

    • WebSocket and HTTP support for IPv6 addresses (note: IPv6 scopes are still not supported) (#844, #1224)
    • Build fixes for NetBSD, OpenBSD, and Android (#1232, #1237)
    • Serious framing error in TLS (regression introduced in 1.3.0) (#1235)
    • nng_msg_clear was clearing the header; now it only clears the body (#1252)
    • Use-after-free segfault in rep protocol (#1241)
    • NNG_OPT_RECONNMAXT zero did not prevent exponential backoff (#1230)
    • Use-after-free in TLS (#1239)
    • Hangs in nng_close fixed (#1236, #1219)
    • Fixes to ease inclusion in other projects
    • Numerous minor doc bugs fixed
    • Various test suite bugs fixed

    Also there are two minor feature enhancements:

    • Support for obtaining the peer process ID using IPC on modern macOS
    • nngcat now supports data from standard input when the file is specified as "-" (#1007)

    It is recommended that all users using v1.3.0 upgrade to v1.3.1.

    Source code(tar.gz)
    Source code(zip)
  • v1.3.0(Mar 1, 2020)

    Features

    • Support for TLS 1.3 and external TLS providers. There is now an API whereby external "engines" can be developed for different TLS implementations. The first of these, for wolfSSL, is available. Set this with the NNG_TLS_ENGINE cmake option. The default is still to use Mbed TLS. The wolfSSL plugin is available under a different license (GPLv3 or commercial), and also provides support for TLS 1.3 and FIPS 140-2 capable solutions.

    • Message cloning and related performance improvements. This is not a feature, so much as a rather large improvement in terms of performance. All workloads will see some benefit -- some will see substantial benefits.

    • Numerous other performance improvements. Much effort was made to reducing allocations, improving cache effectiveness, and eliminating extra context switches. This work is not yet done, but this is a big step in the right direction.

    • HTTP Server support for "non-exclusive" registration -- a given handler may be registered as a fallback handler (e.g. for a directory), even if more specific handlers are registered.

    • Performance test programs grew more options to select different protocols and to change the URL to test across different transports.

    Notable Bug Fixes

    • Thread count is limited. Previously we would spawn potentially vast numbers of threads based on the number of available cores. By default we set an upper limit on this that is about 20 threads. Tune this with the NNG_MAX_TASKQ_WORKERS cmake option.

    • Raw mode fixes for XREQ and XRESPONDENT. These protocols used the raw mode inconsistently, leaving the header in the message body. This is corrected and the protocol headers are kept in the message headers. There is some small risk that some applications broken, but we are not aware of any that used RAW mode to parse message headers.

    • HTTP Server root URL handling had a few issues which are resolved.

    • Some platforms had difficult building due to inconsistencies in the handling of atomics.

    • Numerous test suites uncovered small (rare) races, etc. The tests themselves were often racy. Fixes to both NNG and the tests have been made, while increasing overall test coverage.

    • REP protocol with SENDFD was inconsistent (#1088).

    Other Changes

    • Polyamorous Pair v1 mode is changed, such that a new API call is needed to use it. Further, this mode will likely be removed in a future release. Note that this mode also does not work with other implementations, nor does it support operation with nng_device().

    • Maximum hop count across nng_device() proxies is now limited to 15. This limit should be sufficient for all reasonable configurations, and forcing this allowed us to to inline the header for performance reasons.

    • The nng_msg_options support was removed. It was not used for anything.

    Source code(tar.gz)
    Source code(zip)
  • v1.2.6(Feb 6, 2020)

    This is another release to fix builds for older compiles without support for C11 atomics.

    If you were able to build prior releases, there is no need to update to this release. Generally speaking, the use of older compilers will give less than ideal results; if at all possible upgrading to modern compilers is strongly recommended. The performance different is likely to be significant.

    Source code(tar.gz)
    Source code(zip)
  • v1.2.5(Jan 27, 2020)

    This release fixes a mistake that prevented the code from building for people on older compilers or older operating systems. Users of v1.2.4 do not need to upgrade.

    Source code(tar.gz)
    Source code(zip)
  • v1.2.4(Jan 13, 2020)

    This fix contains an URGENT bug fix, and all users using v1.2.x where x <= 3 should upgrade at their earliest opportunity.

    The bugs fixed are:

    • #1132 Masking error in LMQ leads to corruption
    • #1131 (openindiana) compile error
    • fix reported versions in nng.h and CMakeLists.txt

    The critical bug here is #1132 which leads to use after free heap corruption, and unpredictable results once the receive or send queue wraps. The other two bugs are less critical, but should help Solaris and illumos users, and correct a bug where we reported wrong the ABI version (which could cause compatibility problems when mixing older versions with v1.2.x.)

    Source code(tar.gz)
    Source code(zip)
  • v1.2.3(Dec 31, 2019)

    This release fixes a number of important bugs. It's highly recommended that folks using v1.2.0, v1.2.1, or v1.2.2 upgrade to this release.

    Specifically fixed:

    • #1079 Use after free in tcp dialer
    • #1075 WebSocket use after free
    • #1064 Potential deadlock in statistics code
    • #1065 resolver leaks work structures

    There were also several fixes to the test suite included with these changes.

    Source code(tar.gz)
    Source code(zip)
  • v1.2.0(Dec 22, 2019)

    Version 1.2 is a minor feature release of NNG.

    • Notably it includes support for non-SP protocol usages -- for example NNG can be used to make generic websocket or HTTP applications now.
    • The default maximum receive size is lifted. (If you use NNG on an untrusted network, please set an explicit limit instead of relying on the defaults!)
    • Substantial work on performance. Most protocols should see a nice boost.
    • Numerous bugs fixed.

    Hopefully the next release won't be so far in the future. Thanks to everyone for their patience.

    Source code(tar.gz)
    Source code(zip)
  • v1.2.2(Dec 28, 2019)

    This release fixes two bugs that were affecting some people:

    • Crashes in the websocket code. (#986) Folks using the websocket transport are highly encouraged to update. This bug could have led to either application crashes or data corruption in transit.

    • Windows client connections not supporting NNG_OPT_LOCADDR (#1062)

    This release also includes changes to the way tests are created and managed, so that more tests are run, they are easier to write, easier to collect coverage for, and easier to diagnose when they fail. This won't impact built libraries, but it should have a big improvement on the overall quality of the library over time. This will be an ongoing work.

    We're also making better use of GitHub actions to test on more systems, and more configurations. While this has helped to uncover some real bugs in the library, it has also started uncovering brittleness in the test suites themselves. This is the source of a lot of false failures; we will continue working on this going forward.

    Source code(tar.gz)
    Source code(zip)
  • v1.2.1(Dec 24, 2019)

    Version 1.2.1 is mainly being released to correct a problem with the semantic versioning in the shared library interfaces. It also corrects a problem that may have affect non-mainstream platforms, without support for atomic operations.

    Other changes should not affect library users.

    Source code(tar.gz)
    Source code(zip)
  • v1.1.1(Nov 22, 2018)

    Version 1.1.1 patch release.

    This fixes a few problems with 1.1.0.

    • The version number at build time was misreported as 1.0.1. It will now be 1.1.1.
    • Support for use in CMake scenarios involving add_subdirectory
    • Fix for the bug report URL if NNG crashes
    • Fix for a crash if a remote websocket peer sends PING requests
    Source code(tar.gz)
    Source code(zip)
  • v1.1.0(Nov 10, 2018)

    There are numerous bug fixes and improvements in this since 1.0.1.

    DNS resolution is done asynchronously at dial time, leading to better self healing in the face of changing DNS records (and the possibility to use DNS for round-robin load balancing.)

    Better peer identification is possible with IPC based transports.

    The HTTP framework has better support for cancellation, and the HTTP client has a much friendlier ability to execute transactions. Additionally, the HTTP client can now support servers that insist on sending chunked transfer encodings.

    The ZeroTier transport received a lot of work, so that it is far more stable, and supports more properties relating to peer identification. There are (undocumented) options to tune the local IP addresses used in ZeroTier as well. Also, the entire configuration process for ZeroTier is much saner.

    A statistics API is available to obtain statistics about the application. Unfortunately, only a few statistics have been added -- more will be added in coming releases.

    More modern CMake style is used to provide much more correct use from CMake projects - it should no longer be necessary to call find_package(Threads) or similar.

    Better support for more platforms (older versions of Linux, Alpine Linux, QNX).

    A variety of stability fixes and performance improvements.

    A number of documentation improvements and corrections have been made as well.

    Source code(tar.gz)
    Source code(zip)
  • v1.1.0-rc(Nov 6, 2018)

    This is a release candidate for 1.1.0.

    There are numerous bug fixes and improvements in this since 1.0.1.

    DNS resolution is done asynchronously at dial time, leading to better self healing in the face of changing DNS records (and the possibility to use DNS for round-robin load balancing.)

    Better peer identification is possible with IPC based transports.

    The HTTP framework has better support for cancellation, and the HTTP client has a much friendlier ability to execute transactions. Additionally, the HTTP client can now support servers that insist on sending chunked transfer encodings.

    The ZeroTier transport received a lot of work, so that it is far more stable, and supports more properties relating to peer identification. There are (undocumented) options to tune the local IP addresses used in ZeroTier as well. Also, the entire configuration process for ZeroTier is much saner.

    A statistics API is available to obtain statistics about the application. Unfortunately, only a few statistics have been added -- more will be added in coming releases.

    More modern CMake style is used to provide much more correct use from CMake projects - it should no longer be necessary to call find_package(Threads) or similar.

    A number of documentation improvements have been made as well.

    Source code(tar.gz)
    Source code(zip)
  • v1.0.1(Jul 5, 2018)

    This is a bug fix release, and contains fixes for several important bugs. Users of macOS and BSD operating systems should see a huge reduction in CPU overhead as a critical kqueue processing bug is fixed. The IPC transport also got a critical fix to eliminate accidental deletion of the socket incorrectly.

    There is one feature that snuck in (my bad!) which is that it is now possible to customize the HTTP error page -- see nng_http_server_set_error_file() or nng_http_server_set_error_page() for details.

    A few other minor bugs were fixed as well, including a leak in the URL, an inability to tune dialer reconnection timers individually.

    More stabilization work and feature work is pending, but these issues are important enough that we felt it worthwhile to push this release out while we finish up the work on the other issues.

    Source code(tar.gz)
    Source code(zip)
  • v1.0.0(Jun 8, 2018)

    This is the first "stable" release of NNG. While there are many more things to do in NNG, we are recommending users being transitioning to NNG away from nanomsg.

    Source code(tar.gz)
    Source code(zip)
  • v1.0.0-rc.1(Jun 2, 2018)

    This is a release candidate. At this point we consider ourselves ready for 1.0.0 final release. This does have some changes in it, so we will wait a little bit before the final release:

    Changes since beta.2:

    • Honors BUILD_SHARED_LIBS CMake setting (see below)
    • Support for Android added
    • Instructions for building with Xcode for iOS (no code changes were needed)
    • Support for enabling sanitizers (address, thread, memory, undefined) via NNG_SANITIZER
    • Fixes for bugs found with sanitizers (use-after-free, leaks)
    • Simplified synchronization primitives on POSIX (no fallback needed)
    • CMake exported target (easier integration within CMake projects) & demos
    • Shared library symbol visibility is reduced by default
    • Documentation updates (various) found during editing

    Essentially this project should be easier to use, provided you have a sane CMake installation. It attempts to be a well-behaved citizen in CMake-land, so should be easier to nest into other projects.

    The one very important change is that we use the platform-specific defaults to decide whether to build shared or static libraries, and only build either shared or static -- not both. The determination of which to build is driven by the BUILD_SHARED_LIBS macro, which is user-accessible via CMake. This is more closely aligned with usual CMake practice.

    Also, if you were using our library to access private symbols, that is a no-no, and may not work anymore, as the shared library is now more careful to only export the symbols we have intended for external use. (Build and use a static library if you need to work around that, but you really should only be using the public symbols.) This was already the default behavior for Windows.

    Source code(tar.gz)
    Source code(zip)
  • v1.0.0-beta.2(May 22, 2018)

    This is the second beta release of NNG 1.0.0.

    NNG is now on a functional code freeze. Furthermore, the ABI is now frozen (library version 1.0.0).

    We will very carefully evaluate any new bugs found before this is released for GA; only severe bugs will be considered for integration at this time. If such a bug is found and a fix is required, we will release another beta release.

    Permitted changes include test programs, demonstration programs, and documentation fixes and improvements. Also, minor changes to improve the release process, packaging, and non-runtime aspects of the library may be permitted, subject to review.

    We expect to release one more change, which will be a release candidate (RC1) before final release. More details about release plans will be posted to the mailing list.

    What's changed since 1.0.0-beta.1?

    • Library ABI is now 1.0.0
    • Websocket transport now honors NNG_OPT_RECVMAXSIZE
    • Legacy mode headers are now more compatible with legacy nanomsg
    • SUB protocol is more aggressive at dropping undeliverable messages
    • Compat mode supports NN_TCP_NODELAY
    • NNG_OPT_TCPNODELAY and NNG_OPT_KEEPALIVE defaults are set at socket creation
    • Bug fix in the websocket URL parsing (if no path is given to websocket URL)
    • SO_REUSEADDR is now set on listening TCP sockets on POSIX systems
    • Various improvements and corrections to documentation

    Thanks for your continued testing and support!

    Source code(tar.gz)
    Source code(zip)
  • v1.0.0-beta.1(May 18, 2018)

    Good news! NNG 1.0.0 has reached development complete, and is now entering its first beta period.

    During this beta period, no new features will be introduced into the this release. Furthermore, changes to the functional code of the library itself will be restricted to bug fixes. Any bugs requiring non-trivial changes to the library will necessitate another beta cycle before entering the release candidate phase.

    Permitted changes include test programs, demonstration programs, and documentation fixes and improvements. Also, some changes to improve the release process, packaging, and non-runtime aspects of the library are permissible.

    What's new since 0.9.0?

    • NNG_TCP_OPT_NODELAY option (and legacy compat)
    • Socket, pipe, and similar types are now structural instead of integers
    • Pipe notification callbacks. See nng_pipe_notify(3) for details.
    • Peer identification option for IPC transport (see nng_ipc(7) for details.)
    • Solaris and illumos support, including scalable port event based poller.
    • New options to nngcat(1) to control repeat count and message size.
    • Significant changes that should reduce lock contention and increase performance and scalability.
    • New tcp4://, tcp6://, tls+tcp4://, and tls+tcp6:// scheme support. See nng_tcp(7) and nng_tls(7).

    Numerous bugs have been fixed, supported by much more extensive testing and significant improvements to the test suite itself.

    Thank you for helping us ensure the highest quality release by testing this out!

    Source code(tar.gz)
    Source code(zip)
  • 0.9.0(Apr 25, 2018)

    The big change for this release is context support used for REQ/REP to SURVEYOR/RESPONDENT. With this change, there is very little reason for raw sockets to be used, except in creation of devices and obscure test and debug scenarios.

    A few bugs, some serious, have been fixed -- most notably a serious regression in REQ/REP caused by the original context work -- this turned out to have sweeping ramifications, and small changes were made across the code base to resolve the key issue (which was the need to separate nni_aio_start into two separate functions, which are now nni_aio_begin and nni_aio_schedule. Note that these two functions are for internal use only.)

    Additionally, the compatible API is now documented.

    At this point we consider nng "feature complete" for a 1.0 release; we will be focusing on performance, bugs, test cases, and demonstration code for the rest of the lead-up to 1.0. (We do have post-1.0 features planned, of course!)

    Note that there are still some edge case features missing from nng which are present in nanomsg -- statistics, pipe priorities, certain socket options (TCP_NODELAY for example), and security attributes (named pipes on Windows only). We may still add support for some of these missing things before 1.0 depending on risk assessment. Otherwise we will do them in the following release. (If any of these features are a stopper for you, and you are using them with nanomsg, please let us know!)

    Source code(tar.gz)
    Source code(zip)
  • 0.8.0(Apr 12, 2018)

    This release has what should be the final breaking API changes prior to 1.0. Specifically, the RAW mode sockets are no longer established by NNG_OPT_RAW -- which has become a read-only option, but are instead created by new constructors, such as nng_req0_open_raw().

    Additionally, this release introduces a major new capability -- contexts. The details of this are located in the nng_ctx(5) manual page, as well as an example. An example showing the use of contexts with the HTTP server API to create a REST API to REQ/REP gateway exists in the demo/rest directory. At the moment contexts are only available for REQ and REP protocols -- addition of support for contexts with SURVEYOR and RESPONDENT is planned for the next release.

    Note that users of the REQ and REP protocols may notice differences in behavior around queueing and flow control -- in particular REQ will probably apply backpressure now, so that attempts to send requests will block if there are no pipes available (and the pollable status will reflect this.)

    Significant performance improvements have been made -- users may see up to 20% better performance compared to 0.7.0, depending on numerous factors. More work in this area is still planned.

    A number of other issues (bugs) have been fixed as well.

    We would be especially grateful for any extra testing, especially of the REQ and REP protocols and the new context facility.

    Source code(tar.gz)
    Source code(zip)
  • 0.7.0(Mar 19, 2018)

    This release has some breaking changes for users of properties such as NNG_OPT_RAW and NNG_OPT_PAIR1_POLY, as well as users of the ZeroTier transport.

    Most of the other changes in this release amount to massive documentation improvements.

    However, Linux now uses epoll(), thanks to a contribution by Liam Staskawicz, which should lead to greater scalability.

    There is a new nng_sleep_aio() API, and a demo/async program that demonstrates using the async AIO features.

    Please also note that the documentation on the website and on Leanpub has all been updated to reflect this.

    We are getting much closer to a 1.0 release, so I'm really grateful for testing and review feedback!

    Thanks!

    Source code(tar.gz)
    Source code(zip)
  • 0.6.0(Mar 11, 2018)

    This release involves primarily bug fixes for the ZeroTier transport, and documentation updates.

    Applications using the ZeroTier transport option NNG_ZT_NETWORK_STATUS should be aware that this option has changed; code updates will be required. See the nng_zerotier(7) man page for more details.

    Source code(tar.gz)
    Source code(zip)
  • 0.5.0(Mar 2, 2018)

    This release adds the following:

    • nngcat utility
    • many manual pages
    • kqueue based BSD poller (performance & scalability)
    • public HTTP API
    • Improvements to the TLS API
    • Numerous bug fixes

    We believe this release is getting reasonably close to being ready to stabilize for a public API.

    Testing and review is requested!

    Source code(tar.gz)
    Source code(zip)
  • 0.2.0(Dec 31, 2017)

    This version adds quite a lot since 0.1.0, but most notable is the addition of websocket (ws://) and websocket secure (wss://) support.

    Please be careful and don't use this in production yet, but it is suitable for development and testing. There might still be API breakage in the future, but things are looking more and more stable now.

    Source code(tar.gz)
    Source code(zip)
  • 0.1(Nov 21, 2017)

    This is the first "pre-release" of nng. A lot may still change (even the project name!), but most of the core functionality is now working well enough for us to post this for users to test against.

    Please note that 0.1 means what it says -- you should not depend on this for anything, and we will probably break things in future releases -- there are no semantic version guarantees here.

    Source code(tar.gz)
    Source code(zip)
Owner
nanomsg
Nanomsg Project
nanomsg
Harsh Badwaik 1 Dec 19, 2021
pugixml is a Light-weight, simple and fast XML parser for C++ with XPath support

pugixml is a C++ XML processing library, which consists of a DOM-like interface with rich traversal/modification capabilities, an extremely fast XML parser which constructs the DOM tree from an XML file/buffer, and an XPath 1.0 implementation for complex data-driven tree queries. Full Unicode support is also available, with Unicode interface variants and conversions between different Unicode encodings (which happen automatically during parsing/saving).

Arseny Kapoulkine 3.3k Jan 8, 2023
Poseidon OS (POS) is a light-weight storage OS

Poseidon OS Poseidon OS (POS) is a light-weight storage OS that offers the best performance and valuable features over storage network. POS exploits t

null 59 Jan 5, 2023
A light-weight Flutter Engine Embedder based on HADK ,which for Android devices that runs without any java code

flutter-hadk A light-weight Flutter Engine Embedder based on HADK ,which for Android devices that runs without any java code 1.Build by android-ndk-to

null 12 Jun 15, 2022
Light-weight UNIX backdoor

JadedWraith Lightweight UNIX backdoor for ethical hacking. Useful for red team engagements and CTFs. Something I wrote a few years ago as part of a ga

null 128 Aug 24, 2022
Analytics In Real-time (AIR) is a light-weight system profiling tool

Analytics In Real-time Analytics In Real-time (AIR) is a light-weight system profiling tool that provides a set of APIs for profiling performance, lat

null 2 Dec 14, 2022
Ducktape is an Open source Light weight 2d Game Engine that gives utmost priority to user convenience.

Ducktape is an Open source Light weight 2d Game Engine that gives utmost priority to user convenience. It is written in c++ and uses SFML and Box2d for graphics and physics respectively.

Ducktape 102 Dec 30, 2022
qpSWIFT is a light-weight sparse quadratic programming solver

qpSWIFT Light-weight sparse Quadratic Programming Solver Introduction qpSWIFT is light-weight sparse Quadratic Programming solver targetted for embedd

qpSWIFT 72 Dec 17, 2022
A light-weight music Discord bot using Orca.

What's the "Music Discord bot with C"? A light-weight music Discord bot using Orca for it's bot. It's easy to use and install. How to download and use

ThePedro 12 Dec 7, 2022
A very simple and light-weight drawing app made with qt and C++.

Blackboard A very simple and light-weight drawing app made with qt and C++. It supports tablet and pen pressure with the help of QTabletEvents. So you

null 1 Nov 15, 2021
Fast and Light-weight path smoothing methods for vehicles

path_smoother About Fast and Light-weight path smoothing methods for vehicles Denpendencies This project has been tested on Ubuntu 18.04. sudo apt-get

MingwangZhao 4 Dec 1, 2021
Simple, Fast, Light weight

Welcome To PradoshOS Github! Index Main heading Setup Step 1 Step 2 Step 3 Step 4 Compilation of Bootloader Compilation of Kernel Compilation of Userl

S Pradosh 11 Nov 12, 2022
A light-weight json parser.

pson pson is a lightweight parser and it support six type, null , bool, number, string, array, object, and it can parse the encoding of UTF-8. It's fa

pinK 17 Dec 1, 2022
zlib replacement with optimizations for "next generation" systems.

zlib-ng zlib data compression library for the next generation systems Maintained by Hans Kristian Rosbach aka Dead2 (zlib-ng àt circlestorm dót org) C

zlib-ng 13 Dec 29, 2022
zlib replacement with optimizations for "next generation" systems.

zlib-ng zlib data compression library for the next generation systems Maintained by Hans Kristian Rosbach aka Dead2 (zlib-ng àt circlestorm dót org) C

zlib-ng 13 Dec 29, 2022
zlib replacement with optimizations for "next generation" systems.

zlib-ng zlib data compression library for the next generation systems Maintained by Hans Kristian Rosbach aka Dead2 (zlib-ng àt circlestorm dót org) C

zlib-ng 1.2k Dec 31, 2022
This repository accompanies Ray Tracing Gems II: Next Generation Rendering with DXR, Vulkan, and OptiX

Apress Source Code This repository accompanies Ray Tracing Gems II: Next Generation Rendering with DXR, Vulkan, and OptiX by Adam Marrs, Peter Shirley

Apress 684 Dec 29, 2022
FluidNC - The next generation of motion control firmware

FluidNC (CNC Controller) For ESP32 Introduction FluidNC is the next generation of Grbl_ESP32. It has a lot of improvements over Grbl_ESP32 as listed b

null 683 Jan 3, 2023