uTorrent Transport Protocol library

Related tags

BitTorrent libutp
Overview

libutp - The uTorrent Transport Protocol library.

Copyright (c) 2010 BitTorrent, Inc.

uTP is a TCP-like implementation of LEDBAT documented as a BitTorrent extension in BEP-29. uTP provides reliable, ordered delivery while maintaining minimum extra delay. It is implemented on top of UDP to be cross-platform and functional today. As a result, uTP is the primary transport for uTorrent peer-to-peer connections.

uTP is written in C++, but the external interface is strictly C (ANSI C89).

The Interface

The uTP socket interface is a bit different from the Berkeley socket API to avoid the need for our own select() implementation, and to make it easier to write event-based code with minimal buffering.

When you create a uTP socket, you register a set of callbacks. Most notably, the on_read callback is a reactive callback which occurs when bytes arrive off the network. The write side of the socket is proactive, and you call UTP_Write to indicate the number of bytes you wish to write. As packets are created, the on_write callback is called for each packet, so you can fill the buffers with data.

The libutp interface is not thread-safe. It was designed for use in a single-threaded asyncronous context, although with proper synchronization it may be used from a multi-threaded environment as well.

See utp.h for more details and other API documentation.

Example

See ucat.c. Build with:

make ucat

Building

uTP has been known to build on Windows with MSVC and on linux and OS X with gcc. On Windows, use the MSVC project files (utp.sln, and friends). On other platforms, building the shared library is as simple as:

make

To build one of the examples, which will statically link in everything it needs from libutp:

cd utp_test && make

Packaging and API

The libutp API is considered unstable, and probably always will be. We encourage you to test with the version of libutp you have, and be mindful when upgrading. For this reason, it is probably also a good idea to bundle libutp with your application.

License

libutp is released under the MIT license.

Related Work

Research and analysis of congestion control mechanisms can be found here.

Comments
  • Leaving SO_SNDBUF to the default causes transfer to hang

    Leaving SO_SNDBUF to the default causes transfer to hang

    I'm seeing a case where if I leave SO_SNDBUF on the sending socket to the default for version 1 (~3MB) or set it to a to 1MB, a utp transfer hangs and never. I was able to reproduce with utp_send and utp_recv by just commenting out this line:

    https://github.com/bittorrent/libutp/blob/master/utp_file/utp_send.cpp#L367

    and transferring a large file ~300MB. If I set the SO_SNDBUF to 100*300 the problem goes away. Here are the log entries from utp_send right before the hang:

    [3625] 0x00851b88: rtt:19 avg:23 var:3 rto:500 [3625] 0x00851b88: acks:2 acked_bytes:2764 seq_nr:44962 cur_window:1382 cur_window_packets:1 quota:-21293032 [3625] 0x00851b88: CheckTimeouts timeout:500 max_window:464223 cur_window:1382 quota:-21293032 state:CONNECTED_FULL cur_window_packets:1 bytes_since_ack:0 ack_time:-1879048172 [3686] 0x00851b88: CheckTimeouts timeout:439 max_window:464223 cur_window:1382 quota:-21293032 state:CONNECTED_FULL cur_window_packets:1 bytes_since_ack:0 ack_time:-1879048111 [3718] recv 127.0.0.1:7777 len:20 id:41 [3718] recv id:41 seq_nr:41 ack_nr:44961 [3718] 0x00851b88: recv processing [3718] 0x00851b88: Got ST_STATE. seq_nr:41 ack_nr:44961 state:CONNECTED_FULL version:1 timestamp:7658418 reply_micro:3940078 [3718] 0x00851b88: acks:1 acked_bytes:1382 seq_nr:44962 cur_window:1382 cur_window_packets:1 relative_seqnr:0 max_window:464223 min_rtt:112 rtt:23 [3718] 0x00851b88: actual_delay:3940078 our_delay:0 their_delay:0 off_target:99 max_window:464231 delay_base:3939998 delay_sum:0 target_delay:100 acked_bytes:1382 cur_window:0 scaled_gain:8.923907 rtt:23 rate:9284620 quota:-20991287 wnduser:3670016 rto:500 timeout:407 get_microseconds:3718642 cur_window_packets:1 packet_size:1382 their_delay_base:4291027386 their_actual_delay:4291027503 [3718] 0x00851b88: fast_resend_seq_nr:44962 [3718] 0x00851b88: got ack for:44961 (pkt_size:1382 need_resend:0) [3718] 0x00851b88: rtt:112 avg:35 var:24 rto:500 [3718] 0x00851b88: acks:1 acked_bytes:1382 seq_nr:44962 cur_window:0 cur_window_packets:0 quota:-20991287 [3718] 0x00851b88: CheckTimeouts timeout:500 max_window:464231 cur_window:0 quota:-20991287 state:CONNECTED_FULL cur_window_packets:0 bytes_since_ack:0 ack_time:-1879048079

    I'm not sure I follow why this is the case. Should we always set SO_SNDBUF?

    opened by bassam 13
  • Use _WIN32 instead of WIN32 to check for Windows

    Use _WIN32 instead of WIN32 to check for Windows

    MinGW defines both _WIN32 and WIN32 (and may even be the only compiler doing so). Microsoft and Intel compilers only define _WIN32. Use the common one to eliminate the need in defining WIN32 explicitly.

    opened by mikedld 9
  • Allow overrides of PACKED_ATTRIBUTE preprocessor macro

    Allow overrides of PACKED_ATTRIBUTE preprocessor macro

    This prevents libutp code from defining PACKED_ATTRIBUTE if it has already been defined. This greatly reduces noisy gcc warnings since to override definition - avoids gcc warnings about redefining a preprocessor macro when building client products since this macro is defined elsewhere in client code and has different conditions to its definition than this definition.

    @xercesblue @angitbit please review.

    opened by rmcdonald 8
  • Improved Makefiles

    Improved Makefiles

    shared library, grab environment AR, CXX, *FLAGS, ..., all for packaging purposes so I can make the transmission p2p client use the system copy of libutp

    would help a much if this was committed. grab the patch from here:

    http://dev.gentoo.org/~ssuominen/libutp-Makefile.patch

    opened by ssuominengentoo 8
  • -ansi in Makefile and snprintf()

    -ansi in Makefile and snprintf()

    The -ansi parameter caused snprintf() to not be defined (in a very strict environment)... and compilation of utp.cpp fails.

    snprintf() is defined in C99 but that is not a valid option for g++ (i.e. -std=c99 is the same as using no -ansi, but gets a warning).

    opened by rb07 8
  • PyuTp under Snow Leopard

    PyuTp under Snow Leopard

    I'm currently trying to test your implementation of uTP using the Python wrapper you provided. In doing so I encountered two problems: the first being that the struct.unpack call would throw this exception:

    IPAddr(struct.unpack("L", socket.inet_aton(ip))[0]) struct.error: unpack requires a string argument of length 8

    This at "sockaddr_types.py" line 80.

    However, I managed to solve that by changing the format of the unpack (fmt) to network(big endian) "!L".

    After that, I did not get that message anymore BUT python crashes due to a segmentation fault in the underlying utp implementation. I'm particularly expert in programming python applications which interface with C/C++ but I guess a wrong pointer is passed to the C uTP library. The library crashes when calling:

    utp.UTP_SetCallbacks(self.utp_socket, ctypes.byref(f), ctypes.py_object(self))

    at line 92 in the Socket class in file utp_socket.py. I attach to this e-mail the stack trace of the error.

    I'm currently using the pre-installed Python 2.6 under Mac OS Snow Leopard latest version. I tried also the last Python 2.7 from the official releases and I got the same kind of behavior. Then I tried to install it in a Linux machine and all went fine. However, I do really need to get this working on all platforms, windows included.

    Thank you very much. I really appreciate your work.

    opened by roverso 6
  • Centos 6.3 Build Error:

    Centos 6.3 Build Error: "conflicting types for ‘UTP_Write’"

    https://github.com/cfpp2p/libutp/commit/39d0263191ce6cccfdb72fb9f10ba4f5f51bedf9

    This bug was submitted by mmain Transmission ticket #5232 I'm passing it upstream.


    I have been successfully using Transmission on Centos 5.8, but now need to
     upgrade to Centos 6.3 and have started building new machine with 6.3 loaded
     and I am unable to build Transmission from source without following error
     during make(I also tried 2.75 with same
    result)
    
    tr-utp.c:65: error: conflicting types for ‘UTP_Write’
    ../third-party/libutp/utp.h:116: note:
     previous declaration of ‘UTP_Write’ was here
    
    make[1]: * [tr-utp.o] Error 1
    make[1]: Leaving directory `/root/transmission-2.76/libtransmission'
    make: * [all-recursive] Error 1
    
    ...
    
    Changed 16 hours ago by x190
    
    UTP_Write() is type bool. Maybe utp.h needs the following bit of voodoo that
     was added to transmission.h.
    
    #if !defined (__cplusplus)
     #ifdef HAVE_STDBOOL_H
      #include <stdbool.h>
     #elif !defined (__bool_true_false_are_defined)
      #define bool uint8_t
      #define true 1
      #define false 0
     #endif
    #endif
    
    
    comment:5 Changed 13 hours ago by mmain
    
    Yes, this worked a treat. Many Thanks
    
    
    comment:6 Changed 12 hours ago by jordan
    
    x190: that's the great idea, you ought to submit that upstream to the
     libutp project on github.
    
    opened by cfpp2p 5
  • µTP fails on compilers that don't honor

    µTP fails on compilers that don't honor "pragma pack"

    ticket mirrored from downstream: https://trac.transmissionbt.com/ticket/4260

    At least on the NAS devices Conceptronic CH3SNAS and DNS-323, µTP is not working. These devices run libuClibc 0.9.29, and Transmission was compiled with gcc 4.1.3.

    Transmission does not crash, but the log is filled with "Unexpected UDP packet" messages.

    The reason for this is that the structs defined in libutp don't have the correct size on this platform.

    I've attached a patch to fix the problem, although I can't speak for other platforms, so this should be tested beforehand.

    We actually found this issue a while ago during an IRC session (I believe livings was involved), and we found the solution right then and there. But apparently it was not checked into the trunk / reported upstream.

    opened by jordanl 5
  • Protect against broken implementations of monotonic time

    Protect against broken implementations of monotonic time

    This keeps Transmission from crashing on Mac OS X/powerpc. It seems like a good idea in any case. Please see https://trac.transmissionbt.com/ticket/4090 .

    The alternative would be to work around such issues in getMicroseconds itself, but I'd rather keep getMicroseconds as simple as possible -- it's already a mess as it is.

    opened by jech 5
  • Portable version of GetMicroseconds, take two

    Portable version of GetMicroseconds, take two

    This was brutally rebased.

    I've taken into account two of your comments, Greg, and ignored the one about calling gettime twice. The two system calls are only performed the first time around, which doesn't cost much, and the alternative makes the code somewhat more messy. There's no doubt for me this is the right solution.

    --jch

    opened by jech 5
  • Allow appending CXXFLAGS from command line

    Allow appending CXXFLAGS from command line

    This allows appending CXXFLAGS from the command line when building libutp.

    $ make before patch:

    g++ -c -DPOSIX -I . -I utp_config_lib -fno-exceptions -fno-rtti -Wall -g  utp.cpp
    g++ -c -DPOSIX -I . -I utp_config_lib -fno-exceptions -fno-rtti -Wall -g  utp_utils.cpp
    rm -f libutp.a
    ar q libutp.a utp.o utp_utils.o
    ar: creating libutp.a
    ranlib libutp.a
    

    $ make after patch:

    g++ -c -DPOSIX -I . -I utp_config_lib -fno-exceptions -fno-rtti -Wall -g  utp.cpp
    g++ -c -DPOSIX -I . -I utp_config_lib -fno-exceptions -fno-rtti -Wall -g  utp_utils.cpp
    rm -f libutp.a
    ar q libutp.a utp.o utp_utils.o
    ar: creating libutp.a
    ranlib libutp.a
    

    make CXXFLAGS=-fPIC before patch:

    g++ -c -DPOSIX -I . -I utp_config_lib -fPIC utp.cpp
    g++ -c -DPOSIX -I . -I utp_config_lib -fPIC utp_utils.cpp
    rm -f libutp.a
    ar q libutp.a utp.o utp_utils.o
    ar: creating libutp.a
    ranlib libutp.a
    

    make CXXFLAGS=-fPIC after patch:

    g++ -c -DPOSIX -I . -I utp_config_lib -fno-exceptions -fno-rtti -Wall -g -fPIC utp.cpp
    g++ -c -DPOSIX -I . -I utp_config_lib -fno-exceptions -fno-rtti -Wall -g -fPIC utp_utils.cpp
    rm -f libutp.a
    ar q libutp.a utp.o utp_utils.o
    ar: creating libutp.a
    ranlib libutp.a
    
    opened by robgjansen 4
  • Member access within misaligned address for type 'UTPSocketKeyData', which requires 8 byte alignment

    Member access within misaligned address for type 'UTPSocketKeyData', which requires 8 byte alignment

    Capture d’écran 2022-12-01 à 22 42 59

    Using UndefinedBehaviorSanitizer, we systematically find runtime errors of the like:

    SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /Users/User/Developer/libutp/utp_internal.cpp:2574:77 in /Users/User/Developer/libutp/utp_internal.cpp:2574:77: runtime error: store to misaligned address 0x000106f5c504 for type 'UTPSocket *', which requires 8 byte alignment

    opened by Coeur 0
  • libutp in a multi-thread application

    libutp in a multi-thread application

    Hello,

    Thank you for your efforts on this library. It's a lot of work.

    This is not really a bug report, it could be seems as a feature request, or more likely, it's a call for advice.

    I'm trying to use libutp in a multi-thread applications with multiple UTP communication over a single UDP socket. I have read the warning in the paragraph Interface of the readme.

    I have one instance of:

    ` socketUDP = socket(AF_INET, SOCK_DGRAM, IPPROTO_UDP); bind(socketUDP, (struct sockaddr*)(&si_me), sizeof(si_me)); ctx = utp_init(2); utp_set_callback(ctx, ...);

    struct pollfd pollfds[1]; pollfds[0].fd = socketUDP; pollfds[0].events = POLLIN; while (1) { poll(pollfds, 1, -1); if (pollfds[0].revents & POLLIN) { recvfrom(socketUDP, utp_process_udp(ctx, socket_data, len, (struct sockaddr *)&src_addr, addrlen); } } `

    I have multiple instances from different threads:

    utp_socket *sock = utp_create_socket(ctx); utp_connect(sock, (struct sockaddr *)&sin, sizeof(sin)); utp_write(sock, ...);

    I have in place a system NOT to mix the different 'write' buffers.

    Nevertheless, the library doesn't like it and I get crashes especially when utp_write happen at the same time.

    Is there an example or a paper suggesting how to use libutp in a multi-thread app?

    opened by gregoiregentil 0
  • Is there any format spec for utp version 0?

    Is there any format spec for utp version 0?

    Now I check the utp_internal.cpp, it seems that there is only version 1 in it. But today I want to identify utp protocol, and I know that mostly p2p client use utp version1 (? and I google for version 0 format, in wireshark it mentioned the foramt https://github.com/wireshark/wireshark/blob/73ecb86f4cb3b1737765885d028f2083e06eef1e/epan/dissectors/packet-bt-utp.c#L64 but it didn't mention the flags in it, and I check the history of utp.cpp. Maybe the flags combined of type and version (? Is there anyone know the format of utp version 0 or any resource about it?

    opened by joe820912boy 0
  • refactor: make utp_write() buf arg a const*

    refactor: make utp_write() buf arg a const*

    No functional changes; just a minor const-correctness change.

    Use case: client code calls utp_write() or utp_writev() from a function that receives the data in const form, but currently has to cast away the const with const_cast<void*>(data) to match the current libutp API.

    opened by ckerr 0
  • Assertion `mtu_floor <= mtu_ceiling' failed

    Assertion `mtu_floor <= mtu_ceiling' failed

    When an icmp fragmentation message arrives in utp_process_icmp_fragmentation() and the next_hop_mtu is less than mtu_floor, the assert(mtu_floor <= mtu_ceiling); in mtu_search_update() will go off.

    opened by ghazel 0
libTorrent BitTorrent library

LibTorrent Copyright (C) 2005-2014, Jari Sundell LICENSE GNU GPL, see COPYING. "libtorrent/src/utils/sha_fast.{cc,h}" is originally from the Mozil

Jari Sundell 844 Dec 10, 2022
GnuTLS implements the TLS/SSL (Transport Layer Security aka Secure Sockets Layer) protocol

GnuTLS implements the TLS/SSL (Transport Layer Security aka Secure Sockets Layer) protocol

Jonathan Bastien-Filiatrault 3 Jun 3, 2021
QUIC, a multiplexed stream transport over UDP

QUIC, a multiplexed stream transport over UDP QUIC is an experimental protocol aimed at reducing web latency over that of TCP. On the surface, QUIC is

Devsisters Corp. 1.7k Dec 31, 2022
Wavelike and Particlelike Phonon Transport (WPPT) Solver

'WPPT': A solver for Wave- and Particle-like Phonon Transport (WPPT) Authors: Zhongwei Zhang [email protected] / [email protected], Inst

Zhongwei Zhanng 10 Oct 13, 2022
Mongoose Embedded Web Server Library - a multi-protocol embedded networking library with TCP/UDP, HTTP, WebSocket, MQTT built-in protocols, async DNS resolver, and non-blocking API.

Mongoose - Embedded Web Server / Embedded Networking Library Mongoose is a networking library for C/C++. It implements event-driven non-blocking APIs

Cesanta Software 9k Jan 1, 2023
A cross-platform protocol library to communicate with iOS devices

libimobiledevice A library to communicate with services on iOS devices using native protocols. Features libimobiledevice is a cross-platform software

libimobiledevice 5.4k Dec 30, 2022
A protocol buffers library for C

PBC PBC is a google protocol buffers library for C without code generation. Quick Example package tutorial; message Person { required string name =

云风 1.6k Dec 28, 2022
FreeRDP is a free remote desktop protocol library and clients

FreeRDP is a free implementation of the Remote Desktop Protocol (RDP), released under the Apache license. Enjoy the freedom of using your software wherever you want, the way you want it, in a world where interoperability can finally liberate your computing experience.

null 7.7k Dec 31, 2022
Eclipse Paho C Client Library for the MQTT Protocol

Eclipse Paho C Client Library for the MQTT Protocol This repository contains the source code for the Eclipse Paho MQTT C client library. This code bui

null 2 Apr 27, 2022
A C library for RP2040-powered boards to control LCDs via the I2C protocol

picoi2clcd A C library to control I2C LCD displays using the Rapsberry Pi Pico (or really any board using the RP2040 microcontroller). License All of

Kosmas Raptis 2 Oct 16, 2021
A C++ wrapper library for the monome serialosc protocol

serialoscpp A C++ wrapper library for the monome serialosc protocol Credits oscpack: Ross Bencina http://www.rossbencina.com/code/oscpack serialosc: 2

Isabel 1 Nov 2, 2021
Multi-protocol Port Mapping client library

libplum - Multi-protocol Port Mapping client library libplum (Port Lightweight and Universal Mapping) is a library allowing to forward ports on Networ

Paul-Louis Ageneau 18 Dec 26, 2022
QuantumGate is a peer-to-peer (P2P) communications protocol, library and API written in C++.

About QuantumGate is a peer-to-peer (P2P) communications protocol, library and API. The long-term goal for QuantumGate is to become a platform for dis

Karel Donk 96 Dec 20, 2022
Async non-blocking multi-protocol networking library for C/C++

Fossa: Multi-Protocol Networking Library Note: As of September 21st 2015, Fossa project has been merged back into Mongoose project Fossa is a multi-pr

Cesanta Software 420 Dec 14, 2022
XQUIC Library released by Alibaba is a cross-platform implementation of QUIC and HTTP/3 protocol.

XQUIC 简体中文文档 README-zh-CN Introduction XQUIC Library released by Alibaba is … … a client and server implementation of QUIC and HTTP/3 as specified by

Alibaba 1.4k Dec 29, 2022
Protocol Buffers with small code size

Nanopb - Protocol Buffers for Embedded Systems Nanopb is a small code-size Protocol Buffers implementation in ansi C. It is especially suitable for us

null 3.3k Dec 31, 2022
Protocol Buffers - Google's data interchange format

Protocol Buffers - Google's data interchange format Copyright 2008 Google Inc. https://developers.google.com/protocol-buffers/ Overview Protocol Buffe

Protocol Buffers 57.5k Dec 29, 2022
Protocol Buffers implementation in C

Overview This is protobuf-c, a C implementation of the Google Protocol Buffers data serialization format. It includes libprotobuf-c, a pure C library

null 2.2k Dec 26, 2022
C/C++ language server supporting multi-million line code base, powered by libclang. Emacs, Vim, VSCode, and others with language server protocol support. Cross references, completion, diagnostics, semantic highlighting and more

Archived cquery is no longer under development. clangd and ccls are both good replacements. cquery cquery is a highly-scalable, low-latency language s

Jacob Dufault 2.3k Jan 7, 2023
Google Protocol Buffers tools (C code generator).

About Google Protocol Buffers tools in Python 3.6+. C source code generator. Rust source code generator ( ?? ?? ?? under construction ?? ?? ?? ). prot

Erik Moqvist 51 Nov 29, 2022