Tool for inspecting, creating and editing BitTorrent metafiles.

Overview

build Copr build status GitHub release (latest SemVer) C++ standard Codacy Badge License: MIT

A commandline tool for creating, inspecting and modifying bittorrent metafiles.

Features | Documentation | Packages | Building | License

Features

  • Creating bittorrent metafiles.
  • Inspecting bittorrent metafiles.
  • Verifying bittorrent metafiles against local data.
  • Editing existing bittorrent metafiles.
  • Support for the new v2 and hybrid protocols .
  • Support for tracker abbreviations.
  • Support for announce substitution parameters.
  • Fast multi-buffer hashing with Intel ISA-L.

Status

This project is under development. The commandline interface can change at any release prior to 1.0.0.

Performance

Following test were performed on a in in-memory filesystem with 1 MiB piece size and as target a folder with a few linux iso's totaling 19.0 GiB:

  • 9.0G | CentOS-Stream-8-x86_64-20201211-dvd1.iso
  • 2.0G | Fedora-Workstation-Live-x86_64-33-1.2.iso
  • 4.0G | openSUSE-Leap-15.2-DVD-x86_64.iso
  • 2.6G | ubuntu-20.04.1-desktop-amd64.iso
  • 998M | ubuntu-20.10-live-server-amd64.iso

The CPU is an Intel i7-7700HQ in a Dell XPS 15-9560 machine.

Benchmark

Documentation

Documentation is hosted on Github Pages.

Packages

Binary and source packages for Fedora 32, Fedora 33, Fedora 34, Fedora Rawhide and Centos stream are available in a COPR repo.

sudo dnf copr enable fbdtemme/torrenttools
sudo dnf install torrenttools

Binary and source packages for Ubuntu 20.04, Ubuntu 20.10, Ubuntu 21.04 are available in a launchpad repo.

sudo add-apt-repository ppa:fbdtemme/torrenttools
sudo apt-get update
sudo apt install torrenttools

A windows installer is available as an asset on the release page.

Building

This library depends on following projects:

Almost all dependencies can be fetched from github during configure time or can be installed manually. OpenSSL has to be installed on the system in advance.

Installing build dependencies

Ubuntu 20.04

sudo apt install build-essential git cmake g++-10 libtbb2 libtbb-dev libssl-dev 

Fedora 33

sudo dnf install cmake make g++ git openssl-devel libtbb-devel

Configuration

Option Type Description
TORRENTTOOLS_BUILD_TESTS Bool Build tests.
TORRENTTOOLS_BUILD_DOCS Bool Build documentation.
TORRENTTOOLS_INSTALL Bool Generate an install target.
DOTTORRENT_CRYPTO_MULTIBUFFER Bool Enable fast multi buffer hashing.

Building

This project requires C++20. Currently only GCC 10 is supported.

This project can be build as every other project which makes use of the CMake build system.

mkdir build
cd build
cmake -DCMAKE_BUILD_TYPE=Release ..
cmake --build . --target torrenttools

Installation

Installing the project:

sudo cmake --install . --component torrentttools

License

Distributed under the MIT license. See LICENSE for more information.

Comments
  • Infinite loop parsing LS_COLORS with 256 colors

    Infinite loop parsing LS_COLORS with 256 colors

    Hi, I 'm using torrenttools-0.3.2 and it works fine on my centos 7 server. But when i upgrade it to torrenttools-0.5.1 it is not working any more. It stuck on this

    Scanning target directory: 0 files processed
    Sorting file list... Done.
    Adding files to metafile... Done.
    
    

    ldd /usr/bin/torrenttools

    linux-vdso.so.1 =>  (0x00007ffeb21f0000)
    libtbb.so.2 => /lib64/libtbb.so.2 (0x00007f596e813000)
    libcrypto.so.1.1 => /usr/local/openssl/lib/libcrypto.so.1.1 (0x00007f596e34d000)
    libstdc++.so.6 => /usr/local/gcc8/lib64/libstdc++.so.6 (0x00007f596dfc9000)
    libm.so.6 => /lib64/libm.so.6 (0x00007f596dcc7000)
    libgcc_s.so.1 => /usr/local/gcc8/lib64/libgcc_s.so.1 (0x00007f596daaf000)
    libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f596d893000)
    libc.so.6 => /lib64/libc.so.6 (0x00007f596d4c5000)
    libdl.so.2 => /lib64/libdl.so.2 (0x00007f596d2c1000)
    librt.so.1 => /lib64/librt.so.1 (0x00007f596d0b9000)
    libz.so.1 => /lib64/libz.so.1 (0x00007f596cea3000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f596ea55000)
    

    I attached strace.log please take a look. Please any advise, thank you. strace.log

    bug 
    opened by vuduyhoang 9
  • macOS build fails with  dyld: lazy symbol binding failed: Symbol not found

    macOS build fails with dyld: lazy symbol binding failed: Symbol not found

    macOS fails with the following error

    [email protected] ~/Downloads> /usr/local/bin/torrenttools
    dyld: lazy symbol binding failed: Symbol not found: ___stack_logging_early_finished
      Referenced from: /Library/torrenttools/bin/../lib/libSystem.B.dylib
      Expected in: /Library/torrenttools/bin/../lib/libSystem.B.dylib
    
    dyld: Symbol not found: ___stack_logging_early_finished
      Referenced from: /Library/torrenttools/bin/../lib/libSystem.B.dylib
      Expected in: /Library/torrenttools/bin/../lib/libSystem.B.dylib
    
    fish: Job 1, '/usr/local/bin/torrenttools' terminated by signal SIGABRT (Abort)
    

    This is on an M1 mac with Rosetta installed

    opened by newadventure079 9
  • Reached max memory?

    Reached max memory?

    I use the include filter to select 1700 files from a folder with many more. A 700 file filter is no problem. I get the error Error: re2 compiler out of memory. I have 5GB available so I think I have enough for the regexp, but perhaps there's a configured cap somewhere?

    What about giving an option of increasing the max memory usage for re2?

    cat /proc/meminfo
    MemTotal:        8040328 kB
    MemFree:          134448 kB
    MemAvailable:    5704240 kB
    
    opened by mbalax11 5
  • Error: std::bad_alloc

    Error: std::bad_alloc

    Hey! The project looked extremely promising so I had to give it a go. Unfortunately I have run into an error on my first torrent creator.

    torrenttools create "E:\Docker\Deluge\Data\linux.iso" --piece-size 4M --announce "XXX" -p on -o "E:\Docker\Deluge\Data\linux.torrent"

    16.00 EiB in 0 directories, 1 files

    Error: std::bad_alloc

    bug windows 
    opened by RemuxKid 5
  • Build fails because of fmt

    Build fails because of fmt

    Hi, I tried building following the instructions but it fails: torrenttools/src/tracker_database.cpp:7:10: fatal error: fmt/args.h: No such file or directory

    I am on Arch Linux, and fmt is installed: fmt 7.1.3-1

    build 
    opened by quietvoid 5
  • config.yml mentions Windows and Linux but not macOS

    config.yml mentions Windows and Linux but not macOS

    macOS is left out of the following

    # This is an example configuration file.
    # It should be copied to the suitable directory for your OS and modified there.
    # Linux : $HOME/.config/torrenttools/config.yml
    # Windows : %AppData%/torrenttools/config.yml
    
    documentation 
    opened by newadventure079 4
  • how to double-check verify?

    how to double-check verify?

    I'm having a problem verifying some large torrents that a (doubled, tripled) verified by transmission but have either their unique large file or the last of their files listed as 99% by torrenttools.

    I was wondering if I had a way to get detailed output and/or instructions to debug this ?

    [I then picked torf (from your performance chart) and verified one torrent. It agrees with transmission so I'm inclined to think I found a bug. How can I help?]

    bug 
    opened by diligiant 3
  • Single Announce URL not getting added

    Single Announce URL not getting added

    Using: torrenttools create -a "https://blah1.com" -p -l 21 -v '1' -o ~/somewhere/ ~/somewhere/something

    Results in no announce URL being added to the metafile

    Announces:


    Using: torrenttools create -a "https://blah1.com" "https://blah2.com" -p -l 21 -v '1' -o ~/somewhere/ ~/somewhere/something

    Results in both URLs being added to the metafile

    Announces: tier 1 - https://blah1.com tier 2 - https://blah2.com

    bug 
    opened by SpookyStuff 3
  • torrenttools edit (--announce) crashing on Windows

    torrenttools edit (--announce) crashing on Windows

    On the Windows version, I am using the command torrenttools edit -a "http://newtracker.url/announce.php" "A.Release.by.a-GROUP.torrent", which crashes, producing no output. Afterwards, using torrenttools info "A.Release.by.a-GROUP.torrent" shows that the announce URL has not been edited. I have also tried using the -o option, as well as using a named tracker (which doesn't update any of the relevant parts of the torrent).

    On Linux, this command works as intended.

    Event Viewer log:

    Faulting application name: torrenttools.exe, version: 0.0.0.0, time stamp: 0x610e32a1
    Faulting module name: unknown, version: 0.0.0.0, time stamp: 0x00000000
    Exception code: 0xc0000005
    Fault offset: 0x00007ff7ab3c0000
    Faulting process ID: 0x3680
    Faulting application start time: 0x01d797908bc46f20
    Faulting application path: C:\Program Files\torrenttools\torrenttools.exe
    Faulting module path: unknown
    Report ID: 541fa805-e161-4d41-8cea-92aeb9bce5e1
    Faulting package full name: 
    Faulting package-relative application ID: 
    
    opened by RibShark 2
  • Unknown parsing error on Win10

    Unknown parsing error on Win10

    Hi,

    Regardless of whether I use the default config.yml, or some configured one, I am getting this error.

    Error: [json.exception.parse_error.101] parse error at line 10, column 15: syntax error while parsing object - unexpected string literal; expected '}'

    If I edit line 10 of the config.yml to enclose the {pid}, like so pid: {example_pid}

    Then I receive this error. Error: yaml-cpp: error at line 10, column 10: bad conversion

    opened by Audionut 2
  • Publishing an official Docker image?

    Publishing an official Docker image?

    Thanks for what looks to be a fantastic tool.

    There are some cases where having a Docker image to easily run torrenttools might be more convenient / desirable than the existing COPR / launchpad repos.

    Would you be willing to consider publishing an official Docker image that is updated as part of releases?

    enhancement 
    opened by past-due 2
  • Excessive memory usage for creating huge torrents (> 2 TiB)

    Excessive memory usage for creating huge torrents (> 2 TiB)

    Trying to create a torrents with ~200,000 files, 2 TiB in total, and the memory usage surged up to 12 GB when hashing. ~~The progress isn't moving and stuck at 0 B/s.~~ Any ideas for optimizing this?

    Edit: The progress began moving after a few minutes, ~~at a reasonable rate.~~ But the memory usage is still high.

    Edit2: No. The progress is actually broken. It first went extremely fast to 100%, and freezes. But the network activity shows the progress is still running.

    Thanks in advance.

    image

    image

    opened by cqjjjzr 0
  • aur package won't build

    aur package won't build

    Commands to reproduce: yay -S torrenttools Errors:

    /usr/include/fmt/core.h:1757:7: note: ‘formattable’ evaluates to false
    make[3]: *** [external/dottorrent/CMakeFiles/dottorrent.dir/build.make:132: external/dottorrent/CMakeFiles/dottorrent.dir/src/chunk_reader.cpp.o] Error 1
    make[3]: *** [external/dottorrent/CMakeFiles/dottorrent.dir/build.make:104: external/dottorrent/CMakeFiles/dottorrent.dir/src/chunk_hasher_single_buffer.cpp.o] Error 1
    make[3]: *** [external/dottorrent/CMakeFiles/dottorrent.dir/build.make:118: external/dottorrent/CMakeFiles/dottorrent.dir/src/chunk_processor_base.cpp.o] Error 1
    make[2]: *** [CMakeFiles/Makefile2:1258: external/dottorrent/CMakeFiles/dottorrent.dir/all] Error 2
    make[1]: *** [CMakeFiles/Makefile2:1109: CMakeFiles/torrenttools.dir/rule] Error 2
    make: *** [Makefile:577: torrenttools] Error 2
    ==> ERROR: A failure occurred in build().
        Aborting...
     -> error making: torrenttools
    

    Aur package version: torrenttools-0.6.0-1

    opened by Czechball 1
  • Could not build on Debian Sid/testing

    Could not build on Debian Sid/testing

    The current build fail at many steps.

    1. I assume that Bencode recently changed it's master branch to main. But the torrentools makefile still search for a master tag in external/bencode.cmake: GIT_TAG master

      • Solution: by changing it to "main".
    2. Then is fails because upstream errors with bencode compilation. I opened an issue about this upstream : Bencode: Build errors with deps and instructions

      • Current solution: by compiling and installing the deps manually.
    3. Then it fails because it does not find nlohmann/json.

      • Current solution: by compiling and installing manually.
    [ 81%] Building CXX object CMakeFiles/torrenttools.dir/src/common.cpp.o
    In file included from /home/tomasz/Builds/torrenttools/include/common.hpp:8,
                     from /home/tomasz/Builds/torrenttools/src/common.cpp:4:
    /home/tomasz/Builds/torrenttools/include/tracker_database.hpp:11:10: fatal error: nlohmann/json.hpp: Aucun fichier ou dossier de ce type
       11 | #include <nlohmann/json.hpp>
          |          ^~~~~~~~~~~~~~~~~~~
    compilation terminated.
    
    1. Some compilation errors with a pattern in lines 50 & 65 of include/file_matcher.hpp :
      • Unresolved: I stopped here because I'm not good enough with C++
    [ 81%] Building CXX object CMakeFiles/torrenttools.dir/src/create.cpp.o                                                                                                                      
    In file included from /home/tomasz/Builds/torrenttools/src/create.cpp:22:                                                                                                                    
    /home/tomasz/Builds/torrenttools/include/file_matcher.hpp: In member function ‘void torrenttools::file_matcher::allow_extension(std::string_view)’:                                          
    /home/tomasz/Builds/torrenttools/include/file_matcher.hpp:50:31: error: unable to find string literal operator ‘operator""_format’ with ‘const char [7]’, ‘long unsigned int’ arguments      
       50 |         std::string pattern = ".*.{}$"_format(extension);                                                                                                                            
          |                               ^~~~~~~~~~~~~~~                                                                                                                                        
    /home/tomasz/Builds/torrenttools/include/file_matcher.hpp: In member function ‘void torrenttools::file_matcher::block_extension(std::string_view)’:                                          
    /home/tomasz/Builds/torrenttools/include/file_matcher.hpp:65:31: error: unable to find string literal operator ‘operator""_format’ with ‘const char [8]’, ‘long unsigned int’ arguments      
       65 |         std::string pattern = ".*\\.{}$"_format(extension);                                                                                                                          
          |                               ^~~~~~~~~~~~~~~~~
    
    
    1. It's not an error but you should add this dependancy in the Readme : termcontrol
    opened by tkapias 0
  • ISA-L Crypto support

    ISA-L Crypto support

    Hi

    I have installed "sudo apt install torrenttools" and then installed ISA-L Crypto from their github. does that mean i have torrenttools with ISA-L Crypto support? or am i still using openssl

    opened by manifesto1 1
  • Error: parse error: invalid character read: value limit exceeded reading newly created hybrid torrent

    Error: parse error: invalid character read: value limit exceeded reading newly created hybrid torrent

    I used torrenttools to create a hybrid torrent on about 1TiB of data. The resulting file is about 20MB and can't be opened by qBittorrent ("the v1 and v2 metadata does not match"), but actually even torrenttools info out.torrent fails with an exception:

    Catchpoint 1 (exception thrown), 0x00007ffff76a91b1 in __cxxabiv1::__cxa_throw (obj=0x555555989dd0, tinfo=0x55555593f598 <typeinfo for bencode::parsing_error>, dest=0x55555568b080 <bencode::parsing_error::~parsing_error()>) at ../../../../src/libstdc++-v3/libsupc++/eh_throw.cc:81
    81      ../../../../src/libstdc++-v3/libsupc++/eh_throw.cc: No such file or directory.
    (gdb) bt
    #0  0x00007ffff76a91b1 in __cxxabiv1::__cxa_throw (obj=0x555555989dd0, tinfo=0x55555593f598 <typeinfo for bencode::parsing_error>, dest=0x55555568b080 <bencode::parsing_error::~parsing_error()>) at ../../../../src/libstdc++-v3/libsupc++/eh_throw.cc:81
    #1  0x000055555574f93e in bencode::decode_view (view="d8:announce34:udp://tracker.coppersurfer.tk:696913:announce-listll34:udp://tracker.coppersurfer.tk:6969el40:udp://tracker.leechers-paradise.org:6969el28:udp://exodus.desync.com:6969el42:udp://tracker."...)
        at /code/build/torrenttools/build/_deps/bencode-src/include/bencode/detail/decode_view.hpp:24
    #2  0x000055555574d1ca in dottorrent::read_metafile (view="d8:announce34:udp://tracker.coppersurfer.tk:696913:announce-listll34:udp://tracker.coppersurfer.tk:6969el40:udp://tracker.leechers-paradise.org:6969el28:udp://exodus.desync.com:6969el42:udp://tracker."...)
        at /code/build/torrenttools/build/_deps/dottorrent-src/src/metafile.cpp:262
    #3  0x000055555574d7d6 in dottorrent::load_metafile (path=filesystem::path "/some/redacted/path/out.torrent" = {...}) at /code/build/torrenttools/build/_deps/dottorrent-src/src/metafile.cpp:324
    #4  0x0000555555674291 in run_info_app (main_options=..., options=...) at /code/build/torrenttools/src/info.cpp:52
    #5  0x000055555568dd9c in main (argc=3, argv=0x7fffffffd408) at /code/build/torrenttools/src/main.cpp:71
    

    The exception message is Error: parse error: invalid character read at position 9302219: value limit exceeded.
    Here is the (lightly redacted) start of the log, during the torrent creation:

    Scanning target directory: 56509 files processed
    Sorting file list... Done.
    Adding files to metafile... Done.
    Metafile:          /some/redacted/path/out.torrent
    Protocol version:  v1 + v2 (hybrid)
    Piece size:        4 MiB (4194304 bytes)
    Piece count:       263162
    Created by:        torrenttools/0.6.2
    Created on:        2022-07-18 19:46:21 UTC
    Private:           false
    Name:              test
    Source:            
    Comment:           
    Announce-urls:     tier 1  - udp://tracker.coppersurfer.tk:6969
                       tier 2  - udp://tracker.leechers-paradise.org:6969
                       tier 3  - udp://exodus.desync.com:6969
                       tier 4  - udp://tracker.opentrackr.org:1337/announce
                       tier 5  - udp://tracker.uw0.xyz:6969/announce
    DHT nodes:         
    Web seeds:         
    HTTP seeds:        
    Similar torrents:  
    Collections:       Test collection
    Other info fields: 
    
    Files:
    
    Metafile contains more than 1000 files: skipping file tree ...
    
      1 TiB in 6516 directories, 57385 files
    
    
    Hashing files...
    [progress snipped]
    

    I can provide the 20MB output torrent privately if necessary.

    opened by htnyquist 0
Releases(v0.6.2)
Owner
null
Transmission is a fast, easy, and free BitTorrent client

About Transmission is a fast, easy, and free BitTorrent client. It comes in several flavors: A native Mac OS X GUI application GTK+ and Qt GUI applica

Bread 102 Dec 24, 2022
BitTorrent DHT library

The files dht.c and dht.h implement the variant of the Kademlia Distributed Hash Table (DHT) used in the Bittorrent network (``mainline'' variant). T

Juliusz Chroboczek 654 Dec 24, 2022
an efficient feature complete C++ bittorrent implementation

libtorrent is an open source C++ library implementing the BitTorrent protocol, along with most popular extensions, making it suitable for real world d

Arvid Norberg 4.3k Dec 24, 2022
libTorrent BitTorrent library

LibTorrent Copyright (C) 2005-2014, Jari Sundell LICENSE GNU GPL, see COPYING. "libtorrent/src/utils/sha_fast.{cc,h}" is originally from the Mozil

Jari Sundell 844 Dec 10, 2022
qBittorrent - A BitTorrent client in Qt

qBittorrent - A BitTorrent client in Qt Description: qBittorrent is a bittorrent client programmed in C++ / Qt that uses libtorrent (sometimes called

qBittorrent project 18.5k Jan 6, 2023
CatFrida is a macOS tool for inspecting a running iOS app.

CatFrida CatFrida is a macOS tool for inspecting a running iOS app. Building with frida-swift, CatFrida provide an awesome easy way to dive into an ap

neilwu 105 Dec 6, 2022
This is a collection of tools for creating and manipulating BitTorrent v2 torrent files

torrent tools This is a collection of tools for creating and manipulating BitTorrent v2 torrent files. torrent-new can create hybrid torrents, but the

Arvid Norberg 9 Nov 12, 2022
Tiny and cheap robot car for inspecting sewer pipes >= 125 mm. With pan servo for the ESP32-Cam module

ESP32-Cam Sewer inspection car Version 1.0.0 - work in progress Based on esp32-cam-webserver by Owen Carter. Additional Features Pan servo for the ESP

Armin 5 Nov 6, 2022
OpenShot Video Library (libopenshot) is a free, open-source C++ library dedicated to delivering high quality video editing, animation, and playback solutions to the world

OpenShot Video Library (libopenshot) is a free, open-source C++ library dedicated to delivering high quality video editing, animation, and playback solutions to the world

OpenShot Studios, LLC 936 Jan 4, 2023
Immediate mode 3D gizmo for scene editing and other controls based on Dear Imgui

ImGuizmo Latest stable tagged version is 1.83. Current master version is 1.84 WIP. What started with the gizmo is now a collection of dear imgui widge

Cedric Guillemet 2.3k Dec 27, 2022
This project aims to simplify creation of basic Arduino programs by just editing a UI on Android.

ArdUI A video explanation If you are more a fun of video explanation go to this youtube video Project aim This project aims to simplify creation of ba

Targist 21 Nov 19, 2022
Scintilla is a free source code editing component

Scintilla is a free source code editing component

The Lua Language distribution [Archived] 33 Sep 9, 2022
A Visual Studio extension that provides enhanced support for editing High Level Shading Language (HLSL) files

HLSL Tools for Visual Studio This extension is for Visual Studio 2017 / 2019. Go here for the Visual Studio Code extension. HLSL Tools is a Visual Stu

Tim Jones 433 Dec 27, 2022
A lightweight, self-contained library for gizmo editing commonly found in many game engines

This project is a lightweight, self-contained library for gizmo editing commonly found in many game engines. It includes mechanisms for manipulating 3d position, rotation, and scale. Implemented in C++11, the library does not perform rendering directly and instead provides a per-frame buffer of world-space triangles.

Dimitri Diakopoulos 392 Dec 31, 2022
Transmission is a fast, easy, and free BitTorrent client.

Official Transmission BitTorrent client repository

Transmission 8.3k Jan 3, 2023
Transmission is a fast, easy, and free BitTorrent client

About Transmission is a fast, easy, and free BitTorrent client. It comes in several flavors: A native Mac OS X GUI application GTK+ and Qt GUI applica

Bread 102 Dec 24, 2022
BitTorrent DHT library

The files dht.c and dht.h implement the variant of the Kademlia Distributed Hash Table (DHT) used in the Bittorrent network (``mainline'' variant). T

Juliusz Chroboczek 654 Dec 24, 2022
an efficient feature complete C++ bittorrent implementation

libtorrent is an open source C++ library implementing the BitTorrent protocol, along with most popular extensions, making it suitable for real world d

Arvid Norberg 4.3k Dec 24, 2022
libTorrent BitTorrent library

LibTorrent Copyright (C) 2005-2014, Jari Sundell LICENSE GNU GPL, see COPYING. "libtorrent/src/utils/sha_fast.{cc,h}" is originally from the Mozil

Jari Sundell 844 Dec 10, 2022
A bittorrent plugin for VLC.

vlc-bittorrent (Bittorrent plugin for VLC) What is this? With vlc-bittorrent, you can open a .torrent file or magnet link with VLC and stream any medi

Johan Gunnarsson 348 Jan 5, 2023