ldd as a tree with an option to bundle dependencies into a single folder

Overview

Test AUR version

libtree

A tool that:

  • 🌳 turns ldd into a tree
  • ☝️ explains why shared libraries are found and why not
  • 📦 optionally deploys executables and dependencies into a single directory

example

Installation

Download the latest release from GitHub.

Static executable

wget -qO libtree https://github.com/haampie/libtree/releases/download/v2.0.0/libtree_x86_64
chmod +x libtree
./libtree $(which man)

Static executable + optional dependencies

wget -qO libtree.tar.gz https://github.com/haampie/libtree/releases/download/v2.0.0/libtree_x86_64.tar.gz
mkdir libtree
tar -xf libtree.tar.gz -C libtree
export PATH="$PWD/libtree:$PATH"
libtree $(which man)

Deploying binaries + dependencies into a folder

$ libtree $(which man) -d man.bundle --chrpath --strip
man
├── libmandb-2.9.1.so [runpath]
│   ├── libman-2.9.1.so [runpath]
│   │   ├── libpipeline.so.1 [ld.so.conf]
│   │   └── libseccomp.so.2 [ld.so.conf]
│   └── libgdbm.so.6 [ld.so.conf]
├── libman-2.9.1.so (collapsed) [runpath]
└── libpipeline.so.1 (collapsed) [ld.so.conf]

Deploying to "man.bundle/usr"
"/usr/bin/man" => "man.bundle/usr/bin/man"
"/usr/lib/man-db/libmandb-2.9.1.so" => "man.bundle/usr/lib/libmandb-2.9.1.so"
"/usr/lib/man-db/libman-2.9.1.so" => "man.bundle/usr/lib/libman-2.9.1.so"
"/usr/lib/x86_64-linux-gnu/libpipeline.so.1.5.2" => "man.bundle/usr/lib/libpipeline.so.1.5.2"
  creating symlink "man.bundle/usr/lib/libpipeline.so.1"
"/usr/lib/x86_64-linux-gnu/libseccomp.so.2.5.1" => "man.bundle/usr/lib/libseccomp.so.2.5.1"
  creating symlink "man.bundle/usr/lib/libseccomp.so.2"
"/usr/lib/x86_64-linux-gnu/libgdbm.so.6.0.0" => "man.bundle/usr/lib/libgdbm.so.6.0.0"
  creating symlink "man.bundle/usr/lib/libgdbm.so.6"

$ tree man.bundle/
man.bundle/
└── usr
    ├── bin
    │   └── man
    └── lib
        ├── libgdbm.so.6 -> libgdbm.so.6.0.0
        ├── libgdbm.so.6.0.0
        ├── libman-2.9.1.so
        ├── libmandb-2.9.1.so
        ├── libpipeline.so.1 -> libpipeline.so.1.5.2
        ├── libpipeline.so.1.5.2
        ├── libseccomp.so.2 -> libseccomp.so.2.5.1
        └── libseccomp.so.2.5.1

3 directories, 9 files

Verbose output

By default, certain standard dependencies are not shown. For more verbose output use

  • libtree -v $(which man) to show skipped libraries without their children
  • libtree -a $(which apt-get) to show the full recursive list of libraries

Use the --path or -p flags to show paths rather than sonames:

  • libtree -p $(which tar)

Changing search paths

libtree follows the rules of ld.so to locate libraries, but does not use ldconfig's cache. Instead it parses /etc/ld.so.conf at runtime. In fact you can change the search path config by setting --ldconf mylibs.conf. Search paths can be added as well via LD_LIBRARY_PATH="path1:path2:$LD_LIBRARY_PATH" libtree ....

Building

  • From source:
    git clone https://github.com/haampie/libtree.git
    cd libtree
    mkdir build
    cd build
    cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_PREFIX_PATH="/path/to/cxxopts;/path/to/elfio;/path/to/termcolor" ..
    make -j
    make install
  • Using spack:
    spack install libtree +chrpath +strip
    spack load libtree
    

Known issues

  • When deploying libs with libtree app -d folder.bundle --chrpath, the runpaths are only changed when the binaries already have an rpath or runpath. This is a limitation of chrpath. Another option is to use patchelf instead, but this tool is known to break binaries sometimes.
Comments
  • Different values to `ldd` in Gentoo Prefix

    Different values to `ldd` in Gentoo Prefix

    I was trying to use libtree for some testing under Gentoo Prefix. I know we may need to patch CUDA to look in the prefix but I was surprised that ldd and lddtree don't return the same information:

    [[email protected] release]$ ldd ./deviceQuery
            linux-vdso.so.1 (0x00007ffd78ed6000)
            librt.so.1 => /cvmfs/pilot.eessi-hpc.org/2021.06/compat/linux/x86_64/lib64/librt.so.1 (0x0000153ce5eac000)
            libpthread.so.0 => /cvmfs/pilot.eessi-hpc.org/2021.06/compat/linux/x86_64/lib64/libpthread.so.0 (0x0000153ce5e8c000)
            libdl.so.2 => /cvmfs/pilot.eessi-hpc.org/2021.06/compat/linux/x86_64/lib64/libdl.so.2 (0x0000153ce5e86000)
            libstdc++.so.6 => /cvmfs/pilot.eessi-hpc.org/2021.06/compat/linux/x86_64/usr/lib/gcc/x86_64-pc-linux-gnu/11.1.0/libstdc++.so.6 (0x0000153ce5c71000)
            libm.so.6 => /cvmfs/pilot.eessi-hpc.org/2021.06/compat/linux/x86_64/lib64/libm.so.6 (0x0000153ce5b30000)
            libgcc_s.so.1 => /cvmfs/pilot.eessi-hpc.org/2021.06/compat/linux/x86_64/usr/lib/gcc/x86_64-pc-linux-gnu/11.1.0/libgcc_s.so.1 (0x0000153ce5b13000)
            libc.so.6 => /cvmfs/pilot.eessi-hpc.org/2021.06/compat/linux/x86_64/lib64/libc.so.6 (0x0000153ce5958000)
            /cvmfs/pilot.eessi-hpc.org/2021.06/compat/linux/x86_64/lib64/ld-linux-x86-64.so.2 (0x0000153ce5ec3000)
    [[email protected] release]$ lddtree ./deviceQuery
    deviceQuery => ./deviceQuery (interpreter => /cvmfs/pilot.eessi-hpc.org/2021.06/compat/linux/x86_64/lib64/ld-linux-x86-64.so.2)
        librt.so.1 => /lib64/librt.so.1
        libpthread.so.0 => /lib64/libpthread.so.0
        libdl.so.2 => /lib64/libdl.so.2
        libstdc++.so.6 => /lib64/libstdc++.so.6
        libm.so.6 => /lib64/libm.so.6
        libgcc_s.so.1 => /lib64/libgcc_s.so.1
        libc.so.6 => /lib64/libc.so.6
        ld-linux-x86-64.so.2 => /cvmfs/pilot.eessi-hpc.org/2021.06/compat/linux/x86_64/lib64/ld-linux-x86-64.so.2
    [[email protected] release]$ which ldd
    /cvmfs/pilot.eessi-hpc.org/2021.06/compat/linux/x86_64/usr/bin/ldd
    

    Which is correct here? Is it that libtree is using the system linker rather than the one specified in the executable?

    opened by ocaisa 17
  • Rewrite in C

    Rewrite in C

    Given complaints like #40, conflicting requests about always using system versions of dependencies, as well as people using libtree on old systems without std::filesystem #36, I think it doesn't hurt to rewrite libtree in C without any dependencies. That means anyone can pick their favorite build system, such as make or even gcc *.c.

    I had already started doing so: https://github.com/haampie/libtree-in-c and hopefully it can be done over the Christmas holidays.

    The C version also includes some more fancy reports when libraries are missing, as well as more feedback about rpaths (in what parent's rpath was the library found?)

    Screenshot from 2021-12-03 23-20-24

    opened by haampie 10
  • Better error handling?

    Better error handling?

    libtree: 4f16a1be6a591424a8b1b1658bb61a64495aee0d

    There is lacking error handling in libtree, for example.

    # missing file
    $ ./libtree foo ; echo $?
    1
    
    # broken symlink
    $ ./libtree /usr/lib64/libXvMCgallium.so ; echo $?
    1
    $ file /usr/lib64/libXvMCgallium.so
    /usr/lib64/libXvMCgallium.so: broken symbolic link to libXvMCnouveau.so
    
    # static archive
    $ ./libtree /usr/lib64/libc.a ; echo $?
    1
    

    It would be good if there were appropriate warnings or error messages in these cases. Its currently not at all obvious why libtree fails when executing it on large number of files which happen to have such examples.

    3.x 
    opened by orbea 8
  • Add USE_SYSTEM_DEPS flag and install target

    Add USE_SYSTEM_DEPS flag and install target

    If its -DUSE_SYSTEM_DEPS=ON the deps folder is ignored and only cppglob is linked. Else the libraries at deps are build and linked.

    Also added the deps/cxxopts/include to the include directories because in Arch linux, cxxopts puts the cxxopts.hpp header in /usr/include/

    opened by otreblan 8
  • libtree, the name

    libtree, the name

    Thanks for this software, it's nice. However, wouldn't you want to rename it to lddtree? libletter names are usually for library software.

    https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1004927

    opened by alexmyczko 7
  • Segfault with another llvm library

    Segfault with another llvm library

    libtree: d51185d9fa83dd0525ef135bb3c40391f42cb932

    $ ./libtree /usr/lib64/libLLVMScalarOpts.so
    libLLVMScalarOpts.so.12 
    ├── libLLVMAggressiveInstCombine.so.12 [runpath]
    │   ├── libLLVMTransformUtils.so.12 [runpath]
    │   │   ├── libLLVMAnalysis.so.12 [runpath]
    │   │   │   ├── libLLVMObject.so.12 [runpath]
    │   │   │   │   ├── libLLVMBitReader.so.12 [runpath]
    │   │   │   │   │   ├── libLLVMCore.so.12 [runpath]
    =================================================================
    ==24611==ERROR: AddressSanitizer: heap-use-after-free on address 0x61d000000820 at pc 0x000000499aa7 bp 0x7ffdcb36b3a0 sp 0x7ffdcb36ab68
    READ of size 10 at 0x61d000000820 thread T0
        #0 0x499aa6 in __asan_memcpy /tmp/llvm-12.0.0.src/build/../projects/compiler-rt/lib/asan/asan_interceptors_memintrinsics.cpp:22:3
        #1 0x4d2808 in string_table_store /tmp/libtree/libtree.c:317:5
        #2 0x4d4f87 in interpolate_variables /tmp/libtree/libtree.c:518:9
        #3 0x4cf9c5 in recurse /tmp/libtree/libtree.c:1136:13
        #4 0x4d5dd0 in check_search_paths /tmp/libtree/libtree.c:427:17
        #5 0x4d151d in recurse /tmp/libtree/libtree.c:1253:9
        #6 0x4d5dd0 in check_search_paths /tmp/libtree/libtree.c:427:17
        #7 0x4d151d in recurse /tmp/libtree/libtree.c:1253:9
        #8 0x4d5dd0 in check_search_paths /tmp/libtree/libtree.c:427:17
        #9 0x4d151d in recurse /tmp/libtree/libtree.c:1253:9
        #10 0x4d5dd0 in check_search_paths /tmp/libtree/libtree.c:427:17
        #11 0x4d151d in recurse /tmp/libtree/libtree.c:1253:9
        #12 0x4d5dd0 in check_search_paths /tmp/libtree/libtree.c:427:17
        #13 0x4d151d in recurse /tmp/libtree/libtree.c:1253:9
        #14 0x4d5dd0 in check_search_paths /tmp/libtree/libtree.c:427:17
        #15 0x4d151d in recurse /tmp/libtree/libtree.c:1253:9
        #16 0x4d5dd0 in check_search_paths /tmp/libtree/libtree.c:427:17
        #17 0x4d151d in recurse /tmp/libtree/libtree.c:1253:9
        #18 0x4cc86c in print_tree /tmp/libtree/libtree.c:1463:22
        #19 0x4cc4b3 in main /tmp/libtree/libtree.c:1629:12
        #20 0x7f4b6977c01c in __libc_start_main (/lib64/libc.so.6+0x2401c)
        #21 0x41f369 in _start /tmp/glibc-2.33/csu/../sysdeps/x86_64/start.S:120
    
    0x61d000000820 is located 1952 bytes inside of 2050-byte region [0x61d000000080,0x61d000000882)
    freed by thread T0 here:
        #0 0x49aa19 in realloc /tmp/llvm-12.0.0.src/build/../projects/compiler-rt/lib/asan/asan_malloc_linux.cpp:164:3
        #1 0x4d2cf1 in string_table_maybe_grow /tmp/libtree/libtree.c:307:17
        #2 0x4d279d in string_table_store /tmp/libtree/libtree.c:316:5
        #3 0x4d4f87 in interpolate_variables /tmp/libtree/libtree.c:518:9
        #4 0x4cf9c5 in recurse /tmp/libtree/libtree.c:1136:13
        #5 0x4d5dd0 in check_search_paths /tmp/libtree/libtree.c:427:17
        #6 0x4d151d in recurse /tmp/libtree/libtree.c:1253:9
        #7 0x4d5dd0 in check_search_paths /tmp/libtree/libtree.c:427:17
        #8 0x4d151d in recurse /tmp/libtree/libtree.c:1253:9
        #9 0x4d5dd0 in check_search_paths /tmp/libtree/libtree.c:427:17
        #10 0x4d151d in recurse /tmp/libtree/libtree.c:1253:9
        #11 0x4d5dd0 in check_search_paths /tmp/libtree/libtree.c:427:17
        #12 0x4d151d in recurse /tmp/libtree/libtree.c:1253:9
        #13 0x4d5dd0 in check_search_paths /tmp/libtree/libtree.c:427:17
        #14 0x4d151d in recurse /tmp/libtree/libtree.c:1253:9
        #15 0x4d5dd0 in check_search_paths /tmp/libtree/libtree.c:427:17
        #16 0x4d151d in recurse /tmp/libtree/libtree.c:1253:9
        #17 0x4d5dd0 in check_search_paths /tmp/libtree/libtree.c:427:17
        #18 0x4d151d in recurse /tmp/libtree/libtree.c:1253:9
        #19 0x4cc86c in print_tree /tmp/libtree/libtree.c:1463:22
        #20 0x4cc4b3 in main /tmp/libtree/libtree.c:1629:12
        #21 0x7f4b6977c01c in __libc_start_main (/lib64/libc.so.6+0x2401c)
    
    previously allocated by thread T0 here:
        #0 0x49aa19 in realloc /tmp/llvm-12.0.0.src/build/../projects/compiler-rt/lib/asan/asan_malloc_linux.cpp:164:3
        #1 0x4d2cf1 in string_table_maybe_grow /tmp/libtree/libtree.c:307:17
        #2 0x4d4397 in string_table_copy_from_file /tmp/libtree/libtree.c:325:9
        #3 0x4cfc25 in recurse /tmp/libtree/libtree.c:1153:9
        #4 0x4d5dd0 in check_search_paths /tmp/libtree/libtree.c:427:17
        #5 0x4d151d in recurse /tmp/libtree/libtree.c:1253:9
        #6 0x4d5dd0 in check_search_paths /tmp/libtree/libtree.c:427:17
        #7 0x4d151d in recurse /tmp/libtree/libtree.c:1253:9
        #8 0x4d5dd0 in check_search_paths /tmp/libtree/libtree.c:427:17
        #9 0x4d151d in recurse /tmp/libtree/libtree.c:1253:9
        #10 0x4cc86c in print_tree /tmp/libtree/libtree.c:1463:22
        #11 0x4cc4b3 in main /tmp/libtree/libtree.c:1629:12
        #12 0x7f4b6977c01c in __libc_start_main (/lib64/libc.so.6+0x2401c)
    
    SUMMARY: AddressSanitizer: heap-use-after-free /tmp/llvm-12.0.0.src/build/../projects/compiler-rt/lib/asan/asan_interceptors_memintrinsics.cpp:22:3 in __asan_memcpy
    Shadow bytes around the buggy address:
      0x0c3a7fff80b0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
      0x0c3a7fff80c0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
      0x0c3a7fff80d0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
      0x0c3a7fff80e0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
      0x0c3a7fff80f0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
    =>0x0c3a7fff8100: fd fd fd fd[fd]fd fd fd fd fd fd fd fd fd fd fd
      0x0c3a7fff8110: fd fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
      0x0c3a7fff8120: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
      0x0c3a7fff8130: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
      0x0c3a7fff8140: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
      0x0c3a7fff8150: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
    Shadow byte legend (one shadow byte represents 8 application bytes):
      Addressable:           00
      Partially addressable: 01 02 03 04 05 06 07 
      Heap left redzone:       fa
      Freed heap region:       fd
      Stack left redzone:      f1
      Stack mid redzone:       f2
      Stack right redzone:     f3
      Stack after return:      f5
      Stack use after scope:   f8
      Global redzone:          f9
      Global init order:       f6
      Poisoned by user:        f7
      Container overflow:      fc
      Array cookie:            ac
      Intra object redzone:    bb
      ASan internal:           fe
      Left alloca redzone:     ca
      Right alloca redzone:    cb
      Shadow gap:              cc
    ==24611==ABORTING
    
    opened by orbea 7
  • Segfaults when using many command-line arguments

    Segfaults when using many command-line arguments

    libtree: https://github.com/haampie/libtree/commit/7f7a39f5399142a5c91b05d637a593f7d8eb2009

    Reading symbols from ./libtree...
    (gdb) r /usr/lib64/jollygood/*
    Starting program: /media/gittings/forks/libtree/libtree /usr/lib64/jollygood/*
    /usr/lib64/jollygood/bsnes.so 
    /usr/lib64/jollygood/cega.so 
    /usr/lib64/jollygood/gambatte.so 
    /usr/lib64/jollygood/genplus.so 
    /usr/lib64/jollygood/jollycv.so 
    /usr/lib64/jollygood/mednafen.so 
    ├── libz.so.1 [ld.so.conf]
    ├── libFLAC.so.8 [ld.so.conf]
    │   └── libogg.so.0 [ld.so.conf]
    └── libzstd.so.1 [ld.so.conf]
    /usr/lib64/jollygood/mesens.so 
    /usr/lib64/jollygood/mgba.so 
    /usr/lib64/jollygood/nestopia.so 
    /usr/lib64/jollygood/picodrive.so 
    /usr/lib64/jollygood/prosystem.so 
    /usr/lib64/jollygood/sameboy.so 
    free(): invalid size
    
    Program received signal SIGABRT, Aborted.
    0x00007ffff7e02848 in raise () from /lib64/libc.so.6
    (gdb) bt
    #0  0x00007ffff7e02848 in raise () from /lib64/libc.so.6
    #1  0x00007ffff7de9526 in abort () from /lib64/libc.so.6
    #2  0x00007ffff7e48248 in __libc_message () from /lib64/libc.so.6
    #3  0x00007ffff7e5066a in malloc_printerr () from /lib64/libc.so.6
    #4  0x00007ffff7e5207c in _int_free () from /lib64/libc.so.6
    #5  0x00007ffff7e55ce4 in free () from /lib64/libc.so.6
    #6  0x00007ffff7e3d83f in [email protected]@GLIBC_2.2.5 () from /lib64/libc.so.6
    #7  0x0000000000402c2b in recurse (
        current_file=0x7fffffffe62a "/usr/lib64/jollygood/vecx.so", depth=0, s=0x7fffffffde78, 
        parent_bits=EITHER, reason=...) at libtree.c:1154
    #8  0x00000000004019e3 in print_tree (pathc=13, pathv=0x7fffffffe120, s=0x7fffffffde78)
        at libtree.c:1460
    #9  0x0000000000401945 in main (argc=14, argv=0x7fffffffe120) at libtree.c:1625
    

    Running the library on its own does not produce a segfault.

    $ ./libtree /usr/lib64/jollygood/vecx.so
    /usr/lib64/jollygood/vecx.so
    

    Although I don't think its related for reference the libraries are from this project.

    https://gitlab.com/jgemu

    opened by orbea 6
  • Cannot find libc on Fedora

    Cannot find libc on Fedora

    I've been testing libtree on Fedora Linux distro and have been having issues trying to get the integration test to pass. Other binaries have a similar problem too:

    $ libtree /bin/ls
    ls
    ├── libselinux.so.1 not found: RPATH (empty) LD_LIBRARY_PATH (empty) RUNPATH (empty) /etc/ld.so.conf (empty)
    ├── libcap.so.2 not found: RPATH (empty) LD_LIBRARY_PATH (empty) RUNPATH (empty) /etc/ld.so.conf (empty)
    └── libc.so.6 not found: RPATH (empty) LD_LIBRARY_PATH (empty) RUNPATH (empty) /etc/ld.so.conf (empty)
    

    I believe the issue is due to the difference between Fedora and Ubuntu with the /etc/ld.so.conf.d directory, which is used by /etc/ld.so.conf.

    On Fedora, there is no entry for libc:

    $ docker run --rm fedora:32 ls -la /etc/ld.so.conf.d
    total 8
    drwxr-xr-x 2 root root 4096 Mar 21 01:08 .
    drwxr-xr-x 1 root root 4096 May 30 21:48 ..
    

    However, on Ubuntu (both 20.04 and 18.04), there are entries to find libc:

    $ docker run --rm ubuntu:18.04 ls -al /etc/ld.so.conf.d
    total 16
    drwxr-xr-x 2 root root 4096 Mar 11 21:04 .
    drwxr-xr-x 1 root root 4096 May 30 21:50 ..
    -rw-r--r-- 1 root root   44 Jan 27  2016 libc.conf
    -rw-r--r-- 1 root root  100 Apr 16  2018 x86_64-linux-gnu.conf
    
    $ docker run --rm ubuntu:18.04 cat /etc/ld.so.conf.d/libc.conf
    # libc default configuration
    /usr/local/lib
    
    $ docker run --rm ubuntu:18.04 cat /etc/ld.so.conf.d/x86_64-linux-gnu.conf
    # Multiarch support
    /usr/local/lib/x86_64-linux-gnu
    /lib/x86_64-linux-gnu
    /usr/lib/x86_64-linux-gnu
    

    I think we could resolve this in a reasonable way by searching the output of the binary's loader (interpreter) output.

    For example, in the integration test, we have the main binary (compiled on my Fedora 32 machine):

    $ file main
    main: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=d452578c100a02abb811e0e6cc8775d72aee6edb, for GNU/Linux 3.2.0, not stripped
    

    We can use the interpreter (loader) path as our starting point and execute the following:

    $ LD_DEBUG=libs /lib64/ld-linux-x86-64.so.2 --inhibit-cache ~/src/libtree/build/libtree
         55849:     find library=libstdc++.so.6 [0]; searching
         55849:      search path=/lib64/tls/haswell/x86_64:/lib64/tls/haswell:/lib64/tls/x86_64:/lib64/tls:/lib64/haswell/x86_64:/lib64/haswell:/lib64/x86_64:/lib64:/usr/lib64/tls/haswell/x86_64:/usr/lib64/tls/haswell:/usr/lib64/tls/x86_64:/usr/lib64/tls:/usr/lib64/haswell/x86_64:/usr/lib64/haswell:/usr/lib64/x86_64:/usr/lib64                (system search path)
         55849:       trying file=/lib64/tls/haswell/x86_64/libstdc++.so.6
         55849:       trying file=/lib64/tls/haswell/libstdc++.so.6
         55849:       trying file=/lib64/tls/x86_64/libstdc++.so.6
         55849:       trying file=/lib64/tls/libstdc++.so.6
         55849:       trying file=/lib64/haswell/x86_64/libstdc++.so.6
         55849:       trying file=/lib64/haswell/libstdc++.so.6
         55849:       trying file=/lib64/x86_64/libstdc++.so.6
         55849:       trying file=/lib64/libstdc++.so.6
    ....
         55849:     initialize program: /home/ekilmer/src/libtree/build/libtree
         55849:
         55849:
         55849:     transferring control: /home/ekilmer/src/libtree/build/libtree
         55849:
    Show the dependency tree of binaries and optionally bundle them into a single folder.
    Usage:
      libtree [OPTION...] binary [more binaries...]
    
      -h, --help     Print usage
          --version  Print version info
    
     A. Locating libs options:
      -p, --path          Show the path of libraries instead of their SONAME
      -v, --verbose       Show the skipped libraries without their children
      -a, --all           Show the skipped libraries and their children
      -l, --ldconf arg    Path to custom ld.conf to test settings (default:
                          /etc/ld.so.conf)
      -s, --skip arg      Skip library and its dependencies from being deployed
                          or inspected
          --platform arg  Platform used for interpolation in rpaths (default:
                          x86_64)
    
     B. Copying libs options:
      -d, --destination arg  OPTIONAL: When a destination is set to a folder, all
                             binaries and their dependencies are copied over
          --strip            Call strip on binaries when deploying
          --chrpath          Call chrpath on binaries when deploying
    
         55849:
         55849:     calling fini: /home/ekilmer/src/libtree/build/libtree [0]
    ....
    

    and parse the search path= line to add directories and search for the missing libraries.

    NOTE Running ld-linux-x86-64.so.2 is equivalent to running the actual program, so we want to make sure the program exists and does a no-op. I would suggest just running our own libtree program to collect these paths. I was also thinking of maybe doing /bin/true, but that's not guaranteed to be there, and running the target program probably isn't a good idea.

    Unfortunately, I haven't found a way to just have ld-linux-x86-64.so.2 (or the 32-bit version) print its search paths and do nothing else. That would be best case scenario.

    One issue with this method would be if someone tries to run a 64-bit build of libtree on a 32-bit file. Using the target file's 32-bit loader would result in a mismatch with the 64-bit libtree. I'm not sure of a robust way to collect the loader's search paths without potentially causing damage, since using the loader is equivalent to running a program.

    Here is a sample of 32-bit loader output on my 64-bit Fedora VM:

            96:      search path=/lib/tls/i686/sse2:/lib/tls/i686:/lib/tls/sse2:/lib/tls:/lib/i686/sse2:/lib/i686:/lib/sse2:/lib:/usr/lib/tls/i686/sse2:/usr/lib/tls/i686:/usr/lib/tls/sse2:/usr/lib/tls:/usr/lib/i686/sse2:/usr/lib/i686:/usr/lib/sse2:/usr/lib              (system search path)
    

    I've also opened a Draft Pull Request (#26) that tests Fedora in CI, so we can test commits there.

    opened by ekilmer 6
  • In tests where libtree is expected to fail the tests fail for real

    In tests where libtree is expected to fail the tests fail for real

    Hello!

    I'm trying to package libtree for GNU Guix and there's a problem with the tests.

    In some tests libtree is expected to fail (and that's fine) but the failure brings down the whole build as a non-zero exit status returned to the operating system.

    I tried to call libtree on the test auto-generated program called exe after the fail and it failed indeed with the code 28 as you can see:

    $ libtree /tmp/guix-build-libtree-3.0.1.drv-0/source/tests/07_origin_is_relative_to_symlink_location_not_realpath/exe 
    exe 
    └── libg.so [runpath]
        └── libf.so not found
            ┊ Paths considered in this order:
            ┊ 1. rpath is skipped because runpath was set
            ┊ 2. LD_LIBRARY_PATH was not set
            ┊ 3. runpath:
            ┊    /tmp/guix-build-libtree-3.0.1.drv-0/source/tests/07_origin_is_relative_to_symlink_location_not_realpath/b
            ┊ 4. ld config files:
            ┊    /usr/lib/x86_64-linux-gnu/libfakeroot
            ┊    /usr/local/lib
            ┊    /usr/local/lib/x86_64-linux-gnu
            ┊    /lib/x86_64-linux-gnu
            ┊    /usr/lib/x86_64-linux-gnu
            ┊    /lib32
            ┊    /usr/lib32
            ┊ 5. Standard paths:
            ┊    /lib
            ┊    /lib64
            ┊    /usr/lib
            ┊    /usr/lib64
    Error [/tmp/guix-build-libtree-3.0.1.drv-0/source/tests/07_origin_is_relative_to_symlink_location_not_realpath/exe]: Not all dependencies were found
    $ echo $?
    28
    

    Please re-write the tests in such way that tests with expected failures will return zero status to the OS.

    Thanks!

    opened by artyom-poptsov 4
  • Dependencies should be part of your repo

    Dependencies should be part of your repo

    I'd like to build this tool for my Ubuntu 20.04 system on arm64 (aarch64), but it's not worth the effort.

    So the tool requires 3 libraries to build. Where are they? They're not git submodules, even though I bet they ~~should've~~ could've embedded them easily into your repo as submodules and made people who check out your repo able to build it without wasting their time going on a treasure hunt. That's what submodules are for.

    Option 2, if you don't want to learn git submodules, is trivial: just dump them in a 3rdparty/ subdir.

    Option 3, if you're not happy with any of the above, could be to fetch them as part of CMake. I know of a few C++ devs who gave up on C++ ever having a decent/standard package manager, and who fetch+build dependencies when you run cmake.

    Anyway, I read this CMakeLists file, and your dependencies are so obscure, Ubuntu doesn't have them, which means Debian doesn't have them, or any of the Ubuntu derivatives. That's like 80% of all Linux desktop users right there! If I thought you had reason to embed your deps before, man, does this cement it.

    Anyway, I ended up losing interest. I dont want to spend 30 minutes installing 3 libraries from source (and likely have to install THOSE libraries' dependencies manually) just to satisfy my curiosity about some random tool I clicked on HN at 2AM.

    I know C++ is a language with an ecosystem that's still struggling to enter the year 2000, but you should try to make it easier on users.

    Also, today I learned C++ has yet another package manager, Speck. C++'s already struggling with 3 or so package managers competing for relevancy, at a time it absolutely needs 1 to succeed and all others to drop dead. And yet, someone out there decided C++ needed another package manager. And someone else thought they oughta use it over the ~~less unpopular~~ more popular alternatives Conan or vcpkg. "year of usable C++" is starting to sound like "year of the Linux desktop" :P

    I probably could've built the tool in the time it took me to write this, but felt I would explain why I didn't, partially to give you the perspective of the average user (and believe me, they are as lazy as I am, if not more, since they wouldn't bother writing this post ), but tbh mostly because mocking C++ is a more fun task to do at 2AM than installing random libraries.

    Anyway, remember the golden rule: your project should be buildable by running a single command. Your dependencies aren't PostgreSQL, it's OK to embed a 3000 lines of C++ into your repo.

    opened by jerkstorecaller 4
  • Link failure on GCC8 (missing std::filesystem symbols)

    Link failure on GCC8 (missing std::filesystem symbols)

    Regrettably, my second attempt to install libtree (after #31 was fixed) also failed (gcc 8.3, rhel7):

    /projects/spack/lib/spack/env/gcc/g++ -O2 -g -DNDEBUG -rdynamic CMakeFiles/libtree.dir/src/main.cpp.o CMakeFiles/libtree_lib.dir/src/deploy.cpp.o CMakeFiles/libtree_lib.dir/src/deps.cpp.o CMakeFiles/libtree_lib.dir/src/elf.cpp.o CMakeFiles/libtree_lib.dir/src/exec.cpp.o CMakeFiles/libtree_lib.dir/src/glob.cpp.o CMakeFiles/libtree_lib.dir/src/ld.cpp.o -o libtree  -Wl,-rpath,::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: 
    CMakeFiles/libtree.dir/src/main.cpp.o: In function `std::filesystem::__cxx11::path::path<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::filesystem::__cxx11::path>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::filesystem::__cxx11::path::format)':
    /projects/spack/opt/spack/gcc-4.8.5/gcc/wsnp2ss/include/c++/8.3.0/bits/fs_path.h:184: undefined reference to `std::filesystem::__cxx11::path::_M_split_cmpts()'
    

    spack-build-out.txt

    It looks like you may need to explicitly link to c++17fs, see discussion on https://gitlab.kitware.com/cmake/cmake/-/issues/17834 which makes it pretty clear that only GCC<=8 needs this special target link 🙄

    2.x 
    opened by sethrj 4
  • Feature request: show size info

    Feature request: show size info

    can libtree show all libraries it depends on with size info, and a summary of all those sizes? very useful for embedded system where storage space is very important.

    opened by laoshaw 0
  • Avoid implicit function declarations in tests

    Avoid implicit function declarations in tests

    Future compilers are likely to reject implicit function declarations by default, causing these tests to fail. Also replace () with (void) where appropriate in the changed tests.

    Related to:

    • https://fedoraproject.org/wiki/Changes/PortingToModernC
    • https://fedoraproject.org/wiki/Toolchain/PortingToModernC
    opened by fweimer-rh 0
  • Add cpu arch conditions to 05_32_bits

    Add cpu arch conditions to 05_32_bits

    This should solve #78 as well as make check on other platforms that does not support -m32 or -m64 options.

    Tested on Arch Linux x86_64 and Debian riscv64.

    I've only included part of platforms here, including x86_64 and i386, since I don't know the exact output of uname -m on those platforms. Feel free to append more platforms to the list.

    opened by hack3ric 1
  • 'make check' fails on '08_nodeflib'

    'make check' fails on '08_nodeflib'

    Hello,

    when I try to run tests, '08_nodeflib' fails:

    /tmp/guix-build-libtree-3.1.1.drv-0/source/tests/08_nodeflib$ make check
    echo 'int f(){return 1;}' | cc -z nodefaultlib -Wl,--no-as-needed -shared -Wl,-soname,lib_nodefaultlib.so -o lib_nodefaultlib.so -x c -
    echo 'int g(){return 1;}' | cc -Wl,--no-as-needed -shared -Wl,-soname,lib_defaultlib.so -o lib_defaultlib.so -x c -
    echo 'extern int f(); extern int g(); int main(){return f() + g();}' | cc -z nodefaultlib -o exe_a -Wl,--enable-new-dtags '-Wl,-rpath,$ORIGIN' -x c - -L. -l_nodefaultlib -l_defaultlib
    echo 'extern int g(); int main(){return g();}' | cc -z nodefaultlib -o exe_b -Wl,--disable-new-dtags '-Wl,-rpath,$ORIGIN' -x c - -L. -l_defaultlib
    ! ../../libtree -vvv exe_a # should likely not find libc
    exe_a 
    ├── lib_nodefaultlib.so [runpath]
    │   └── libc.so.6 [runpath]
    │       └── ld-linux-x86-64.so.2 [ld.so.conf]
    ├── lib_defaultlib.so [runpath]
    │   └── libc.so.6 [runpath]
    │       └── ld-linux-x86-64.so.2 [ld.so.conf]
    └── libc.so.6 [runpath]
        └── ld-linux-x86-64.so.2 [ld.so.conf]
    make: *** [Makefile:33: check] Error 1
    

    As far as I can see, the issue is that libtree finds libc when it shouldn't. I'm using the following compiler and linker versions:

    $ gcc --version
    gcc (GCC) 10.3.0
    Copyright (C) 2020 Free Software Foundation, Inc.
    This is free software; see the source for copying conditions.  There is NO
    warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
    
    $ ld --version
    GNU ld (GNU Binutils) 2.37
    Copyright (C) 2021 Free Software Foundation, Inc.
    This program is free software; you may redistribute it under the terms of
    the GNU General Public License version 3 or (at your option) a later version.
    This program has absolutely no warranty.
    

    The tests ran on Ubuntu 20.04.4 LTS

    Thanks, avp

    opened by artyom-poptsov 1
  • Sysroot support

    Sysroot support

    Hello, I am wondering more about why the --sysroot option was removed (https://github.com/haampie/libtree/commit/05ec3ac5e8f65083fd25fccb0380c11e30d63e89)?

    My use-case is applying libtree to binaries/libraries from extracted firmware image filesystems where I might not be able to compile and run libtree on the device, and this feature would be very nice to have.

    I implemented support (also rudimentary) for this myself before I noticed the aforementioned commit that reverted this functionality.

    Thank you!

    opened by ekilmer 2
Releases(v3.1.1)
Owner
Harmen Stoppels
Harmen Stoppels
FoxRaycaster, optimized, fixed and with a CUDA option

Like FoxRaycaster(link) but with a nicer GUI, bug fixes, more optimized and with CUDA. Used in project: Code from FoxRaycaster, which was based on thi

Błażej Roszkowski 2 Oct 21, 2021
A fast, distributed, high performance gradient boosting (GBT, GBDT, GBRT, GBM or MART) framework based on decision tree algorithms, used for ranking, classification and many other machine learning tasks.

Light Gradient Boosting Machine LightGBM is a gradient boosting framework that uses tree based learning algorithms. It is designed to be distributed a

Microsoft 14.5k Jan 5, 2023
APFS module for linux, with experimental write support (out-of-tree repository)

Apple File System ================= The Apple File System (APFS) is the copy-on-write filesystem currently used on all Apple devices. This module pro

APFS for Linux 260 Jan 4, 2023
DLPrimitives/OpenCL out of tree backend for pytorch

Pytorch OpenCL backend based on dlprimitives DLPrimitives-OpenCL out of tree backend for pytorch It is only beginning, but you can train some vision n

Artyom Beilis 89 Dec 27, 2022
C++11 wrapper for the LMDB embedded B+ tree database library.

lmdb++: a C++11 wrapper for LMDB This is a comprehensive C++ wrapper for the LMDB embedded database library, offering both an error-checked procedural

D.R.Y. C++ 263 Dec 27, 2022
Training and Evaluating Facial Classification Keras Models using the Tensorflow C API Implemented into a C++ Codebase.

CFace Training and Evaluating Facial Classification Keras Models using the Tensorflow C API Implemented into a C++ Codebase. Dependancies Tensorflow 2

null 7 Oct 18, 2022
Scalable, Portable and Distributed Gradient Boosting (GBDT, GBRT or GBM) Library, for Python, R, Java, Scala, C++ and more. Runs on single machine, Hadoop, Spark, Dask, Flink and DataFlow

eXtreme Gradient Boosting Community | Documentation | Resources | Contributors | Release Notes XGBoost is an optimized distributed gradient boosting l

Distributed (Deep) Machine Learning Community 23.6k Jan 3, 2023
CubbyDNN - Deep learning framework using C++17 in a single header file

CubbyDNN CubbyDNN is C++17 implementation of deep learning. It is suitable for deep learning on limited computational resource, embedded systems and I

Chris Ohk 30 Aug 16, 2022
A program to backup all of your game savefiles on your system, neatly, and into a single folder.

Savefile Saver I created this project as a solution to a simple, but annoying problem: Backing up my game savefiles. I wanted to be able to copy all o

Dominic Esposito 6 Oct 24, 2022
Square Root Bundle Adjustment for Large-Scale Reconstruction

Square Root Bundle Adjustment for Large-Scale Reconstruction

Nikolaus Demmel 205 Dec 20, 2022
C++ implementation of R*-tree, an MVR-tree and a TPR-tree with C API

libspatialindex Author: Marios Hadjieleftheriou Contact: [email protected] Revision: 1.9.3 Date: 10/23/2019 See http://libspatialindex.org for full doc

null 633 Dec 28, 2022
This is like Inverting Binary Tree, but instead of a Binary Tree it's a File Tree.

Invert File Tree in C++ This is like Inverting Binary Tree, but instead of the Binary Tree it's a File Tree. This is intended as a simple exercise to

Tsoding 12 Nov 23, 2022
An intrusive C++17 implementation of a Red-Black-Tree, a Weight Balanced Tree, a Dynamic Segment Tree and much more!

This is Ygg (short for Yggdrasil), a C++17 implementation of several intrusive data structures: several balanced binary search trees: a red-black Tree

Lukas Barth 98 Dec 25, 2022
Embed read-only filesystems into any C++11 program w. a single header, zero dependencies and zero modifications to your code

c-embed Embed read-only filesystems into any C++11 program w. a single header, zero dependencies and zero modifications to your code. Usage c-embed al

Nick McDonald 9 Dec 29, 2022
A small C library with that portably invokes native file open, folder select and file save dialogs.

Cross platform (Windows, Mac, Linux) native file dialog library with C and C++ bindings, based on mlabbe/nativefiledialog.

Bernard Teo 299 Dec 23, 2022
Protect files under a specific folder from deleting or moving by explorer.exe.

Explorer-Delete-Protection Protect files under a specific folder from deleting or moving by explorer.exe. Requierments: Microsoft Detours Library - ht

null 6 Nov 14, 2022
libcurses and dependencies taken from netbsd and brought into a portable shape (at least to musl or glibc)

netbsd-libcurses portable edition this is a port of netbsd's curses library for usage on Linux systems (tested and developed on sabotage linux, based

null 124 Nov 7, 2022
Lightweight C++ command line option parser

Release versions Note that master is generally a work in progress, and you probably want to use a tagged release version. Version 3 breaking changes I

null 3.3k Dec 30, 2022
Lightweight C++ command line option parser

Release versions Note that master is generally a work in progress, and you probably want to use a tagged release version. Version 3 breaking changes I

null 3.4k Jan 4, 2023