Automated upstream mirror for bpftool stand-alone build.

Overview

bpftool

This is a mirror of bpf-next Linux source tree's tools/bpf/bpftool directory, plus its few dependencies from under kernel/bpf/, and its supporting header files.

All the gory details of syncing can be found in scripts/sync-kernel.sh script.

Some header files in this repo (include/linux/*.h) are reduced versions of their counterpart files at bpf-next's tools/include/linux/*.h to make compilation successful.

BPF/bpftool usage and questions

Please check out the manual pages for documentation about bpftool. A number of example invocations are also displayed in this blog post.

All general BPF questions, including kernel functionality, bpftool features and usage, should be sent to [email protected] mailing list. You can subscribe to it here and search its archive here. Please search the archive before asking new questions. It very well might be that this was already addressed or answered before.

[email protected] is monitored by many more people and they will happily try to help you with whatever issue you have. This repository's PRs and issues should be opened only for dealing with issues pertaining to specific way this bpftool mirror repo is set up and organized.

Dependencies

Required:

  • libelf
  • zlib

Optional:

  • libbfd (for dumping JIT-compiled program instructions)
  • libcap (for better feature probing)
  • kernel BTF information (for profiling programs or showing PIDs of processes referencing BPF objects)
  • clang/LLVM (idem)

Build build

Initialize libbpf submodule

This repository uses libbpf as a submodule. You can initialize it when cloning bpftool:

$ git clone --recurse-submodules https://github.com/libbpf/bpftool.git

Alternatively, if you have already cloned the repository, you can initialize the submodule by running the following command from within the repository:

$ git submodule update --init

Build bpftool

To build bpftool:

$ cd src
$ make

To build and install bpftool on the system:

$ cd src
# make install

Building bpftool in a separate directory is supported via the OUTPUT variable:

$ mkdir /tmp/bpftool
$ cd src
$ OUTPUT=/tmp/bpftool make

Most of the output is suppressed by default, but detailed building logs can be displayed by passing V=1:

$ cd src
$ make V=1

Build bpftool's man pages

The man pages for bpftool can be built with:

$ cd docs
$ make

They can be installed on the system with:

$ cd docs
# make install

License

This work is dual-licensed under the GNU GPL v2.0 (only) license and the BSD 2-clause license. You can choose between one of them if you use this work.

SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)

Comments
  • argv[0] usage in print-outs

    argv[0] usage in print-outs

    Some prints use argv[0] as the name of the tool, e.g. bpftool --version. I don't like that.

    I find it weird e.g. that the output should change depending on how you call the tool – via $PATH or by absolute path. But we even faced a practical issue with that: https://github.com/NixOS/nixpkgs/pull/195934

    So I suggest that you consider changing this. You can get inspired in the linked pull request or not.

    opened by vcunat 9
  • failed to find skeleton map '.rodata.str1.1'

    failed to find skeleton map '.rodata.str1.1'

    Code like bpf_printk("hello\n") will generate a section '.rodata.str1.1'

    When I compile the above code with old version bpftool (the bpftool is installed by apt install in Ubuntu 20.10 kernel 5.8), the generated binary works well.

    But when I use the same environment with the latest bpftool version to compile the same code, the generated binary report the error "libbpf: failed to find skeleton map '.rodata.str1.1'".

    opened by rainkin1993 8
  • bpftool incorrectly prints

    bpftool incorrectly prints "true" for a false boolean

    Version:

    bpftool --version
    bpftool v6.7.0
    using libbpf v0.7
    features: libbpf_strict
    

    This happens only in a 32-bit environment and not 64-bit. I have a BPF array with value that looks like this:

    struct map_value {
       int a;
       int b;
       short c;
       bool d;
    };
    
    struct {
       __uint( type, BPF_MAP_TYPE_ARRAY );
       __uint( max_entries, 1 );
       __type( key, uint32_t );
       __type( value, struct map_value );
    } my_map SEC( ".maps" );
    

    bpftool map dump shows:

    bpftool map dump name my_map --pretty
    [{
            "key": ["0x00","0x00","0x00","0x00"
            ],
            "value": ["0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00"
            ],
            "formatted": {
                "key": 0,
                "value": {
                    "a": 0,
                    "b": 0,
                    "c": 0,
                    "d": true
                }
            }
        }
    ]
    

    The raw bytes look right (this is the initial state of the array) but the formatted output is wrong.

    bug 
    opened by atapie 6
  • Question: Getting map type info from bpf programs.

    Question: Getting map type info from bpf programs.

    Hi, I am working on a small tool which needs to identify map type (HASH/ARRAY etc) from bpf programs. I have seen BTF which is able to provide type of the key and value of the map but not the type of the map itself (which is the value of the map type variable). So what I want to ask is is there a way to identity the type of map being used by looking at the BPF programs.

    Not sure if this is the correct place to ask this question, if not please let me know. Thanks in advance.

    question 
    opened by dushyantbehl 4
  • mirror: Add Dockerfile: run bpftool in a container

    mirror: Add Dockerfile: run bpftool in a container

    I often need to use bpftool on systems where it is not installed and cannot be installed. With this Dockerfile I can build and push a container image containing bpftool:

         $ docker build -f Dockerfile -t bpftool .
    

    And use it on systems without bpftool:

         $ docker run --rm -ti --privileged --pid=host bpftool prog
    

    cc @mauriciovasquezbernal

    opened by alban 4
  • [bpftool] remove BPF_OBJ_NAME_LEN restriction when looking up bpf

    [bpftool] remove BPF_OBJ_NAME_LEN restriction when looking up bpf

    program by name

    bpftool was limiting the length of names to BPF_OBJ_NAME_LEN.

    Since https://github.com/libbpf/bpftool/commit/61833a284f48b90f6802c141c8356de64bb41e10 we can get the full program name from BTF.

    This diffs remove the restriction of name length when running bpftool prog show name ${name}.

    Test: Tested against some internal program names that were longer than BPF_OBJ_NAME_LEN, here a redacted example of what was ran to test.

    $ sudo bpftool prog show name some_long_program_name
    Error: can't parse name
    $ sudo ./bpftool prog show name some_long_program_name
    123456789: tracing  name some_long_program_name  tag taghexa  gpl ....
    ...
    ...
    ...
    
    opened by chantra 3
  • CO-RE feature detection not working

    CO-RE feature detection not working

    On Ubuntu focal I see:

    $ make
    ...                        libbfd: [ on  ]
    ...        disassembler-four-args: [ on  ]
    ...                          zlib: [ on  ]
    ...                        libcap: [ on  ]
    ...               clang-bpf-co-re: [ OFF ]
    

    After looking in Makefile.features it is looking for BTF_KIND_VAR, but I don't see that:

    
    $ clang --version
    clang version 10.0.0-4ubuntu1
    Target: x86_64-pc-linux-gnu
    Thread model: posix
    InstalledDir: /usr/bin
    
    [~/scm/bpftool/src (master)]
    $ printf '%s\n' 'struct s { int i; } __attribute__((preserve_access_index)); struct s foo;' | clang -g -target bpf -S -o - -x c -  | grep BTF_KIND_
            .long   1                       # BTF_KIND_STRUCT(id = 1)
            .long   5                       # BTF_KIND_INT(id = 2)
    

    Am I not configured correctly or should this check be more loose?

    opened by pmmccorm 3
  • Check zlib feature error

    Check zlib feature error

    Makefile.feature may check requirement lib like as zlib,but if clang's version 11.0.0 it's not work well. some code need escape character like \n you need add like that:

    -        $(shell echo $(ZLIB_PROBE) | \
    +        $(shell echo -e $(ZLIB_PROBE) | \
           $(CC) $(CFLAGS) -Wall -Werror $(LDFLAGS) -x c - -lz -S -o - >/dev/null 2>&1 \
         && echo 1)
    

    or it output error like that:

    <stdin>:1:18: error: extra tokens at end of #include directive [-Werror,-Wextra-tokens]
    #include <zlib.h>\n int main(void) {  z_stream zs;      inflateInit(&zs);       return 0; }
                     ^
                     //
    1 error generated.
    

    and other echo need echo -e replace

    Makefile.include need add EXTRA_WARNINGS in clang version is 11.0.0

          +        -Wno-unused-command-line-argument \
    
    bug 
    opened by aguaiyoung 3
  • bpftool incorrectly shows helper support for bpf_getsockopt

    bpftool incorrectly shows helper support for bpf_getsockopt

    Output:

    # bpftool feature probe | grep sockopt
    eBPF program_type cgroup_sockopt is available
    	- bpf_setsockopt
    	- bpf_getsockopt
    	- bpf_setsockopt
    	- bpf_getsockopt
    eBPF helpers supported for program type cgroup_sockopt:
    	- bpf_setsockopt
    	- bpf_getsockopt
    

    Expected output:

    # bpftool feature probe | grep sockopt
    eBPF program_type cgroup_sockopt is available
    	- bpf_setsockopt
    	- bpf_getsockopt
    eBPF helpers supported for program type cgroup_sockopt:
    

    Version:

    # bpftool --version
    bpftool v5.10.140
    

    Kernel Version:

    # uname -r
    5.10.0-14-amd64
    

    The helper bpf_getsockopt did not get merged into kernel until 5.12 due to my research. Cilium is using bpftool to check whether this helper exist. On kernel 5.10, bpftool show this helper exist, so cilium tries to use this helper and crashed because it does not exist. I tried bpftool on kernel 5.4.214, it correctly show the absence of the helper.

    opened by kuroa-me 2
  • May need add if_link.h in include/uapi/linux/ ?

    May need add if_link.h in include/uapi/linux/ ?

    make error build in clang 11.0.0 like that: netlink_dumper.c:50:14: error: use of undeclared identifier 'XDP_ATTACHED_MULTI' if (mode == XDP_ATTACHED_MULTI) { ^ netlink_dumper.c:55:24: error: use of undeclared identifier 'IFLA_XDP_SKB_PROG_ID'; did you mean 'IFLA_XDP_PROG_ID'? xdp_dump_prog_id(tb, IFLA_XDP_SKB_PROG_ID, "generic", true); ^~~~~~~~~~~~~~~~~~~~ IFLA_XDP_PROG_ID maybe miss if_link.h in usersapce? like add libbpf that https://github.com/libbpf/libbpf/tree/master/include/uapi/linux

    bug 
    opened by aguaiyoung 2
  • Bpftool sync 2022-11-16

    Bpftool sync 2022-11-16

    Pull latest libbpf from mirror and sync bpftool repo with kernel, up to the commits used for libbpf sync. This is an automatic update performed by calling the sync script from this repo:

    $ ./scripts/sync-kernel.sh . <path/to/>linux
    

    Additionally, add the relevant feature detection probe for the LLVM library which is now the default for disassembling JIT-compiled programs, and update the CI workflow accordingly.

    opened by qmonnet 1
  • the new introduced prog_attach_flags are not compatible

    the new introduced prog_attach_flags are not compatible

    https://github.com/libbpf/bpftool/blob/43b5daa16a4b8afd71bba491950b54219fc35988/src/cgroup.c#L235-L245 prog_attach_flags is the latest field in uapi but not exist in older kernel
    https://github.com/torvalds/linux/blob/7d2a07b769330c34b4deabeed939325c77a7ec2f/include/uapi/linux/bpf.h#L1400-L1407 and in kernel it will ensure all fields after it's last field(prog_cnt) must be zero

    /* helper macro to check that unused fields 'union bpf_attr' are zero */
    #define CHECK_ATTR(CMD) \
    	memchr_inv((void *) &attr->CMD##_LAST_FIELD + \
    		   sizeof(attr->CMD##_LAST_FIELD), 0, \
    		   sizeof(*attr) - \
    		   offsetof(union bpf_attr, CMD##_LAST_FIELD) - \
    		   sizeof(attr->CMD##_LAST_FIELD)) != NULL
    
    #define BPF_PROG_QUERY_LAST_FIELD query.prog_cnt
    
    static int bpf_prog_query(const union bpf_attr *attr,
    			  union bpf_attr __user *uattr)
    {
    	if (!capable(CAP_NET_ADMIN))
    		return -EPERM;
    	if (CHECK_ATTR(BPF_PROG_QUERY))
    		return -EINVAL;
             ...
    }  
    

    It seems the upstream kernel doesn't have the compatibility issue because libbpf/bpftool is as the part of it's source.

    opened by mrpre 1
  • Failed to build bpftool from source

    Failed to build bpftool from source

    clang \
            -I. \
            -I/home/ubuntu/bpf/bpftool/include/uapi/ \
            -I/home/ubuntu/bpf/bpftool/src/bootstrap/libbpf/include \
            -g -O2 -Wall -target bpf -c skeleton/pid_iter.bpf.c -o pid_iter.bpf.o
    skeleton/pid_iter.bpf.c:47:14: error: incomplete definition of type 'struct bpf_perf_link'
            perf_link = container_of(link, struct bpf_perf_link, link);
                        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    
    opened by chenhengqi 8
  • libbpf submodule

    libbpf submodule

    Is there a plan to drop the libbpf submodule, and/or to allow building bpftool against the stand-alone libbpf? Right now this requires modifications to the Makefile, making life of people trying to package bpftool harder than it needs to be. Also, the release tarball does not include libbpf, so it can't be built without extra actions.

    opened by stintel 1
Releases(v7.0.0)
  • v7.0.0(Sep 1, 2022)

    Note Bpftool's version number is aligned on libbpf's version number (with an offset). The change of major version number for the current release reflects libbpf's bump (v1.0.0), but is not synonym of breaking changes in bpftool.

    New features

    • Implement bpftool cgroup tree for BPF_LSM_CGROUP
    • Use strings provided by libbpf for program types, map types, attach types, link types, instead of redefining them in bpftool, including for parsing these types from the command line
    • Add bpftool feature list_builtins to list program, map, attach, or link types known to libbpf (but independent from what the system supports), to allow iterating on these types in scripts, and use this in bash completion
    • Support passing longer strings for referencing programs by name on the command line, now that the kernel can use longer names with BTF
    • Add support for BTF_KIND_ENUM64 in BTF
    • Add support for BTF_KIND_RESTRICT for bpftool gen min_core_btf
    • Support BPF_CORE_TYPE_MATCHES relocations, for bpftool gen min_core_btf
    • Indicate that a pinned object is a BPF link (when relevant) instead of “unknown” when passing a pinned path to an incorrect object type on the command line

    Bug fixes

    • Fix compilation with libbfd 2.39 and newer, following a change of interface of the function used to initialise the context for the disassembler and which needs an additional printer for styled output
    • Fix feature-libbfd-liberty-z detection in the mirror's Makefile
    • Fix bootstrap for cross-compilation, by passing down the host ar to libbpf's Makefile
    • Clean errno at the beginning of the program execution to clean up after the checks performed by libcap < 2.63 and avoid breaking the batch mode
    • Check for NULL pointer in codegen_asserts() to avoid a NULL pointer dereference on skeleton generation
    • Revert the switch to libbpf's API to lift the memlock rlimit restriction, as detection for cgroup-based memory accounting would fail in some corner cases, and implement instead a more thorough probe in bpftool to check whether we need to lift the rlimit
    • Do not return value from void function in skeleton, which would produce a warning

    Other internal changes

    • Remove zlib feature test from Makefile
    • Do not print the result of the disassembler-four-args feature test when building bpftool
    • Replace the use of multiple sizeof() by ARRAY_SIZE() on the array of metrics in prog.c
    • Remove outdated attach_type_name forward declaration in main.h
    • Fix a typo in a comment, which would reference the wrong library (libpcap/libcap)
    • CI: Build on several runners (Ubuntu 18.04, 20.04, 22.04)
    • README.md: Fix a broken link

    Known bugs

    • Some features requiring the use of “skeletons” (feature-clang-bpf-co-re compilation option) require kernel 5.15 or more recent, with CONFIG_PERF_EVENTS set, to compile bpftool (https://github.com/libbpf/bpftool/issues/17).
    • Map dumps with BTF information may print erroneous values for bool fields inside of a struct (https://github.com/libbpf/bpftool/issues/38).

    Full Changelog: https://github.com/libbpf/bpftool/compare/v6.8.0...v7.0.0

    Source code(tar.gz)
    Source code(zip)
  • v6.8.0(May 17, 2022)

    New features

    • Implement bpftool gen min_core_btf to generate the minimal BTF file necessary to load a program
    • Add C++-specific skeleton wrappers
    • Restore support for probing BPF features on hardware (such as Netronome SmartNICs)
    • Add bpf_cookie to bpftool link output
    • Add support for generating sub-skeletons for “incomplete” BPF object files
    • Defaults to sysfs vmlinux when dumping BTF objects and no base BTF was specified
    • Extend skeletons for LSM programs to create links
    • Add name for trace_kprobe_multi attach type
    • Add name for syscall program types
    • Add names for xdp, perf_event, kprobe_multi link types
    • Add distinction, for feature probes, between reversible or permanent disabling of bpf() syscall for unprivileged users
    • Handle better the probing of BPF helpers for programs types that are not available
    • Assert type sizes in BPF skeletons
    • Expand autogeneration warnings in skeletons by mentioning bpftool

    Bug fixes

    • Fix error when looking up for keys in map with no associated BTF information
    • Fix pretty-print dump for maps with no associated BTF information
    • Fix formatting of bytes_memlock for JSON map listing
    • Show process PIDs for BPF maps even if there is no associated BTF id or the map is frozen
    • Add the description of missing commands in the top-level bpftool.8 man page
    • Handle libbpf_probe_prog_type() and libbpf_probe_bpf_helper() errors correctly in feature probing

    Other internal changes

    • Switch to libbpf's API to lift the rlimit restriction, on legacy systems without memcg-based memory accounting for BPF
    • Replace the use of libc's reallocarray() with the implementation from libbpf
    • Remove redundant slashes in Makefile

    Known bugs

    • Some features requiring the use of “skeletons” (feature-clang-bpf-co-re compilation option) require kernel 5.15 or more recent, with CONFIG_PERF_EVENTS set, to compile bpftool (https://github.com/libbpf/bpftool/issues/17).

    Full Changelog: https://github.com/libbpf/bpftool/compare/v6.7.0...v6.8.0

    Source code(tar.gz)
    Source code(zip)
  • v6.7.0(Feb 18, 2022)

Owner
null
Easy automated syncing between your computers and your MEGA Cloud Drive

Easy automated syncing between your computers and your MEGA Cloud Drive

Mega Limited 1.3k Jan 7, 2023
Mirror of Apache C++ Standard Library

$Id$ Apache C++ Standard Library (STDCXX) 5.0.0 ------------------------------------------ 0 Index -------- Inde

The Apache Software Foundation 56 Oct 6, 2022
Header only library for writing build recipes in C.

nobuild Header only library for writing build recipes in C. Main idea The idea is that you should not need anything but a C compiler to build a C proj

Tsoding 110 Dec 24, 2022
This repo contains Direct3D 9, Direct3D 10, a few Direct3D 11, and DirectSound C++ samples from the legacy DirectX SDK updated to build using the Windows 10 SDK and the Microsoft.DXSDK.D3DX NuGet package

DirectX SDK Legacy Samples This repo contains Direct3D 9, Direct3D 10, a few Direct3D 11, and DirectSound samples that originally shipped in the legac

Chuck Walbourn 44 Jan 2, 2023
Build a redis client by cpp:)

Redix A light redis client implement by c++. Develop Environment cmake:3.16.3 g++:9.3.0 os:ubuntu 20.04.01 reids:5.0.7 Welcome to see my talk. Benchma

null 3 Apr 15, 2022
Nin - It's metal to my ears. Build system in Go.

nin An experimental fork of ninja translated in Go. Currently a toy. Are you serious? Yeah. Marc-Antoine, isn't it a stupid idea? Yeah. When Google wa

M-A 10 Nov 3, 2022
Zenotech 6 Oct 21, 2022
Custom ESPHome Component for generic Sit-Stand-Desks

ESPHomeGenericSitStandDesk I have one of those generic relatively cheap Sit Stand Desks. In an effort to monitor my desk usage I developed this overki

Till 18 Dec 27, 2022
wl-mirror - a simple Wayland output mirror client

wl-mirror - a simple Wayland output mirror client wl-mirror attempts to provide a solution to sway's lack of output mirroring by mirroring an output o

Ferdinand Bachmann 118 Dec 29, 2022
Wtf Riot? I just want to close League of Legends and live my life. Leave me alone. F*ck corporate adware.

RiotKiller Wtf Riot? Anyways... This application launches League of Legends by calling RiotClientServices.exe --launch-product=league_of_legends --lau

Alejandro Romano 1 Mar 29, 2022
Free in the Dark, a Alone in the Dark engine reimplementation.

FITD Free in the Dark, a Alone in the Dark engine reimplementation. Long overdue source code update from the version that was released on source forge

null 47 Dec 20, 2022
Generic force-feedback vibrator HAL for upstream haptics drivers

vibrator-ff, a generic vibrator HAL for force feedback haptics. Most downstream haptics / vibrator drivers are implemented as LED class devices, this

AOSP Mainline 4 Feb 2, 2022
upstream module that allows nginx to communicate directly with PostgreSQL database.

About ngx_postgres is an upstream module that allows nginx to communicate directly with PostgreSQL database. Configuration directives postgres_server

RekGRpth 1 Apr 29, 2022
Unofficial git mirror of SQLite sources (see link for build instructions)

SQLite Source Repository This repository contains the complete source code for the SQLite database engine. Some test scripts are also included. Howeve

null 2k Dec 25, 2022
Minimal Linux Live (MLL) is a tiny educational Linux distribution, which is designed to be built from scratch by using a collection of automated shell scripts. Minimal Linux Live offers a core environment with just the Linux kernel, GNU C library, and Busybox userland utilities.

Minimal Linux Live (MLL) is a tiny educational Linux distribution, which is designed to be built from scratch by using a collection of automated shell scripts. Minimal Linux Live offers a core environment with just the Linux kernel, GNU C library, and Busybox userland utilities.

John Davidson 1.3k Jan 8, 2023
An open source python library for automated feature engineering

"One of the holy grails of machine learning is to automate more and more of the feature engineering process." ― Pedro Domingos, A Few Useful Things to

alteryx 6.4k Jan 7, 2023
Easy automated syncing between your computers and your MEGA Cloud Drive

Easy automated syncing between your computers and your MEGA Cloud Drive

Mega Limited 1.3k Jan 7, 2023
Powerful automated tool for reverse engineering Unity IL2CPP binaries

Powerful automated tool for reverse engineering Unity IL2CPP binaries

Katy 2.1k Jan 7, 2023
Utility on top of the Flutter Driver API that facilitates measuring the performance of your app in an automated way created by Very Good Ventures 🦄

Very Good Performance Developed with ?? by Very Good Ventures ?? Utility on top of the Flutter Driver API that facilitates measuring the performance o

Very Good Open Source 80 Dec 22, 2022
Proof-of-concept implementation for the paper "Osiris: Automated Discovery of Microarchitectural Side Channels" (USENIX Security'21)

Osiris This repository contains the implementation of the Osiris framework discussed in the research paper "Osiris: Automated Discovery of Microarchit

CISPA 41 Nov 11, 2022