Source-code based coverage for eBPF programs actually running in the Linux kernel

Overview

bpfcov

Source-code based coverage for eBPF programs actually running in the Linux kernel

This project provides 2 main components:

  1. libBPFCov.so - an out-of-tree LLVM pass to instrument your eBPF programs for coverage.
  2. bpfcov - a CLI to collect source-based coverage from your eBPF programs.
Source-based code coverage for BPF raw tracepoints Source-based code coverage for BPF LSM programs HTML coverage index for multiple eBPF programs
HTML coverage report for eBPF programs HTML coverage report for eBPF programs JSON report for multiple eBPF programs
LCOV info file from multiple eBPF programs HTML line coverage report for eBPF programs HTML line coverage report for eBPF programs

Overview

This section aims to provide a high-level overiew of the steps you need to get started with bpfcov.

  1. Compile the LLVM pass obtaining libBPFCov.so
  2. Instrument your eBPF program by compiling it and by running the LLVM pass (libBPFCov.so) on it
  3. Build the userspace code of your eBPF application
  4. Execute your eBPF application in the kernel through the bpfcov run ... command
  5. Generate the .profraw file from the run through the bpfcov gen ... command
    1. Having a .profraw makes this tool fully interoperable
    2. Having a .profraw allows you to generate a variety of coverage reports in different formats
  6. Use the LLVM toolchain to create coverage reports as documented in the LLVM docs

In case you are impatient and want to jump straight into getting your hands dirty, then the examples directory contains a few dummy eBPF programs to showcase what bpfcov does.

It basically automates steps 2 and 3. Its README contains more details.

While the README of the cli directory gives you more details about the steps 4 and 5 (and also 6).

Usage

Here I will highlight the manual steps to use it.

I suggest you to automate most of them like I did in the examples Makefile.

Anyway, assuming you have built the LLVM pass, you can then use your fresh libBPFCov.so to instrument your eBPF programs for coverage (steps 2 and 3 above).

How to do it?

First, you need to compile your eBPF program almost as usual but to LLVM IR...

clang -g -O2 \
    -target bpf -D__TARGET_ARCH_x86 -I$(YOUR_INCLUDES) \
    -fprofile-instr-generate -fcoverage-mapping \
    -emit-llvm -S \
    -c program.bpf.c \
    -o program.bpf.ll

Notice it doesn't matter if you use the textual (*.ll) or the binary form (*.bc). Obviously, the former is more readable.

The same logic applies to opt: by default it generates *.bc. Using the -S flag you can obtain the output in textual form (*.ll).

Anyhow, it's time to run the LLVM pass on the LLVM IR we obtained.

Let's do it:

opt -load-pass-plugin $(BUILD_DIR)/lib/libBPFCov.so -passes="bpf-cov" \
    -S program.bpf.ll \
    -o program.bpf.cov.ll

We should have obtained a new LLVM IR that's now valid and loadable from the BPF VM in the Linux kernel. Almost there, YaY!

From it, we can obtain a valid BPF ELF now:

llc -march=bpf -filetype=obj -o cov/program.bpf.o program.bpf.cov.ll

While we are at it, it is also worth running the LLVM pass again (with a flag) to obtain another BPF ELF containing all the profiling and coverage mapping info. It will come in handy later with llvm-cov.

opt -load $(BUILD_DIR)/lib/libBPFCov.so -strip-initializers-only -bpf-cov \
    program.bpf.ll | \
    llc -march=bpf -filetype=obj -o cov/program.bpf.obj

At this point, we can compile our userspace application loading the eBPF instrumented program (cov/program.bpf.o).

Doing this when using libbpf and skeletons is very easy. Nothing different from the common steps: bpftool, cc, etc.

In the examples directory, you can find further explainations.

So assuming we got our instrumented binary ready (cov/program), we can run it via the bpfcov CLI.

sudo ./bpfcov run cov/program
# Wait for it to exit, or stop it with CTRL+C
sudo ./bpfcov gen --unpin cov/program

Again, in case you wanna know more about these 2 steps, refer this time to the CLI README.

Now we have a magic cov/program.profraw file...

And we can use the LLVM toolchain to generate very fine-grained coverage reports like those in the screenshots!

Refer to the LLVM docs to learn how to do it.

But no worries, it's just about invoking llvm-profdata and llvm-cov:

lvm-profdata merge -sparse cov/program.profraw -o cov/program.profdata
llvm-cov show \
    --format=html \
    --show-line-counts-or-regions --show-region-summary --show-branch-summary \
    --instr-profile=cov/profdata.profdata \
    -object cov/program.bpf.obj \
    --output-dir=cov/html_report

Anyayws, bpfcov also provides you an opinionated shortcut command to generate HTML, JSON, and LCOV coverage reports:

./bpfcov out --format=html cov/program.profraw

Development Environment

In order to build the BPFCov library (libBPFCov.so) you will need:

  • LLVM 12+
  • CMake 3.13.4+
  • C++ compiler that supports C++14

In order to use it, you will need:

  • clang 12 (to generate the input LLVM files)
  • its opt binary to run the LLVM pass

This project has been tested on 5.15 Linux kernels.

Building

Build as follows:

mkdir -p build && cd build
cmake -DLT_LLVM_INSTALL_DIR=/path/to/llvm/installation ..
make

Notice that the LT_LLVM_INSTALL_DIR variable should be set to the root of either the installation (usually /usr) or the build directory of LLVM.

It is used to locate the corresponding LLVMConfig.cmake script that is used to set the include and the library paths.

Testing

TBD

To run the tests you will need to install llvm-lit.

Usually, you can install it with pip:

pip install lit

Running the tests is as simple as:

lit build/test
You might also like...
bpflock - eBPF driven security for locking and auditing Linux machines

bpflock - Lock Linux machines bpflock - eBPF driven security for locking and auditing Linux machines. This is a Work In Progress: bpflock is currently

Drogon: A C++14/17 based HTTP web application framework running on Linux/macOS/Unix/Windows
Drogon: A C++14/17 based HTTP web application framework running on Linux/macOS/Unix/Windows

English | 简体中文 | 繁體中文 Overview Drogon is a C++14/17-based HTTP application framework. Drogon can be used to easily build various types of web applicat

Dolphin is an emulator for running GameCube and Wii games on Windows, Linux, macOS, and recent Android devices.

Dolphin is a GameCube / Wii emulator, allowing you to play games for these two platforms on PC with improvements.

GNUWeebBot - High-performance bot Telegram, running on Linux environment.

High-performance bot Telegram, running on Linux environment, written in C. Core Features Event Debug Event Logger Modules Telegram debug info.

SimpleSockets is a lightweight set of classes that allow developers to implement IP based network programs.

------------------------------------------------------------------------------------------ * History -------------------------------------------------

eBPF bytecode assembler and compiler

An eBPF bytecode assembler and compiler that * Assembles the bytecode to object code. * Compiles the bytecode to C macro preprocessors. Symbolic

Example how to run eBPF probes without a usermode process using fentry

Pinning eBPF Probes Simple example to demonstrate how to pin kernel function and syscall probes. Overview From my reading of the kernel code, KProbe a

eBPF implementation that runs on top of Windows
eBPF implementation that runs on top of Windows

eBPF for Windows eBPF is a well-known technology for providing programmability and agility, especially for extending an OS kernel, for use cases such

ebpfkit-monitor is a tool that detects and protects against eBPF powered rootkits

ebpfkit-monitor ebpfkit-monitor is an utility that you can use to statically analyse eBPF bytecode or monitor suspicious eBPF activity at runtime. It

Comments
  • failed to get EHDR from /sys/kernel/btf/vmlinux when compiling examples

    failed to get EHDR from /sys/kernel/btf/vmlinux when compiling examples

    Hello! First of all, by having a look at README, this tool looks so cool!

    I am following the instructions to compile the examples under examples/src. I generated libBPFCov.so. My bpftool version is v5.4.162. When running make I get:

    make: command: Command not found
      MKDIR    vmlinux/
      VMLINUX  vmlinux/vmlinux.h
    libbpf: failed to get EHDR from /sys/kernel/btf/vmlinux
    Error: failed to load BTF from /sys/kernel/btf/vmlinux: Unknown error -4001
    make: *** [Makefile:77: /home/ubuntu/dev/bpfcov/examples/src/.output/vmlinux/vmlinux.h] Error 95
    make: *** Deleting file '/home/ubuntu/dev/bpfcov/examples/src/.output/vmlinux/vmlinux.h'
    

    Apart from the Command not found error, looks like I cannot get EHDR from the vm "embedded" btf. My machine is: Linux 5.11.0-1025-aws #27~20.04.1-Ubuntu SMP Fri Jan 7 13:09:56 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

    Also, I tried to use bpftool with version 5.11.22, however it doesn't have the "skeleton" feature:

    bpftool version -j
    {"version":"5.11.22","features":{"libbfd":false,"skeletons":false}}
    

    Might be that v5.4.162, does not support skeleton as well. However I do not get such an indication from Makefile. My gut feeling is that the issue is with bpftool version. However, version 5.11.22 looks like the most updated for my OS, and if this one does not work, do you know which version might support skeleton feature? Or maybe, the problem might not be related to bpftool version. Thank you!

    opened by mtcherni95 1
Owner
elastic
elastic
A Rust crate that simplifies the integration of Rust and eBPF programs written in C.

This crate simplifies the compilation of eBPF programs written in C integrating clang with Rust and the cargo build system with functions that can be

Simone Margaritelli 19 Mar 16, 2022
A collection of eBPF programs demonstrating bad behavior

Bad BPF A collection of malicious eBPF programs that make use of eBPF's ability to read and write user data in between the usermode program and the ke

pat_h/to/file 285 Nov 25, 2022
A Linux Host-based Intrusion Detection System based on eBPF.

eHIDS 介绍 eBPF内核技术实现的HIDS demo. 功能实现: TCP网络数据捕获 UDP网络数据捕获 uprobe方式的DNS信息捕获 进程数据捕获 uprobe方式实现JAVA的RASP命令执行场景事件捕获 eBPF的go框架实现,针对kprobe\uprobe挂载方式,多类型even

CFC4N 288 Nov 24, 2022
Linux Application Level Firewall based on eBPF and NFQUEUE.

eBPFSnitch eBPFSnitch is a Linux Application Level Firewall based on eBPF and NFQUEUE. It is inspired by OpenSnitch, and Douane, but utilizing modern

Harpo Roeder 664 Nov 19, 2022
eBPF-based EDR for Linux

ebpf-edr A proof-of-concept eBPF-based EDR for Linux Seems to be working fine with the 20 basic rules implemented. Logs the alerts to stdout at the mo

null 15 Nov 9, 2022
Extract TLS session keys from running programs

Tlskeydump Tlskeydump extracts TLS key material from processes at runtime so that packet captures containing TLS-encrypted data can be decrypted and a

Konstantinos Tsanaktsidis 7 Sep 5, 2022
Linux Terminal Service Manager (LTSM) is a set of service programs that allows remote computers to connect to a Linux operating system computer using a remote terminal session (over VNC or RDP)

Linux Terminal Service Manager (LTSM) is a set of service programs that allows remote computers to connect to a Linux operating system computer using a remote terminal session (over VNC)

null 33 Nov 28, 2022
Parca-agent - eBPF based always-on profiler auto-discovering targets in Kubernetes and systemd, zero code changes or restarts needed!

Parca Agent Parca Agent is an always-on sampling profiler that uses eBPF to capture raw profiling data with very low overhead. It observes user-space

Parca 224 Nov 17, 2022
A virtual network Differential GNSS server-client project using Precise Point Positioning (PPP). Global coverage. Without physical base station construction needed. An open-source virtual base station approach.

Virtual-Network-DGNSS-Project This project is the software implementation for a publicly available, open-source, client/server VN-DGNSS implementation

null 13 Oct 24, 2022
libsinsp, libscap, the kernel module driver, and the eBPF driver sources

falcosecurity/libs As per the OSS Libraries Contribution Plan, this repository has been chosen to be the new home for libsinsp, libscap, the kernel mo

Falco 127 Nov 23, 2022