USENIX 2021 - Nyx: Greybox Hypervisor Fuzzing using Fast Snapshots and Affine Types

Related tags

Miscellaneous Nyx
Overview

Nyx: Greybox Hypervisor Fuzzing using Fast Snapshots and Affine Types

Nyx is fast full-VM snapshot fuzzer for type-2 hypervisors. It's built upon kAFL, Redqueen and Hyper-Cube. The fuzzer is based on our paper (slides, recording):

@inproceedings {nyx,
author = {Sergej Schumilo and Cornelius Aschermann and Ali Abbasi and Simon W{\"o}r-ner and Thorsten Holz},
title = {Nyx: Greybox Hypervisor Fuzzing using Fast Snapshots and Affine Types},
booktitle = {30th {USENIX} Security Symposium ({USENIX} Security 21)},
year = {2021},
url = {https://www.usenix.org/conference/usenixsecurity21/presentation/schumilo},
}

Please note:

This release is based on our academic prototype implementation from early 2020 and it is quite outdated and has several known bugs, specific limitations and gets unstable in specific use cases (e.g. this version does not use the more stabile libxdc Intel-PT decoder).

If you are interested in using Nyx or one of Nyx's components for bug hunting or to build your project upon, we are recommending to use Nyx++ instead. Nyx++ is a mainted, improved version of Nyx with all of its latest and greatest features. Other than that, we are not planning to maintain this prototype implementation anymore. So please do yourself a favor and use Nyx++ instead.

Setup:

Install dependencies:

The following commands will install all required dependencies (assuming an Ubuntu 20.04):

sudo apt-get install gcc-8 libisoburn1 libgcc-8-dev-i386-cross libx32gcc-8-dev-i386-cross build-essential curl qemu-utils python3-pip python-is-python3
pip install msgpack
sudo apt-get remove libcapstone3

# install rust
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

# install capstone-v4
git clone https://github.com/aquynh/capstone.git && \
cd capstone && \
git checkout v4 && \
make && \
sudo make install && \
cd -

# install all QEMU build dependencies (requires deb-src URIs in your /etc/apt/sources.list file)
sudo apt-get build-dep qemu

KVM-PT

One of Nyx's main components is KVM-PT. This standalone LKM is based on KVM (Kernel 4.20-rc4), but implements multiple features on top of KVM required to build a fast coverage-guided ring -1 fuzzer. KVM-PT provides an Intel-PT tracing module for vCPUs (instead of Intel-PT tracing per logical CPU), nested hypercalls (which are fully transparent to L1), an in-kernel dirty-memory tracker and many more other hacks.

  1. Install the following kernel and reboot your system. Ensure that you are using the right kernel by using uname:

    cd /tmp/ && \
    wget -c https://kernel.ubuntu.com/~kernel-ppa/mainline/v4.19/linux-headers-4.19.0-041900_4.19.0-041900.201810221809_all.deb && \
    wget -c https://kernel.ubuntu.com/~kernel-ppa/mainline/v4.19/linux-headers-4.19.0-041900-generic_4.19.0-041900.201810221809_amd64.deb && \
    wget -c https://kernel.ubuntu.com/~kernel-ppa/mainline/v4.19/linux-image-unsigned-4.19.0-041900-generic_4.19.0-041900.201810221809_amd64.deb && \
    wget -c https://kernel.ubuntu.com/~kernel-ppa/mainline/v4.19/linux-modules-4.19.0-041900-generic_4.19.0-041900.201810221809_amd64.deb && \
    sudo dpkg -i *.deb && \
    sudo reboot
    
    
  2. After the system reboot verify that your system is using kernel 4.19 (by using uname -a). Otherwise, you can manually set kernel 4.19 in your grub configuration by adding the following line to /etc/default/grub and by generating a new GRUB config via update-grub:

GRUB_DEFAULT="Advanced options for Ubuntu>Ubuntu, with Linux 4.19.0-041900-generic"
  1. Execute load.sh in KVM-PT/ to compile and load KVM-PT.
  2. Check if the /dev/kvm-pt device node has been created.

QEMU-PT:

QEMU-PT is the user-space counterpart of KVM-PT. It is based on QEMU 4.2.50 and implements userspace handlers for hypercalls, fast snapshots and a custom Intel-PT decoder.

  1. Run sh compile_qemu_pt.h in QEMU-PT/ to build QEMU-PT. `

Build GRUB:

  1. Move to nyx_fuzzer/hypertrash_spec/build/hypertrash_os/grub/ and unpack grub.zip (unzip grub.zip). In case you want to compile GRUB by yourself instead of using our precompiled binaries, you can use install.sh to do so (gcc-6 is required).

Nyx-Fuzzer:

  1. Move to nyx_fuzzer/rust_fuzzer and run cargo build --release to build the fuzzer.

Getting Started:

bhyve (AHCI) Fuzzing:

  1. Move to the Targets/bhyve/VM folder.
  2. Execute prepare.sh. This script will download an ISO image of FreeBSD 11-3, prepare a virtual disk and spawn a QEMU instance. Connect via VNC (localhost:5900) and install FreeBSD. Install the following packages: base-dbg, kernel-dbg, ib32, ports. Create the follwing users: root / password: root, user / password: userand add user to wheelgroup.
  3. Once the OS installation has finished, reboot the VM and execute copy_files.shon the host (in another shell). Connect via SSH (localhost:2222) and run install.sh as root. Once the script has finished and the OS has shutdown, you can terminate QEMU via CTRL+c.
  4. Run create_pre_snapshot.sh and connect via VNC. Login as rootand run /home/user/loader. QEMU will create a pre-snapshot image and terminate.
  5. Create a config file by executing cp Targets/bhyve/config_template.ron Targets/bhyve/config.ron and adjust all paths in Targets/bhyve/config.ron (use absolute paths).
  6. Run make_sharedir.sh in nyx_fuzzer/hypertrash_spec/.
  7. Move to nyx-fuzzer/rust_fuzzer and execute cargo run --release -- -c ../../Targets/bhyve/config.ron to start the fuzzer.
  8. Use the following command to show all findings: cat /tmp/workdir_bhyve/corpus/crash/*.log | sort | uniq -c

qemu (legacy XHCI) Fuzzing:

  1. Move to the Targets/qemu/VM folder.
  2. Execute prepare.sh. This script will download an ISO image of Ubuntu 18-04 LTS, prepare a virtual disk and spawn a QEMU instance. Connect via VNC (localhost:5900) and install Ubuntu. Create the following users: user: user / password: userand add install openssh-server.
  3. Once the OS installation has finished, reboot the VM and execute copy_install_files.shon the host (in another shell). Connect via SSH (localhost:2223) and run ./install.sh as root. Once the install script is done and the OS has shutdown, you can terminate QEMU via CTRL+c.
  4. Run create_pre_snapshot.sh and connect via VNC. Login as rootand run /home/user/loader. QEMU will create a pre-snapshot image and exit.
  5. Create a config file by executing cp Targets/qemu/config_template.ron Targets/qemu/config.ron and adjust all paths in Targets/qemu/config.ron (use absolute paths).
  6. Run make_sharedir.sh in nyx_fuzzer/hypertrash_spec/.
  7. Move to Targets/qemu/agent and run build_sharedir.sh.
  8. Move to nyx-fuzzer/rust_fuzzer and execute cargo run --release -- -c ../../Targets/qemu/config.ron to start the fuzzer.
  9. Use the following command to show all findings: cat /tmp/workdir_qemu/corpus/crash/*.log | sort | uniq -c

License

Nyx is provided under GPLv2 license.

Free Software Hell Yeah!

Proudly provided by:

You might also like...
Little driver for detect UM/KM debugger and Hypervisor
Little driver for detect UM/KM debugger and Hypervisor

MAJESTY-technologies Little driver for protecthion. The driver is still under development, so you can submit your ideas! I write it's for manual map d

TAFuzzer: Effective and Efficient Targeted Fuzzing framework for Smart Contract Vulnerability Detection (CCS2022a Under Review).

TAFuzzer An effective and efficient targeted fuzzing framework for smart contract vulnerability detection. Requirements TAFuzzer is supported on Linux

This Program Enables And Disables Hyper-V Hypervisor So You Can Use Other Virtualisation Tools Such As (VMware, VirtualBox) Simultaneously.

Hyper-V-Switch This Program Enables And Disables Hyper-V Hypervisor So You Can Use Other Virtualisation Tools Such As (VMware, VirtualBox) Simultaneou

A refactored Proof-of-concept originally developed in 2017 to print all function calls with their arguments data types and values using Ptrace during program execution.

print-function-args-debugger A refactored Proof-of-concept originally developed in 2017 to print all function calls with their arguments data types an

General purpose power controller, capable of driving soldering irons using different voltages and probe types.
General purpose power controller, capable of driving soldering irons using different voltages and probe types.

All-purpose Power Micro Controller This general purpose power micro controller features: Wheatstone Bridge front-end New Texas Instruments INA823 inst

CollabFuzz: A Framework for Collaborative Fuzzing

Collaborative Fuzzing Design In this cooperative framework, the fuzzers collaborate using a centralized scheduler.

Owfuzz: a WiFi protocol fuzzing tool
Owfuzz: a WiFi protocol fuzzing tool

owfuzz owfuzz: a WiFi protocol fuzzing tool using openwifi. Openwifi is an open-source WiFi protocol stack based on SDR that is fully compatible with

BSOD: Binary-only Scalable fuzzing Of device Drivers
BSOD: Binary-only Scalable fuzzing Of device Drivers

bsod-kernel-fuzzing This repository contains the implementations described in "BSOD: Binary-only Scalable fuzzing Of device Drivers". The paper and th

OSS-Fuzz - continuous fuzzing for open source software.
OSS-Fuzz - continuous fuzzing for open source software.

OSS-Fuzz: Continuous Fuzzing for Open Source Software Fuzz testing is a well-known technique for uncovering programming errors in software. Many of th

Comments
  • Differential fuzzing support

    Differential fuzzing support

    Thinking about using Nyx to do some differential fuzzing between kernel versions. I'm happy to implement this support myself, but wanted to ask here: what would be the best way to go about this for contributing? Don't want to make massive changes, submit a PR, and find out that there was a preferred way to do it. :grin:

    To that end, here are the main changes I think I would make; if you have a preferred method, please let me know. I'll use intuition or documentation for the rest.

    • [ ] final state hypercall/buffer (for when target executes successfully and we want to extract the output for comparison)
    • [ ] flags or cmdline for differential mode (i.e. preparing for 2..n targets instead of 1)
    • [ ] parallelism or no parallelism for executing the targets? (should the targets be run sequentially or in parallel? I'd do parallel but not sure what internal changes would need to be made)
    • [ ] which module the comparison of outputs takes place
    • [ ] other changes I missed? (not too familiar with the project code just yet)

    Additionally, if there are any design constraints regarding what differential fuzzing looks like here, let me know and I'll adjust. My personal approach will look something like:

    1. for each target:
      1. run the target with fuzz input
      2. if it crashes, stash crash
      3. if it doesn't crash, stash output state
    2. if any crashed, report crashes and quit (unless ignoring crashes; if so, keep going)
    3. compare each output of non-crashed; if any are different, report and quit (unless ignoring differences; if so, keep going)
    4. mutate and go to 1
    opened by VTCAKAVSMoACE 1
  • How can Nyx get branch coverage by

    How can Nyx get branch coverage by "GCov" profiler?

    Hi guys,

    Nyx is super cool project. Applying nested virtualized fuzzing with Intel-PT is promising work. By the way, we are planning to check how much Nyx covers each branch coverage of virtual devices. However, we have trouble with collecting branch coverage using the "GCov" profiler. Could you share how Nyx modifies to get branch coverage?

    Thank you in advance!

    opened by ir0nc0w 1
  • Can this version of Nyx fuzz targets besides bhyve AHCI and qemu legacy XHCI?

    Can this version of Nyx fuzz targets besides bhyve AHCI and qemu legacy XHCI?

    Great work! I am learning how Nyx works and want to fuzz the latest qemu e1000 device. I notice the function “make_legacy_e1000”, so I guess this version of Nyx can support e1000 as well.

    It seems I need to change the following positions.

    1. Targets/qemu/agent/src/run.sh
    LD_PRELOAD=./hypertrash_crash_detector qemu-system-x86_64 -nographic -cdrom hypertrash.iso -m 100 -enable-kvm -device virtio-gpu-pci -device nec-usb-xhci -nographic
    
    1. Targets/qemu/misc/install.sh
    wget https://download.qemu.org/qemu-4.2.0.tar.xz
    
    tar xf qemu-4.2.0.tar.xz
    cd qemu-4.2.0
    
    ./configure --target-list=x86_64-softmmu --disable-werror --disable-capstone --enable-sanitizers
    make
    

    Do I miss anything?

    Keep great work!

    opened by cyruscyliu 0
Owner
Chair for Sys­tems Se­cu­ri­ty
Chair for Sys­tems Se­cu­ri­ty
Leo Hypervisor. Type 1 hypervisor on Raspberry Pi 4 machine.

Leo Hypervisor Leo. Type 1 hypervisor on Raspberry Pi 4 machine. Set Up This setup was tested on Debian Install Required Dependencies and Toolchain su

Mushahid Hussain 15 Dec 5, 2022
Arbitrary Precision provides C++ long integer types that behave as basic integer types. This library aims to be intuitive and versatile in usage, rather than fast.

Arbitrary Precision (AP) Cross-platform and cross-standard header-only arbitrary precision arithmetic library. Currently it offers integer types that

null 17 Sep 28, 2022
Proof-of-concept implementation for the paper "Osiris: Automated Discovery of Microarchitectural Side Channels" (USENIX Security'21)

Osiris This repository contains the implementation of the Osiris framework discussed in the research paper "Osiris: Automated Discovery of Microarchit

CISPA 41 Nov 11, 2022
Exploring the Design Space of Page Management for Multi-Tiered Memory Systems (USENIX ATC'21)

AutoTiering This repo contains the kernel code in the following paper: Exploring the Design Space of Page Management for Multi-Tiered Memory Systems (

Computer Systems Laboratory @ Ajou University 23 Dec 20, 2022
The artifact associated with our ISSTA 2021 paper "Seed Selection for Successful Fuzzing"

Seed Selection for Successful Fuzzing The artifact associated with our ISSTA 2021 paper "Seed Selection for Successful Fuzzing". While our primary art

HexHive 34 Dec 7, 2022
rax/RAX is a C++ extension library designed to provide new, fast, and reliable cross-platform class types.

rax rax/RAX is a C++ extension library designed to provide cross-platform new, fast, and reliable class types for different fields such as work with I

MaxHwoy 5 May 2, 2022
StochFuzz - Sound and Cost-effective Fuzzing of Stripped Binaries by Incremental and Stochastic Rewriting

StochFuzz: A New Solution for Binary-only Fuzzing StochFuzz is a (probabilistically) sound and cost-effective fuzzing technique for stripped binaries.

Zhuo Zhang 164 Dec 5, 2022
Gunyah is a Type-1 hypervisor designed for strong security, performance and modularity.

Gunyah is a Type-1 hypervisor, meaning that it is independent of any high-level OS kernel, and runs in a higher CPU privilege level. It does not depend on any lower-privileged OS kernel/code for its core functionality. This increases its security and can support a much smaller trusted computing base than a Type-2 hypervisor.

Qualcomm Innovation Center 84 Jan 2, 2023
Some hypervisor research notes. There is also a useful exploit template that you can use to verify / falsify any assumptions you may make while auditing code, and for exploit development.

Introduction Over the past few weeks, I've been doing some hypervisor research here and there, with most of my focus being on PCI device emulation cod

Faith 130 Nov 18, 2022
User-mode trap-and-emulate hypervisor for RISC-V

URVirt URVirt is a U-mode trap-and-emulate hypervisor for RISC-V. Related projects RVirt is an S-mode trap-and-emulate hypervisor for RISC-V. It runs

null 12 Sep 16, 2022