Cross-connect Linux interfaces with XDP

Related tags

xdp-xconnect
Overview

Cross-connect Linux interfaces with XDP redirect

Go Report Card GoDoc License: MIT Build Status

xdp-xconnect daemon is a long-running process that uses a YAML file as its configuration API. For example:

links:
    eth0: tap0
    veth2: veth3

Given the above YAML file, local Linux interfaces will be cross-connected (eth0<->tap0 and veth2<->veth3) with the following command:

sudo xdp-xconnect -conf config.yaml

This command will block, listening to any changes to the file and will perform "warm" reconfigurations on the fly.

Note: due to its nature (loading eBPF progs, maps and interacting with netlink), xdp-xconnect requires NET_ADMIN capabilities (root privileges are used for simplicity).

Theory

Each pair of interfaces will have an eBPF program attached to its XDP hook and will use bpf_redirect_map eBPF helper function to redirect packets directly to the receive queue of the peer interface. For mode details read this.

Prerequisites

Linux Kernel version > 4.14 (introduced veth XDP support) Go

Installation

Binary:

go get github.com/networkop/xdp-xconnect

Docker:

docker pull networkop/xdp-xconnect

Usage

Binary:

sudo xdp-xconnect -conf input.yaml

Docker:

docker run --net host -v$(pwd):/xc --privileged networkop/xdp-xconnect -conf /xc/input.yaml

Go code:

import "github.com/networkop/xdp-xconnect/pkg/xdp"

func main() {

    input := map[string]string{"eth1":"tap1"}

    app, err := xdp.NewXconnectApp(input)
    // handle error

    updateCh := make(chan map[string]string, 1)
    
    app.Launch(ctx, updateCh)
}

Demo

Create three network namespaces and three veth links:

sudo ip link add dev xconnect-1 type veth peer name xc-1
sudo ip link add dev xconnect-2 type veth peer name xc-2
sudo ip link add dev xconnect-3 type veth peer name xc-3
sudo ip netns add ns1
sudo ip netns add ns2
sudo ip netns add ns3

Move one side of each veth link into a correponding namespace and configure an IP address from 169.254.0.0/16 subnet:

sudo ip link set xc-1 netns ns1
sudo ip link set xc-2 netns ns2
sudo ip link set xc-3 netns ns3
sudo ip netns exec ns1 ip addr add 169.254.1.10/24 dev xc-1
sudo ip netns exec ns2 ip addr add 169.254.1.20/24 dev xc-2
sudo ip netns exec ns3 ip addr add 169.254.1.30/24 dev xc-3

Bring up both sides of veth links

sudo ip link set dev xconnect-1 up
sudo ip link set dev xconnect-2 up
sudo ip link set dev xconnect-3 up
sudo ip netns exec ns1 ip link set dev xc-1 up
sudo ip netns exec ns2 ip link set dev xc-2 up
sudo ip netns exec ns3 ip link set dev xc-3 up

At this point there should be no connectivity between IPs in individual namespaces, i.e. the following commands will return no output:

sudo ip netns exec ns1 ping 169.254.1.20 &
sudo ip netns exec ns1 ping 169.254.1.30 &

Start the xdp-xconnect app with and connect xconnect-1 to itself (see first.yaml):

cp testdata/first.yaml testdata/input.yaml
sudo go run ./main.go -conf testdata/input.yaml &

2021/03/03 20:08:41 Parsing config file: testdata/input.yaml
2021/03/03 20:08:41 App configuration: {Links:map[xconnect-1:xconnect-1]}

Now update the configuration by replacing the file to connect the first two interfaces (see second.yaml):

cp testdata/second.yaml testdata/input.yaml

2021/03/03 20:12:16 Parsing config file: testdata/input.yaml
2021/03/03 20:12:16 Error parsing the configuration file: EOF
2021/03/03 20:12:16 Parsing config file: testdata/input.yaml
2021/03/03 20:12:16 App configuration: {Links:map[xconnect-1:xconnect-2]}
64 bytes from 169.254.1.20: icmp_seq=113 ttl=64 time=0.068 ms
64 bytes from 169.254.1.20: icmp_seq=114 ttl=64 time=0.068 ms
64 bytes from 169.254.1.20: icmp_seq=115 ttl=64 time=0.075 ms

This proves that the first and second links are now connected. Now swap the connection over to the third link (see third.yaml):

cp testdata/third.yaml testdata/input.yaml

2021/03/03 20:13:53 Parsing config file: testdata/input.yaml
2021/03/03 20:13:53 Error parsing the configuration file: EOF
2021/03/03 20:13:53 Parsing config file: testdata/input.yaml
2021/03/03 20:13:53 App configuration: {Links:map[xconnect-1:xconnect-3]
64 bytes from 169.254.1.30: icmp_seq=207 ttl=64 time=0.075 ms
64 bytes from 169.254.1.30: icmp_seq=208 ttl=64 time=0.070 ms
64 bytes from 169.254.1.30: icmp_seq=209 ttl=64 time=0.071 ms
64 bytes from 169.254.1.30: icmp_seq=210 ttl=64 time=0.071 ms

Ping replies now come from the third link. Let's see what happens if we provide the malformed input (see bad.yaml)

cp testdata/bad.yaml testdata/input.yaml

2021/03/03 20:15:46 Parsing config file: testdata/input.yaml
2021/03/03 20:15:46 App configuration: {Links:map[xconnect-1:xconnect-asd]}
2021/03/03 20:15:46 Error updating eBPF: 1 error occurred:                                                                                                                   
	* Link not found

64 bytes from 169.254.1.30: icmp_seq=317 ttl=64 time=0.065 ms
64 bytes from 169.254.1.30: icmp_seq=318 ttl=64 time=0.065 ms
64 bytes from 169.254.1.30: icmp_seq=319 ttl=64 time=0.064 m

The app detected the error and cross-connect continues working with its last known configuration. Finally, we can terminate the application which will cleanup all of the configured state:

fg
[1]  + 1095444 running    sudo go run ./main.go -conf testdata/input.yaml
^C
2021/03/03 20:16:44 Received syscall:interrupt
2021/03/03 20:16:44 ctx.Done

Don't forget to cleanup test interfaces and namespaces:

sudo ip link del dev xconnect-1
sudo ip link del dev xconnect-2
sudo ip link del dev xconnect-3
sudo ip netns del ns1
sudo ip netns del ns2
sudo ip netns del ns3

Additional Reading and References

https://github.com/xdp-project/xdp-tutorial

https://docs.cilium.io/en/stable/bpf/

https://qmonnet.github.io/whirl-offload/2020/04/12/llvm-ebpf-asm/

https://github.com/takehaya/goxdp-template

https://github.com/hrntknr/nfNat

https://github.com/takehaya/Vinbero

https://github.com/tcfw/vpc

https://github.com/florianl/tc-skeleton

https://github.com/cloudflare/rakelimit

https://github.com/b3a-dev/ebpf-geoip-demo

https://github.com/lmb/ship-bpf-with-go

Issues
Owner
Michael Kashin
Michael Kashin
Michael Kashin
Ultra fast and low latency asynchronous socket server & client C++ library with support TCP, SSL, UDP, HTTP, HTTPS, WebSocket protocols and 10K connections problem solution

CppServer Ultra fast and low latency asynchronous socket server & client C++ library with support TCP, SSL, UDP, HTTP, HTTPS, WebSocket protocols and

Ivan Shynkarenka 669 Sep 12, 2021
ESP8266 WiFi Connection manager with fallback web configuration portal

ESP8266 WiFi Connection manager with fallback web configuration portal

null 4.8k Sep 18, 2021
C++ Kite Connect API library / SDK

Kite Connect API C++ client Overview Dependencies Getting dependencies Linux Others & uWS v0.14 Building & installation Examples REST API Ticker Docum

Zerodha Technology 19 Sep 18, 2021
Open hardware to measure EC and pH, drive pumps, and otherwise manage a mid-size hydroponic grow over Wi-Fi.

Hydromisc This is a single PCBA with all the necessary I/O to automate a typical small to mid-size hydroponic grow, controllable over Wi-Fi

null 39 Sep 20, 2021
eBPF implementation that runs on top of Windows

eBPF for Windows eBPF is a well-known technology for providing programmability and agility, especially for extending an OS kernel, for use cases such

Microsoft 1.1k Sep 20, 2021
reids client suport corotine

date title linkTitle author menu 2021-08-18 coro_redis 开发手记 coro_redis 开发手记 asura9527 ([@gqw](https://gqw.github.io)) sidebar name identifier parent w

asura9527 3 Sep 2, 2021
从零开始重写sylar C++高性能分布式服务器框架

sylar-from-scratch 从零开始重写sylar C++高性能分布式服务器框架。 在线文档:从零开始重写sylar C++高性能分布式服务器框架。 模块概述 日志模块 支持流式日志风格写日志和格式化风格写日志,支持日志格式自定义,日志级别,多日志分离等等功能。

rs232 21 Sep 18, 2021
Wangle is a framework providing a set of common client/server abstractions for building services in a consistent, modular, and composable way.

Wangle C++ networking library Wangle is a library that makes it easy to build protocols, application clients, and application servers. It's like Netty

Facebook 2.8k Sep 17, 2021
Cross-platform library for building Telegram clients

TDLib (Telegram Database library) is a cross-platform library for building Telegram clients. It can be easily used from almost any programming language.

Telegram Library 4k Sep 13, 2021
LibVNCServer/LibVNCClient are cross-platform C libraries that allow you to easily implement VNC server or client functionality in your program.

LibVNCServer: A library for easy implementation of a VNC server. Copyright (C) 2001-2003 Johannes E. Schindelin If you already used LibVNCServer, you

null 747 Sep 9, 2021
Local OXID Resolver (LCLOR) : Research and Tooling

hazmat5 Local OXID Resolver (LCLOR) : Research and Tooling Welcome to a repository on my research into DCOM's Local OXID Resolution mechanisms, and RP

Alex Ionescu 13 May 25, 2021
mTCP: A Highly Scalable User-level TCP Stack for Multicore Systems

README mTCP is a highly scalable user-level TCP stack for multicore systems. mTCP source code is distributed under the Modified BSD License. For more

null 1.7k Sep 19, 2021
Drogon: A C++14/17 based HTTP web application framework running on Linux/macOS/Unix/Windows

English | 简体中文 | 繁體中文 Overview Drogon is a C++14/17-based HTTP application framework. Drogon can be used to easily build various types of web applicat

An Tao 6.1k Sep 12, 2021
Zyre - an open-source framework for proximity-based peer-to-peer applications

Zyre - Local Area Clustering for Peer-to-Peer Applications Linux & MacOSX Windows Contents Overview Scope and Goals Ownership and License Using Zyre B

The ZeroMQ project 756 Sep 13, 2021
Professional ARP Spoofer written with C

professional-arp-spoofer Professional ARP Spoofer written with C Note: You can sniff the traffic between any two machines in the network with the help

Noob0x 4 Jul 14, 2021
Enabling the Windows Subsystem for Linux to include support for Wayland and X server related scenarios

Welcome to WSLg WSLg is short for Windows Subsystem for Linux GUI and the purpose of the project is to enable support for running Linux GUI applicatio

Microsoft 6.1k Sep 22, 2021
HTTP and WebSocket built on Boost.Asio in C++11

HTTP and WebSocket built on Boost.Asio in C++11 Branch Linux/OSX Windows Coverage Documentation Matrix master develop Contents Introduction Appearance

Boost.org 3.1k Sep 12, 2021
Brynet - Header Only Cross platform high performance TCP network library using C++ 11.

Brynet Header Only Cross platform high performance TCP network library using C++ 11. Build status Windows : Linux/MacOS : Features Header only Cross p

IronsDu 751 Sep 18, 2021
Linux Terminal Service Manager (LTSM) is a set of service programs that allows remote computers to connect to a Linux operating system computer using a remote terminal session (over VNC or RDP)

Linux Terminal Service Manager (LTSM) is a set of service programs that allows remote computers to connect to a Linux operating system computer using a remote terminal session (over VNC)

null 13 Sep 20, 2021