OverlayBD
Accelerated Container Image
Accelerated Container Image is an open-source implementation of paper "DADI: Block-Level Image Service for Agile and Elastic Application Deployment. USENIX ATC'20". It is a solution of remote container image by supporting fetching image data on-demand without downloading and unpacking the whole image before a container running.
At the heart of the acceleration is OverlayBD, which provides a merged view of a sequence of block-based layers as an block device. This repository is a component of Accelerated Container Image, provides an implementation of OverlayBD and as a third-party backing-store of tgt, which is an user space iSCSI target framework.
Getting Started
Building
Requirements
To build OverlayBD, the following dependencies are required:
-
CMake >= 3.8+
-
gcc/g++ >= 7+
-
Libaio, libcurl and openssl runtime and development libraries.
- CentOS/Fedora:
sudo yum install libaio-devel libcurl-devel openssl-devel - Debian/Ubuntu:
sudo apt install libcurl4-openssl-dev libssl-dev libaio-dev
- CentOS/Fedora:
-
The Linux target framework (tgt)
- CentOS/Fedora:
yum install scsi-target-utils, epel maybe required. - Debian/Ubuntu:
apt install tgt
- CentOS/Fedora:
-
iscsi-initiator-utils
- CentOS/Fedora:
sudo yum install iscsi-initiator-utils - Debian/Ubuntu:
sudo apt install open-iscsi
- CentOS/Fedora:
Build
You need git to checkout the source code:
git clone https://github.com/alibaba/overlaybd.git
The whole project is managed by CMake.
cd overlaybd
mkdir build
cd build
cmake ..
make -j
sudo make install
# restart tgt service to reload backing-store
sudo systemctl restart tgtd
A liboverlaybd.so file is installed to /usr/lib{64}/tgt/backing-store,tgtd service has to be restarted to load this dynamic library as a backing-store module. OverlayBD related command-line tools are installed to /opt/overlaybd/bin/.
Configure
The default configuration files are installed after make install.
Default configure file tgt-overlaybd.json is installed to /etc/overlaybd/.
{
"logLevel": 1,
"registryCacheDir": "/opt/overlaybd/registry_cache",
"registryCacheSizeGB": 1,
"credentialFilePath": "/opt/overlaybd/cred.json",
"ioEngine": 1,
"download": {
"enable": true,
"delay": 120,
"delayExtra": 30,
"maxMBps": 100
}
}
| Field | Description |
|---|---|
| logLevel | DEBUG 0, INFO 1, WARN 2, ERROR 3 |
| ioEngine | IO engine used to open local files: psync 0, libaio 1, posix aio 2. |
| registryCacheDir | The cache directory for remote image data. |
| registryCacheSizeGB | The max size of cache, in GB. |
| credentialFilePath | The credential used for fetching images on registry. /opt/overlaybd/cred.json is the default value. |
| download.enable | Whether background downloading is enabled or not. |
| download.delay | The seconds waiting to start downloading task after the OverlayBD device launched. |
| download.delayExtra | A random extra delay is attached to delay, avoiding too many tasks started at the same time. |
| download.maxMBps | The speed limit in MB/s for a downloading task. |
NOTE:
downloadis the config for background downloading. After an OverlayBD device is lauched, a background task will be running to fetch the whole blobs into local directories. After downloading, I/O requests are directed to local files. Different to other options, download config is reloaded when a device lauching.
Here is an example of credential file described by credentialFilePath field.
{
"auths": {
"hub.docker.com": {
"username": "username",
"password": "password"
},
"hub.docker.com/hello/world": {
"auth": "dXNlcm5hbWU6cGFzc3dvcmQK"
}
}
}
Credentials are reloaded when authentication is required. Credentials have to be updated before expiration if temporary credential is used, otherwise OverlayBD keeps reloading until a valid credential is set.
Important: The corresponding credential has to be set before launching devices.
Usage
OverlayBD is working together with overlaybd-snapshotter and ctr plugin. See EXAMPLES
Licenses
- OverlayBD is released under the General Public License, Version 2.0.
2 Dec 11, 2021
4 Dec 21, 2021
3 Jan 20, 2022
154 Nov 24, 2022
4 Aug 14, 2021
32 Dec 15, 2022
47 Jul 27, 2022
2.1k Jan 7, 2023
17 Mar 16, 2022
35 Nov 10, 2022
15 Nov 28, 2022
35 Dec 10, 2022
10.8k Dec 29, 2022
764 Jan 2, 2023
39 Dec 21, 2022
1.3k Jan 5, 2023
69 Dec 9, 2022