Boki: Stateful Serverless Computing with Shared Logs [SOSP '21]

Overview

Boki

Boki is a research FaaS runtime for stateful serverless computing with shared logs. Boki exports the shared log API to serverless functions, allowing them to manage states with strong consistency, durability, and fault tolerance. Boki uses Nightcore as the runtime for serverless functions.

Boki is the pronunciation of "簿記", meaning bookkeeping in Japanese.

Building Boki

Under Ubuntu 20.04, building Boki needs following dependencies installed:

sudo apt install g++ make cmake pkg-config autoconf automake libtool curl unzip

Once installed, build Boki with:

./build_deps.sh
make -j $(nproc)

Kernel requirements

Boki uses io_uring for asynchronous I/Os. io_uring is a very new feature in the Linux kernel (introduced in 5.1), and evolves rapidly with newer Linux kernel version.

Boki require Linux kernel 5.10 or later to run.

Boki support libraries

As presented in our SOSP paper, we build BokiFlow, BokiStore and BokiQueue for serverless use cases of transactional workflows, durable object storage, and message queues.

slib directory in this repository contains implementations of BokiStore and BokiQueue. For BokiFlow, check out workloads/workflow directory in ut-osa/boki-benchmarks repository.

Running Boki's evaluation workloads

A separate repository ut-osa/boki-benchmarks includes scripts and detailed instructions on running evaluation workloads presented in our SOSP paper.

Limitations of the current prototype

The shared log API is only exported to functions written in Go.

You might also like...
Single header C++ logging library. It is extremely powerful, extendable, light-weight, fast performing, thread and type safe and consists of many built-in features. It provides ability to write logs in your own customized format. It also provide support for logging your classes, third-party libraries, STL and third-party containers etc. Single header C++ logging library. It is extremely powerful, extendable, light-weight, fast performing, thread and type safe and consists of many built-in features. It provides ability to write logs in your own customized format. It also provide support for logging your classes, third-party libraries, STL and third-party containers etc.
Receive and process logs from the Linux kernel.

Netconsd: The Netconsole Daemon This is a daemon for receiving and processing logs from the Linux Kernel, as emitted over a network by the kernel's ne

Tiny ANSI C lib for logs

logger.c An ANSI C (C86) lib for logs Easy to use and easy. Build You can build this lib or copy/paste sources files in your project. cd build make

jvm-monitor is a lightweight monitoring tool that logs all the local variables whenever exceptions occur.

jvm-monitor jvm-monitor is a Java agent attached to a Java VM (virtual machine), which logs all the local variables when exceptions occur. Rationales

Shows Unreal Engine logs in-game using ImGui
Shows Unreal Engine logs in-game using ImGui

BYG Imgui Logger Displays Unreal's UE_LOG output in an ImGui window. Heavily based on the Console example from imgui_demo.cpp included with ImGui. Fea

Viewing files of technological logs 1C (WinAPI)
Viewing files of technological logs 1C (WinAPI)

YellowViewer Viewing files of technological logs 1C (WinAPI) 1C technological log files viewer (WinAPI). Works with large files. Minimal memory consum

A software C library designed to extract data attributes from network packets, server logs, and from structured events in general, in order to make them available for analysis

MMT-DPI A software C library desinged to extract data attributes from network packets, server logs, and from structured events in general, in odrder t

Sysmon event simulation utility which can be used to simulate the attacks to generate the Sysmon Event logs for testing the EDR detections and correlation rules by Blue teams.

SysmonSimulator SysmonSimulator is an Open source Windows event simulation utility created in C language, that can be used to simulate most of the att

Header-only, event based, tiny and easy to use libuv wrapper in modern C++ - now available as also shared/static library!

Do you have a question that doesn't require you to open an issue? Join the gitter channel. If you use uvw and you want to say thanks or support the pr

Shared-Memory Parallel Graph Partitioning for Large K

KaMinPar The graph partitioning software KaMinPar -- Karlsruhe Minimal Graph Partitioning. KaMinPar is a shared-memory parallel tool to heuristically

Implementation of System V shared memory (a type of inter process communication) in xv6 operating system.

NOTE: we have stopped maintaining the x86 version of xv6, and switched our efforts to the RISC-V version (https://github.com/mit-pdos/xv6-riscv.git)

In Memory Property Graph Server using the Shared Nothing design from Seastar.

RageDB In Memory Property Graph Server using the Shared Nothing design from Seastar. The RageDB server can host multiple Graphs. The graphs are access

Header-only, event based, tiny and easy to use libuv wrapper in modern C++ - now available as also shared/static library!

Do you have a question that doesn't require you to open an issue? Join the gitter channel. If you use uvw and you want to say thanks or support the pr

Cross-platform shared memory stream/buffer, header-only library for IPC in C/C++.
Cross-platform shared memory stream/buffer, header-only library for IPC in C/C++.

libsharedmemory libsharedmemory is a small C++11 header-only library for using shared memory on Windows, Linux and macOS. libsharedmemory makes it eas

A tool to test if a shared library is dlopen'ble

A tool to test if a shared library is dlopen'ble

A linux library to get the file path of the currently running shared library. Emulates use of Win32 GetModuleHandleEx/GetModuleFilename.

whereami A linux library to get the file path of the currently running shared library. Emulates use of Win32 GetModuleHandleEx/GetModuleFilename. usag

A command-line tool to extract dylib files from the dyld shared cache file.

DyldExtractor A command-line tool to extract dylib files from the dyld shared cache file. Starting with macOS 11, standalone binaries of system librar

A late bound, hope-for-the-best dyld shared cache extractor

yolo_dsc A late bound, hope-for-the-best dyld shared cache extractor why? There are other dsc_extract utilities. They usually require some combination

Comments
  • Cannot build boki

    Cannot build boki

    The dependencies directories are placeholders for their corresponding git repos.

    1. how is protobuf to be built? the build script does not fulfill the requirements for the generated makefile
    2. is GSL the gnu scientific library or some other package
    3. Is http-parser the node-js repo
    4. what tags should be checked out for the respective dependencies?

    Thanks, Dwight Browne

    opened by dbrownecu 0
  • Ordering layer evaluation

    Ordering layer evaluation

    Hi,

    I would like to stress-test the ordering layer of the system (where the storage tier, etc. would be mocked). Could you please let me know where the code of the ordering layer resides (in which files, etc.)?

    Thanks, Dimitra

    opened by dgiantsidi 2
Owner
Operating Systems and Architecture
Operating Systems and Architecture
An efficient C++17 GPU numerical computing library with Python-like syntax

MatX - Matrix Primitives Library MatX is a modern C++ library for numerical computing on NVIDIA GPUs. Near-native performance can be achieved while us

NVIDIA Corporation 549 Sep 9, 2022
GPTPU: General-Purpose Computing on (Edge) Tensor Processing Units

GPTPU: General-Purpose Computing on (Edge) Tensor Processing Units Welcome to the repository of ESCAL @ UCR's GPTPU project! We aim at demonstrating t

Extreme Storage and Computer Architecture Lab 32 Jul 31, 2022
Experimental and Comparative Performance Measurements of High Performance Computing Based on OpenMP and MPI

High-Performance-Computing-Experiments Experimental and Comparative Performance Measurements of High Performance Computing Based on OpenMP and MPI 实验结

Jiang Lu 1 Nov 27, 2021
SMID, Parallel computing of CNN

Parallel Computing in Deep Reference Network 1. Introduction Deep neural networks are made up of a number of layers of linked nodes, each of which imp

null 1 Dec 22, 2021
4eisa40 GPU computing : exploiting the GPU to execute advanced simulations

GPU-computing 4eisa40 GPU computing : exploiting the GPU to execute advanced simulations Activities Parallel programming Algorithms Image processing O

Ecam 4MIN repositories 2 Jan 10, 2022
MACE is a deep learning inference framework optimized for mobile heterogeneous computing platforms.

Mobile AI Compute Engine (or MACE for short) is a deep learning inference framework optimized for mobile heterogeneous computing on Android, iOS, Linux and Windows devices.

Xiaomi 4.7k Sep 22, 2022
Compressed Log Processor (CLP) is a free tool capable of compressing text logs and searching the compressed logs without decompression.

CLP Compressed Log Processor (CLP) is a tool capable of losslessly compressing text logs and searching the compressed logs without decompression. To l

null 55 Sep 16, 2022
Serverless SQLite database read from and write to Object Storage Service, run on FaaS platform.

serverless-sqlite Serverless SQLite database read from and write to Object Storage Service, run on FaaS platform. NOTES: This repository is still in t

老雷 7 May 12, 2022
StateAFL: A Greybox Fuzzer for Stateful Network Servers

StateAFL: A Coverage-Driven (Greybox) Fuzzer for Stateful Network Protocols StateAFL is a fuzzer designed for network servers. It extends the original

null 107 Sep 16, 2022
Stateful pointer - Use unused bits in your pointer

Stateful Pointer Library Sometimes space is tight! What if you could squeeze extra state into a pointer at (almost) no additional cost? A pointer occu

Hans Dembinski 40 Jun 7, 2020