C++20 coroutines-based cooperative multitasking library

Overview

πŸ” Coop

Coop is a C++20 coroutines-based library to support cooperative multitasking in the context of a multithreaded application. The syntax will be familiar to users of async and await functionality in other programming languages. Users do not need to understand the C++20 coroutines API to use this library.

Features

  • Ships with a default affinity-aware two-priority threadsafe task scheduler.
  • The task scheduler is swappable with your own
  • Supports scheduling of user-defined code and OS completion events (e.g. events that signal after I/O completes)
  • Easy to use, efficient API, with a small and digestible code footprint (hundreds of lines of code, not thousands)

Tasks in Coop are eager as opposed to lazy, meaning that upon suspension, the coroutine is immediately dispatched for execution on a worker with the appropriate affinity. While there are many benefits to structuring things lazily (see this excellent talk), Coop opts to do things the way it does because:

  • Coop was designed to interoperate with existing job/task graph systems
  • Coop was originally written within the context of a game engine, where exceptions were not used
  • For game engines, having a CPU-toplogy-aware dispatch mechanism is extremely important (consider the architecture of, say, the PS5)

While game consoles don't (yet) support C++20 fully, the hope is that options like Coop will be there when the compiler support gets there as well.

Limitations

If your use case is too far abreast of Coop's original use case (as above), you may need to do more modification to get Coop to behave the way you want. The limitations to consider below are:

  • Requires a recent C++20 compiler and code that uses Coop headers must also use C++20
  • The "event_t" wrapper around Win32 events doesn't have equivalent functionality on other platforms yet (it's provided as a reference for how you might handle your own overlapped IO)
  • The Clang implementation of the coroutines API at the moment doesn't work with the GCC stdlib++, so use libc++ instead
  • Clang on Windows does not yet support the MSVC coroutines runtime due to ABI differences
  • Coop ignores the problem of unhandled exceptions within scheduled tasks

If the above limitations make Coop unsuitable for you, consider the following libraries:

  • CppCoro - A coroutine library for C++
  • Conduit - Lazy High Performance Streams using Coroutine TS
  • folly::coro - a developer-friendly asynchronous C++ framework based on Coroutines TS

Building and Running the Tests

When configured as a standalone project, the built-in scheduler and tests are enabled by default. To configure and build the project from the command line:

mkdir build
cd build
cmake .. # Supply your own generator if you don't want the default generator
cmake --build .
./test/coop_test

Integration Guide

If you don't intend on using the built in scheduler, simply copy the contents of the include folder somewhere in your include path.

Otherwise, the recommended integration is done via cmake. For the header only portion, link against the coop target.

If you'd like both headers and the scheduler implementation, link against both coop and coop_scheduler.

Drop this quick cmake snippet somewhere in your CMakeLists.txt file to make both of these targets available.

include(FetchContent)

FetchContent_Declare(
    coop
    GIT_REPOSITORY https://github.com/jeremyong/coop.git
    GIT_TAG master
    GIT_SHALLOW ON
)
FetchContent_MakeAvailable(coop)

Usage

To write a coroutine, you'll use the task_t template type.

coop::task_t<> simple_coroutine()
{
    co_await coop::suspend();

    // Fake some work with a timer
    std::this_thread::sleep_for(std::chrono::milliseconds{50});
}

The first line with the coop::suspend function will suspend the execution of simple_coroutine and the next line will continue on a different thread.

To use this coroutine from another coroutine, we can do something like the following:

coop::task_t<> another_coroutine()
{
    // This will cause `simple_coroutine` to be scheduled on a thread different to this one
    auto task = simple_coroutine();

    // Do other useful work

    // Await the task when we need it to finish
    co_await task;
}

Tasks can hold values to be awaited on.

coop::task_t<int> coroutine_with_data()
{
    co_await coop::suspend();

    // Do some work
    int result = some_expensive_simulation();

    co_return result;
}

When the task above is awaited via the co_await operator, what results is the int returned via co_return. Of course, passing other types is possible by changing the first template parameter of task_t.

Tasks let you do multiple async operations simultaneously, for example:

coop::task_t<> my_task(int ms)
{
    co_await coop::suspend();

    // Fake some work with a timer
    std::this_thread::sleep_for(std::chrono::milliseconds{ms});
}

coop::task_t<> big_coroutine()
{
    auto t1 = my_task(50);
    auto t2 = my_task(40);
    auto t3 = my_task(80);

    // 3 invocations of `my_task` are now potentially running concurrently on different threads

    do_something_useful();

    // Block until t2 is done
    co_await t2;

    // Right now, t1 and t3 are *potentially* still running

    do_something_else();

    co_await t1;
    co_await t3;

    // Now, all three tasks are complete
}

One thing to keep in mind is that after awaiting a task, the thread you resume on is not necessarily the same thread you were on originally.

What if you want to await a task from main or some other execution context that isn't a coroutine? For this, you can make a joinable task and join it.

coop::task_t<void, true> joinable_coroutine()
{
    co_await coop::suspend();

    // Fake some work with a timer
    std::this_thread::sleep_for(std::chrono::milliseconds{50});
}

int main(int argc, char** argv)
{
    auto task = joinable_coroutine();
    // The timer is now running on a different thread than the main thread

    // Pause execution until joinable_coroutine is finished on whichever thread it was scheduled on
    task.join();

    return 0;
}

Note that currently, there is some overhead associated with spawning a joinable task because it creates new event objects instead of reusing event handles from a pool.

The coop::suspend function takes additional parameters that can set the CPU affinity mask, priority (only 0 and 1 are supported at the moment, with 1 being the higher priority), and file/line information for debugging purposes.

The task_t template type also takes an additional type that should implement the TaskControl concept. This is currently useful if you intend on overriding the allocation and deallocation behavior of the coroutine frames.

In addition to awaiting tasks, you can also await the event_t object. While this currently only supports Windows, this lets a coroutine suspend execution until an event handle is signaled - a powerful pattern for doing async I/O.

coop::task_t<> wait_for_event()
{
    // Suppose file_reading_code produces a Win32 HANDLE which will get signaled whenever the file
    // read is ready
    coop::event_t event{file_reading_code()};

    // Do something else while the file is reading

    // Suspend until the event gets signaled
    co_await event;
}

In the future, support may be added for epoll and kqueue abstractions.

Convenience macro COOP_SUSPEND#

The full function signature of the suspend function is the following:

template <Scheduler S = scheduler_t>
inline auto suspend(S& scheduler                             = S::instance(),
                    uint64_t cpu_mask                        = 0,
                    uint32_t priority                        = 0,
                    source_location_t const& source_location = {}) noexcept

and you must await the returned result. Instead, you can use the family of macros and simply write

COOP_SUSPEND();

if you are comfortable with the default behavior. This macro will supply __FILE__ and __LINE__ information to the source_location paramter to get additional tracking. Other macros with numerical suffixes to COOP_SUSPEND are also provided to allow you to override a subset of parameters as needed.

(Optional) Override default allocators

By default, coroutine frames are allocated via operator new and operator delete. Remember that the coroutine frames may not always allocate if the compiler can prove the allocation isn't necessary. That said, if you'd like to override the allocator with your own (for tracking purposes, or to use a different more specialized allocator), simply provide a TaskControl concept conforming type as the third template parameter of task_t. The full template type signature of a task_t is as follows:

template <typename T = void, bool Joinable = false, TaskControl C = task_control_t>
class task_t;

The first template parameter refers to the type that should be co_returned by the coroutine. The Joinable parameter indicates whether this task should create a binary_semaphore which is signaled on completion (and provides the task_t::join method to wait on the semaphore). The last parameter is any type that has an alloc and free function. By default, the TaskControl type is the one below:

struct task_control_t final
{
    static void* alloc(size_t size)
    {
        return operator new(size);
    }

    static void free(void* ptr)
    {
        operator delete(ptr);
    }
};

(Optional) Use your own scheduler

Coop is designed to be a pretty thin abstraction layer to make writing async code more convenient. If you already have a robust scheduler and thread pool, you don't have to use the one provided here. The coop::suspend function is templated and accepts an optional first parameter to a class that implements the Scheduler concept. To qualify as a Scheduler, a class only needs to implement the following function signature:

    void schedule(std::coroutine_handle<> coroutine,
                  uint64_t cpu_affinity             = 0,
                  uint32_t priority                 = 0,
                  source_location_t source_location = {});

Then, at the opportune time on a thread of your choosing, simply call coroutine.resume(). Remember that when implementing your own scheduler, you are responsible for thread safety and ensuring that the "usual" bugs (like missed notifications) are ironed out. You can ignore the cpu affinity and priority flags if you don't need this functionality (i.e. if you aren't targeting a NUMA).

Hack away

The source code of Coop is pretty small all things considered, with the core of its functionality contained in only a few hundred lines of commented code. Feel free to take it and adapt it for your use case. This was the route taken as opposed to making every design aspect customizable (which would have made the interface far more complicated).

Additional Resources

To learn more about coroutines in C++20, please do visit this awesome compendium of resources compiled by @MattPD.

You might also like...
A C++20 coroutine library based off asyncio
A C++20 coroutine library based off asyncio

kuro A C++20 coroutine library, somewhat modelled on Python's asyncio Requirements Kuro requires a C++20 compliant compiler and a Linux OS. Tested on

C++20 Coroutine-Based Synchronous Parser Combinator Library

This library contains a monadic parser type and associated combinators that can be composed to create parsers using C++20 Coroutines.

A C++17 message passing library based on MPI

MPL - A message passing library MPL is a message passing library written in C++17 based on the Message Passing Interface (MPI) standard. Since the C++

DwThreadPool - A simple, header-only, dependency-free, C++ 11 based ThreadPool library.
DwThreadPool - A simple, header-only, dependency-free, C++ 11 based ThreadPool library.

dwThreadPool A simple, header-only, dependency-free, C++ 11 based ThreadPool library. Features C++ 11 Minimal Source Code Header-only No external depe

C++14 coroutine-based task library for games

SquidTasks Squid::Tasks is a header-only C++14 coroutine-based task library for games. Full project and source code available at https://github.com/we

checkedthreads: no race condition goes unnoticed! Simple API, automatic load balancing, Valgrind-based checking

checkedthreads checkedthreads is a fork-join parallelism framework for C and C++ providing: Automated race detection using debugging schedulers and Va

SymQEMU: Compilation-based symbolic execution for binaries

SymQEMU This is SymQEMU, a binary-only symbolic executor based on QEMU and SymCC. It currently extends QEMU 4.1.1 and works with the most recent versi

RocketOS is a Unix based OS that uses legacy BIOS and GRUB and is written in C17. It is being developed for educational purposes primarily, but it still is a serious project. It is currently in its infancy.

RocketOS What is RocketOS? RocketOS is a Unix based OS that uses legacy BIOS and GRUB and is written in C17. It is being developed for educational pur

C++-based high-performance parallel environment execution engine for general RL environments.
C++-based high-performance parallel environment execution engine for general RL environments.

EnvPool is a highly parallel reinforcement learning environment execution engine which significantly outperforms existing environment executors. With

Comments
  • task_control_t seems undefined

    task_control_t seems undefined

    The promise type uses task_control_t to control allocations.

    https://github.com/jeremyong/coop/blob/06da5d03068a5e36927d3b8cd2215d5e8a0820ba/include/coop/detail/promise.hpp#L164

    But task has no definition for this. Am I missing something here?

    opened by arBmind 1
Owner
Jeremy Ong
Principal Engineer at Warner Bros. Primary interests: Rendering/graphics Machine learning/AI Networking Low-level optimization/performance tuning
Jeremy Ong
Header-Only C++20 Coroutines library

CPP20Coroutines Header-Only C++20 Coroutines library This repository aims to demonstrate the capabilities of C++20 coroutines. generator Generates val

null 16 Aug 15, 2022
Cppcoro - A library of C++ coroutine abstractions for the coroutines TS

CppCoro - A coroutine library for C++ The 'cppcoro' library provides a large set of general-purpose primitives for making use of the coroutines TS pro

Lewis Baker 2.6k Dec 30, 2022
Coro - Single-header library facilities for C++2a Coroutines

coro This is a collection of single-header library facilities for C++2a Coroutines. coro/include/ co_future.h Provides co_future<T>, which is like std

Arthur O'Dwyer 66 Dec 6, 2022
Modern concurrency for C++. Tasks, executors, timers and C++20 coroutines to rule them all

concurrencpp, the C++ concurrency library concurrencpp is a tasking library for C++ allowing developers to write highly concurrent applications easily

David Haim 1.2k Jan 3, 2023
Discrete-event simulation in C++20 using coroutines

SimCpp20 SimCpp20 is a discrete-event simulation framework for C++20. It is similar to SimPy and aims to be easy to set up and use. Processes are defi

Felix SchΓΌtz 34 Nov 15, 2022
Open source PHP extension for Async IO, Coroutines and Fibers

Swoole is an event-driven asynchronous & coroutine-based concurrency networking communication engine with high performance written in C++ for PHP. Ope

Open Swoole 684 Jan 4, 2023
C++14 asynchronous allocation aware futures (supporting then, exception handling, coroutines and connections)

Continuable is a C++14 library that provides full support for: lazy async continuation chaining based on callbacks (then) and expression templates, ca

Denis Blank 771 Dec 20, 2022
Termite-jobs - Fast, multiplatform fiber based job dispatcher based on Naughty Dogs' GDC2015 talk.

NOTE This library is obsolete and may contain bugs. For maintained version checkout sx library. until I rip it from there and make a proper single-hea

Sepehr Taghdisian 35 Jan 9, 2022
A library for enabling task-based multi-threading. It allows execution of task graphs with arbitrary dependencies.

Fiber Tasking Lib This is a library for enabling task-based multi-threading. It allows execution of task graphs with arbitrary dependencies. Dependenc

RichieSams 796 Dec 30, 2022
OpenCL based GPU accelerated SPH fluid simulation library

libclsph An OpenCL based GPU accelerated SPH fluid simulation library Can I see it in action? Demo #1 Demo #2 Why? Libclsph was created to explore the

null 47 Jul 27, 2022