Async GRPC with C++20 coroutine support

Overview

agrpc

Build an elegant GRPC async interface with C++20 coroutine and libunifex (target for C++23 executor).

Get started

mkdir build && cd build
conan install ..
cmake -DCMAKE_CXX_FLAGS:STRING=-fcoroutines ..
cmake --build .

Work in progress

One glance:

native async grpc agrpc
class CallData {
 public:
  void Proceed() {
    if (status_ == CREATE) {
      status_ = PROCESS;
      service_->RequestSayHello(&ctx_, &request_,
                                &responder_,
                                cq_, cq_, this);
    } else if (status_ == PROCESS) {
      new CallData(service_, cq_);
  
      std::string prefix("Hello ");
      reply_.set_message(prefix + request_.name());

      status_ = FINISH;
      responder_.Finish(reply_, Status::OK, this);
    } else {
      GPR_ASSERT(status_ == FINISH);
      delete this;
    }
  }

 private:
  // ...
  enum CallStatus { CREATE, PROCESS, FINISH };
  CallStatus status_{CREATE};  // The current serving state.
};

void HandleRpcs() {
  new CallData(&service_, cq_.get());
  void* tag;  // uniquely identifies a request.
  bool ok;
  while (true) {
    GPR_ASSERT(cq_->Next(&tag, &ok));
    GPR_ASSERT(ok);
    static_cast<CallData*>(tag)->Proceed();
  }
}
while (true) {
  grpc::ServerContext server_context;
  helloworld::HelloRequest request;
  grpc::ServerAsyncResponseWriter<helloworld::HelloReply> writer{
      &server_context};
  bool request_ok = co_await agrpc::AsyncRequest(
      grpc_context.get_scheduler(),
      &helloworld::Greeter::AsyncService::RequestSayHello,
      service, server_context, request, writer);
  if (!request_ok)
    co_return;
  helloworld::HelloReply response;
  response.set_message("Hello " + request.name());
  co_await agrpc::AsyncFinish(grpc_context.get_scheduler(), writer,
                              response, grpc::Status::OK);
}

Benchmark

1 CPU server

name req/s avg. latency 90 % in 95 % in 99 % in avg. cpu avg. memory
cpp_agrpc 35385 26.74 ms 39.48 ms 49.60 ms 64.11 ms 92.43% 28.78 MiB
cpp_grpc_st 35364 26.03 ms 48.25 ms 58.39 ms 72.63 ms 80.95% 21.57 MiB
cpp_grpc_mt 35068 26.23 ms 48.16 ms 58.60 ms 72.72 ms 80.29% 18.05 MiB
rust_tonic_st 33407 25.96 ms 54.54 ms 60.15 ms 68.41 ms 66.22% 15.34 MiB
dotnet_grpc 32125 26.78 ms 53.29 ms 66.82 ms 81.76 ms 95.1% 94.31 MiB
rust_tonic_mt 31428 29.06 ms 90.03 ms 97.46 ms 106.46 ms 69.87% 13.52 MiB
cpp_grpc_callback 12283 80.54 ms 103.30 ms 107.25 ms 178.37 ms 100.27% 406.88 MiB
go_grpc 5600 176.37 ms 306.34 ms 397.40 ms 598.96 ms 100.41% 44.02 MiB

2 CPU server

name req/s avg. latency 90 % in 95 % in 99 % in avg. cpu avg. memory
rust_tonic_mt 77236 11.97 ms 30.99 ms 44.78 ms 68.00 ms 191.33% 17.42 MiB
cpp_agrpc 72084 13.50 ms 20.83 ms 23.73 ms 31.08 ms 202.99% 86.8 MiB
cpp_grpc_mt 68099 14.18 ms 34.95 ms 39.29 ms 45.88 ms 203.08% 61.64 MiB
dotnet_grpc 63479 13.98 ms 20.35 ms 26.62 ms 61.63 ms 201.58% 175.97 MiB
rust_tonic_st 61962 15.83 ms 19.73 ms 21.05 ms 24.13 ms 102.46% 16.32 MiB
cpp_grpc_st 50593 19.47 ms 23.59 ms 25.26 ms 29.66 ms 102.15% 19.53 MiB
cpp_grpc_callback 29612 32.49 ms 74.33 ms 78.51 ms 85.74 ms 202.37% 444.44 MiB
go_grpc 12050 80.52 ms 188.05 ms 194.19 ms 286.45 ms 200.78% 45.11 MiB

Goal

Maybe something like C# interface.

public override async Task RouteChat(Grpc.Core.IAsyncStreamReader<RouteNote> requestStream,
    Grpc.Core.IServerStreamWriter<RouteNote> responseStream,
    Grpc.Core.ServerCallContext context)
{
    while (await requestStream.MoveNext())
    {
        var note = requestStream.Current;
        List<RouteNote> prevNotes = AddNoteForLocation(note.Location, note);
        foreach (var prevNote in prevNotes)
        {
            await responseStream.WriteAsync(prevNote);
        }
    }
}

Reference

Issues
  • Why cpp_agrpc is faster than grpc_callback?

    Why cpp_agrpc is faster than grpc_callback?

    Hi, as i know coroutine just change the style of async programming, don't improve performance, but i noticed your benchmark, the coroutine version is much faster than callback version on multiple cores server, why? What optimizations you did?

    opened by qicosmos 3
Owner
Yuchao Zhang
speech synthesis/machine learning
Yuchao Zhang
Async++ concurrency framework for C++11

Async++ Async++ is a lightweight concurrency framework for C++11. The concept was inspired by the Microsoft PPL library and the N3428 C++ standard pro

Amanieu d'Antras 1.1k Aug 15, 2022
Open source PHP extension for Async IO, Coroutines and Fibers

Swoole is an event-driven asynchronous & coroutine-based concurrency networking communication engine with high performance written in C++ for PHP. Ope

Open Swoole 521 Aug 6, 2022
Powerful multi-threaded coroutine dispatcher and parallel execution engine

Quantum Library : A scalable C++ coroutine framework Quantum is a full-featured and powerful C++ framework build on top of the Boost coroutine library

Bloomberg 447 Jul 25, 2022
:copyright: Concurrent Programming Library (Coroutine) for C11

libconcurrent tiny asymmetric-coroutine library. Description asymmetric-coroutine bidirectional communication by yield_value/resume_value native conte

sharow 350 Aug 4, 2022
Single header asymmetric stackful cross-platform coroutine library in pure C.

minicoro Minicoro is single-file library for using asymmetric coroutines in C. The API is inspired by Lua coroutines but with C use in mind. The proje

Eduardo Bart 313 Aug 15, 2022
A golang-style C++ coroutine library and more.

CO is an elegant and efficient C++ base library that supports Linux, Windows and Mac platforms. It pursues minimalism and efficiency, and does not rely on third-party library such as boost.

Alvin 2.7k Aug 7, 2022
A C++20 coroutine library based off asyncio

kuro A C++20 coroutine library, somewhat modelled on Python's asyncio Requirements Kuro requires a C++20 compliant compiler and a Linux OS. Tested on

null 17 Jul 19, 2022
C++20 Coroutine-Based Synchronous Parser Combinator Library

This library contains a monadic parser type and associated combinators that can be composed to create parsers using C++20 Coroutines.

null 41 Aug 7, 2022
Cppcoro - A library of C++ coroutine abstractions for the coroutines TS

CppCoro - A coroutine library for C++ The 'cppcoro' library provides a large set of general-purpose primitives for making use of the coroutines TS pro

Lewis Baker 2.4k Aug 14, 2022
Mx - C++ coroutine await, yield, channels, i/o events (single header + link to boost)

mx C++11 coroutine await, yield, channels, i/o events (single header + link to boost). This was originally part of my c++ util library kit, but I'm se

Grady O'Connell 4 Sep 21, 2019
Elle - The Elle coroutine-based asynchronous C++ development framework.

Elle, the coroutine-based asynchronous C++ development framework Elle is a collection of libraries, written in modern C++ (C++14). It contains a rich

Infinit 463 Jul 29, 2022
A go-style coroutine library in C++11 and more.

cocoyaxi English | 简体中文 A go-style coroutine library in C++11 and more. 0. Introduction cocoyaxi (co for short), is an elegant and efficient cross-pla

Alvin 2.7k Aug 5, 2022
C++14 coroutine-based task library for games

SquidTasks Squid::Tasks is a header-only C++14 coroutine-based task library for games. Full project and source code available at https://github.com/we

Tim Ambrogi Saxon 58 Jul 25, 2022
A library OS for Linux multi-process applications, with Intel SGX support

Graphene Library OS with Intel SGX Support A Linux-compatible Library OS for Multi-Process Applications NOTE: We are in the middle of transitioning ou

The Gramine Project 241 Aug 10, 2022
EnkiTS - A permissively licensed C and C++ Task Scheduler for creating parallel programs. Requires C++11 support.

Support development of enkiTS through Github Sponsors or Patreon enkiTS Master branch Dev branch enki Task Scheduler A permissively licensed C and C++

Doug Binks 1.3k Aug 6, 2022
Mini-async-log-c - Mini async log C port. Now with C++ wrappers.

Description A C11/C++11 low-latency wait-free producer (when using Thread Local Storage) asynchronous textual data logger with type-safe strings. Base

null 66 Dec 7, 2021
Coroutine - C++11 single .h asymmetric coroutine implementation via ucontext / fiber

C++11 single .h asymmetric coroutine implementation API in namespace coroutine: routine_t create(std::function<void()> f); void destroy(routine_t id);

null 379 Jun 30, 2022
The C based gRPC (C++, Python, Ruby, Objective-C, PHP, C#)

gRPC - An RPC library and framework gRPC is a modern, open source, high-performance remote procedure call (RPC) framework that can run anywhere. gRPC

grpc 35.2k Aug 8, 2022
Asynchronous gRPC with Boost.Asio executors

asio-grpc This library provides an implementation of boost::asio::execution_context that dispatches work to a grpc::CompletionQueue. Making it possibl

Dennis 138 Aug 8, 2022
gRPC - An RPC library and framework Baind Unity 3D Project

Unity 3D Compose for Desktop and Android, a modern UI framework for C ++ , C# that makes building performant and beautiful user interfaces easy and enjoyable.

Md Raihan 4 May 19, 2022
Async++ concurrency framework for C++11

Async++ Async++ is a lightweight concurrency framework for C++11. The concept was inspired by the Microsoft PPL library and the N3428 C++ standard pro

Amanieu d'Antras 1.1k Aug 15, 2022
Mongoose Embedded Web Server Library - a multi-protocol embedded networking library with TCP/UDP, HTTP, WebSocket, MQTT built-in protocols, async DNS resolver, and non-blocking API.

Mongoose - Embedded Web Server / Embedded Networking Library Mongoose is a networking library for C/C++. It implements event-driven non-blocking APIs

Cesanta Software 8.6k Aug 8, 2022
Async & Concurrent Servers implemented in C

Concurrent servers in c Imlementation of concurrent severs in c from scratch using this awesome blog as a tutorial. Project Structure . ├── readme.md

Rupanshu Yadav 7 Jun 21, 2021
Example programs for Talking Async videos

Talking Async Example programs for the Talking Async videos, which can be found on YouTube. Episode 1: Why C++20 is the Awesomest Language for Network

null 96 Aug 2, 2022
A PostgreSQL extension providing an async networking interface accessible via SQL using a background worker and curl.

pg_net is a PostgreSQL extension exposing a SQL interface for async networking with a focus on scalability and UX.

Supabase 41 Jun 19, 2022
An SQLite binding for node.js with built-in encryption, focused on simplicity and (async) performance

Description An SQLite (more accurately SQLite3MultipleCiphers) binding for node.js focused on simplicity and (async) performance. When dealing with en

mscdex 14 May 15, 2022
Open source PHP extension for Async IO, Coroutines and Fibers

Swoole is an event-driven asynchronous & coroutine-based concurrency networking communication engine with high performance written in C++ for PHP. Ope

Open Swoole 521 Aug 6, 2022
A C++ async HTTP client library to use in asynchronous applications while communicating with REST services.

libashttp An asynchronous HTTP library using Boost.ASIO as the backend. This project is licensed under: Usage Here is a example usage which is taken f

Tolga Hoşgör 52 Jul 3, 2022
C++11 Lightweight Redis client: async, thread-safe, no dependency, pipelining, multi-platform

C++11 Lightweight Redis client: async, thread-safe, no dependency, pipelining, multi-platform

Simon Ninon 981 Aug 10, 2022