Async GRPC with C++20 coroutine support

Related tags

Concurrency and Multithreading async executor coroutines grpc cpp20
Overview

agrpc

Build an elegant GRPC async interface with C++20 coroutine and libunifex (target for C++23 executor).

Get started

mkdir build && cd build
conan install ..
cmake -DCMAKE_CXX_FLAGS:STRING=-fcoroutines ..
cmake --build .

Work in progress

One glance:

native async grpc agrpc 
class CallData {
 public:
  void Proceed() {
    if (status_ == CREATE) {
      status_ = PROCESS;
      service_->RequestSayHello(&ctx_, &request_,
                                &responder_,
                                cq_, cq_, this);
    } else if (status_ == PROCESS) {
      new CallData(service_, cq_);
  
      std::string prefix("Hello ");
      reply_.set_message(prefix + request_.name());

      status_ = FINISH;
      responder_.Finish(reply_, Status::OK, this);
    } else {
      GPR_ASSERT(status_ == FINISH);
      delete this;
    }
  }

 private:
  // ...
  enum CallStatus { CREATE, PROCESS, FINISH };
  CallStatus status_{CREATE};  // The current serving state.
};

void HandleRpcs() {
  new CallData(&service_, cq_.get());
  void* tag;  // uniquely identifies a request.
  bool ok;
  while (true) {
    GPR_ASSERT(cq_->Next(&tag, &ok));
    GPR_ASSERT(ok);
    static_cast<CallData*>(tag)->Proceed();
  }
}
 
while (true) {
  grpc::ServerContext server_context;
  helloworld::HelloRequest request;
  grpc::ServerAsyncResponseWriter<helloworld::HelloReply> writer{
      &server_context};
  bool request_ok = co_await agrpc::AsyncRequest(
      grpc_context.get_scheduler(),
      &helloworld::Greeter::AsyncService::RequestSayHello,
      service, server_context, request, writer);
  if (!request_ok)
    co_return;
  helloworld::HelloReply response;
  response.set_message("Hello " + request.name());
  co_await agrpc::AsyncFinish(grpc_context.get_scheduler(), writer,
                              response, grpc::Status::OK);
}

Benchmark

1 CPU server

name req/s avg. latency 90 % in 95 % in 99 % in avg. cpu avg. memory
cpp_agrpc 35385 26.74 ms 39.48 ms 49.60 ms 64.11 ms 92.43% 28.78 MiB
cpp_grpc_st 35364 26.03 ms 48.25 ms 58.39 ms 72.63 ms 80.95% 21.57 MiB
cpp_grpc_mt 35068 26.23 ms 48.16 ms 58.60 ms 72.72 ms 80.29% 18.05 MiB
rust_tonic_st 33407 25.96 ms 54.54 ms 60.15 ms 68.41 ms 66.22% 15.34 MiB
dotnet_grpc 32125 26.78 ms 53.29 ms 66.82 ms 81.76 ms 95.1% 94.31 MiB
rust_tonic_mt 31428 29.06 ms 90.03 ms 97.46 ms 106.46 ms 69.87% 13.52 MiB
cpp_grpc_callback 12283 80.54 ms 103.30 ms 107.25 ms 178.37 ms 100.27% 406.88 MiB
go_grpc 5600 176.37 ms 306.34 ms 397.40 ms 598.96 ms 100.41% 44.02 MiB

2 CPU server

name req/s avg. latency 90 % in 95 % in 99 % in avg. cpu avg. memory
rust_tonic_mt 77236 11.97 ms 30.99 ms 44.78 ms 68.00 ms 191.33% 17.42 MiB
cpp_agrpc 72084 13.50 ms 20.83 ms 23.73 ms 31.08 ms 202.99% 86.8 MiB
cpp_grpc_mt 68099 14.18 ms 34.95 ms 39.29 ms 45.88 ms 203.08% 61.64 MiB
dotnet_grpc 63479 13.98 ms 20.35 ms 26.62 ms 61.63 ms 201.58% 175.97 MiB
rust_tonic_st 61962 15.83 ms 19.73 ms 21.05 ms 24.13 ms 102.46% 16.32 MiB
cpp_grpc_st 50593 19.47 ms 23.59 ms 25.26 ms 29.66 ms 102.15% 19.53 MiB
cpp_grpc_callback 29612 32.49 ms 74.33 ms 78.51 ms 85.74 ms 202.37% 444.44 MiB
go_grpc 12050 80.52 ms 188.05 ms 194.19 ms 286.45 ms 200.78% 45.11 MiB

Goal

Maybe something like C# interface.

public override async Task RouteChat(Grpc.Core.IAsyncStreamReader<RouteNote> requestStream,
    Grpc.Core.IServerStreamWriter<RouteNote> responseStream,
    Grpc.Core.ServerCallContext context)
{
    while (await requestStream.MoveNext())
    {
        var note = requestStream.Current;
        List<RouteNote> prevNotes = AddNoteForLocation(note.Location, note);
        foreach (var prevNote in prevNotes)
        {
            await responseStream.WriteAsync(prevNote);
        }
    }
}

Reference

You might also like...
Elle - The Elle coroutine-based asynchronous C++ development framework.
Elle - The Elle coroutine-based asynchronous C++ development framework.

Elle, the coroutine-based asynchronous C++ development framework Elle is a collection of libraries, written in modern C++ (C++14). It contains a rich

Infinit 466 Jan 1, 2023
A go-style coroutine library in C++11 and more.
A go-style coroutine library in C++11 and more.

cocoyaxi English | 简体中文 A go-style coroutine library in C++11 and more. 0. Introduction cocoyaxi (co for short), is an elegant and efficient cross-pla

Alvin 3.1k Dec 27, 2022
C++14 coroutine-based task library for games

SquidTasks Squid::Tasks is a header-only C++14 coroutine-based task library for games. Full project and source code available at https://github.com/we

Tim Ambrogi Saxon 74 Nov 30, 2022
A library OS for Linux multi-process applications, with Intel SGX support

Graphene Library OS with Intel SGX Support A Linux-compatible Library OS for Multi-Process Applications NOTE: We are in the middle of transitioning ou

The Gramine Project 323 Jan 4, 2023
EnkiTS - A permissively licensed C and C++ Task Scheduler for creating parallel programs. Requires C++11 support.
EnkiTS - A permissively licensed C and C++ Task Scheduler for creating parallel programs. Requires C++11 support.

Support development of enkiTS through Github Sponsors or Patreon enkiTS Master branch Dev branch enki Task Scheduler A permissively licensed C and C++

Doug Binks 1.4k Dec 27, 2022
Mini-async-log-c - Mini async log C port. Now with C++ wrappers.

Description A C11/C++11 low-latency wait-free producer (when using Thread Local Storage) asynchronous textual data logger with type-safe strings. Base

null 69 Nov 9, 2022
Coroutine - C++11 single .h asymmetric coroutine implementation via ucontext / fiber

C++11 single .h asymmetric coroutine implementation API in namespace coroutine: routine_t create(std::functionvoid() f); void destroy(routine_t id);

null 390 Dec 20, 2022
The C based gRPC (C++, Python, Ruby, Objective-C, PHP, C#)

gRPC - An RPC library and framework gRPC is a modern, open source, high-performance remote procedure call (RPC) framework that can run anywhere. gRPC

grpc 36.6k Dec 30, 2022
Asynchronous gRPC with Boost.Asio executors

asio-grpc This library provides an implementation of boost::asio::execution_context that dispatches work to a grpc::CompletionQueue. Making it possibl

Dennis 180 Dec 31, 2022
gRPC - An RPC library and framework Baind Unity 3D Project

Unity 3D Compose for Desktop and Android, a modern UI framework for C ++ , C# that makes building performant and beautiful user interfaces easy and enjoyable.

Md Raihan 4 May 19, 2022
Async++ concurrency framework for C++11

Async++ Async++ is a lightweight concurrency framework for C++11. The concept was inspired by the Microsoft PPL library and the N3428 C++ standard pro

Amanieu d'Antras 1.1k Dec 30, 2022
Mongoose Embedded Web Server Library - a multi-protocol embedded networking library with TCP/UDP, HTTP, WebSocket, MQTT built-in protocols, async DNS resolver, and non-blocking API.
Mongoose Embedded Web Server Library - a multi-protocol embedded networking library with TCP/UDP, HTTP, WebSocket, MQTT built-in protocols, async DNS resolver, and non-blocking API.

Mongoose - Embedded Web Server / Embedded Networking Library Mongoose is a networking library for C/C++. It implements event-driven non-blocking APIs

Cesanta Software 9k Jan 1, 2023
Async & Concurrent Servers implemented in C

Concurrent servers in c Imlementation of concurrent severs in c from scratch using this awesome blog as a tutorial. Project Structure . ├── readme.md

Rupanshu Yadav 7 Jun 21, 2021
Example programs for Talking Async videos

Talking Async Example programs for the Talking Async videos, which can be found on YouTube. Episode 1: Why C++20 is the Awesomest Language for Network

null 123 Dec 28, 2022
A PostgreSQL extension providing an async networking interface accessible via SQL using a background worker and curl.

pg_net is a PostgreSQL extension exposing a SQL interface for async networking with a focus on scalability and UX.

Supabase 49 Dec 14, 2022
An SQLite binding for node.js with built-in encryption, focused on simplicity and (async) performance

Description An SQLite (more accurately SQLite3MultipleCiphers) binding for node.js focused on simplicity and (async) performance. When dealing with en

mscdex 14 May 15, 2022
Open source PHP extension for Async IO, Coroutines and Fibers

Swoole is an event-driven asynchronous & coroutine-based concurrency networking communication engine with high performance written in C++ for PHP. Ope

Open Swoole 684 Jan 4, 2023
A C++ async HTTP client library to use in asynchronous applications while communicating with REST services.

libashttp An asynchronous HTTP library using Boost.ASIO as the backend. This project is licensed under: Usage Here is a example usage which is taken f

Tolga Hoşgör 53 Dec 17, 2022
C++11 Lightweight Redis client: async, thread-safe, no dependency, pipelining, multi-platform
C++11 Lightweight Redis client: async, thread-safe, no dependency, pipelining, multi-platform

C++11 Lightweight Redis client: async, thread-safe, no dependency, pipelining, multi-platform

Simon Ninon 1k Jan 8, 2023
Comments
  • Why cpp_agrpc is faster than grpc_callback?

    Why cpp_agrpc is faster than grpc_callback?

    Hi, as i know coroutine just change the style of async programming, don't improve performance, but i noticed your benchmark, the coroutine version is much faster than callback version on multiple cores server, why? What optimizations you did?

    opened by qicosmos 3
  • Fix clock_gettime for Apple clang

    Fix clock_gettime for Apple clang

    The PR fixes clock_gettime lack of type conversion error for Apple clang version 13.1.6 (clang-1316.0.21.2.5):

    /Users/genertorg/git/agrpc/agrpc/base/chrono.cc:31:3: error: no matching function for call to 'clock_gettime'
  clock_gettime(type, &ts);
  ^~~~~~~~~~~~~
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.3.sdk/usr/include/time.h:178:5: note: candidate function not viable: no known conversion from 'int' to 'clockid_t' for 1st argument
int clock_gettime(clockid_t __clock_id, struct timespec *__tp);
    ^
1 error generated.
make[2]: *** [agrpc/base/CMakeFiles/agrpc_base_chrono.dir/chrono.cc.o] Error 1
make[1]: *** [agrpc/base/CMakeFiles/agrpc_base_chrono.dir/all] Error 2
    opened by genert 0
Owner
Yuchao Zhang
speech synthesis/machine learning
Yuchao Zhang
GitHub
Async++ concurrency framework for C++11

Async++ Async++ is a lightweight concurrency framework for C++11. The concept was inspired by the Microsoft PPL library and the N3428 C++ standard pro

Amanieu d'Antras 1.1k Dec 30, 2022
Open source PHP extension for Async IO, Coroutines and Fibers

Swoole is an event-driven asynchronous & coroutine-based concurrency networking communication engine with high performance written in C++ for PHP. Ope

Open Swoole 684 Jan 4, 2023
Powerful multi-threaded coroutine dispatcher and parallel execution engine

Quantum Library : A scalable C++ coroutine framework Quantum is a full-featured and powerful C++ framework build on top of the Boost coroutine library

Bloomberg 491 Dec 30, 2022
:copyright: Concurrent Programming Library (Coroutine) for C11

libconcurrent tiny asymmetric-coroutine library. Description asymmetric-coroutine bidirectional communication by yield_value/resume_value native conte

sharow 350 Sep 2, 2022
Single header asymmetric stackful cross-platform coroutine library in pure C.

minicoro Minicoro is single-file library for using asymmetric coroutines in C. The API is inspired by Lua coroutines but with C use in mind. The proje

Eduardo Bart 405 Dec 29, 2022
A golang-style C++ coroutine library and more.

CO is an elegant and efficient C++ base library that supports Linux, Windows and Mac platforms. It pursues minimalism and efficiency, and does not rely on third-party library such as boost.

Alvin 3.1k Jan 5, 2023
A C++20 coroutine library based off asyncio

kuro A C++20 coroutine library, somewhat modelled on Python's asyncio Requirements Kuro requires a C++20 compliant compiler and a Linux OS. Tested on

null 19 Nov 9, 2022
C++20 Coroutine-Based Synchronous Parser Combinator Library

This library contains a monadic parser type and associated combinators that can be composed to create parsers using C++20 Coroutines.

null 46 Dec 17, 2022
Cppcoro - A library of C++ coroutine abstractions for the coroutines TS

CppCoro - A coroutine library for C++ The 'cppcoro' library provides a large set of general-purpose primitives for making use of the coroutines TS pro

Lewis Baker 2.6k Dec 30, 2022
Mx - C++ coroutine await, yield, channels, i/o events (single header + link to boost)

mx C++11 coroutine await, yield, channels, i/o events (single header + link to boost). This was originally part of my c++ util library kit, but I'm se

Grady O'Connell 4 Sep 21, 2019