cross platform subprocess library for c++ similar to design of python subprocess

Overview

subprocess

cross platform subprocess library for c++ similar to design of python subprocess. See subprocess documentation for further documentation.

supports

  • very python like style of subprocess. With very nice syntax for c++20.
  • Connect output of process A to input of process B. However not pretty API for this.
  • Environment utilities to make it easy to get/set environment variables. as easy as subprocess::cenv["MY_VAR"] = "value".
  • subprocess::EnvGuard that will save the environment and reload it when scope block ends, making it easy to have a temporary environment. Obviously this is not thread safe as environment variable changes effects process wide.
  • Get a copy of environment so you can modify a std::map as you please for use in a thread safe manner of environment and pass it along to subprocesses.
  • cross-platform find_program
  • find_program has special handling of "python3" on windows making it easy to find python3 executable. It searches the path for python and inspects it's version so that find_program("python3") is cross-platform.
  • Supports connecting process stdin, stdout, stderr to C++ streams making redirection convenient. stdin can be connected with a std::string too.

Shakey elements

  • The os error level exceptions is still changing. I'm thinking of having an OSError subclass to abstract the OS differences.

requirements

  • c++17
  • linked with support for threading, filesystem

Integration

Adhoc

  1. copy files in src/cpp to your project.
  2. add the top folder as include.
  3. make sure cpp files are compiled.
  4. add #include <subprocess.hpp> to start using in source files.

Teaport

add this to your dependencies:

"subprocess": "0.4.+"

Todo add to cocoapods and perhaps others.

Examples

#include <subprocess.hpp>
#include <thread>
#include <cstring>

void simple() {
    using subprocess::CompletedProcess;
    using subprocess::RunBuilder;
    using subprocess::PipeOption;
    // quick echo it, doesn't capture
    subprocess::run({"echo", "hello", "world"});

    // simplest capture output.
    CompletedProcess process = subprocess::run({"echo", "hello", "world"},
        RunBuilder().cout(PipeOption::pipe));

    // simplest sending data example
    process = subprocess::run({"cat"},
        RunBuilder().cin("hello world\n"));

    // simplest send & capture
    process = subprocess::run({"cat"},
        RunBuilder().cin("hello world").cout(PipeOption::pipe));
    std::cout << "captured: " << process.cout << '\n';

    // capture stderr too.
    process = subprocess::run({"echo", "hello", "world"},
        RunBuilder().cerr(PipeOption::pipe)
        .cout(PipeOption::pipe)
        .check(true) // will throw CalledProcessError if returncode != 0.
    );

    // there is no cerr so it will be empty
    std::cout << "cerr was: " << process.cerr << "\n";

#if __cplusplus >= 202002L
    // capture output. You can do this syntax if you have C++20
    process = subprocess::run({"echo", "hello", "world"}, {
        .cout = PipeOption::pipe,
        // make true to throw exception
        .check = false
    });

    std::cout << "captured: " << process.cout << '\n';
#endif
}


void popen_examples() {
    using subprocess::CompletedProcess;
    using subprocess::RunBuilder;
    using subprocess::Popen;
    using subprocess::PipeOption;

    // simplest example
    // capture is enabled by default
    Popen popen = subprocess::RunBuilder({"echo", "hello", "world"})
        .cout(PipeOption::pipe).popen();
    char buf[1024] = {0}; // initializes everything to 0
    subprocess::pipe_read(popen.cout, buf, 1024);
    std::cout << buf;
    // the destructor will call wait on your behalf.
    popen.close();


    // communicate with data
    popen = subprocess::RunBuilder({"cat"}).cin(PipeOption::pipe)
        .cout(PipeOption::pipe).popen();
    /*  if we write more data than the buffer, we would dead lock if the subprocess
        is deadlocked trying to write. So we spin a new thread for writing. When
        you provide buffers for cin, internally the library spins it's own thread.
    */
    std::thread write_thread([&]() {
        subprocess::pipe_write(popen.cin, "hello world\n", std::strlen("hello world\n"));
        // no more data to send. If we don't close we may run into a deadlock as
        // we are looking to read for more.
        popen.close_cin();
    });

    for (auto& c : buf)
        c = 0;

    subprocess::pipe_read(popen.cout, buf, 1024);
    std::cout << buf;
    popen.close();
    if (write_thread.joinable())
        write_thread.join();
}
int main(int argc, char** argv) {
    std::cout << "running basic examples\n";
    simple();
    std::cout << "running popen_examples\n";
    popen_examples();
    return 0;
}

Deviations

  • On windows terminating a process sends CTRL_BREAK_EVENT instead of hard termination. You can send a SIGKILL and it will do a hard termination as expected. Becareful as this may kill your process as it's sent to the process group. See send_signal for more details.
  • cin, cout, cerr variable names are used instead of stdin, stdout, stderr as std* are macros and cannot be used as names in C++.

current progress

All tests pass on linux & mac. Most pass under mingw & MSVC.

Changelog

0.4.0

  • CTRL_BREAK_EVENT is sent for SIGTERM & terminate() functions on windows.
  • fixed invalid handles when launching a python script that then launches new processes.
  • new kIsWin32 constant to help avoid ifdef use.
  • Documentation wording to be more confident as the library is looking pretty good, and I haven't felt like changing much of the API.

0.3.0

  • fixed MSVC issues & compiles
  • documentation should be complete. Please report any missing

0.2.0

  • omg setting check=true is fixed. What a typo
You might also like...
C++React: A reactive programming library for C++11.

C++React is reactive programming library for C++14. It enables the declarative definition of data dependencies between state and event flows. Based on

A library for enabling task-based multi-threading. It allows execution of task graphs with arbitrary dependencies.

Fiber Tasking Lib This is a library for enabling task-based multi-threading. It allows execution of task graphs with arbitrary dependencies. Dependenc

The C++ Standard Library for Parallelism and Concurrency

Documentation: latest, development (master) HPX HPX is a C++ Standard Library for Concurrency and Parallelism. It implements all of the corresponding

A C++ library of Concurrent Data Structures

CDS C++ library The Concurrent Data Structures (CDS) library is a collection of concurrent containers that don't require external (manual) synchroniza

OpenCL based GPU accelerated SPH fluid simulation library

libclsph An OpenCL based GPU accelerated SPH fluid simulation library Can I see it in action? Demo #1 Demo #2 Why? Libclsph was created to explore the

A header-only C++ library for task concurrency
A header-only C++ library for task concurrency

transwarp Doxygen documentation transwarp is a header-only C++ library for task concurrency. It allows you to easily create a graph of tasks where eve

VexCL is a C++ vector expression template library for OpenCL/CUDA/OpenMP

VexCL VexCL is a vector expression template library for OpenCL/CUDA. It has been created for ease of GPGPU development with C++. VexCL strives to redu

:copyright: Concurrent Programming Library (Coroutine) for C11

libconcurrent tiny asymmetric-coroutine library. Description asymmetric-coroutine bidirectional communication by yield_value/resume_value native conte

Simple and fast C library implementing a thread-safe API to manage hash-tables, linked lists, lock-free ring buffers and queues

libhl C library implementing a set of APIs to efficiently manage some basic data structures such as : hashtables, linked lists, queues, trees, ringbuf

Comments
  • Library not working anymore on Windows for shell commands

    Library not working anymore on Windows for shell commands

    I'm not able to run the library anymore on Windows. Running the basic example from this site gives:

    Exception has occurred: CPP/subprocess::CommandNotFoundError
    Unhandled exception at 0x00007FF9FB204FD9 in process.exe: Microsoft C++ exception: subprocess::CommandNotFoundError at memory location 0x000000CAF711E830.
    
    

    Example stops immediately. vscode debugger:

    subprocess_error

    Whatever you put in the command list, even a simple "cd", doesn't work I'm running on:

    - visual studio community 2019 11.16.17
    - windows 10.0.19043.2006
    

    The library worked flawlessly before. Breaking changes are on one of the this two... HTH

    opened by kabukunz 2
  • std::codecvt_utf8_utf16 is deprecated in C++17

    std::codecvt_utf8_utf16 is deprecated in C++17

    You will have to come up with an alternative way to convert between UTF-8 and UTF-16: https://en.cppreference.com/w/cpp/locale/codecvt_utf8_utf16

    You code will not compile on newer C++ compilers.

    opened by visuve 0
  • fix #5: fatal system error exception in CloseHandle (invalid handle) due to double close of `cin`

    fix #5: fatal system error exception in CloseHandle (invalid handle) due to double close of `cin`

    fix (https://github.com/benman64/subprocess/issues/5) fatal system error exception in CloseHandle (invalid handle) due to double close of cin: once through cin pipe thread's autoclose=true (as it should be), then, when the subprocess has terminated, the cin handle is again closed in the POpen destructor.

    The fix is to 'remember' when cin is being processed with autoclose ON and then forego the second cleanup in the close/destructor code to prevent system exceptions and consequent application failure.


    Since cin is a public POpen member, we chose this route, while initially considering passing cin as a mutable reference into the pipe thread, so it could be nilled on thread termination. This, however, would introduce potential race issues as the pipe thread would then modify the cin member variable out-of-band for thee main thread, which owns the POpen instance.

    The current solution is safe for regular use, including the sample code where userland code writes to the cin pipe and then closes it explicitly using the provided POpeen::close_cin() API.

    opened by GerHobbelt 1
  • fix for Unicode build on Windows.

    fix for Unicode build on Windows.

    Wee bit hacky as it uses the ANSI *A APIs from Windows instead of converting everything to wchar_t/Unicode *W APIs.

    Which should take care of https://github.com/benman64/subprocess/issues/1, BTW.

    opened by GerHobbelt 0
Releases(v0.4.0)
  • v0.4.0(Aug 3, 2020)

    Changes:

    • CTRL_BREAK_EVENT is sent for SIGTERM & terminate() functions on windows.
    • fixed invalid handles when launching a python script that then launches new processes.
    • new kIsWin32 constant to help avoid ifdef use.
    • Documentation wording to be more confident as the library is looking pretty good, and I haven't felt like changing much of the API.
    Source code(tar.gz)
    Source code(zip)
  • v0.2.0(Jul 27, 2020)

Owner
null
Single header asymmetric stackful cross-platform coroutine library in pure C.

minicoro Minicoro is single-file library for using asymmetric coroutines in C. The API is inspired by Lua coroutines but with C use in mind. The proje

Eduardo Bart 368 Nov 18, 2022
Concurrency Kit 2.1k Nov 19, 2022
EAThread - EAThread implements a unified cross-platform interface for multithreaded programming.

EAThread EAThread implements a unified cross-platform interface for multithreaded programming on various platforms. Documentation Please see Introduct

Electronic Arts 256 Nov 24, 2022
Arcana.cpp - Arcana.cpp is a collection of helpers and utility code for low overhead, cross platform C++ implementation of task-based asynchrony.

Arcana.cpp Arcana is a collection of general purpose C++ utilities with no code that is specific to a particular project or specialized technology are

Microsoft 67 Nov 23, 2022
Fiber - A header only cross platform wrapper of fiber API.

Fiber Header only cross platform wrapper of fiber API A fiber is a particularly lightweight thread of execution. Which is useful for implementing coro

Tony Wang 41 Jul 31, 2022
Rpmalloc - Public domain cross platform lock free thread caching 16-byte aligned memory allocator implemented in C

rpmalloc - General Purpose Memory Allocator This library provides a public domain cross platform lock free thread caching 16-byte aligned memory alloc

Mattias Jansson 1.7k Nov 22, 2022
Bolt is a C++ template library optimized for GPUs. Bolt provides high-performance library implementations for common algorithms such as scan, reduce, transform, and sort.

Bolt is a C++ template library optimized for heterogeneous computing. Bolt is designed to provide high-performance library implementations for common

null 358 Nov 14, 2022
oneAPI DPC++ Library (oneDPL) https://software.intel.com/content/www/us/en/develop/tools/oneapi/components/dpc-library.html

oneAPI DPC++ Library (oneDPL) The oneAPI DPC++ Library (oneDPL) aims to work with the oneAPI DPC++ Compiler to provide high-productivity APIs to devel

oneAPI-SRC 641 Nov 16, 2022
ArrayFire: a general purpose GPU library.

ArrayFire is a general-purpose library that simplifies the process of developing software that targets parallel and massively-parallel architectures i

ArrayFire 4k Nov 28, 2022
A C++ GPU Computing Library for OpenCL

Boost.Compute Boost.Compute is a GPU/parallel-computing library for C++ based on OpenCL. The core library is a thin C++ wrapper over the OpenCL API an

Boost.org 1.3k Nov 23, 2022