Header-only C++20 wrapper for MPI 4.0.

Ali Can Demiralp

Last update: Apr 8, 2022

Related tags

Utilities mpi

Overview

MPI

Modern C++20 message passing interface wrapper.

Examples

Initialization:

  mpi::environment environment;
  const auto& communicator = mpi::world_communicator;

Transmitting basic types:

  std::int32_t data = 0;
  if (communicator.rank() == 0)
  {
    data = 42;
    communicator.send(data, 1);
  }
  if (communicator.rank() == 1)
  {
    communicator.receive(data, 0);
  }

Transmitting a container of basic types:

  std::vector
   int32_t> 
   data_container(
   3);
  
   if (communicator.rank() == 
   0)
  {
    data_container = {
   1, 
   2, 
   3};
    communicator.
   send(data_container, 
   1);
  }
  
   if (communicator.rank() == 
   1)
  {
    communicator.
   receive(data_container, 
   0);
  }

Transmitting user-defined aggregates:

  struct user_type
  {
    std::int32_t         id      ;
    std::array<float, 3> position;
  };

  user_type user_object;
  if (communicator.rank() == 0)
  {
    user_object = {42, {0.0f, 1.0f, 2.0f}};
    communicator.send(user_object, 1);
  }
  if (communicator.rank() == 1)
  {
    communicator.receive(user_object, 0);
  }

Transmitting a container of user-defined aggregates:

  std::vector 
   user_object_container(
   2);
  
   if (communicator.rank() == 
   0)
  {
    user_object_container[
   0] = {
   42, {
   0.
   0f, 
   1.
   0f, 
   2.
   0f}};
    user_object_container[
   1] = {
   84, {
   3.
   0f, 
   4.
   0f, 
   5.
   0f}};
    communicator.
   send(user_object_container, 
   1);
  }
  
   if (communicator.rank() == 
   1)
  {
    communicator.
   receive(user_object_container, 
   0);
  }

See the tests for more.

Usage Notes

Define MPI_USE_EXCEPTIONS to check the return values of all viable functions against MPI_SUCCESS and throw an exception otherwise.
Define MPI_USE_RELAXED_TRAITS to prevent the library from checking the types of aggregate elements and triggering static asserts for non-aggregates (useful for e.g. testing).
Compliant types (satisfying mpi::is_compliant) are types whose corresponding mpi::data_type can be automatically generated:
- Arithmetic types (satisfying std::is_arithmetic), enumerations (satisfying std::is_enum), specializations of std::complex are compliant types.
- C-style arrays, std::array, std::pair, std::tuple, and aggregate types (satisfying std::is_aggregate) consisting of other compliant types are also compliant types.
- If your type is none of the above, you can specialize template <> struct mpi::type_traits { static data_type get_data_type() { return DATA_TYPE; } }; manually.
The MPI functions accepting buffers may be used with:
- Compliant types.
- Contiguous sequential containers (i.e. std::string, std::span, std::valarray, std::vector) of compliant types.
Extension functions (starting with MPIX) are not included as they are often implementation-specific. You can nevertheless use them with the wrapper via the native handle getters.

Design Notes

Constructors:
- Copy constructors are deleted unless MPI provides duplication functions (ending with _dup) for the object.
- Move constructors are available whenever possible.
- The object wrappers have two types of constructors:
  - Managed constructors: Construct a new MPI object, and be responsible for its destruction.
  - Unmanaged constructors: Accept an existing MPI object, and not be responsible for its destruction.
Structures:
- Structs are for POD data types. They may only contain member variables, constructors, destructors and assignment operators.
Statics:
- Namespace functions are preferred over static member functions whenever possible.

Reflection:

The data type for aggregates are created automatically through reflection.

Survey of reflection libraries:

Library	Built	Extra step	Manual	Dependencies
https://github.com/rttrorg/rttr	True	False	True	False
https://github.com/boostorg/pfr	False	False	False	False
https://github.com/apolukhin/pfr_non_boost	False	False	False	False
https://github.com/felixguendling/cista	False	False	False	False
https://github.com/veselink1/refl-cpp	False	False	True	False
https://github.com/AustinBrunkhorst/CPP-Reflection	True	True	True	True
https://github.com/billyquith/ponder	True	False	True	False
https://github.com/skypjack/meta	False	False	True	False
https://github.com/Ubpa/USRefl	False	False	True	False
https://github.com/Ubpa/UDRefl	True	False	True	False
https://github.com/garbageslam/visit_struct	False	False	True	False
https://github.com/chakaz/reflang	True	True	False	True
https://github.com/Manu343726/tinyrefl	False	True	False	False
https://github.com/Manu343726/siplasplas	True	True	False	False
https://github.com/preshing/FlexibleReflection	False	False	True	False
https://github.com/Celtoys/clReflect	True	True	True	False
https://github.com/Leandros/metareflect	False	True	True	True
https://github.com/Cylix/Reflex	False	False	True	True
https://github.com/boostorg/describe	False	False	True	True

Decided on PFR after watching https://www.youtube.com/watch?v=abdeAew3gmQ, specifically https://github.com/apolukhin/pfr_non_boost without the boost namespace.
Limitations:
- The type must be an aggregate (satisfy std::is_aggregate):
  - No user-declared or inherited constructors.
  - No private or protected direct non-static data members.
  - No virtual functions.
  - No virtual, private or protected base classes.
- Additionally:
  - No const fields.
  - No references.
  - Static data members are ignored.
- See https://www.boost.org/doc/libs/1_76_0/doc/html/boost_pfr/limitations_and_configuration.html for further detail.

Constants:
- Several MPI constants are unused:
  - MPI_STATUS_IGNORE : Stati are return values and this would require void overloads for every function.
  - MPI_STATUSES_IGNORE: Stati are return values and this would require void overloads for every function.
  - MPI_ARGVS_NULL : This would require a scan over std::vector ::arguments() to see if they all are empty.
  - MPI_WEIGHTS_EMPTY : Prefer an empty container instead.
  - MPI_BOTTOM : Prefer nullptr instead.
  - MPI_ROOT : Unused in default arguments as intercommunicators are rarer than intracommunicators. Can nevertheless be passed to functions.
  - MPI_PROC_NULL : Unused in default arguments as intercommunicators are rarer than intracommunicators. Can nevertheless be passed to functions.

Coverage (list from https://www.open-mpi.org/doc/v4.1/)

Future Work

See https://github.com/acdemiralp/mpi/projects/1

You might also like...

The Ultimate Raylib gaming library wrapper for Nim

NimraylibNow! - The Ultimate Raylib wrapper for Nim The most idiomatic and up-to-date wrapper for Raylib gaming C library. Use this library if you wan

138 Dec 22, 2022

A wrapper around std::variant with some helper functions

1 Oct 30, 2021

Stripped down version of BIGTREETECH-TouchScreenFirmware which only supports ST7920 emulation (Marlin Mode)

Stripped down version of BIGTREETECH-TouchScreenFirmware which only supports ST7920 emulation (Marlin Mode). This project only uses peripheral drivers supplied by the screen manufacturer and uses it's own library to parse the ST7920 commands.

112 Dec 30, 2022

You Only Look Twice: Rapid Multi-Scale Object Detection In Satellite Imagery

YOLT You Only Look Twice: Rapid Multi-Scale Object Detection In Satellite Imagery As of 24 October 2018 YOLT has been superceded by SIMRDWN YOLT is an

618 Dec 23, 2022

A Windows only library for allowing C++ to feel like C#

System What is this? C++ System is a header and Windows only library for interacting with the machine, in the way that C# does, but since C++ is not r

7 Nov 18, 2021

Dead simple C logging library contained in a single header (.h) file

Seethe Logging so simple, you only need to include a single header file. seethe supports 6 different log levels (DEBUG, INFO, NOTICE, WARNING, ERROR,

28 Nov 24, 2022

RapidObj is an easy-to-use, single-header C++17 library that loads and parses Wavefront .obj files.

RapidObj About Integration Prerequisites Manual Integration CMake Integration API RapidObj Result Next Steps OS Support Third Party Tools and Resource

108 Jan 2, 2023

convert elf file to single c/c++ header file

elf-to-c-header Split ELF to single C/C++ header file

2 Nov 4, 2021

Open MPI main development repository

Open MPI The Open MPI Project is an open source Message Passing Interface (MPI) implementation that is developed and maintained by a consortium of aca

1.6k Jan 5, 2023

A C++17 message passing library based on MPI

MPL - A message passing library MPL is a message passing library written in C++17 based on the Message Passing Interface (MPI) standard. Since the C++

115 Dec 28, 2022

Experimental and Comparative Performance Measurements of High Performance Computing Based on OpenMP and MPI

High-Performance-Computing-Experiments Experimental and Comparative Performance Measurements of High Performance Computing Based on OpenMP and MPI 实验结

1 Nov 27, 2021

A program developed using MPI for distributed computation of Histogram for large data and their performance anaysis on multi-core systems

mpi-histo A program developed using MPI for distributed computation of Histogram for large data and their performance anaysis on multi-core systems. T

2 Dec 21, 2021

Parallel implementation of Dijkstra's shortest path algorithm using MPI

1 Jan 21, 2022

Header-only, event based, tiny and easy to use libuv wrapper in modern C++ - now available as also shared/static library!

Do you have a question that doesn't require you to open an issue? Join the gitter channel. If you use uvw and you want to say thanks or support the pr

1.5k Jan 1, 2023

A C++ header-only ZLib wrapper

A C++ ZLib wrapper This C++ header-only library enables the use of C++ standard iostreams to access ZLib-compressed streams. For input access (decompr

242 Jan 5, 2023

Header-only, event based, tiny and easy to use libuv wrapper in modern C++ - now available as also shared/static library!

Do you have a question that doesn't require you to open an issue? Join the gitter channel. If you use uvw and you want to say thanks or support the pr

1.5k Jan 9, 2023

Comments

Incorporate futures, coroutines, and detach callbacks (for requests).

An MPI_Request is a handle for an MPI operation which will complete in the future. A std::future<T> is a handle for a C++ operation which will complete in the future.

They are very much the same. Futures should be incorporated in the implementation of the request wrapper. Specifically std::future<T>::then() which is a part of the C++ concurrency proposal would enable chaining non-blocking operations with a clean syntax.

Callbacks are an alternative to asynchrony; the user provides a function which is ran when the operation completes. Callbacks also enable chaining through nesting, today.

opened by acdemiralp 1
Combined collectives for convenient communication of varying sizes.

Example: A user has to first MPI_Gather the sizes of each vector from each rank, and use that information to allocate space and fill in the sizes / displacements prior to calling MPI_Gatherv.

Provide conveniences which combine such common collective operations into a single call.

Blocked by #2 as chaining via futures or callbacks is necessary for immediate and persistent collectives.

opened by acdemiralp 0
Design and implement a high-level library on top of this one.
Potential primitives:

[x] Distributed FSIO:

Distributed reading/writing of partial/complete data to/from multiple processes without user intervention.

Perhaps just accept HDF5 as a de-facto standard and write a wrapper for it (it is already based on MPI-IO).

Inspiration: HDF5, NetCDF, ADIOS.

Resolution: Accepted HDF5. See https://github.com/acdemiralp/hdf for a modern interface.

[x] Sync Variables:

Variables which are synchronized across all processes (in a communicator?) without explicit calls to collectives.

Inspiration: Unity3D/Unreal/RakNet network SyncVars.

Resolution: Implemented. See mpi/extensions/shared_variable.hpp.

[ ] Sync Memory:

Abstraction of a memory region in which:

An object constructed or destroyed in one process is constructed or destroyed across all processes.

Members of all objects are kept in sync across all processes.

A function called in one process is called across all processes.

Objects can reference / point to each other (pointers are valid for use across processes - virtual pointers).

Builds on sync variables and RPCs.

Inspiration: Unity3D/Unreal maps with networking.

[ ] Distributed Shared Memory:

Treat the combined memory of all processes as one i.e. partitioned global address space (PGAS).

PGAS versions of STL containers.

PGAS versions of <algorithm>.

Probably using mpi::window.

See: DASH: Data Structures and Algorithms with Support for Hierarchical Locality.

See: UPC++: A PGAS Extension for C++

Objection: I doubt that I can create something as elaborate as DASH. I should just accept DASH as a de-facto standard for PGAS in C++. It even supports C++17.

Objection to objection: DASH is bloated to the throat with scripts, makefiles, site-specifics, i.e. a workflow that distrupts the user's own workflow. The user has to take a break from programming his own work, and have to spend time building, installing and making sure that DASH works. I can just provide a few STL style headers to achieve PGAS. Specifically:

<pgas/algorithm>

<pgas/array>

<pgas/iterator>

<pgas/numeric>

[ ] Task Graphs:

Define streamlined inputs and outputs (i.e. resources) for each point-to-point and collective operation (i.e. tasks).

Allow user to construct a directed acyclic graph of resources and tasks.

Automatically place barriers for resources referred by multiple tasks.

This would achieve complex communication operations with understandable code.

Inspiration: Intel TBB

[ ] Remote Procedure Calls:

Functions which are simultaneously called on all processes (in a communicator?)

Inspiration: Unity3D/Unreal/RakNet network commands/RPCs.

See: Mercury: Enabling remote procedure call for high-performance computing

[ ] Socket-like Communication:

Real-time (always on, passive), bidirectional, low-latency communication across all processes (in a communicator?).

Subscribe to a group/topic with a key and a callback which gets called whenever a message with the key is received.

Perhaps a notion of "lobby" and "room" for ranks.

Request/reply, publish/subscribe, pipeline, exclusive pair patterns.

Inspiration: Socket.io, ZeroMQ.

See: MPI/RT-an emerging standard for high-performance real-time systems

help wanted
opened by acdemiralp 0
Convert all viable functions to use MPI_Count once the functions ending with _c become widespread.
Convert all the following functions to use std::int64_t once the big count proposal is available by majority of implementations.

Reasoning for conversion rather than addition: We should not represent a memory address/size with a std::int32_t in the era of 64-bit computers.

Including but not limited to:

[ ] MPIX_Accumulate_c

[ ] MPIX_Allgather_c

[ ] MPIX_Allgatherv_c

[ ] MPIX_Allreduce_c

[ ] MPIX_Alltoall_c

[ ] MPIX_Alltoallv_c

[ ] MPIX_Alltoallw_c

[ ] MPIX_Bcast_c

[ ] MPIX_Bcast_init_c

[ ] MPIX_Bsend_c

[ ] MPIX_Bsend_init_c

[ ] MPIX_Buffer_attach_c

[ ] MPIX_Buffer_detach_c

[ ] MPIX_Exscan_c

[ ] MPIX_Gather_c

[ ] MPIX_Gatherv_c

[ ] MPIX_Get_accumulate_c

[ ] MPIX_Get_c

[ ] MPIX_Get_count_c

[ ] MPIX_Get_elements_c

[ ] MPIX_Iallgather_c

[ ] MPIX_Iallgatherv_c

[ ] MPIX_Iallreduce_c

[ ] MPIX_Ialltoall_c

[ ] MPIX_Ialltoallv_c

[ ] MPIX_Ialltoallw_c

[ ] MPIX_Ibcast_c

[ ] MPIX_Ibsend_c

[ ] MPIX_Iexscan_c

[ ] MPIX_Igather_c

[ ] MPIX_Igatherv_c

[ ] MPIX_Imrecv_c

[ ] MPIX_Ineighbor_allgather_c

[ ] MPIX_Ineighbor_allgatherv_c

[ ] MPIX_Ineighbor_alltoall_c

[ ] MPIX_Ineighbor_alltoallv_c

[ ] MPIX_Ineighbor_alltoallw_c

[ ] MPIX_Irecv_c

[ ] MPIX_Ireduce_c

[ ] MPIX_Ireduce_scatter_block_c

[ ] MPIX_Ireduce_scatter_c

[ ] MPIX_Irsend_c

[ ] MPIX_Iscan_c

[ ] MPIX_Iscatter_c

[ ] MPIX_Iscatterv_c

[ ] MPIX_Isend_c

[ ] MPIX_Issend_c

[ ] MPIX_Mrecv_c

[ ] MPIX_Neighbor_allgather_c

[ ] MPIX_Neighbor_allgatherv_c

[ ] MPIX_Neighbor_alltoall_c

[ ] MPIX_Neighbor_alltoallv_c

[ ] MPIX_Neighbor_alltoallw_c

[ ] MPIX_Op_create_c

[ ] MPIX_Pack_c

[ ] MPIX_Pack_external_c

[ ] MPIX_Pack_external_size_c

[ ] MPIX_Pack_size_c

[ ] MPIX_Put_c

[ ] MPIX_Raccumulate_c

[ ] MPIX_Recv_c

[ ] MPIX_Recv_init_c

[ ] MPIX_Reduce_c

[ ] MPIX_Reduce_local_c

[ ] MPIX_Reduce_scatter_block_c

[ ] MPIX_Reduce_scatter_c

[ ] MPIX_Rget_accumulate_c

[ ] MPIX_Rget_c

[ ] MPIX_Rput_c

[ ] MPIX_Rsend_c

[ ] MPIX_Rsend_init_c

[ ] MPIX_Scan_c

[ ] MPIX_Scatter_c

[ ] MPIX_Scatterv_c

[ ] MPIX_Send_c

[ ] MPIX_Send_init_c

[ ] MPIX_Ssend_c

[ ] MPIX_Ssend_init_c

[ ] MPIX_Type_contiguous_c

[ ] MPIX_Type_create_darray_c

[ ] MPIX_Type_create_hindexed_block_c

[ ] MPIX_Type_create_hindexed_c

[ ] MPIX_Type_create_hvector_c

[ ] MPIX_Type_create_indexed_block_c

[ ] MPIX_Type_create_resized_c

[ ] MPIX_Type_create_struct_c

[ ] MPIX_Type_create_subarray_c

[ ] MPIX_Type_get_contents_c

[ ] MPIX_Type_get_envelope_c

[ ] MPIX_Type_get_extent_c

[ ] MPIX_Type_get_true_extent_c

[ ] MPIX_Type_indexed_c

[ ] MPIX_Type_size_c

[ ] MPIX_Type_vector_c

[ ] MPIX_Unpack_c

[ ] MPIX_Unpack_external_c

[ ] MPIX_Win_allocate_c

[ ] MPIX_Win_allocate_shared_c

[ ] MPIX_Win_create_c

[ ] MPIX_Win_shared_query_c

See https://eurompi.github.io/assets/papers/2020-09-eurompi2020-mpi4.pdf for more functions.
help wanted
opened by acdemiralp 0

Releases(1.3.0)

1.3.0(Aug 10, 2022)

Source code(tar.gz)
Source code(zip)
1.1.0(Nov 21, 2021)

Source code(tar.gz)
Source code(zip)

Header-only C++20 wrapper for MPI 4.0.

Related tags

Overview

MPI

Examples

Usage Notes

Design Notes

Coverage (list from https://www.open-mpi.org/doc/v4.1/)

Future Work

You might also like...

The Ultimate Raylib gaming library wrapper for Nim

A wrapper around std::variant with some helper functions

Stripped down version of BIGTREETECH-TouchScreenFirmware which only supports ST7920 emulation (Marlin Mode)

You Only Look Twice: Rapid Multi-Scale Object Detection In Satellite Imagery

A Windows only library for allowing C++ to feel like C#

Dead simple C logging library contained in a single header (.h) file

RapidObj is an easy-to-use, single-header C++17 library that loads and parses Wavefront .obj files.

convert elf file to single c/c++ header file

Open MPI main development repository

A C++17 message passing library based on MPI

Experimental and Comparative Performance Measurements of High Performance Computing Based on OpenMP and MPI

A program developed using MPI for distributed computation of Histogram for large data and their performance anaysis on multi-core systems

Parallel implementation of Dijkstra's shortest path algorithm using MPI

Header-only, event based, tiny and easy to use libuv wrapper in modern C++ - now available as also shared/static library!

A C++ header-only ZLib wrapper

Header-only, event based, tiny and easy to use libuv wrapper in modern C++ - now available as also shared/static library!

A single file C++ header-only minizip wrapper library

sdl4cpp, header-only C++ wrapper around SDL2.

Header only wrapper around Hex-Rays API in C++20.

Comments

Incorporate futures, coroutines, and detach callbacks (for requests).

Combined collectives for convenient communication of varying sizes.

Design and implement a high-level library on top of this one.

Convert all viable functions to use MPI_Count once the functions ending with _c become widespread.

Releases(1.3.0)

1.3.0(Aug 10, 2022)

1.1.0(Nov 21, 2021)

Owner

Ali Can Demiralp

Small header-only C++ library that helps to initialize Vulkan instance and device object

match(it): A lightweight header-only pattern-matching library for C++17 with macro-free APIs.

A Header-Only Engine that tries to use SFML in a deeper level

C++ header-only library for generic data validation.

a compile-time, header-only, dimensional analysis and unit conversion library built on c++14 with no dependencies.

A header only C++ library that provides type safety and user defined literals for physical units

Small Header only library to parse argv for flags

Header-only lock-free synchronization utilities (one writer, many readers).

This is a Header-only c++ string implentation which specializes in dealing with small strings. 🧵

Header only roguelike rendering library.