QuickCheck clone for C++ with the goal of being simple to use with as little boilerplate as possible.

Related tags

Testing rapidcheck
Overview

RapidCheck Build Status Build status

RapidCheck is a C++ framework for property based testing inspired by QuickCheck and other similar frameworks. In property based testing, you state facts about your code that given certain precondition should always be true. RapidCheck then generates random test data to try and find a case for which the property doesn't hold. If such a case is found, RapidCheck tries to find the smallest case (for some definition of smallest) for which the property is still false and then displays this as a counterexample. For example, if the input is an integer, RapidCheck tries to find the smallest integer for which the property is false.

Sounds interesting? Why don't you read the User Guide to learn more!

Why RapidCheck?

There are existing implementations of property based testing but the ones that I have found are either (in my humble opinion) a bit clunky or are missing essential features such as test case shrinking.

Let's throw together a list of features:

  • Write your properties in an imperative way that makes sense for C++
  • Test case shrinking
  • Great support for STL types, including maps and sets
  • Advanced combinators for creating your own generators
  • Stateful based on commands in the vein of Erlang QuickCheck
  • Integration with popular testing frameworks such as Boost Test, Google Test and Google Mock

Prerequisites and installation

RapidCheck makes extensive use of C++11 and thus requires a compliant compiler. RapidCheck continuous integration builds using Clang 3.5, GCC 4.9 and Visual Studio 2015 so any later versions should also work.

RapidCheck uses CMake and is built like any other CMake project. If your own project uses CMake you can simply have RapidCheck as a subdirectory and add the following to your CMakeLists.txt:

add_subdirectory("path/to/rapidcheck")
target_link_libraries(my_target rapidcheck)

This will give you both linking and include directories.

Quick introduction

A common first example is testing a reversal function. For such a function, double reversal should always result in the original list. In this example we will use the standard C++ std::reverse function:

#include <rapidcheck.h>

#include <vector>
#include <algorithm>

int main() {
  rc::check("double reversal yields the original value",
            [](const std::vector<int> &l0) {
              auto l1 = l0;
              std::reverse(begin(l1), end(l1));
              std::reverse(begin(l1), end(l1));
              RC_ASSERT(l0 == l1);
            });

  return 0;
}

The check function is used to check properties. The first parameter is an optional string which describes the property. The second parameter is a callable object that implements the property, in this case a lambda. Any parameters to the callable (in our case the l0 parameter) will be randomly generated. The RC_ASSERT macro works just like any other assert macro. If the given condition is false, the property has been falsified.

The property above also forms part of a specification of the reversal function: "For any list of integers A, reversing and then reversing again should result in A".

If we were to run this, RapidCheck would (hopefully) output the following:

Using configuration: seed=9928307433081493900

- double reversal yields the original value
OK, passed 100 tests

Here, RapidCheck tells us that it ran 100 test cases and all of them passed. It also tells us the configuration that was used, in particular the random seed. If there was a bug in the implementation of std::reverse we could get the following output instead:

Falsifiable after 12 tests and 10 shrinks

std::tuple<std::vector<int>>:
([1, 0, 0, 0, 0, 0, 0, 0, 0, 0])

main.cpp:17:
RC_ASSERT(l0 == l1)

Expands to:
[1, 0, 0, 0, 0, 0, 0, 0, 0, 0] == [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]

Here RapidCheck tells us that it found a case for which the property does not hold after running 12 tests. When it found this case, it shrunk it 10 times to arrive at the counterexample in the output. The counterexample contains each input value that was used for the failing case along with its type. Since RapidCheck views property arguments as tuples, the type is shown here as std::tuple<std::vector<int>>.

Can you guess what the bug is? The fact that there are exactly 10 items should give a clue. In this case, the bug is that the implementation sets the first element to 0 when l0.size() >= 10. This is also the reason for the initial 0, the problem doesn't manifest when all elements are zero. How did this bug happen? Who knows!

Thanks

Big thanks to my employer, Spotify, for making it possible for me to spend work time improving RapidCheck.

Issues
  • Infinite loop when using rc::gen::container<std::set> with rc::gen::inRange

    Infinite loop when using rc::gen::container with rc::gen::inRange

    As of 59f5f4e024ebac38ec65fd69bbc21682113bbc45, this code does not terminate:

    rc::check("", [] {
      *rc::gen::container<std::set<int>>(rc::gen::inRange(0, 1));
    });
    

    The same thing happens when generating an std::unordered_set, It works when using rc::gen::arbitrary<int>() and it works when generating an std::vector<int>.

    opened by per-gron 11
  • Avoid undefined behavior

    Avoid undefined behavior

    Using GCC 6.3.1 and the -fsanitize=address,undefined compile flags, a project I've been testing using RapidCheck has been throwing the below failures (NB. the paths are munged to be more readable),

    ./include/rapidcheck/detail/BitStream.hpp:60:12: runtime error: shift exponent 64 is too large for 64-bit type 'uint64_t' (aka 'unsigned long')
    SUMMARY: AddressSanitizer: undefined-behavior ./include/rapidcheck/detail/BitStream.hpp:60:12
    
    ./include/rapidcheck/detail/Utility.h:98:37: runtime error: left shift of negative value -2
    SUMMARY: AddressSanitizer: undefined-behavior ./include/rapidcheck/detail/Utility.h:98:37
    

    This PR aims to avoid these undefined bit-shift behaviors and -- in admittedly very loose, "Does it work on my machine?" tests -- seems to do the job.

    opened by pyrrho 10
  • error: ‘const class boost::unit_test::test_case’ has no member named ‘full_name’

    error: ‘const class boost::unit_test::test_case’ has no member named ‘full_name’

    I'm trying to add rapidcheck to my library. I've added the following headers to my file:

    #include <boost/test/unit_test.hpp>
    #include <rapidcheck/boost_test.h>
    

    The Boost version I am using is 1.59.0, and I've tried using 1.61.0 as well. Both give this error

    In file included from test/script_tests.cpp:31:0:
    /home/chris/dev/rapidcheck/extras/boost_test/include/rapidcheck/boost_test.h: In function ‘void rc::detail::checkBoostTest(const string&, Testable&&)’:
    /home/chris/dev/rapidcheck/extras/boost_test/include/rapidcheck/boost_test.h:14:26: error: ‘const class boost::unit_test::test_case’ has no member named ‘full_name’
       metadata.id = testCase.full_name();
                              ^
    make[2]: *** [test/test_test_bitcoin-script_tests.o] Error 1
    make[2]: Leaving directory `/home/chris/dev/bitcoin/src'
    make[1]: *** [all-recursive] Error 1
    make[1]: Leaving directory `/home/chris/dev/bitcoin/src'
    make: *** [all-recursive] Error 1
    
    opened by Christewart 7
  • Make RTTI enable default consistent between headers and library compile

    Make RTTI enable default consistent between headers and library compile

    Problem

    A default build and install of rapidcheck builds the library with RTTI enabled. The headers in rapidcheck/include have RTTI disabled by default.

    Symptom

    Including the headers from rapidcheck/include in a compile that is linking against a default compile of rapidcheck.a results in SIGSEGV failures when an RC_ASSERT macro fails it's condition.

    Expected Behavior

    A default compile of rapidcheck should be usable by default with it's includes.

    Suggested Solution(s)

    1. Invert the RC_USE_RTTI define by changing it to RC_DONT_USE_RTTI, replace the #ifdefs with #ifndefs, and update the CMakeLists.txt to set RC_DONT_USE_RTTI if NOT RC_ENABLE_RTTI.
    2. Change the default setting for RC_USE_RTTI to OFF in the CMakeLists.txt.
    opened by tsmanner 6
  • Segmentation fault when program is falsified, when compiling with optimizations.

    Segmentation fault when program is falsified, when compiling with optimizations.

    The following example of a falsifiable function gives a segfault when the RC_ASSERT receives FALSE:

    #include<iostream>
    #include<rapidcheck.h>
    
    int myadd(int x,int y){
        if(x==7){return 1;}
        else{
            return x+y;
        }
    }
    void issame(int x,int y){
    
    
        bool res = myadd(x,y)==myadd(y,x); 
        
        std::cout<<x<<" , "<<y<<" ---> "<<myadd(x,y)<<" , "<<myadd(y,x)<<" res: "<<res<<std::endl;
        
        RC_ASSERT(res);
    
    }
    
    int main(){
    
        rc::check("issame",issame);
    }
    

    Sometimes, 100 tests run without error (I.E. x hasn't been 7 in these 100 tests). When the function is falsified, it returns a segfault. Note that it doesn't stop the program right away.

    Output:

    Using configuration: seed=13664549568846329301
    
    - issame
    0 , 0 ---> 0 , 0 res: 1
    0 , 0 ---> 0 , 0 res: 1
    0 , 0 ---> 0 , 0 res: 1
    -1 , 0 ---> -1 , -1 res: 1
    0 , -1 ---> -1 , -1 res: 1
    -1 , -2 ---> -3 , -3 res: 1
    -1 , 1 ---> 0 , 0 res: 1
    0 , 1 ---> 1 , 1 res: 1
    0 , 2 ---> 2 , 2 res: 1
    -1 , 1 ---> 0 , 0 res: 1
    -1 , 3 ---> 2 , 2 res: 1
    -1 , 7 ---> 6 , 1 res: 0
    -1 , 7 ---> 6 , 1 res: 0
    0 , 7 ---> 7 , 1 res: 0
    0 , 7 ---> 7 , 1 res: 0
    0 , 0 ---> 0 , 0 res: 1
    0 , 4 ---> 4 , 4 res: 1
    0 , 6 ---> 6 , 6 res: 1
    0 , 7 ---> 7 , 1 res: 0
    Segmentation fault (core dumped)
    
    opened by Xvdgeest 5
  • Add fixed-count overloads for gen::unique(By)

    Add fixed-count overloads for gen::unique(By)

    I've run into the need to generate fixed-size containers with unique elements (for generating valid CSR matrices); this PR adds the functionality into rapidcheck as additional overloads for gen::unique and gen::uniqueBy, similar to what's already there for gen::container.

    opened by Jvanrhijn 5
  • build shared library, allow system install

    build shared library, allow system install

    These changes will

    1. Address Issue #182 by allowing the user to install the library.

    2. Allow the user to create a shared library instead of a static library as follows:

      cmake -DRC_SHARED_LIBRARY=ON .

    You may prefer some other way of doing this, so feel free to reject this pull request.

    opened by mhwombat 5
  • How can I construct an Arbitrary instance for (unions of) C-style arrays?

    How can I construct an Arbitrary instance for (unions of) C-style arrays?

    I'm trying to construct an arbitrary 16-byte array of unsigned chars, representing an IPv6 address. Well, actually I'm trying to generate a Microsoft in6_addr structure, defined as

    typedef struct in6_addr {
        union {
            UCHAR       Byte[16];
            USHORT      Word[8];
        } u;
    } IN6_ADDR, *PIN6_ADDR, FAR *LPIN6_ADDR;
    

    ... but I'm falling at the first hurdle. I can ask for a std::array<Byte, 16> and apply reinterpret_cast in my test code, but I'd like to construct an Arbitrary instance for the type so I can use it to build Arbitrary instances for bigger objects. Is this possible?

    opened by milesgould 5
  • Double fixture initialization with RC_GTEST_FIXTURE_PROP

    Double fixture initialization with RC_GTEST_FIXTURE_PROP

    Excellent work!

    Trying RapidCheck out, I have one issue though. Using the gtest integration causes fixture to be initialized twice.

    The following test case shows the problem:

    
    struct TestFixture : public ::testing::Test {
       static int initCounter;
    
       TestFixture() {
          std::cout << "Creating fixture" << std::endl;
          initCounter ++;
       }
    
       ~TestFixture() {
          std::cout << "Destructing fixture" << std::endl;
          initCounter --;
       }
    };
    
    int TestFixture::initCounter = 0;
    
    RC_GTEST_FIXTURE_PROP( TestFixture, TestInitCount, () ){
       std::cout << "RC_GTEST_FIXTURE_PROP(), initCounter: " << initCounter << std::endl;
    }
    
    TEST_F(TestFixture, ThisIsATest) {
       std::cout << "TEST_F(), initCounter: " << initCounter  << std::endl;
    }
    

    The output becomes:

    Note: Google Test filter = TestFixture.*
    [==========] Running 2 tests from 1 test case.
    [----------] Global test environment set-up.
    [----------] 2 tests from TestFixture
    [ RUN      ] TestFixture.TestInitCount
    Creating fixture
    Creating fixture
    RC_GTEST_FIXTURE_PROP(), initCounter: 2
    Destructing fixture
    Creating fixture
    RC_GTEST_FIXTURE_PROP(), initCounter: 2
    Destructing fixture
    Creating fixture
    RC_GTEST_FIXTURE_PROP(), initCounter: 2
    Destructing fixture
    Destructing fixture
    [       OK ] TestFixture.TestInitCount (1 ms)
    [ RUN      ] TestFixture.ThisIsATest
    Creating fixture
    TEST_F(), initCounter: 1
    Destructing fixture
    [       OK ] TestFixture.ThisIsATest (0 ms)
    [----------] 2 tests from TestFixture (1 ms total)
    
    [----------] Global test environment tear-down
    [==========] 2 tests from 1 test case ran. (1 ms total)
    [  PASSED  ] 2 tests.
    

    In my case, this causes problems with code that has static members that can't be initialized twice.

    opened by frejk 5
  • Crash on dereferencing generator when multiple threads are used

    Crash on dereferencing generator when multiple threads are used

    Hi!

    I encountered a problem with the following code:

    #include <cassert>
    #include <rapidcheck.h>
    #include <thread>
    #include <vector>
    
    int main(int argc, char *argv[])
    {
    	return rc::check("generator", [&] {
    		std::vector<std::thread> threads;
    		for (int i = 0; i < 2; i++) {
    			threads.emplace_back([&] {
    				auto ret = *rc::gen::inRange<unsigned>(0, 10);
    				assert(ret <= 10);
    			});
    		}
    		for (auto &t : threads)
    			t.join();
    	});
    }
    

    Below is output from the program:

    Using configuration: seed=8356093754809374738
    
    - generator
    terminate called after throwing an instance of 'std::runtime_error'
      what():  operator* is not allowed in this context
    

    It looks like it's not possible to use generators from multiple thread. Is this expected? Is there some workaround for this?

    I compiled my code using G++ 9.3.0-17ubuntu1~20.04 like this: g++ test.cpp /usr/lib/x86_64-linux-gnu/librapidcheck.a -lpthread

    opened by igchor 4
  • Improve CMake installation

    Improve CMake installation

    There are a couple of improvements made to the CMake files in order to make the export and import of the CMake targets more useful and, in general, make the library be as easily consumed as an external dependency as it is including it as sub-project - if RapidCheck is to be used in different package managers, this is essential.

    The changes were motivated by the desire to add a port of RapidCheck to VCPKG (a C++ based package manager). Here is a description of the major changes:

    • The install interface directories were not properly set, leading to the need to use "../rapidcheck.h" to access the primary library header. The fix for this is trivial.
    • All the extra integration header-only libraries are now properly installed and exported so that these integration can now be used from an installed bundle of the library.
    • The RC_INSTALL_ALL_EXTRAS CMake option is added to allow for an easier installation of the integrations without the need to require the inclusion and set up of the submodule projects in ext. This can be improved but it should be fine for an initial version.
    • The RC_BUILD_SHARED_LIB is added to have finer control over the type of the RapidCheck library. This is done as to have greater control over what kind of library VCPKG (and possibly other package managers) can build. It does not make much sense to build as a shared library for now (as no symbols are exported explicitly, which is an issue on Windows) but it would be useful to have the infrastructure in place.
    opened by Thorius 4
Simple C testing framework

MrTest Simple C testing framework Usage Copy the mrtest.c and mrtest.h file into your project. In order to use the mrtest main: create a .c file that

Maarten Raasveldt 2 Jul 20, 2022
Simple, fast, accurate single-header microbenchmarking functionality for C++11/14/17/20

ankerl::nanobench ankerl::nanobench is a platform independent microbenchmarking library for C++11/14/17/20. #includ

Martin Leitner-Ankerl 804 Aug 8, 2022
A simple, cross-platform, and continuously integrated C++14 project template

Project Name A simple, cross-platform, and continuously integrated C++14 project template. Making cross platform C++ projects is widely known to be a

Arnav Borborah 62 Jul 27, 2022
✔️The smallest header-only GUI library(4 KLOC) for all platforms

Welcome to GUI-lite The smallest header-only GUI library (4 KLOC) for all platforms. 中文 Lightweight ✂️ Small: 4,000+ lines of C++ code, zero dependenc

null 6.4k Aug 10, 2022
Kernel source for j7y17lte - the goal is to make it as closest to linux-stable sources as possible without breaking OneUI compatibility.

Linux kernel release 3.x <http://kernel.org/> These are the release notes for Linux version 3. Read them carefully, as they tell you what this is al

Exynos7870 1 Oct 28, 2021
Horde3D is a small 3D rendering and animation engine. It is written in an effort to create an engine being as lightweight and conceptually clean as possible.

Horde3D Horde3D is a 3D rendering engine written in C++ with an effort being as lightweight and conceptually clean as possible. Horde3D requires a ful

Volker Vogelhuber 1.3k Aug 15, 2022
Connect 4 clone written with c++ with the RSGL library. Based on my connect 4 clone written in python/pygame and my SDL port of that same repo. Along with 3DS support by SaCode

RSGL-Connect-4 Building linux git clone https://github.com/RSGL-Org/RSGL-Connect-4.git cd RSGL-Connect-4 make ./Connect4 Bulding 3ds (3ds support

RSGL 2 Dec 11, 2021
SDR++ is a cross-platform and open source SDR software with the aim of being bloat free and simple to use.

SDR++ is a cross-platform and open source SDR software with the aim of being bloat free and simple to use.

AlexandreRouma 1.8k Aug 12, 2022
This is kdmapper but it doesn't use ExAllocatePool instead it allocates pages to avoid being in BigPoolTable,

KDMapper without allocating memory in BigPoolTable Original creator https://github.com/z175 Improved by https://github.com/TheCruZ TheCruz has intergr

tygo lokum 33 Jun 12, 2022
A simple C++ 03/11/etc timer class for ~microsecond-precision cross-platform benchmarking. The implementation is as limited and as simple as possible to create the lowest amount of overhead.

plf_nanotimer A simple C++ 03/11/etc timer class for ~microsecond-precision cross-platform benchmarking. The implementation is as limited and simple a

Matt Bentley 89 Jul 15, 2022
This PoC uses two diferent technics for stealing the primary token from all running processes, showing that is possible to impersonate and use whatever token present at any process

StealAllTokens This PoC uses two diferent technics for stealing the primary token from all running processes, showing that is possible to impersonate

lab52.io 43 Jul 27, 2022
std::tuple like methods for user defined types without any macro or boilerplate code

Boost.PFR This is a C++14 library for very basic reflection that gives you access to structure elements by index and provides other std::tuple like me

Antony Polukhin 127 Aug 8, 2022
Type-safe zero-boilerplate interfaces for pure C99, implemented as a single-header library.

Interface99 Type-safe zero-boilerplate interfaces for pure C99, implemented as a single-header library. [ examples/state.c ] #include <interface99.h>

null 186 Aug 4, 2022
32blit SDK boilerplate for the PicoSystem RP2040-based handheld

PicoSystem 32blit Boilerplate This is a basic template for starting 32blit projects for the Pimoroni PicoSystem. It shows a minimal code layout and as

32blit 14 Jul 27, 2022
ESP32 + GitHub Actions + Husarnet. A boilerplate project for ESP32 allowing in-field firmware update using GitHub Actions workflow.

esp32-internet-ota ESP32 + GitHub Actions + Husarnet. A boilerplate project for ESP32 allowing in-field firmware update using GitHub Actions workflow.

Husarnet 29 Aug 4, 2022
Boilerplate-free YAML parsing for C++

AutoYAML AutoYAML is a Clang/LLVM/LibTooling-based program that automatically generates yaml-cpp conversion code for C++ record types. Usage As a moti

Timo Nicolai 10 Nov 12, 2021
A CMake addon that avoids you writing boilerplate code for resource management.

SHader INJ(I)ector SHINJI (originally SHader INJector) is a CMake addon that avoids you writing boilerplate code for resource management and exposes s

Lorenzo Rutayisire 6 Mar 21, 2022
Boilerplate to create a project with: STM32 + Ethernet + micro-ROS + FreeRTOS + Arduino + PlatformIO

micro_ros_stm32_template Boilerplate to create a project with: STM32 + Ethernet + micro-ROS + FreeRTOS + Arduino + PlatformIO Default config STM32F407

Husarion 13 Aug 6, 2022
I was tired of copy-pasting the same boilerplate code over and over across my projects.

Gui Since I was tired of copy-pasting the same boilerplate code over and over across my projects, I decided to make this wrapper that actually also ad

Simone Coco 5 Jan 10, 2022
Boilerplate of a TCP Server written in C++ using the epoll API

tcp-server-boilerplate Boilerplate of a TCP Server written in C++ using the epoll API. Requirements cmake (if you want to use the provided build.sh sc

Leonardo Folgoni 20 Jan 23, 2022
std::tuple like methods for user defined types without any macro or boilerplate code

Boost.PFR This is a C++14 library for very basic reflection that gives you access to structure elements by index and provides other std::tuple like me

Boost.org 1k Aug 7, 2022
A simple PoC to demonstrate that is possible to write Non writable memory and execute Non executable memory on Windows

WindowsPermsPoC A simple PoC to demonstrate that is possible to write Non writable memory and execute Non executable memory on Windows You can build i

Lorenzo Maffia 55 Jul 21, 2022
SIMULATeQCD is a multi-GPU Lattice QCD framework that makes it simple and easy for physicists to implement lattice QCD formulas while still providing the best possible performance.

SIMULATeQCD a SImple MUlti-GPU LATtice code for QCD calculations SIMULATeQCD is a multi-GPU Lattice QCD framework that makes it simple and easy for ph

null 10 Apr 23, 2022
A collection of as simple as possible, modern CMake projects

Modern CMake Examples Overview This repository is a collection of as simple as possible CMake projects (with a focus on installing). The idea is to tr

Tom Hulton-Harrop 967 Aug 7, 2022
A simple, funky, horrible fluid simulator, made possible by OLC's Pixel Game Engine

flOwie A simple, funky, horrible fluid simulator, made possible by OLC's Pixel Game Engine! Not meant for usefulness. The big O is in the name because

null 1 Jan 21, 2022
Pandex is a light but FAST programming language written in C . Pandex goal is that be hard & it's good for eductional goals

The Pandex programming language version 1.0.0.3 Pandex versions release type The Pandex version has four numbers. the first number holds 1 ( or 0 in s

null 8 May 23, 2022
A programming game, in which your goal is to help a group of dwarves establish a small outpost in the middle of a dangerous forest.

"Since they were to come in the days of the power of Melkor, Aulë made the dwarves strong to endure. Therefore they are stone-hard, stubborn, fast in

Alexey Nikolaev 5 Jul 29, 2022