UT: C++20 μ(micro)/Unit Testing Framework

Overview

Boost Licence Version conan Build Status Build Status Coveralls Codacy Badge Try it online

"If you liked it then you "should have put a"_test on it", Beyonce rule

[Boost::ext].UT / μt

| Motivation | Quick Start | Overview | Tutorial | Examples | User Guide | FAQ | Benchmarks |

C++ single header/single module, macro-free μ(micro)/Unit Testing Framework

Motivation

Testing is a very important part of the Software Development, however, C++ doesn't provide any good testing facilities out of the box, which often leads into a poor testing experience for develops and/or lack of tests/coverage in general.

One should treat testing code as production code!

Additionally, well established testing practises such as Test Driven Development (TDD)/Behaviour Driven Development (BDD) are often not followed due to the same reasons.

The following snippet is a common example of testing with projects in C++.

int main() {
  // should sum numbers
  {
    assert(3 == sum(1, 2));
  }
}

There are quite a few problems with the approach above

  • No names for tests (Hard to follow intentions by further readers)
  • No automatic registration of tests (No way to run specific tests)
  • Hard to debug (Assertions don't provide any information why it failed)
  • Hard to scale (No easy path forward for parameterized tests, multiple suites, parallel execution, etc...)
  • Hard to integrate (No easy way to have a custom output such as XML for CI integration)
  • Easy to make mistakes (With implicit casting, floating point comparison, pointer comparison for strings, etc...)
  • Hard to follow good practises such as TDD/BDD (Lack of support for sections and declarative expressions)
  • ...

[Boost::ext].UT is trying to address these issues by simplifying testing experience with a few simple steps:

And you good to go!

Okay, great, but why I would use [Boost::ext].UT over other/similar testing frameworks already available in C++?

Great question! There are a few unique features which makes [Boost::ext].UT worth trying

  • Firstly, it supports all the basic Unit Testing Framework features (automatic registration of tests, assertions, suites, etc...)
  • It's easy to integrate (it's just one header/module)
  • It's macro free which makes testing experience that much nicer (it uses modern C++ features instead, macros are opt-in rather than being compulsory - Can I still use macros?)
  • It's flexible (all parts of the framework such as: runner, reporter, printer can be customized, basically most other Unit Testing Frameworks can be implemented on top of UT primitives)
  • It has smaller learning curve (just a few simple concepts (expect, test, suite))
  • It leverages C++ features to support more complex testing (parameterized)
  • It's faster to compile and execute than similar frameworks which makes it suitable for bigger projects without additional hassle (Benchmarks)
  • It supports TDD/BDD workflows
  • It supports Gherkin specification
  • It supports Spec
  • ...

Sounds intriguing/interesting? Learn more at

Quick Start

https://bit.ly/ut-quick-start (slides)

Overview

Tutorial

    Step 0: Get it...

Get the latest latest header/module from here!

Include/Import

// #include <boost/ut.hpp> // single header
// import boost.ut;        // single module (C++20)

int main() { }

Compile & Run

$CXX main.cpp && ./a.out
All tests passed (0 assert in 0 test)

[Optional] Install it

cmake -Bbuild -H.
cd build && make         # run tests
cd build && make install # install

[Optional] CMake integration

This project provides a CMake config and target. Just load ut with find_package to import the boost::ut target. Linking against this target will add the necessary include directory for the single header file. This is demonstrated in the following example.

find_package(ut REQUIRED)

add_library(my_test my_test.cpp)

target_link_libraries(my_test PRIVATE boost::ut)

[Optional] Conan integration

The boost-ext-ut package is available from Conan Center. Just include it in your project's Conanfile with boost-ext-ut/1.1.8.

    Step 1: Expect it...

Let's write our first assertion, shall we?

int main() {
  boost::ut::expect(true);
}
All tests passed (1 asserts in 0 test)

https://godbolt.org/z/vfx-eB

Okay, let's make it fail now?

int main() {
  boost::ut::expect(1 == 2);
}
main.cpp:4:FAILED [false]
===============================================================================
tests:   0 | 0 failed
asserts: 1 | 0 passed | 1 failed

https://godbolt.org/z/7qTePx

Notice that expression 1 == 2 hasn't been printed. Instead we got false?

Let's print it then?

int main() {
  using namespace boost::ut;
  expect(1_i == 2);
}
main.cpp:4:FAILED [1 == 2]
===============================================================================
tests:   0 | 0 failed
asserts: 1 | 0 passed | 1 failed

https://godbolt.org/z/7MXVzu

Okay, now we have it! 1 == 2 has been printed as expected. Notice the User Defined Literal (UDL) 1_i was used. _i is a compile-time constant integer value

  • It allows to override comparison operators 👍
  • It disallow comparison of different types 👍

See the User-guide for more details.

Alternatively, a terse notation (no expect required) can be used.

int main() {
  using namespace boost::ut::literals;
  using namespace boost::ut::operators::terse;

  1_i == 2; // terse notation
}
main.cpp:7:FAILED [1 == 2]
===============================================================================
tests:   0 | 0 failed
asserts: 1 | 0 passed | 1 failed

https://godbolt.org/z/s77GSm

Other expression syntaxes are also available.

expect(1_i == 2);       // UDL syntax
expect(1 == 2_i);       // UDL syntax
expect(that % 1 == 2);  // Matcher syntax
expect(eq(1, 2));       // eq/neq/gt/ge/lt/le
main.cpp:6:FAILED [1 == 2]
main.cpp:7:FAILED [1 == 2]
main.cpp:8:FAILED [1 == 2]
main.cpp:9:FAILED [1 == 2]
===============================================================================
tests:   0 | 0 failed
asserts: 4 | 0 passed | 4 failed

https://godbolt.org/z/QbgGtc

Okay, but what about the case if my assertion is fatal. Meaning that the program will crash unless the processing will be terminated. Nothing easier, let's just add ! before the expect call to make it fatal.

expect((1 == 2_i) >> fatal); // fatal assertion
expect(1_i == 2);            // not executed
main.cpp:6:FAILED [1 == 2]
===============================================================================
tests:   1 | 1 failed
asserts: 2 | 0 passed | 2 failed

https://godbolt.org/z/WMe8Y1

But my expression is more complex than just simple comparisons. Not a problem, logic operators are also supported in the expect 👍 .

expect(42l == 42_l and 1 == 2_i); // compound expression
main.cpp:5:FAILED [(42 == 42 and 1 == 2)]
===============================================================================
tests:   0 | 0 failed
asserts: 1 | 0 passed | 1 failed

https://godbolt.org/z/aEhX4t

Can I add a custom message though? Sure, expect calls are streamable!

int main() {
  expect(42l == 42_l and 1 == 2_i) << "additional info";
}
main.cpp:5:FAILED [(42 == 42 and 1 == 2)] additional info
===============================================================================
tests:   0 | 0 failed
asserts: 1 | 0 passed | 1 failed

https://godbolt.org/z/v2PDuU

    Step 2: Group it...

Assertions are great, but how to combine them into more cohesive units? Test cases are the way to go! They allow to group expectations for the same functionality into coherent units.

"hello world"_test = [] { };

Alternatively test("hello world") = [] {} can be used.

All tests passed (0 asserts in 1 tests)

https://godbolt.org/z/Bh-EmY

Notice 1 tests but 0 asserts.

Let's make our first end-2-end test case, shall we?

int main() {
  "hello world"_test = [] {
    int i = 43;
    expect(42_i == i);
  };
}
Running "hello world"...
  main.cpp:8:FAILED [42 == 43]
FAILED
===============================================================================
tests:   1 | 1 failed
asserts: 1 | 0 passed | 1 failed

https://godbolt.org/z/Y43mXz

👍 We are done here!

I'd like to nest my tests, though and share setup/tear-down. With lambdas used to represents tests/sections we can easily achieve that. Let's just take a look at the following example.

int main() {
  "[vector]"_test = [] {
    std::vector<int> v(5);

    expect((5_ul == std::size(v)) >> fatal);

    should("resize bigger") = [v] { // or "resize bigger"_test
      mut(v).resize(10);
      expect(10_ul == std::size(v));
    };

    expect((5_ul == std::size(v)) >> fatal);

    should("resize smaller") = [=]() mutable { // or "resize smaller"_test
      v.resize(0);
      expect(0_ul == std::size(v));
    };
  }
}
All tests passed (4 asserts in 1 tests)

https://godbolt.org/z/XWAdYt

Nice! That was easy, but I'm a believer into Behaviour Driven Development (BDD). Is there a support for that? Yes! Same example as above just with the BDD syntax.

int main() {
  "vector"_test = [] {
    given("I have a vector") = [] {
      std::vector<int> v(5);
      expect((5_ul == std::size(v)) >> fatal);

      when("I resize bigger") = [=] {
        mut(v).resize(10);

        then("The size should increase") = [=] {
          expect(10_ul == std::size(v));
        };
      };
    };
  };
}
All tests passed (2 asserts in 1 tests)

https://godbolt.org/z/dnvxsE

On top of that, feature/scenario aliases can be leveraged.

int main() {
  feature("vector") = [] {
    scenario("size") = [] {
      given("I have a vector") = [] {
        std::vector<int> v(5);
        expect((5_ul == std::size(v)) >> fatal);

        when("I resize bigger") = [=] {
          mut(v).resize(10);

          then("The size should increase") = [=] {
            expect(10_ul == std::size(v));
          };
        };
      };
    };
  };
}
All tests passed (2 asserts in 1 tests)

https://godbolt.org/z/T4cWss

Can I use Gherkin? Yeah, let's rewrite the example using Gherkin specification

int main() {
  bdd::gherkin::steps steps = [](auto& steps) {
    steps.feature("Vector") = [&] {
      steps.scenario("*") = [&] {
        steps.given("I have a vector") = [&] {
          std::vector<int> v(5);
          expect((5_ul == std::size(v)) >> fatal);

          steps.when("I resize bigger") = [&] {
            v.resize(10);
          };

          steps.then("The size should increase") = [&] {
            expect(10_ul == std::size(v));
          };
        };
      };
    };
  };

  "Vector"_test = steps |
    R"(
      Feature: Vector
        Scenario: Resize
          Given I have a vector
           When I resize bigger
           Then The size should increase
    )";
}
All tests passed (2 asserts in 1 tests)

https://godbolt.org/z/jb1d8P

Nice, is Spec notation supported as well?

int main() {
  describe("vector") = [] {
    std::vector<int> v(5);
    expect((5_ul == std::size(v)) >> fatal);

    it("should resize bigger") = [v] {
      mut(v).resize(10);
      expect(10_ul == std::size(v));
    };
  };
}
All tests passed (2 asserts in 1 tests)

https://godbolt.org/z/6jKKzT

That's great, but how can call the same tests with different arguments/types to be DRY (Don't Repeat Yourself)? Parameterized tests to the rescue!

int main() {
  for (auto i : std::vector{1, 2, 3}) {
    test("parameterized " + std::to_string(i)) = [i] { // 3 tests
      expect(that % i > 0); // 3 asserts
    };
  }
}
All tests passed (3 asserts in 3 tests)

https://godbolt.org/z/Utnd6X

That's it 😮 ! Alternatively, a convenient test syntax is also provided 👍

int main() {
  "args"_test = [](const auto& arg) {
    expect(arg > 0_i) << "all values greater than 0";
  } | std::vector{1, 2, 3};
}
All tests passed (3 asserts in 3 tests)

https://godbolt.org/z/6FHtpq

Check Examples for further reading.

    Step 3: Scale it...

Okay, but my project is more complex than that. How can I scale? Test suites will make that possible. By using suite in translation units tests defined inside will be automatically registered 👍

suite errors = [] {
  "exception"_test = [] {
    expect(throws([] { throw 0; })) << "throws any exception";
  };

  "failure"_test = [] {
    expect(aborts([] { assert(false); }));
  };
};

int main() { }
All tests passed (2 asserts in 2 tests)

https://godbolt.org/z/_ccGwZ


What's next?

Examples

    Assertions

// operators
expect(0_i == sum());
expect(2_i != sum(1, 2));
expect(sum(1) >= 0_i);
expect(sum(1) <= 1_i);
// message
expect(3_i == sum(1, 2)) << "wrong sum";
// expressions
expect(0_i == sum() and 42_i == sum(40, 2));
expect(0_i == sum() or 1_i == sum()) << "compound";
// matchers
expect(that % 0 == sum());
expect(that % 42 == sum(40, 2) and that % (1 + 2) == sum(1, 2));
expect(that % 1 != 2 or 2_i > 3);
// eq/neq/gt/ge/lt/le
expect(eq(42, sum(40, 2)));
expect(neq(1, 2));
expect(eq(sum(1), 1) and neq(sum(1, 2), 2));
expect(eq(1, 1) and that % 1 == 1 and 1_i == 1);
// floating points
expect(42.1_d == 42.101) << "epsilon=0.1";
expect(42.10_d == 42.101) << "epsilon=0.01";
expect(42.10000001 == 42.1_d) << "epsilon=0.1";
// constant
constexpr auto compile_time_v = 42;
auto run_time_v = 99;
expect(constant<42_i == compile_time_v> and run_time_v == 99_i);
// failure
expect(1_i == 2) << "should fail";
expect(sum() == 1_i or 2_i == sum()) << "sum?";
assertions.cpp:53:FAILED [1 == 2] should fail
assertions.cpp:54:FAILED [(0 == 1 or 2 == 0)] sum?
===============================================================================
tests:   0  | 0 failed
asserts: 20 | 18 passed | 2 failed

https://godbolt.org/z/E1c7G5

    Tests

        Run/Skip/Tag

"run UDL"_test = [] {
  expect(42_i == 42);
};

skip / "don't run UDL"_test = [] {
  expect(42_i == 43) << "should not fire!";
};
All tests passed (1 asserts in 1 tests)
1 tests skipped
test("run function") = [] {
  expect(42_i == 42);
};

skip / test("don't run function") = [] {
  expect(42_i == 43) << "should not fire!";
};
All tests passed (1 asserts in 1 tests)
1 tests skipped
tag("nightly") / tag("slow") /
"performance"_test= [] {
  expect(42_i == 42);
};

tag("slow") /
"run slowly"_test= [] {
  expect(42_i == 43) << "should not fire!";
};
cfg<override> = {.tag = {"nightly"}};
All tests passed (1 asserts in 1 tests)
1 tests skipped

https://godbolt.org/z/X3_kG4

        Sections

"[vector]"_test = [] {
  std::vector<int> v(5);

  expect((5_ul == std::size(v)) >> fatal);

  should("resize bigger") = [=] { // or "resize bigger"_test
    mut(v).resize(10);
    expect(10_ul == std::size(v));
  };

  expect((5_ul == std::size(v)) >> fatal);

  should("resize smaller") = [=]() mutable { // or "resize smaller"_test
    v.resize(0);
    expect(0_ul == std::size(v));
  };
};
All tests passed (4 asserts in 1 tests)

https://godbolt.org/z/cE91bj

        Behavior Driven Development (BDD)

"Scenario"_test = [] {
  given("I have...") = [] {
    when("I run...") = [] {
      then("I expect...") = [] { expect(1_i == 1); };
      then("I expect...") = [] { expect(1 == 1_i); };
    };
  };
};
All tests passed (2 asserts in 1 tests)

https://godbolt.org/z/mNBySr

        Gherkin

int main() {
  bdd::gherkin::steps steps = [](auto& steps) {
    steps.feature("*") = [&] {
      steps.scenario("*") = [&] {
        steps.given("I have a number {value}") = [&](int value) {
          auto number = value;
          steps.when("I add {value} to it") = [&](int value) {
            number += value;
          };
          steps.then("I expect number to be {value}") = [&](int value) {
            expect(that % number == value);
          };
        };
      };
    };
  };

  "Gherkin"_test = steps |
    R"(
      Feature: Number
        Scenario: Addition
          Given I have a number 40
           When I add 2 to it
           Then I expect number to be 42
    )";
}
All tests passed (1 asserts in 1 tests)

https://godbolt.org/z/BP3hyt

        Spec

int main() {
  describe("equality") = [] {
    it("should be equal")     = [] { expect(0_i == 0); };
    it("should not be equal") = [] { expect(1_i != 0); };
  };
}
All tests passed (2 asserts in 1 tests)

https://godbolt.org/z/BXYJ3a

        Parameterized

for (auto i : std::vector{1, 2, 3}) {
  test("parameterized " + std::to_string(i)) = [i] {
    expect(that % i > 0);
  };
}

"args"_test =
   [](auto arg) {
      expect(arg >= 1_i);
    }
  | std::vector{1, 2, 3};

"types"_test =
    []<class T> {
      expect(std::is_integral_v<T>) << "all types are integrals";
    }
  | std::tuple<bool, int>{};

"args and types"_test =
    []<class TArg>(TArg arg) {
      expect(std::is_integral_v<TArg> >> fatal);
      expect(42_i == arg or "is true"_b == arg);
      expect(type<TArg> == type<int> or type<TArg> == type<bool>);
    }
  | std::tuple{true, 42};
All tests passed (14 asserts in 10 tests)

https://godbolt.org/z/4xGGdo

    Suites

namespace ut = boost::ut;

ut::suite errors = [] {
  using namespace ut;

  "throws"_test = [] {
    expect(throws([] { throw 0; }));
  };

  "doesn't throw"_test = [] {
    expect(nothrow([]{}));
  };
};

int main() { }
All tests passed (2 asserts in 2 tests)

https://godbolt.org/z/CFbTP9

    Misc

        Logging

"logging"_test = [] {
  log << "pre";
  expect(42_i == 43) << "message on failure";
  log << "post";
};
Running "logging"...
pre
  logging.cpp:8:FAILED [42 == 43] message on failure
post
FAILED

===============================================================================

tests:   1 | 1 failed
asserts: 1 | 0 passed | 1 failed

https://godbolt.org/z/26fPSY

        Matchers

"matchers"_test = [] {
  constexpr auto is_between = [](auto lhs, auto rhs) {
    return [=](auto value) {
      return that % value >= lhs and that % value <= rhs;
    };
  };

  expect(is_between(1, 100)(42));
  expect(not is_between(1, 100)(0));
};
All tests passed (2 asserts in 1 tests)

https://godbolt.org/z/4qwrCi

        Exceptions/Aborts

"exceptions/aborts"_test = [] {
  expect(throws<std::runtime_error>([] { throw std::runtime_error{""}; }))
    << "throws runtime_error";
  expect(throws([] { throw 0; })) << "throws any exception";
  expect(nothrow([]{})) << "doesn't throw";
  expect(aborts([] { assert(false); }));
};
All tests passed (4 asserts in 1 tests)

https://godbolt.org/z/A2EehK

    Config

        Runner

namespace ut = boost::ut;

namespace cfg {
  class runner {
   public:
    template <class... Ts> auto on(ut::events::test<Ts...> test) { test(); }
    template <class... Ts> auto on(ut::events::skip<Ts...>) {}
    template <class TExpr>
    auto on(ut::events::assertion<TExpr>) -> bool { return true; }
    auto on(ut::events::fatal_assertion) {}
    template <class TMsg> auto on(ut::events::log<TMsg>) {}
  };
} // namespace cfg

template<> auto ut::cfg<ut::override> = cfg::runner{};

https://godbolt.org/z/jdg687

        Reporter

namespace ut = boost::ut;

namespace cfg {
  class reporter {
   public:
    auto on(ut::events::test_begin) -> void {}
    auto on(ut::events::test_run) -> void {}
    auto on(ut::events::test_skip) -> void {}
    auto on(ut::events::test_end) -> void {}
    template <class TMsg> auto on(ut::events::log<TMsg>) -> void {}
    template <class TExpr>
    auto on(ut::events::assertion_pass<TExpr>) -> void {}
    template <class TExpr>
    auto on(ut::events::assertion_fail<TExpr>) -> void {}
    auto on(ut::events::fatal_assertion) -> void {}
    auto on(ut::events::exception) -> void {}
    auto on(ut::events::summary) -> void {}
  };
}  // namespace cfg

template <>
auto ut::cfg<ut::override> = ut::runner<cfg::reporter>{};

https://godbolt.org/z/gsAPKg

        Printer

namespace ut = boost::ut;

namespace cfg {
struct printer : ut::printer {
  template <class T>
  auto& operator<<(T&& t) {
    std::cerr << std::forward<T>(t);
    return *this;
  }
};
}  // namespace cfg

template <>
auto ut::cfg<ut::override> = ut::runner<ut::reporter<cfg::printer>>{};

int main() {
  using namespace ut;
  "printer"_test = [] {};
}

https://godbolt.org/z/XCscF9

User Guide

    API

export module boost.ut; /// __cpp_modules

namespace boost::inline ext::ut::inline v1_1_8 {
  /**
   * Represents test suite object
   */
  struct suite final {
    /**
     * Creates and executes test suite
     * @example suite _ = [] {};
     * @param suite test suite function
     */
    constexpr explicit(false) suite(auto suite);
  };

  /**
   * Creates a test
   * @example "name"_test = [] {};
   * @return test object to be executed
   */
  constexpr auto operator""_test;

  /**
   * Creates a test
   * @example test("name") = [] {};
   * @return test object to be executed
   */
  constexpr auto test = [](const auto name);

  /**
   * Creates a test
   * @example should("name") = [] {};
   * @return test object to be executed
   */
  constexpr auto should = [](const auto name);

  /**
   * Behaviour Driven Development (BDD) helper functions
   * @param name step name
   * @return test object to be executed
   */
  constexpr auto given = [](const auto name);
  constexpr auto when  = [](const auto name);
  constexpr auto then  = [](const auto name);

  /**
   * Evaluates an expression
   * @example expect(42 == 42_i and 1 != 2_i);
   * @param expr expression to be evaluated
   * @param source location https://en.cppreference.com/w/cpp/utility/source_location
   * @return stream
   */
  constexpr OStream& expect(
    Expression expr,
    const std::source_location& location = std::source_location::current()
  );

  struct {
    /**
     * @example (that % 42 == 42);
     * @param expr expression to be evaluated
     */
    [[nodiscard]] constexpr auto operator%(Expression expr) const;
  } that{};

  inline namespace literals {
    /**
     * User defined literals to represent constant values
     * @example 42_i, 0_uc, 1.23_d
     */
    constexpr auto operator""_i;  /// int
    constexpr auto operator""_s;  /// short
    constexpr auto operator""_c;  /// char
    constexpr auto operator""_l;  /// long
    constexpr auto operator""_ll; /// long long
    constexpr auto operator""_u;  /// unsigned
    constexpr auto operator""_uc; /// unsigned char
    constexpr auto operator""_us; /// unsigned short
    constexpr auto operator""_ul; /// unsigned long
    constexpr auto operator""_f;  /// float
    constexpr auto operator""_d;  /// double
    constexpr auto operator""_ld; /// long double

    /**
     * Represents dynamic values
     * @example _i(42), _f(42.)
     */
    constexpr auto _b(bool);
    constexpr auto _c(char);
    constexpr auto _s(short);
    constexpr auto _i(int);
    constexpr auto _l(long);
    constexpr auto _ll(long long);
    constexpr auto _u(unsigned);
    constexpr auto _uc(unsigned char);
    constexpr auto _us(unsigned short);
    constexpr auto _ul(unsigned long);
    constexpr auto _f(float);
    constexpr auto _d(double);
    constexpr auto _ld(long double);

    /**
     * Logical representation of constant boolean (true) value
     * @example "is set"_b     : true
     *          not "is set"_b : false
     */
    constexpr auto operator ""_b;
  } // namespace literals

  inline namespace operators {
    /**
     * Comparison functions to be used in expressions
     * @example eq(42, 42), neq(1, 2)
     */
    constexpr auto eq(Operator lhs, Operator rhs);  /// ==
    constexpr auto neq(Operator lhs, Operator rhs); /// !=
    constexpr auto gt(Operator lhs, Operator rhs);  /// >
    constexpr auto ge(Operator lhs, Operator rhs);  /// >=
    constexpr auto lt(Operator lhs, Operator rhs);  /// <
    constexpr auto le(Operator lhs, Operator rhs);  /// <=

    /**
     * Overloaded comparison operators to be used in expressions
     * @example (42_i != 0)
     */
    constexpr auto operator==;
    constexpr auto operator!=;
    constexpr auto operator>;
    constexpr auto operator>=;
    constexpr auto operator<;
    constexpr auto operator<=;

    /**
     * Overloaded logic operators to be used in expressions
     * @example (42_i != 0 and 1 == 2_i)
     */
    constexpr auto operator and;
    constexpr auto operator or;
    constexpr auto operator not;

    /**
     * Executes parameterized tests
     * @example "parameterized"_test = [](auto arg) {} | std::tuple{1, 2, 3};
     */
    constexpr auto operator|;

    /**
     * Creates tags
     * @example tag("slow") / tag("nightly") / "perf"_test = []{};
     */
    constexpr auto operator/;

    /**
     * Creates a `fatal_assertion` from an expression
     * @example (42_i == 0) >> fatal
     */
    constexpr auto operator>>;
  } // namespace operators

  /**
   * Creates skippable test object
   * @example skip / "don't run"_test = [] { };
   */
  constexpr auto skip = tag("skip");

  struct {
    /**
     * @example log << "message!";
     * @param msg stringable message
     */
    auto& operator<<(Msg msg);
  } log{};

  /**
   * Makes object mutable
   * @example mut(object)
   * @param t object to be mutated
   */
  template<class T> auto mut(const T& t) -> T&;

  /**
   * Default execution flow policy
   */
  class runner {
   public:
    /**
     * @example cfg<override> = {
        .filter  = "test.section.*",
        .colors  = { .none = "" },
        .dry__run = true
       };
     * @param options.filter {default: "*"} runs all tests which names
                                            matches test.section.* filter
     * @param options.colors {default: {
                               .none = "\033[0m",
                               .pass = "\033[32m",
                               .fail  = "\033[31m"
              } if specified then overrides default color values
     * @param options.dry_run {default: false} if true then print test names to be
                                               executed without running them
     */
    auto operator=(options);

    /**
     * @example suite _ = [] {};
     * @param suite() executes suite
     */
    template<class TSuite>
    auto on(ut::events::suite<TSuite>);

    /**
     * @example "name"_test = [] {};
     * @param test.type ["test", "given", "when", "then"]
     * @param test.name "name"
     * @param test.arg parameterized argument
     * @param test() executes test
     */
    template<class... Ts>
    auto on(ut::events::test<Ts...>);

    /**
     * @example skip / "don't run"_test = []{};
     * @param skip.type ["test", "given", "when", "then"]
     * @param skip.name "don't run"
     * @param skip.arg parameterized argument
     */
    template<class... Ts>
    auto on(ut::events::skip<Ts...>);

    /**
     * @example file.cpp:42: expect(42_i == 42);
     * @param assertion.expr 42_i == 42
     * @param assertion.location { "file.cpp", 42 }
     * @return true if expr passes, false otherwise
     */
    template <class TExpr>
    auto on(ut::events::assertion<TExpr>) -> bool;

    /**
     * @example expect((2_i == 1) >> fatal)
     * @note triggered by `fatal`
     *       should std::exit
     */
    auto on(ut::events::fatal_assertion);

    /**
     * @example log << "message"
     * @param log.msg "message"
     */
    template<class TMsg>
    auto on(ut::events::log<TMsg>);

    /**
     * Explicitly runs registered test suites
     * If not called directly test suites are executed with run's destructor
     * @example return run({.report_errors = true})
     * @param run_cfg.report_errors {default: false} if true it prints the summary after runnig
     */
    auto run(run_cfg);

    /**
     * Runs registered test suites if they haven't been explicilty executed already
     */
    ~run();
  };

  /**
   * Default reporter policy
   */
  class reporter {
   public:
    /**
     * @example file.cpp:42: "name"_test = [] {};
     * @param test_begin.type ["test", "given", "when", "then"]
     * @param test_begin.name "name"
     * @param test_begin.location { "file.cpp", 42 }
     */
    auto on(ut::events::test_begin) -> void;

    /**
     * @example "name"_test = [] {};
     * @param test_run.type ["test", "given", "when", "then"]
     * @param test_run.name "name"
     */
    auto on(ut::events::test_run) -> void;

    /**
     * @example "name"_test = [] {};
     * @param test_skip.type ["test", "given", "when", "then"]
     * @param test_skip.name "name"
     */
    auto on(ut::events::test_skip) -> void;

    /**
     * @example "name"_test = [] {};
     * @param test_end.type ["test", "given", "when", "then"]
     * @param test_end.name "name"
     */
    auto on(ut::events::test_end) -> void;

    /**
     * @example log << "message"
     * @param log.msg "message"
     */
    template<class TMsg>
    auto on(ut::events::log<TMsg>) -> void;

    /**
     * @example file.cpp:42: expect(42_i == 42);
     * @param assertion_pass.expr 42_i == 42
     * @param assertion_pass.location { "file.cpp", 42 }
     */
    template <class TExpr>
    auto on(ut::events::assertion_pass<TExpr>) -> void;

    /**
     * @example file.cpp:42: expect(42_i != 42);
     * @param assertion_fail.expr 42_i != 42
     * @param assertion_fail.location { "file.cpp", 42 }
     */
    template <class TExpr>
    auto on(ut::events::assertion_fail<TExpr>) -> void;

    /**
     * @example expect((2_i == 1) >> fatal)
     * @note triggered by `fatal`
     *       should std::exit
     */
    auto on(ut::events::fatal_assertion) -> void;

    /**
     * @example "exception"_test = [] { throw std::runtime_error{""}; };
     */
    auto on(ut::events::exception) -> void;

    /**
     * @note triggered on destruction of runner
     */
    auto on(ut::events::summary) -> void;
  };

  /**
   * Used to override default running policy
   * @example template <> auto cfg<override> = runner<reporter>{};
   */
  struct override {};

  /**
   * Default UT execution policy
   * Can be overwritten with override
   */
  template <class = override> auto cfg = runner<reporter>{};
}

    Configuration

Option Description Example
BOOST_UT_VERSION Current version 1'1'8

FAQ

    How does it work?

suite

/**
 * Reperesents suite object
 * @example suite _ = []{};
 */
struct suite final {
  /**
   * Assigns and executes test suite
   */
  [[nodiscard]] constexpr explicit(false) suite(Suite suite) {
    suite();
  }
};

test

/**
 * Creates named test object
 * @example "hello world"_test
 * @return test object
 */
[[nodiscard]] constexpr Test operator ""_test(const char* name, std::size_t size) {
  return test{{name, size}};
}
/**
 * Represents test object
 */
struct test final {
  std::string_view name{}; /// test case name

  /**
   * Assigns and executes test function
   * @param test function
   */
  constexpr auto operator=(const Test& test) {
    std::cout << "Running... " << name << '\n';
    test();
  }
};

expect

/**
 * Evaluates an expression
 * @example expect(42_i == 42);
 * @param expr expression to be evaluated
 * @param source location https://en.cppreference.com/w/cpp/utility/source_location
 * @return stream
 */
constexpr OStream& expect(
  Expression expr,
  const std::source_location& location = std::source_location::current()
) {
  if (not static_cast<bool>(expr) {
    std::cerr << location.file()
              << ':'
              << location.line()
              << ":FAILED: "
              << expr
              << '\n';
  }

  return std::cerr;
}
/**
 * Creates constant object for which operators can be overloaded
 * @example 42_i
 * @return integral constant object
 */
template <char... Cs>
[[nodiscard]] constexpr Operator operator""_i() -> integral_constant<int, value<Cs...>>;
/**
 * Overloads comparison if at least one of {lhs, rhs} is an Operator
 * @example (42_i == 42)
 * @param lhs Left-hand side operator
 * @param rhs Right-hand side operator
 * @return Comparison object
 */
[[nodiscard]] constexpr auto operator==(Operator lhs, Operator rhs) {
  return eq{lhs, rhs};
}
/**
 * Comparison Operator
 */
template <Operator TLhs, Opeartor TRhs>
struct eq final {
  TLhs lhs{}; // Left-hand side operator
  TRhs rhs{}; // Right-hand side operator

  /**
   * Performs comparison operatation
   * @return true if expression is succesful
   */
  [[nodiscard]] constexpr explicit operator bool() const {
    return lhs == rhs;
  }

  /**
   * Nicely prints the operation
   */
  friend auto operator<<(OStream& os, const eq& op) -> Ostream& {
    return (os << op.lhs << " == " << op.rhs);
  }
};

Sections

/**
 * Convenient aliases for creating test named object
 * @example should("return true") = [] {};
 */
constexpr auto should = [](const auto name) { return test{name}; };

Behaviour Driven Development (BDD)

/**
 * Convenient aliases for creating BDD tests
 * @example feature("Feature") = [] {};
 * @example scenario("Scenario") = [] {};
 * @example given("I have an object") = [] {};
 * @example when("I call it") = [] {};
 * @example then("I should get") = [] {};
 */
constexpr auto feature  = [](const auto name) { return test{name}; };
constexpr auto scenario = [](const auto name) { return test{name}; };
constexpr auto given    = [](const auto name) { return test{name}; };
constexpr auto when     = [](const auto name) { return test{name}; };
constexpr auto then     = [](const auto name) { return test{name}; };

https://godbolt.org/z/6Nk5Mi

Spec

/**
 * Convenient aliases for creating Spec tests
 * @example describe("test") = [] {};
 * @example it("should...") = [] {};
 */
constexpr auto describe = [](const auto name) { return test{name}; };
constexpr auto it       = [](const auto name) { return test{name}; };

Example implementation

Try it online

    Fast compilation times (Benchmarks)?

Implementation

  • Leveraging C++20 features

  • Avoiding unique types for lambda expressions

  template <class Test>
    requires not std::convertible_to<Test, void (*)()>>
  constexpr auto operator=(Test test);

vs

  // Compiles 5x faster because it doesn't introduce a new type for each lambda
  constexpr auto operator=(void (*test)());
  • Type-name erasure (allows types/function memoization)
  eq<integral_constant<42>, int>{ {}, 42 }

vs

  // Can be memoized - faster to compile
  eq<int, int>{42, 42}
  • Limiting preprocessor work

    • Single header/module
    • Minimal number of include files
  • Simplified versions of

    • std::function
    • std::string_view

    C++20 features?

    C++2X integration?

Parameterized tests with Expansion statements (https://wg21.link/P1306r1)

template for (auto arg : std::tuple<int, double>{}) {
  test("types " + std::to_string(arg)) = [arg] {
    expect(type(arg) == type<int> or type(arg) == type<double>);
  };
}
All tests passed (2 asserts in 2 tests)

https://cppx.godbolt.org/z/dMmqmM

    Is standardization an option?

Personally, I believe that C++ standard could benefit from common testing primitives (expect, ""_test) because

  • It lowers the entry-level to the language (no need for third-party libraries)
  • It improves the education aspect (one standard way of doing it)
  • It makes the language more coherent/stable (consistent design with other features, stable API)
  • It makes the testing a first class citizen (shows that the community cares about this aspect of the language)
  • It allows to publish tests for the Standard Library (STL) in the standard way (coherency, easier to extend)
  • It allows to act as additional documentation as a way to verify whether a particular implementation is conforming (quality, self-verification)
  • It helps with establishing standard vocabulary for testing (common across STL and other projects)

    Can I still use macros?

Sure, although please notice that there are negatives of using macros such as

  • Error messages might be not clear and/or point to the wrong line
  • Global scope will be polluted
  • Type safety will be ignored
#define EXPECT(...) ::boost::ut::expect(::boost::ut::that % __VA_ARGS__)
#define SUITE       ::boost::ut::suite _ = []
#define TEST(name)  ::boost::ut::detail::test{"test", name} = [=]() mutable

SUITE {
  TEST("suite") {
    EXPECT(42 == 42);
  };
};

int main() {
  TEST("macro") {
    EXPECT(1 != 2);
  };

  TEST("vector") {
    std::vector<int> v(5);

   EXPECT((5u == std::size(v)) >> fatal) << "fatal";

    TEST("resize bigger") {
      v.resize(10);
      EXPECT(10u == std::size(v));
    };
  };
}
All tests passed (4 asserts in 3 tests)

https://godbolt.org/z/WcEKTr

    What about Mocks/Stubs/Fakes?

Consider using one of the following frameworks

    What about Microbenchmarking?

Example benchmark

Consider using one of the following frameworks

    Related materials/talks?

    How to contribute?

CONTRIBUTING

Benchmarks

Framework Version Standard License Linkage Test configuration
Boost.Test 1.71.0 C++03 Boost 1.0 single header/library static library
GoogleTest 1.10.0 C++11 BSD-3 library static library
Catch 2.10.2 C++11 Boost 1.0 single header CATCH_CONFIG_FAST_COMPILE
Doctest 2.3.5 C++11 MIT single header DOCTEST_CONFIG_SUPER_FAST_ASSERTS
[Boost::ext].UT 1.1.0 C++20 Boost 1.0 single header/module
Include / 0 tests, 0 asserts, 1 cpp file
Assert / 1 test, 1'000'000 asserts, 1 cpp file
Test / 1'000 tests, 0 asserts, 1 cpp file
Suite / 10'000 tests, 0 asserts, 100 cpp files
Suite+Assert / 10'000 tests, 40'000 asserts, 100 cpp files
Suite+Assert+STL / 10'000 tests, 20'000 asserts, 100 cpp files
Incremental Build - Suite+Assert+STL / 1 cpp file change (1'000 tests, 20'000 asserts, 100 cpp files)
Suite+Assert+STL / 10'000 tests, 20'000 asserts, 100 cpp files
(Headers vs Precompiled headers vs C++20 Modules)

https://github.com/cpp-testing/ut-benchmark


Disclaimer [Boost::ext].UT is not an official Boost library.

Comments
  • Source-file-location fails on macOS, giving 'unknown'

    Source-file-location fails on macOS, giving 'unknown'

    Expected Behavior

    The ApprovalTests.cpp UT integration tests pass with the latest version of this project on macOS.

    This combination passes:

    Actual Behavior

    On macOS, with both g++9 homebrew and clang-9 MacPorts, using 515c0803f1ffc2016a84dcfbb737d0ec8075bed4, the UT tests all fail with:

    *****************************************************************************
    *                                                                           *
    * Welcome to Approval Tests.
    *
    * There seems to be a problem with your build configuration.
    * We cannot find the test source file at:
    *   unknown
    *
    * For details on how to fix this, please visit:
    * https://github.com/approvals/ApprovalTests.cpp/blob/master/doc/TroubleshootingMisconfiguredBuild.md
    *                                                                           *
    *****************************************************************************
    

    Steps to Reproduce the Problem

    • Clone ApprovalTest.cpp: 63acddcf2d0833e5e757f801f3f3c821f110c581
    • Copy ut header 515c0803f1ffc2016a84dcfbb737d0ec8075bed4 over ApprovalTests.cpp/third_party/ut/include/boost/ut.hpp
    • Build on Mac
    • Run UT_Tests

    Specifications

    It's failing for me with:

    CMAKE_VERSION = 3.16.2
    CMAKE_GENERATOR = Unix Makefiles
    CMAKE_SOURCE_DIR = /Users/clare/Documents/develop/ApprovalTests/ApprovalTests.cpp
    CMAKE_CURRENT_SOURCE_DIR = /Users/clare/Documents/develop/ApprovalTests/ApprovalTests.cpp
    CMAKE_CURRENT_BINARY_DIR = /Users/clare/Documents/develop/ApprovalTests/ApprovalTests.cpp/cmake-build-debug-g-9-brew
    CMAKE_CXX_COMPILER = /usr/local/Cellar/gcc/9.2.0_2/bin/g++-9
    CMAKE_CXX_COMPILER_ID = GNU
    CMAKE_CXX_COMPILER_VERSION = 9.2.0
    CMAKE_UNITY_BUILD = 
    

    and

    CMAKE_VERSION = 3.16.2
    CMAKE_GENERATOR = Unix Makefiles
    CMAKE_SOURCE_DIR = /Users/clare/Documents/develop/ApprovalTests/ApprovalTests.cpp
    CMAKE_CURRENT_SOURCE_DIR = /Users/clare/Documents/develop/ApprovalTests/ApprovalTests.cpp
    CMAKE_CURRENT_BINARY_DIR = /Users/clare/Documents/develop/ApprovalTests/ApprovalTests.cpp/cmake-build-debug-clang-mp-90-brew
    CMAKE_CXX_COMPILER = /opt/local/bin/clang++-mp-9.0
    CMAKE_CXX_COMPILER_ID = Clang
    CMAKE_CXX_COMPILER_VERSION = 9.0.1
    CMAKE_UNITY_BUILD = 
    

    Likely cause

    There are not many differences between 8004290d195b08eda531eb015ddcda2e11d41ce9 and 515c0803f1ffc2016a84dcfbb737d0ec8075bed4

    I suspect that the cause is this, as in both compiler cases, __APPLE__ is defined:

    @@ -211,10 +211,10 @@ namespace reflection {
     class source_location {
      public:
       [[nodiscard]] static constexpr auto current(
    -#if (__GNUC__ >= 9 or __clang_major__ >= 9)
    +#if ((__GNUC__ >= 9 or __clang_major__ >= 9) and not defined(__APPLE__))
           const char* file = __builtin_FILE(), int line = __builtin_LINE()
     #else
    -      const char* file = {}, int line = {}
    +      const char* file = "unknown", int line = {}
     #endif
               ) noexcept {
         source_location sl{};
    
    opened by claremacrae 19
  • Errors popped up for Visual Studio 16.10 Preview 3

    Errors popped up for Visual Studio 16.10 Preview 3

    I'm posting this because I'm not sure if it's a problem with Visual Studio or Boost.ut or if it's me.

    Expected Behavior

    These worked in the 16.9.x versions of Visual Studio.

    Actual Behavior

    Errors popped up for Visual Studio 16.10 Preview 3

    Steps to Reproduce the Problem

    1. Install Visual Studio 16.10 Preview 3
    2. Try to compile
    3. Error

    Specifications

    • Version: boost.ut trunk
    • Platform: Windows 10 x64
    • Subsystem: ?
    ====================[ Build | all | Debug Windows ]=============================
    "C:\Program Files\JetBrains\CLion 2020.2\bin\cmake\win\bin\cmake.exe" --build D:\dev\ToolsLibrary\cmake-build-debug-windows --target all
    Scanning dependencies of target ToolsLibrary_random_test
    [  7%] Building CXX object tests/src/CMakeFiles/ToolsLibrary_random_test.dir/random_test.cpp.obj
    random_test.cpp
    D:\dev\ToolsLibrary\cmake-build-debug-windows\_deps\ut-src\include\boost/ut.hpp(9): error C3378: a declaration can be exported only from a module interface unit
    D:\dev\ToolsLibrary\cmake-build-debug-windows\_deps\ut-src\include\boost/ut.hpp(9): error C2230: could not find module 'boost.ut'
    D:\dev\ToolsLibrary\cmake-build-debug-windows\_deps\ut-src\include\boost/ut.hpp(10): error C3378: a declaration can be exported only from a module interface unit
    D:\dev\ToolsLibrary\cmake-build-debug-windows\_deps\ut-src\include\boost/ut.hpp(10): error C7635: a module import declaration cannot appear in an 'export' declaration sequence
    D:\dev\ToolsLibrary\cmake-build-debug-windows\_deps\ut-src\include\boost/ut.hpp(16): warning C5244: '#include <ciso646>' in the purview of module 'boost.ut' appears erroneous.  Consider moving that directive before the module declaration, or replace the textual inclusion with 'import <ciso646>;'.
    D:\dev\ToolsLibrary\cmake-build-debug-windows\_deps\ut-src\include\boost/ut.hpp(16): note: to simplify migration, consider the temporary use of /Wv:18 flag with the version of the compiler with which you used to build without warnings
    D:\dev\ToolsLibrary\cmake-build-debug-windows\_deps\ut-src\include\boost/ut.hpp(9): note: see module 'boost.ut' declaration
    D:\dev\ToolsLibrary\cmake-build-debug-windows\_deps\ut-src\include\boost/ut.hpp(55): warning C5244: '#include <array>' in the purview of module 'boost.ut' appears erroneous.  Consider moving that directive before the module declaration, or replace the textual inclusion with 'import <array>;'.
    D:\dev\ToolsLibrary\cmake-build-debug-windows\_deps\ut-src\include\boost/ut.hpp(55): note: to simplify migration, consider the temporary use of /Wv:18 flag with the version of the compiler with which you used to build without warnings
    D:\dev\ToolsLibrary\cmake-build-debug-windows\_deps\ut-src\include\boost/ut.hpp(9): note: see module 'boost.ut' declaration
    D:\dev\ToolsLibrary\cmake-build-debug-windows\_deps\ut-src\include\boost/ut.hpp(56): warning C5244: '#include <iostream>' in the purview of module 'boost.ut' appears erroneous.  Consider moving that directive before the module declaration, or replace the textual inclusion with 'import <iostream>;'.
    D:\dev\ToolsLibrary\cmake-build-debug-windows\_deps\ut-src\include\boost/ut.hpp(56): note: to simplify migration, consider the temporary use of /Wv:18 flag with the version of the compiler with which you used to build without warnings
    D:\dev\ToolsLibrary\cmake-build-debug-windows\_deps\ut-src\include\boost/ut.hpp(9): note: see module 'boost.ut' declaration
    D:\dev\ToolsLibrary\cmake-build-debug-windows\_deps\ut-src\include\boost/ut.hpp(57): warning C5244: '#include <sstream>' in the purview of module 'boost.ut' appears erroneous.  Consider moving that directive before the module declaration, or replace the textual inclusion with 'import <sstream>;'.
    D:\dev\ToolsLibrary\cmake-build-debug-windows\_deps\ut-src\include\boost/ut.hpp(57): note: to simplify migration, consider the temporary use of /Wv:18 flag with the version of the compiler with which you used to build without warnings
    D:\dev\ToolsLibrary\cmake-build-debug-windows\_deps\ut-src\include\boost/ut.hpp(9): note: see module 'boost.ut' declaration
    D:\dev\ToolsLibrary\cmake-build-debug-windows\_deps\ut-src\include\boost/ut.hpp(58): warning C5244: '#include <string_view>' in the purview of module 'boost.ut' appears erroneous.  Consider moving that directive before the module declaration, or replace the textual inclusion with 'import <string_view>;'.
    D:\dev\ToolsLibrary\cmake-build-debug-windows\_deps\ut-src\include\boost/ut.hpp(58): note: to simplify migration, consider the temporary use of /Wv:18 flag with the version of the compiler with which you used to build without warnings
    D:\dev\ToolsLibrary\cmake-build-debug-windows\_deps\ut-src\include\boost/ut.hpp(9): note: see module 'boost.ut' declaration
    D:\dev\ToolsLibrary\cmake-build-debug-windows\_deps\ut-src\include\boost/ut.hpp(60): warning C5244: '#include <vector>' in the purview of module 'boost.ut' appears erroneous.  Consider moving that directive before the module declaration, or replace the textual inclusion with 'import <vector>;'.
    D:\dev\ToolsLibrary\cmake-build-debug-windows\_deps\ut-src\include\boost/ut.hpp(60): note: to simplify migration, consider the temporary use of /Wv:18 flag with the version of the compiler with which you used to build without warnings
    D:\dev\ToolsLibrary\cmake-build-debug-windows\_deps\ut-src\include\boost/ut.hpp(9): note: see module 'boost.ut' declaration
    D:\dev\ToolsLibrary\cmake-build-debug-windows\_deps\ut-src\include\boost/ut.hpp(70): error C3378: a declaration can be exported only from a module interface unit
    D:\dev\ToolsLibrary\src\tl/algorithm.hpp(9): warning C5244: '#include <algorithm>' in the purview of module 'boost.ut' appears erroneous.  Consider moving that directive before the module declaration, or replace the textual inclusion with 'import <algorithm>;'.
    D:\dev\ToolsLibrary\src\tl/algorithm.hpp(9): note: to simplify migration, consider the temporary use of /Wv:18 flag with the version of the compiler with which you used to build without warnings
    D:\dev\ToolsLibrary\cmake-build-debug-windows\_deps\ut-src\include\boost/ut.hpp(9): note: see module 'boost.ut' declaration
    D:\dev\ToolsLibrary\src\tl/algorithm.hpp(10): warning C5244: '#include <cassert>' in the purview of module 'boost.ut' appears erroneous.  Consider moving that directive before the module declaration, or replace the textual inclusion with 'import <cassert>;'.
    D:\dev\ToolsLibrary\src\tl/algorithm.hpp(10): note: to simplify migration, consider the temporary use of /Wv:18 flag with the version of the compiler with which you used to build without warnings
    D:\dev\ToolsLibrary\cmake-build-debug-windows\_deps\ut-src\include\boost/ut.hpp(9): note: see module 'boost.ut' declaration
    D:\dev\ToolsLibrary\src\tl/algorithm.hpp(11): warning C5244: '#include <numeric>' in the purview of module 'boost.ut' appears erroneous.  Consider moving that directive before the module declaration, or replace the textual inclusion with 'import <numeric>;'.
    D:\dev\ToolsLibrary\src\tl/algorithm.hpp(11): note: to simplify migration, consider the temporary use of /Wv:18 flag with the version of the compiler with which you used to build without warnings
    D:\dev\ToolsLibrary\cmake-build-debug-windows\_deps\ut-src\include\boost/ut.hpp(9): note: see module 'boost.ut' declaration
    D:\dev\ToolsLibrary\src\tl/random.hpp(8): warning C5244: '#include <random>' in the purview of module 'boost.ut' appears erroneous.  Consider moving that directive before the module declaration, or replace the textual inclusion with 'import <random>;'.
    D:\dev\ToolsLibrary\src\tl/random.hpp(8): note: to simplify migration, consider the temporary use of /Wv:18 flag with the version of the compiler with which you used to build without warnings
    D:\dev\ToolsLibrary\cmake-build-debug-windows\_deps\ut-src\include\boost/ut.hpp(9): note: see module 'boost.ut' declaration
    NMAKE : fatal error U1077: 'C:\PROGRA~2\MICROS~1\2019\Preview\VC\Tools\MSVC\1429~1.300\bin\Hostx86\x86\cl.exe' : return code '0x2'
    Stop.
    NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio\2019\Preview\VC\Tools\MSVC\14.29.30035\bin\HostX86\x86\nmake.exe"' : return code '0x2'
    Stop.
    NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio\2019\Preview\VC\Tools\MSVC\14.29.30035\bin\HostX86\x86\nmake.exe"' : return code '0x2'
    Stop.
    
    opened by Sebanisu 16
  • Recent CMake changes in this project have broken builds of dependent projects

    Recent CMake changes in this project have broken builds of dependent projects

    Expected Behavior

    1. boost.ut target seems to no longer be created

    That the target boost.ut is still provided, to link against, when this project is obtained via add_subdirectory(), FetchContent() or similar...

    2. include file no longer found: #include <boost/ut.hpp>

    Example code - this is how I obtain the boost.ut project - UtVersion is master:

    https://github.com/claremacrae/ApprovalTests.cpp.CMakeSamples/blob/main/dev_approvals_fetch_content/CMakeLists.txt#L130-L133

    FetchContent_Declare(boost.ut
            GIT_REPOSITORY https://github.com/boost-ext/ut.git
            GIT_TAG ${UtVersion})
    FetchContent_MakeAvailable(boost.ut)
    

    The repo goes on to use the target boost.ut.

    This directory and GitHub action exists to help the ApprovalTests.cpp project have early detection of any changes in test frameworks that might affect our users, when we do our next release.

    This has been working for some months, and was still working on the 24th of Feb.

    Actual Behavior

    The next build, on 27th Feb at 01:10 GMT gave these errors

    1. in clang builds:

    Fetching boost.ut...
    CMake Error at CMakeLists.txt:137 (target_compile_options):
      Cannot specify compile options for target "boost.ut" which is not built by
      this project.
    

    2. in gcc builds:

    Building CXX object approvaltests.cpp_build/tests/UT_Tests/CMakeFiles/UT_Tests.dir/UTApprovalTestTests.cpp.o
    In file included from /home/runner/work/ApprovalTests.cpp.CMakeSamples/ApprovalTests.cpp.CMakeSamples/ApprovalTests.cpp/ApprovalTests/ApprovalTests.hpp:68,
                     from /home/runner/work/ApprovalTests.cpp.CMakeSamples/ApprovalTests.cpp.CMakeSamples/ApprovalTests.cpp/tests/UT_Tests/UTApprovalTestTests.cpp:3:
    /home/runner/work/ApprovalTests.cpp.CMakeSamples/ApprovalTests.cpp.CMakeSamples/ApprovalTests.cpp/ApprovalTests/../ApprovalTests/integrations/ut/UTApprovals.h:14:10: fatal error: boost/ut.hpp: No such file or directory
       14 | #include <boost/ut.hpp>
          |          ^~~~~~~~~~~~~~
    compilation terminated.
    approvaltests.cpp_build/tests/UT_Tests/CMakeFiles/UT_Tests.dir/build.make:81: recipe for target 'approvaltests.cpp_build/tests/UT_Tests/CMakeFiles/UT_Tests.dir/UTApprovalTestTests.cpp.o' failed
    

    Steps to Reproduce the Problem

    Run this on a unix box with a recent version of either linux or gcc installed:

    git clone https://github.com/claremacrae/ApprovalTests.cpp.CMakeSamples
    cd ApprovalTests.cpp.CMakeSamples
    git clone https://github.com/approvals/ApprovalTests.cpp.git
    cd dev_approvals_fetch_content
    ./build.sh
    

    Wait until all the dependencies have been cloned, and then one of the errors will occur - depending on whether you are using gcc or clang.

    Specifications

    The failure was detected in a GitHub Actions build. The failing builds can be seen here:

    https://github.com/claremacrae/ApprovalTests.cpp.CMakeSamples/actions/runs/604457152

    • Linux Clang 6, 8, 9, and 10
    • Linux gcc 9 and 10
    • macOS Xcode 10.3, 11.7 and 12.

    It's likely that the ones that passed are skipping the building of boost.ut

    opened by claremacrae 13
  • Conan Center

    Conan Center

    What do you think about making a [Boost-ext].UT package available in ConanCenter? I would like to able to handle the dependency through Conan, and having it available in ConanCenter seems the easiest way to do so.

    Their instructions for doing so are available here. I found an example recipe for a header-only library on ConanCenter here.

    opened by jwillikers 12
  • Approval Test Integration Bug

    Approval Test Integration Bug

    Hi,

    I really like the [Boost].UT framework. It's clean and easy to use and does away with macros. Thanks for developing it!

    Anyways, I want to use approval tests (https://github.com/approvals/ApprovalTests.cpp) in a project I'm working on, and I'm implementing the integration between UT <> ApprovalTests.cpp. I followed the guidelines (https://github.com/approvals/ApprovalTests.cpp/blob/master/doc/SupportingNewTestFramework.md#top) and the UT reporter example (https://godbolt.org/z/gsAPKg). I made this simple example:

    #include <string>
    
    #include "ut.hpp"
    #include "ApprovalTests.hpp"
    
    namespace ut = boost::ut;
    
    namespace ApprovalTests
    {
        namespace cfg {
            class reporter : public ut::reporter<ut::printer> {
                private:
                    TestName currentTest;
    
                public:
                    auto on(ut::events::test_begin test_begin) -> void {
                        std::string name = test_begin.name;
                        currentTest.sections.emplace_back(name);
                        ut::reporter<ut::printer>::on(test_begin);
                    }
                    auto on(ut::events::test_run test_run) -> void { 
                        ut::reporter<ut::printer>::on(test_run);
                    }
                    auto on(ut::events::test_skip test_skip) -> void { ut::reporter<ut::printer>::on(test_skip); }
                    auto on(ut::events::test_end test_end) -> void {
                        while (!currentTest.sections.empty()) {
                            currentTest.sections.pop_back();
                        }
                        ut::reporter<ut::printer>::on(test_end);
                    }
                    template <class TMsg>
                    auto on(ut::events::log<TMsg>) -> void {}
                    template <class TLocation, class TExpr>
                    auto on(ut::events::assertion_pass<TLocation, TExpr> location) -> void {
                        currentTest.setFileName(location.location.file_name());
                        ApprovalTestNamer::currentTest(&currentTest);
                        ut::reporter<ut::printer>::on(location);
                    }
                    template <class TLocation, class TExpr>
                    auto on(ut::events::assertion_fail<TLocation, TExpr> fail) -> void { ut::reporter<ut::printer>::on(fail); }
                    auto on(ut::events::fatal_assertion fatal) -> void { ut::reporter<ut::printer>::on(fatal); }
                    auto on(ut::events::exception exception) -> void { ut::reporter<ut::printer>::on(exception); }
                    auto on(ut::events::summary summary) -> void { ut::reporter<ut::printer>::on(summary); }
            };
        }  // namespace cfg
    }
    
    template <>
    auto ut::cfg<ut::override> = ut::runner<ApprovalTests::cfg::reporter>{};
    
    int main()
    {
        using namespace ut;
        using namespace ApprovalTests;
    
    	"Approval"_test = []() {
            //expect(true);
            expect(nothrow([] { Approvals::verify("Approval Tests can verify text via the golder master method"); }));
    	};
    
    }
    

    The issue is unless the line containing expect(true); is uncommented it doesn't work. It complains that currentTest was not configured. This seems more like a UT issue than an ApprovalTests issue.

    Can anyone help? I'm not familiar enough with the UT code logic to track this issue. Or am I wrong and this is an ApprovalTests.cpp issue?

    I appreciate any help.

    opened by lp55 11
  • fix cmake install config package

    fix cmake install config package

    use CPM.cmake to install the project cleanup cmake file format cmake file fix compile errors fix some clang-tidy warnings add some Notes to source

    Problem:

    • dit not compile on OSX with gcc-10, clang-11, AppleClangV12

    Solution:

    • use target_link_library()
    • and change some compiler flags
    • correct source code

    Issue: this fix https://github.com/boost-ext/ut/issues/420 too

    Reviewers: @

    opened by ClausKlein 9
  • :bug: :art: [exception] Tests that throw unexpected exceptions appear…

    :bug: :art: [exception] Tests that throw unexpected exceptions appear…

    … to pass, by returning 0 exit status, Fix #299

    Problem:

    • fails counter is not properly incremented in the case when an exception is thrown.
    • There are unused boolean exception variables.

    Solution:

    • Update fails count in exception handling.
    • Remove them.
    opened by krzysztof-jusiak 8
  • Improve readability and glancability of docs in README.txt

    Improve readability and glancability of docs in README.txt

    Expected Behavior

    That it's easy to search text, and see the whole structure, of the docs on the front page:

    https://github.com/boost-experimental/ut

    Actual Behavior

    • Most sections on the front page are collapsed, and some require two levels of expansion of triangles to see the content
    • Searching in the browser (Ctrl+F) doesn't find text in collapsed sections
    • I've seen someone who is more familiar with UT than me still click on 2 or 3 triangles before finding the code example they wanted to show me

    Specifications

    • Testing in both Firefox and Chrome (Windows 10)

    Suggestions

    • Perhaps provide a button at the top that opens all the triangles all the way down
    • Or just remove the triangles and collapsing altogether
    opened by claremacrae 8
  • Conan version 1.1.8 is broken on newer msvc.

    Conan version 1.1.8 is broken on newer msvc.

    I don't think Conan will push out a version with out a new numbered release. Since 1.1.8 is from Aug 2020. I was wondering if you could push out a new build using the current version of the code. So that Conan could release it.

    We needed a way to turn Modules off. And it only exists in the current code it doesn't exist in the current released build.

    opened by Sebanisu 7
  • Build errors on Visual Studio with clang-cl compiler when using boost.ut's CMakeLists.txt

    Build errors on Visual Studio with clang-cl compiler when using boost.ut's CMakeLists.txt

    Expected Behavior

    Boost.ut builds and tests pass with VS2019 MSVC with clang-cl compiler

    I know this, as we have been building that with the Approval Tests integration, which works file with this compiler.

    Actual Behavior

    Builds fail, with the error:

      Building Custom Rule D:/a/ApprovalTests.cpp.Builds/ApprovalTests.cpp.Builds/_deps/ut-src/example/CMakeLists.txt
    clang-cl : error : unknown argument ignored in clang-cl: '-pedantic' [-Werror,-Wunknown-argument] [D:\a\ApprovalTests.cpp.Builds\ApprovalTests.cpp.Builds\_deps\ut-build\example\BDD.vcxproj]
    clang-cl : error : unknown argument ignored in clang-cl: '-pedantic-errors' [-Werror,-Wunknown-argument] [D:\a\ApprovalTests.cpp.Builds\ApprovalTests.cpp.Builds\_deps\ut-build\example\BDD.vcxproj]
    

    For an example failure, see https://github.com/claremacrae/ApprovalTests.cpp.Builds/runs/370748554

    Steps to Reproduce the Problem

    1. cmake -G "Visual Studio 16 2019" -T "clangcl"
    2. build all of Boost.ut, including examples

    Specifications

    • Version: 8004290d195b08eda531eb015ddcda2e11d41ce9
    • Platform: 'The CXX compiler identification is Clang 9.0.0 with MSVC-like command-line'
    • Subsystem: ?

    Nature of fix

    See https://github.com/approvals/ApprovalTests.cpp/blob/v.7.0.0/tests/UT_Tests/CMakeLists.txt#L17 for the type of check to use...

    Someone previously reported this problem with ApprovalTests.cpp on this environment, and provided a fix which used windows-style compiler warnings if using clang-cl...

    opened by claremacrae 7
  • Print type name in failed templated test

    Print type name in failed templated test

    It would be really nice if a failing template test prints the type name for which it failed.

    So for this code,

    "test"_test = []<class T>{
      expect(std::same_as<bool, T>);
    } | std::tuple<bool, int>{};
    

    ut should print something like

    Running "test<bool>"...PASSED
    Running "test<int>"...
      test.cpp:2:FAILED [false]
    FAILED
    
    opened by jessestricker 6
  • How to override the runner's MaxPathSize template argument?

    How to override the runner's MaxPathSize template argument?

    Sorry, this is more of a question than an issue.

    I don´t understand how I can override the MaxPathSize template argument to the default test runner while also being able to modify the filter value. The default path max of 16 is not a lot to make descriptive test names. I tried to combine the examples to the best of my ability, but always end up with some template-related errors.

    Can you help me out?

    Kind regards

    opened by SanderVocke 1
  • `expect()` does not work with non-copiable types

    `expect()` does not work with non-copiable types

    Expected Behavior

    #include <boost/ut.hpp>
    #include <memory>
    
    using namespace boost::ut;
    
    int main() {
      "test"_test = []() {
        std::unique_ptr<int> p;
        expect(that % p == nullptr);
      };
    }
    

    Should compile.

    Actual Behavior

    Does not compile:

    In file included from test.cpp:1:
    ../boost/ut.hpp: In instantiation of ‘constexpr auto boost::ext::ut::v1_1_8::expect(const TExpr&, const boost::ext::ut::v1_1_8::reflection::source_location&) [with TExpr = boost::ext::ut::v1_1_8::detail::eq_<std::unique_ptr<int>, std::nullptr_t>; typename boost::ext::ut::v1_1_8::type_traits::requires_<(is_op_v<TExpr> || is_convertible_v<TExpr, bool>)>::type <anonymous> = 0]’:
    test.cpp:9:31:   required from here
    ../boost/ut.hpp:2080:50: error: use of deleted function ‘boost::ext::ut::v1_1_8::detail::eq_<std::unique_ptr<int>, std::nullptr_t>::eq_(const boost::ext::ut::v1_1_8::detail::eq_<std::unique_ptr<int>, std::nullptr_t>&)’
     2080 |   return detail::expect_<TExpr>{detail::on<TExpr>(
          |                                 ~~~~~~~~~~~~~~~~~^
     2081 |       events::assertion<TExpr>{.expr = expr, .location = sl})};
          |       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    ../boost/ut.hpp:648:8: note: ‘boost::ext::ut::v1_1_8::detail::eq_<std::unique_ptr<int>, std::nullptr_t>::eq_(const boost::ext::ut::v1_1_8::detail::eq_<std::unique_ptr<int>, std::nullptr_t>&)’ is implicitly deleted because the default definition would be ill-formed:
      648 | struct eq_ : op {
          |        ^~~
    ../boost/ut.hpp:648:8: error: use of deleted function ‘std::unique_ptr<_Tp, _Dp>::unique_ptr(const std::unique_ptr<_Tp, _Dp>&) [with _Tp = int; _Dp = std::default_delete<int>]’
    In file included from /usr/include/c++/10/memory:83,
                     from test.cpp:2:
    /usr/include/c++/10/bits/unique_ptr.h:468:7: note: declared here
      468 |       unique_ptr(const unique_ptr&) = delete;
          |       ^~~~~~~~~~
    In file included from test.cpp:1:
    ../boost/ut.hpp:2081:62: error: no matching function for call to ‘boost::ext::ut::v1_1_8::detail::expect_<boost::ext::ut::v1_1_8::detail::eq_<std::unique_ptr<int>, std::nullptr_t> >::expect_(<brace-enclosed initializer list>)’
     2081 |       events::assertion<TExpr>{.expr = expr, .location = sl})};
          |                                                              ^
    ../boost/ut.hpp:1568:22: note: candidate: ‘constexpr boost::ext::ut::v1_1_8::detail::expect_<T>::expect_(bool) [with T = boost::ext::ut::v1_1_8::detail::eq_<std::unique_ptr<int>, std::nullptr_t>]’
     1568 |   constexpr explicit expect_(bool value) : value_{value} { cfg::wip = {}; }
          |                      ^~~~~~~
    ../boost/ut.hpp:1568:22: note:   conversion of argument 1 would be ill-formed:
    ../boost/ut.hpp:1567:8: note: candidate: ‘constexpr boost::ext::ut::v1_1_8::detail::expect_<boost::ext::ut::v1_1_8::detail::eq_<std::unique_ptr<int>, std::nullptr_t> >::expect_(const boost::ext::ut::v1_1_8::detail::expect_<boost::ext::ut::v1_1_8::detail::eq_<std::unique_ptr<int>, std::nullptr_t> >&)’
     1567 | struct expect_ {
          |        ^~~~~~~
    ../boost/ut.hpp:1567:8: note:   conversion of argument 1 would be ill-formed:
    ../boost/ut.hpp:1567:8: note: candidate: ‘constexpr boost::ext::ut::v1_1_8::detail::expect_<boost::ext::ut::v1_1_8::detail::eq_<std::unique_ptr<int>, std::nullptr_t> >::expect_(boost::ext::ut::v1_1_8::detail::expect_<boost::ext::ut::v1_1_8::detail::eq_<std::unique_ptr<int>, std::nullptr_t> >&&)’
    ../boost/ut.hpp:1567:8: note:   conversion of argument 1 would be ill-formed:
    ../boost/ut.hpp: In instantiation of ‘constexpr boost::ext::ut::v1_1_8::detail::that_::expr<T>::expr(const T&) [with T = std::unique_ptr<int>]’:
    ../boost/ut.hpp:1540:18:   required from ‘constexpr auto boost::ext::ut::v1_1_8::detail::that_::operator%(const T&) const [with T = std::unique_ptr<int>]’
    test.cpp:9:19:   required from here
    ../boost/ut.hpp:1497:47: error: use of deleted function ‘std::unique_ptr<_Tp, _Dp>::unique_ptr(const std::unique_ptr<_Tp, _Dp>&) [with _Tp = int; _Dp = std::default_delete<int>]’
     1497 |     constexpr explicit expr(const T& t) : t_{t} {}
          |                                               ^
    In file included from /usr/include/c++/10/memory:83,
                     from test.cpp:2:
    /usr/include/c++/10/bits/unique_ptr.h:468:7: note: declared here
      468 |       unique_ptr(const unique_ptr&) = delete;
          |       ^~~~~~~~~~
    In file included from test.cpp:1:
    ../boost/ut.hpp: In instantiation of ‘constexpr boost::ext::ut::v1_1_8::detail::eq_<TLhs, TRhs>::eq_(const TLhs&, const TRhs&) [with TLhs = std::unique_ptr<int>; TRhs = std::nullptr_t]’:
    ../boost/ut.hpp:1503:25:   required from ‘constexpr auto boost::ext::ut::v1_1_8::detail::that_::expr<T>::operator==(const TRhs&) const [with TRhs = std::nullptr_t; T = std::unique_ptr<int>]’
    test.cpp:9:24:   required from here
    ../boost/ut.hpp:668:12: error: use of deleted function ‘std::unique_ptr<_Tp, _Dp>::unique_ptr(const std::unique_ptr<_Tp, _Dp>&) [with _Tp = int; _Dp = std::default_delete<int>]’
      668 |         }()} {}
          |            ^
    In file included from /usr/include/c++/10/memory:83,
                     from test.cpp:2:
    /usr/include/c++/10/bits/unique_ptr.h:468:7: note: declared here
      468 |       unique_ptr(const unique_ptr&) = delete;
          |       ^~~~~~~~~~
    

    Steps to Reproduce the Problem

    1. Attempt to compile the code above

    Specifications

    • Version: 1.1.9
    • Platform: Linux x86_64
    • Subsystem: g++ v10.3.0
    opened by cschreib 0
  • Parameterized tests require creating (unused) instances of the tested types.

    Parameterized tests require creating (unused) instances of the tested types.

    Expected Behavior

    #include <boost/ut.hpp>
    #include <array>
    #include <tuple>
    
    using namespace boost::ut;
    
    struct my_expensive_type {
      std::array<int, 1'000'000> data;
    };
    
    int main() {
      "test"_test = []<typename T>() { expect(that % sizeof(T) > 0); } |
                    std::tuple<my_expensive_type>{};
    }
    

    This should compile and run, perhaps with a change to the API (i.e., not instantiate tuple).

    Actual Behavior

    This compiles, but generates a segfault at runtime (stack exhausted).

    Steps to Reproduce the Problem

    1. Compile and run the test above.

    Specifications

    • Version: 1.1.9
    • Platform: Linux x86_64
    • Subsystem: g++ v10.3.0

    Extra information

    The example above is contrived, but the issue is real. This test should not create any instance of the tested type, since no instance is actually used. Actual instances may be created inside the test body, but this should be left up to the user to decide. Here the type cannot be created on the stack, hence cannot be used in parameterized tests.

    opened by cschreib 0
  • Parameterized tests do not compile for non-copiable types

    Parameterized tests do not compile for non-copiable types

    Expected Behavior

    #include <boost/ut.hpp>
    #include <memory>
    #include <tuple>
    
    using namespace boost::ut;
    
    int main() {
      "test"_test = []<typename T>() {
        expect(that % sizeof(T) == sizeof(typename T::element_type*));
      } | std::tuple<std::unique_ptr<int>>{};
    }
    

    Should compile.

    Actual Behavior

    Does not compile:

    In file included from test.cpp:1:
    ../boost/ut.hpp: In instantiation of ‘constexpr auto boost::ext::ut::v1_1_8::operators::operator|(const F&, const T<Ts ...>&) [with F = main()::<lambda()>; T = std::tuple; Ts = {std::unique_ptr<int, std::default_delete<int> >}; typename boost::ext::ut::v1_1_8::type_traits::requires_<(! is_container_v<T<Ts ...> >)>::type <anonymous> = 0]’:
    test.cpp:10:40:   required from here
    ../boost/ut.hpp:1850:10: error: use of deleted function ‘constexpr std::tuple<_Elements>::tuple(const std::tuple<_Elements>&) [with _Elements = {std::unique_ptr<int, std::default_delete<int> >}]’
     1850 |   return [f, t](const auto name) {
          |          ^~~~~~~~~~~~~~~~~~~~~~~~~
     1851 |     apply(
          |     ~~~~~~
     1852 |         [f, name](const auto&... args) {
          |         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     1853 |           (detail::on<F>(events::test<F, Ts>{.type = "test",
          |           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     1854 |                                              .name = name,
          |                                              ~~~~~~~~~~~~~
     1855 |                                              .tag = {},
          |                                              ~~~~~~~~~~
     1856 |                                              .location = {},
          |                                              ~~~~~~~~~~~~~~~
     1857 |                                              .arg = args,
          |                                              ~~~~~~~~~~~~
     1858 |                                              .run = f}),
          |                                              ~~~~~~~~~~~
     1859 |            ...);
          |            ~~~~~
     1860 |         },
          |         ~~
     1861 |         t);
          |         ~~~
     1862 |   };
          |   ~       
    In file included from /usr/include/c++/10/ranges:46,
                     from /usr/include/c++/10/bits/ranges_algobase.h:38,
                     from /usr/include/c++/10/bits/ranges_uninitialized.h:36,
                     from /usr/include/c++/10/memory:69,
                     from test.cpp:2:
    /usr/include/c++/10/tuple:665:17: note: ‘constexpr std::tuple<_Elements>::tuple(const std::tuple<_Elements>&) [with _Elements = {std::unique_ptr<int, std::default_delete<int> >}]’ is implicitly deleted because the default definition would be ill-formed:
      665 |       constexpr tuple(const tuple&) = default;
          |                 ^~~~~
    /usr/include/c++/10/tuple:665:17: error: use of deleted function ‘constexpr std::_Tuple_impl<_Idx, _Head>::_Tuple_impl(const std::_Tuple_impl<_Idx, _Head>&) [with long unsigned int _Idx = 0; _Head = std::unique_ptr<int>]’
    /usr/include/c++/10/tuple:366:17: note: ‘constexpr std::_Tuple_impl<_Idx, _Head>::_Tuple_impl(const std::_Tuple_impl<_Idx, _Head>&) [with long unsigned int _Idx = 0; _Head = std::unique_ptr<int>]’ is implicitly deleted because the default definition would be ill-formed:
      366 |       constexpr _Tuple_impl(const _Tuple_impl&) = default;
          |                 ^~~~~~~~~~~
    /usr/include/c++/10/tuple:366:17: error: use of deleted function ‘constexpr std::_Head_base<_Idx, _Head, false>::_Head_base(const std::_Head_base<_Idx, _Head, false>&) [with long unsigned int _Idx = 0; _Head = std::unique_ptr<int>]’
    /usr/include/c++/10/tuple:132:17: note: ‘constexpr std::_Head_base<_Idx, _Head, false>::_Head_base(const std::_Head_base<_Idx, _Head, false>&) [with long unsigned int _Idx = 0; _Head = std::unique_ptr<int>]’ is implicitly deleted because the default definition would be ill-formed:
      132 |       constexpr _Head_base(const _Head_base&) = default;
          |                 ^~~~~~~~~~
    /usr/include/c++/10/tuple:132:17: error: use of deleted function ‘std::unique_ptr<_Tp, _Dp>::unique_ptr(const std::unique_ptr<_Tp, _Dp>&) [with _Tp = int; _Dp = std::default_delete<int>]’
    In file included from /usr/include/c++/10/memory:83,
                     from test.cpp:2:
    /usr/include/c++/10/bits/unique_ptr.h:468:7: note: declared here
      468 |       unique_ptr(const unique_ptr&) = delete;
          |       ^~~~~~~~~~
    

    Steps to Reproduce the Problem

    1. Attempt to compile the code above

    Specifications

    • Version: 1.1.9
    • Platform: Linux x86_64
    • Subsystem: g++ v10.3.0

    Extra information

    The current syntax requires instantiating a std::tuple, which creates instances of each of the listed types, which are then copied around by the Boost UT internals. The copies can easily avoided. The instances, not so much (not without changing your API, see https://github.com/boost-ext/ut/issues/539), even though they are not actually necessary.

    opened by cschreib 0
  •  declaration of 'something' with internal linkage cannot be exported

    declaration of 'something' with internal linkage cannot be exported

    Expected Behavior

    ut being precompiled as a module

    Actual Behavior

    header isn't being precompiled as a prebuilt module unit

    ./zero/tests/deps/ut.hpp:379:23: error: declaration of 'is_container_v' with internal linkage cannot be exported
    static constexpr auto is_container_v =
                          ^
    ./zero/tests/deps/ut.hpp:383:23: error: declaration of 'has_user_print' with internal linkage cannot be exported
    static constexpr auto has_user_print = is_valid<T>(
                          ^
    ./zero/tests/deps/ut.hpp:423:23: error: declaration of 'is_convertible_v' with internal linkage cannot be exported
    static constexpr auto is_convertible_v = __is_convertible_to(From, To);
                          ^
    ./zero/tests/deps/ut.hpp:383:23: error: declaration of 'has_user_print<char>' with internal linkage cannot be exported
    static constexpr auto has_user_print = is_valid<T>(
                          ^
    ./zero/tests/deps/ut.hpp:379:23: error: declaration of 'is_container_v<char>' with internal linkage cannot be exported
    static constexpr auto is_container_v =
                          ^
    ./zero/tests/deps/ut.hpp:383:23: error: declaration of 'has_user_print<const char *>' with internal linkage cannot be exported
    static constexpr auto has_user_print = is_valid<T>(
                          ^
    ./zero/tests/deps/ut.hpp:379:23: error: declaration of 'is_container_v<const char *>' with internal linkage cannot be exported
    static constexpr auto is_container_v =
                          ^
    ./zero/tests/deps/ut.hpp:379:23: error: declaration of 'is_container_v<std::__wrap_iter<const std::string_view *> &>' with internal linkage cannot be exported
    

    Steps to Reproduce the Problem

    1. clang++ -stdlib=libc++ -std=c++20 -D__cpp_modules -fmodules --precompile -x c++-module ./zero/tests/deps/ut.hpp -o ./zero/tests/deps/ut.pcm

    Specifications

    • Version: 1.1.9
    • Platform: Linux
    • Subsystem: Arch/Manjaro
    • Compiler version: Clang 15.0.2 - gnu-linux

    EDIT

    Also, using new bash profiles on wandbox produces another kind of errors. Permalink on the doc it's ok, but newer versions contains clang incompatiblities with ut

    https://wandbox.org/permlink/NZyBkZddbKTEktGV

    opened by Pyzyryab 0
Releases(v1.1.9)
  • v1.1.8(Aug 2, 2020)

    [Boost::ext].μt - C++ single header/module, macro-free μ(micro)/Unit Testing Framework

    • Additions

      • Support for tagging tests
        • https://github.com/boost-ext/ut/commit/c6c6975119a07dda6bca4fca436fa1a3ae1b6afd
      • Support for GCC-10/Clang-10/XCode-11.6
        • https://github.com/boost-ext/ut/commit/16991dbe06ec23280fe763828046ea3a784b6134
      • Support for https://conan.io
        • https://github.com/boost-ext/ut/commit/8a16f76c9af8dc3d8c9b332e14d8815d1d09471e
      • Support for https://mesonbuild.com
        • https://github.com/boost-ext/ut/commit/e87c120f6080e250935325b9d382c51a82bc42d0
      • CMake improvements
        • https://github.com/boost-ext/ut/commit/7f4a91eb841a5d897017185100d2af035a40a261
    • Examples

      • Gherkin example
        • https://github.com/boost-ext/ut/commit/95902253bc938ed0fdb5b1f222db757b232efeb6
      • Micro-benchmarking
        • https://github.com/boost-ext/ut/commit/d04cc3f81cd561e54e52a5dd9e988ad4430a43c8
    • Breaking Changes

      • Fatal assertions with expressions (!expect is deprecated)
        • https://github.com/boost-ext/ut/commit/ce21fa39dd05fa0cdddbedb896691545e8758f0c
      • Use Boost::ext.UT instead of [Boost].UT
        • https://github.com/boost-ext/ut/commit/d58714991cded6153aef4ee1f9b79989f497bc8a
      • true_b and false_b have been removed (instead named ""_b should be used)
        • https://github.com/boost-ext/ut/commit/3ccb45986d9334428bdb521fa2ceaa95ac914ec3
    • Bug Fixes

      • https://github.com/boost-experimental/ut/issues?utf8=%E2%9C%93&q=is%3Aissue+is%3Aclosed+closed%3A2020-03-15..2020-08-02
    • Contributors

    Source code(tar.gz)
    Source code(zip)
  • v1.1.7(Mar 15, 2020)

    [Boost].μt - C++ single header/module, macro-free μ(micro)/Unit Testing Framework

    • Examples

      • terse
        • https://github.com/boost-experimental/ut/commit/06a1cf951b8454b7deeca4ff55c92c5bdc483056
      • spec
        • https://github.com/boost-experimental/ut/commit/72e4bf8d737a37277e3cbe46c8db5f8fb9e5fa13
      • ""_b named boolean expression
        • https://github.com/boost-experimental/ut/commit/5cc7950dfc54da90b06bb0bae601e17740c07574
    • Breaking Changes

      • BDD requires a namespace - ut::bdd * https://github.com/boost-experimental/ut/commit/cb8a12a0b2f1b40cd794bec95c4e3ea4ad5bfcbc
    • Bug Fixes

      • https://github.com/boost-experimental/ut/issues?utf8=%E2%9C%93&q=is%3Aissue+is%3Aclosed+closed%3A2020-01-12..2020-03-15
    • Contributors

    Source code(tar.gz)
    Source code(zip)
  • v1.1.6(Jan 12, 2020)

    [Boost].μt - C++ single header, macro-free μ(micro)/Unit Testing Framework

    • Additions

      • clang-cl support

        • https://github.com/boost-experimental/ut/blob/master/.appveyor.yml#L20
      • apple-clang and osx support

        • https://github.com/boost-experimental/ut/blob/master/.travis.yml#L64
      • Gitter chat

        • https://gitter.im/boost-experimental/ut?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge
      • Examples

        • parallel runner
          • https://github.com/boost-experimental/ut/commit/79539521c0e6bdce7b3159b9897e792509a18078
        • any_of matcher
          • https://github.com/boost-experimental/ut/commit/9c6194516a65977dd5c224d7549e3b33977008f5
        • Parameterized tests with language syntax
          • https://github.com/boost-experimental/ut/commit/429db0aecbccbe8fcd298c52ceecce00a5c899b2
      • CMake modernization and improvements

    • Breaking Changes

      • Removal of ut::matcher * https://github.com/boost-experimental/ut/commit/cb8a12a0b2f1b40cd794bec95c4e3ea4ad5bfcbc
    • Bug Fixes

      • https://github.com/boost-experimental/ut/issues?utf8=%E2%9C%93&q=is%3Aissue+is%3Aclosed+closed%3A2019-12-23..2020-01-12
    • Contributors

    Source code(tar.gz)
    Source code(zip)
  • v1.1.5(Dec 23, 2019)

    [Boost].μt - C++ single header, macro-free μ(micro)/Unit Testing Framework

    • Additions

      • Integrations
      • Source location with ""_test
        • https://github.com/boost-experimental/ut/commit/eb948358187fd164825141380dd545d4831902d0
      • Apply C++20 designated initializers
        • https://github.com/boost-experimental/ut/commit/9156b463ef6da76f2ad3d94a8fc8c87a154dc8a4
      • [Benchmarks] Incremental build
        • https://github.com/boost-experimental/ut/commit/2f08258a9f14e4a1f708f39452db9fafa8a14a43
    • Breaking Changes

      • Removal of TLocation template parameter in runner/reporter
        • https://github.com/boost-experimental/ut/commit/94220e8c95b323349bfd94ef30b2568916fb1421
    • Bug Fixes

      • https://github.com/boost-experimental/ut/issues?utf8=%E2%9C%93&q=is%3Aissue+is%3Aclosed+closed%3A2019-12-06..2019-12-23
    Source code(tar.gz)
    Source code(zip)
  • v1.1.4(Dec 6, 2019)

    [Boost].μt - C++ single header, macro-free μ(micro)/Unit Testing Framework

    • Additions

      • Quick Start
        • https://bit.ly/boost-ut-quick-start
      • Motivation
        • https://github.com/boost-experimental/ut/commit/ebbb4ae5b6a21a3d6dce0e3bfc59b351b2033b1f
      • should - for sections
        • https://github.com/boost-experimental/ut/commit/c2f93a9edbca0187f30e02352ba9c47876d82479
      • mut - for sections (immutiblity by default)
        • https://github.com/boost-experimental/ut/commit/c9d58dea9a11fc0c5b4cf1de45556ca93c85626b
      • Example - Template Meta-Programming (TMP)
        • https://github.com/boost-experimental/ut/commit/79d6b675b97f73c9868d57ea263e463fb4c9f598
    • Bug Fixes

      • https://github.com/boost-experimental/ut/issues?utf8=%E2%9C%93&q=is%3Aissue+is%3Aclosed+closed%3A2019-11-30..2019-12-06
    Source code(tar.gz)
    Source code(zip)
  • v1.1.3(Nov 30, 2019)

    [Boost].μt - C++ single header, macro-free μ(micro)/Unit Testing Framework

    • Additions

      • Colors in compound expressions
        • https://github.com/boost-experimental/ut/commit/4e603de6a85b9be54d13598c0a658fd22b64b510
      • Fatal assertions on per test case bases
        • https://github.com/boost-experimental/ut/commit/7cf2430036fedeb209e8d7a436ca67df6baaa173
      • Ability to override/disable default colors
        • https://github.com/boost-experimental/ut/commit/4384ded95a871340aa6a7736f866254971a6b4fe
      • Overview
        • https://bit.ly/boost-ut-docs
    • Bug Fixes

      • https://github.com/boost-experimental/ut/issues?utf8=%E2%9C%93&q=is%3Aissue+is%3Aclosed+closed%3A2019-11-27..2019-11-30
    Source code(tar.gz)
    Source code(zip)
  • v1.1.2(Nov 27, 2019)

    [Boost].μt - C++ single header, macro-free μ(micro)/Unit Testing Framework

    "If you liked it then you "should have put a"_test on it", Beyonce rule

    • Additions

      • C++17 support
        • https://github.com/boost-experimental/ut/commit/6b0a685d33cd2070934fdeb5688d62d9f714ad9d
      • Alternative test("name") = [] {}; syntax
        • https://github.com/boost-experimental/ut/commit/76d11db4d09ad1eb595f8d003470a5d28d08d039
      • README
        • Tutorial
          • https://github.com/boost-experimental/ut/tree/master#tutorial
        • FAQ
          • https://github.com/boost-experimental/ut/tree/master#faq
      • Overview
        • https://bit.ly/boost-ut-docs
      • Benchmarks / Boost.Test-1.71.0
        • https://github.com/boost-experimental/ut#benchmarks
    • Bug Fixes

      • https://github.com/boost-experimental/ut/issues?utf8=%E2%9C%93&q=is%3Aissue+is%3Aclosed+closed%3A2019-11-20..2019-11-27
    Source code(tar.gz)
    Source code(zip)
  • v1.1.1(Nov 21, 2019)

    [Boost].μt - C++20 single header, macro-free μ(micro)/Unit Testing Framework


    • Additions

      • Updated benchmarks
        • https://github.com/cpp-testing/ut-benchmark
      • Alternative assertion syntax
        • that %
          • expect(that % 1 == 2) https://godbolt.org/z/Df2nrN
        • eq/ne/lt/le/gt/ge
          • expect(eq(1, 2)) -> https://godbolt.org/z/Df2nrN
      • README
        • How it works?
          • https://github.com/boost-experimental/ut/tree/master#how-it-works
    • Bug Fixes

      • https://github.com/boost-experimental/ut/issues?utf8=%E2%9C%93&q=is%3Aissue+is%3Aclosed+closed%3A2019-11-14..2019-11-20
    Source code(tar.gz)
    Source code(zip)
  • v1.1.0(Nov 14, 2019)

    [Boost].μt - C++20 single header, macro-free μ(micro)/Unit Testing Framework


    • Additions

      • Updated benchmarks
        • https://github.com/cpp-testing/ut-benchmark
      • Support for -fno-exceptions
        • https://github.com/boost-experimental/ut/commit/facfe9785d401126d5de9f0f1b7c93f4cdf1cc0c
      • BOOST_UT_VERSION
        • https://github.com/boost-experimental/ut/commit/9f849e606f99c26ed3bbf2a42e083f3ffea0ed56
      • Faster compilation times of multiple file test suites with BOOST_UT_INTERFACE/BOOST_UT_IMPLEMENTATION
        • https://github.com/boost-experimental/ut/commit/c52160fcd0b515f21f092be07777931124f0b230
    • Bug Fixes

      • https://github.com/boost-experimental/ut/issues?utf8=%E2%9C%93&q=is%3Aissue+is%3Aclosed+closed%3A2019-11-03..2019-11-13
    Source code(tar.gz)
    Source code(zip)
  • v1.0.1(Nov 3, 2019)

    [Boost].μt - C++20 single header, macro-free μ(micro)/Unit Testing Framework


    • Additions

      • Support for C++20 modules
        • https://github.com/boost-experimental/ut/blob/master/include/boost/ut.mpp
        • Examples
          • https://wandbox.org/permlink/yMnlEUeRHjQePSPw
      • Benchmarks
        • https://github.com/cpp-testing/ut-benchmark
    • Improvements

      • Support for colors in the output
        • https://github.com/boost-experimental/ut/commit/ce5b29381983792177536874f01d68ca3a5cd860
      • Faster compilation times of _test
        • https://github.com/boost-experimental/ut/commit/9b7ec0312802e119f77d71c558e194fd8f97a540
    • Bug Fixes

      • https://github.com/boost-experimental/ut/issues?utf8=%E2%9C%93&q=is%3Aissue+is%3Aclosed+closed%3A2019-10-25..2019-11-03
    • Contributions

      • https://github.com/rianquinn
    Source code(tar.gz)
    Source code(zip)
  • v1.0.0(Oct 26, 2019)

    [Boost].μt - C++20 single header, macro-free μ(micro)/unit test framework

    • Additions
      • Support for Clang-9.0
      • Support for GCC-9
      • Support for MSVC-2019
    Source code(tar.gz)
    Source code(zip)
Owner
boost::ext
Modern C++ libraries (not officially in Boost)
boost::ext
A unit testing framework for C

Check Table of Contents About Installing Linking Packaging About Check is a unit testing framework for C. It features a simple interface for defining

null 923 Nov 27, 2022
CppUTest unit testing and mocking framework for C/C++

CppUTest CppUTest unit testing and mocking framework for C/C++ More information on the project page Slack channel: Join if link not expired Getting St

CppUTest 1.1k Nov 26, 2022
Minimal unit testing framework for C

MinUnit Minunit is a minimal unit testing framework for C/C++ self-contained in a single header file. It provides a way to define and configure test s

David Siñuela Pastor 451 Dec 5, 2022
A lightweight unit testing framework for C++

Maintenance of UnitTest++, recently sporadic, is officially on hiatus until 26 November 2020. Subscribe to https://github.com/unittest-cpp/unittest-cp

UnitTest++ 509 Nov 24, 2022
🧪 single header unit testing framework for C and C++

?? utest.h A simple one header solution to unit testing for C/C++. Usage Just #include "utest.h" in your code! The current supported platforms are Lin

Neil Henning 544 Dec 1, 2022
Simple Unit Testing for C

Unity Test Copyright (c) 2007 - 2021 Unity Project by Mike Karlesky, Mark VanderVoord, and Greg Williams Welcome to the Unity Test Project, one of the

Throw The Switch 2.8k Dec 4, 2022
A C++ micro-benchmarking framework

Nonius What is nonius? Nonius is an open-source framework for benchmarking small snippets of C++ code. It is very heavily inspired by Criterion, a sim

Nonius 338 Nov 28, 2022
A modern, C++-native, header-only, test framework for unit-tests, TDD and BDD - using C++11, C++14, C++17 and later (or C++03 on the Catch1.x branch)

Catch2 v3 is being developed! You are on the devel branch, where the next major version, v3, of Catch2 is being developed. As it is a significant rewo

Catch Org 15.9k Nov 25, 2022
A modern, C++-native, header-only, test framework for unit-tests, TDD and BDD - using C++11, C++14, C++17 and later (or C++03 on the Catch1.x branch)

Catch2 v3 is being developed! You are on the devel branch, where the next major version, v3, of Catch2 is being developed. As it is a significant rewo

Catch Org 15.9k Dec 4, 2022
The fastest feature-rich C++11/14/17/20 single-header testing framework

master branch Windows All dev branch Windows All doctest is a new C++ testing framework but is by far the fastest both in compile times (by orders of

Viktor Kirilov 4.4k Dec 5, 2022
Googletest - Google Testing and Mocking Framework

GoogleTest OSS Builds Status Announcements Release 1.10.x Release 1.10.x is now available. Coming Soon Post 1.10.x googletest will follow Abseil Live

Google 28.5k Dec 5, 2022
C++ Benchmark Authoring Library/Framework

Celero C++ Benchmarking Library Copyright 2017-2019 John Farrier Apache 2.0 License Community Support A Special Thanks to the following corporations f

John Farrier 723 Nov 24, 2022
test framework

Photesthesis This is a small, experimental parameterized-testing tool. It is intended to be used in concert with another unit-testing framework (eg. C

Graydon Hoare 11 Jun 2, 2021
A simple framework for compile-time benchmarks

Metabench A simple framework for compile-time microbenchmarks Overview Metabench is a single, self-contained CMake module making it easy to create com

Louis Dionne 162 Oct 23, 2022
The C Unit Testing Library on GitHub is a library designed for easy unit testing in C

The C Unit Testing Library on GitHub is a library designed for easy unit testing in C. It was written by Brennan Hurst for the purpose of providing a J-Unit-like testing framework within C for personal projects.

null 1 Oct 11, 2021
UT: C++20 μ(micro)/Unit Testing Framework

"If you liked it then you "should have put a"_test on it", Beyonce rule [Boost::ext].UT / μt | Motivation | Quick Start | Overview | Tutorial | Exampl

boost::ext 929 Nov 26, 2022
UT: C++20 μ(micro)/Unit Testing Framework

"If you liked it then you "should have put a"_test on it", Beyonce rule UT / μt | Motivation | Quick Start | Overview | Tutorial | Examples | User Gui

boost::ext 931 Dec 3, 2022
A micro unit-testing library for C/C++

µ-test A micro unit testing framework for C/C++ to get you up and running with unit-testing ASAP (even without libc). Usage Simply include the C and h

Trevor McKay 1 Dec 8, 2021
Tau - A Micro (1k lines of code) Unit Test Framework for C/C++

Tau τ A Micro Unit Testing Framework for >C11/C++11 projects, with the promise of always being tiny - about 1k lines of code. This framework is a much

Jason Dsouza 109 Dec 3, 2022
A testing micro framework for creating function test doubles

Fake Function Framework (fff) A Fake Function Framework for C Hello Fake World! Capturing Arguments Return Values Resetting a Fake Call History Defaul

Mike Long 540 Dec 1, 2022