Asynchronous Low Latency C++ Logging Library

Odysseas Georgoudis

Last update: Dec 20, 2022

Overview

Quill

Asynchronous Low Latency C++ Logging Library

Introduction
Features
Performance
Supported Platforms And Compilers
Basic Usage
CMake Integration
Documentation
License

homebrew	vcpkg	conan
`brew install quill`	`vcpkg install quill`	`quill/[>=1.2.3]`

Introduction

Quill is a cross-platform low latency logging library based on C++14.

The main goals of the library are:

Simplicity A small example code snippet should be enough to get started and use most of features.
Performance Ultra low latency. No formatting on the hot-path, asynchronous only mode. No hot-path allocations for fundamental types, enums and strings (including std::string and std::string_view). Any other custom or user defined type gets copy constructed with the formatting done on a backend worker thread.
Convenience Ease application monitoring/debugging. Latency is equal to latencies of binary loggers, but the produced log is in human readable form.

Features

Log anything - Blazing fast. See Benchmarks.
Format outside of the hot-path in a backend logging thread. For non-built-in types ostream::operator<<() is called on a copy of the object by the backend logging thread. Unsafe to copy non-trivial user defined are detected in compile time. Those types can be tagged as safe-to-copy to avoid formatting them on the hot path. See User Defined Types.
Custom formatters. Logs can be formatted based on a user specified pattern. See Formatters.
Support for log stack traces. Store log messages in a ring buffer and display later on a higher severity log statement or on demand. See Backtrace Logging.
Various logging targets. See Handlers.
- Console logging with colours support.
- File Logging
- Rotating log files
- Time rotating log files
- Custom Handlers
Filters for filtering log messages. See Filters.
guaranteed non-blocking or non-guaranteed logging. In non-guaranteed mode there is no heap allocation of a new queue but log messages can be dropped. See FAQ.
Support for wide character logging and wide character filenames (Windows only).
Log statements in timestamp order even when produced by different threads. This makes debugging easier in multi-threaded applications.
Log levels can be completely stripped out at compile time reducing if branches.
Clean warning-free codebase even on high warning levels.
Crash safe behaviour with a build-in signal handler.
Type safe python style API with compile type checks and built-in support for logging STL types/containers by using the excellent {fmt} library.

Performance

Log Numbers

The following message is logged 100'000 times per thread LOG_INFO(logger, "Logging int: {}, int: {}, double: {}", i, j, d).

1 Thread

Library	50th	75th	90th	95th	99th	99.9th	Worst
Quill, Dual Queue Enabled, Bounded Queue	20	22	25	27	34	67	118
Quill, Dual Queue Enabled, Unbounded Queue	20	24	28	30	36	69	134
Quill, Dual Queue Disabled, Unbounded Queue	23	31	35	37	52	75	134
PlatformLab NanoLog	19	22	23	25	28	64	128
MS BinLog	46	47	48	49	82	129	353
Reckless	69	70	74	80	106	133	5908477
Iyengar NanoLog	112	122	134	147	207	337	597293
spdlog	288	312	328	345	670	914	4794
g3log	2434	2886	2992	3055	3178	3338	5579

4 Threads

Library	50th	75th	90th	95th	99th	99.9th	Worst
Quill, Dual Queue Enabled, Bounded Queue	19	22	26	30	46	67	343
Quill, Dual Queue Enabled, Unbounded Queue	20	22	26	30	46	66	242
Quill, Dual Queue Disabled, Unbounded Queue	21	28	33	38	64	79	4207
PlatformLab NanoLog	18	21	23	24	42	58	223
MS BinLog	45	46	48	49	75	125	271
Reckless	93	117	125	129	163	358	30186464
Iyengar NanoLog	115	156	191	214	299	441	1142546
spdlog	335	537	714	796	1081	1534	27378
g3log	2834	2929	3032	3132	5081	5994	26563

Log Numbers and Large Strings

The following message is logged 100'000 times per thread LOG_INFO(logger, "Logging int: {}, int: {}, string: {}", i, j, large_string). The large string is over 35 characters to avoid short string optimisation of std::string

1 Thread

Library	50th	75th	90th	95th	99th	99.9th	Worst
Quill, Dual Queue Enabled, Bounded Queue	26	31	37	40	48	73	136
Quill, Dual Queue Enabled, Unbounded Queue	28	33	39	42	59	79	138
Quill, Dual Queue Disabled, Unbounded Queue	195	211	224	232	251	283	325
PlatformLab NanoLog	29	34	39	54	57	77	178
MS BinLog	51	53	56	58	75	131	343
Reckless	94	96	102	117	155	176	6924627
Iyengar NanoLog	111	119	127	131	172	319	9553
spdlog	231	253	269	282	587	806	1055
g3log	1122	1142	1368	2293	2516	2698	4051

4 Threads

Library	50th	75th	90th	95th	99th	99.9th	Worst
Quill, Dual Queue Enabled, Bounded Queue	27	32	38	45	59	82	440
Quill, Dual Queue Enabled, Unbounded Queue	26	32	38	46	60	106	3866
Quill, Dual Queue Disabled, Unbounded Queue	203	226	243	252	269	300	4278
PlatformLab NanoLog	29	34	47	55	59	69	289
MS BinLog	51	54	58	62	96	133	375
Reckless	115	134	157	172	216	377	30721857
Iyengar NanoLog	111	121	137	175	231	811	32100
spdlog	266	309	593	672	911	1329	13600
g3log	1816	2207	2389	2497	3457	4211	8365

The benchmarks are done on Linux (Ubuntu/RHEL) with GCC 8.1.

Each thread is pinned on a different cpu. Unfortunately the cores are not isolated. Running the backend logger thread in the same CPU as the caller hot-path threads, slows down the log message processing on the backend logging thread and will cause the SPSC queue to fill faster and re-allocate.

Continuously Logging messages in a loop makes the consumer (backend logging thread) unable to follow up and the queue will have to re-allocate or block for most logging libraries expect very high throughput binary loggers like PlatformLab Nanolog. Therefore, a different approach was followed that suits more to a real time application:

20 messages are logged in a loop.
calculate/store the average latency for those messages.
wait between 1-2 ms.
repeat for n iterations.

I run each logger benchmark four times and the above latencies are the second best result.

The benchmark code and results can be found here.

Supported Platforms And Compilers

Quill requires a C++14 compiler. Minimum required versions of supported compilers are shown in the below table.

Compiler	Notes
GCC	version >= 5.0
Clang	version >= 5.0
MSVC++	version >= 14.3

Platform	Notes
Linux	Ubuntu, RHEL, Centos, Fedora
Windows	Windows 10 - version 1607, Windows Server 2016
macOS	Tested with Xcode 9.4

Basic usage

#include "quill/Quill.h"

int main()
{
  quill::enable_console_colours();
  quill::start();

  quill::Logger* logger = quill::get_logger();
  logger->set_log_level(quill::LogLevel::TraceL3);

  // enable a backtrace that will get flushed when we log CRITICAL
  logger->init_backtrace(2, quill::LogLevel::Critical);

  LOG_BACKTRACE(logger, "Backtrace log {}", 1);
  LOG_BACKTRACE(logger, "Backtrace log {}", 2);

  LOG_INFO(logger, "Welcome to Quill!");
  LOG_ERROR(logger, "An error message. error code {}", 123);
  LOG_WARNING(logger, "A warning message.");
  LOG_CRITICAL(logger, "A critical error.");
  LOG_DEBUG(logger, "Debugging foo {}", 1234);
  LOG_TRACE_L1(logger, "{:>30}", "right aligned");
  LOG_TRACE_L2(logger, "Positional arguments are {1} {0} ", "too", "supported");
  LOG_TRACE_L3(logger, "Support for floats {:03.2f}", 1.23456);
}

Output

CMake-Integration

External

Building and Installing Quill as Static Library

git clone https://github.com/odygrd/quill.git
mkdir cmake_build
cd cmake_build
make install

Note: To install in custom directory invoke cmake with -DCMAKE_INSTALL_PREFIX=/quill/install-dir/

Building and Installing Quill as Static Library With External `libfmt`

cmake -DCMAKE_PREFIX_PATH=/my/fmt/fmt-config.cmake-directory/ -DQUILL_FMT_EXTERNAL=ON -DCMAKE_INSTALL_PREFIX=/quill/install-dir/'

Then use the library from a CMake project, you can locate it directly with find_package()

Directory Structure

my_project/
├── CMakeLists.txt
├── main.cpp

CMakeLists.txt

# Set only if needed - quill was installed under a custom non-standard directory
set(CMAKE_PREFIX_PATH /test_quill/usr/local/)

find_package(quill REQUIRED)

# Linking your project against quill
add_executable(example main.cpp)
target_link_libraries(example PRIVATE quill::quill)

main.cpp

See basic usage

Embedded

To embed the library directly, copy the source folder to your project and call add_subdirectory() in your CMakeLists.txt file

Directory Structure

my_project/
├── quill/            (source folder)
├── CMakeLists.txt
├── main.cpp

CMakeLists.txt

cmake_minimum_required(VERSION 3.1.0)
project(my_project)

set(CMAKE_CXX_STANDARD 14)

add_subdirectory(quill)

add_executable(my_project main.cpp)
target_link_libraries(my_project PRIVATE quill::quill)

main.cpp

See basic usage

Documentation

Advanced usage and additional documentation can be found in the wiki pages.

The examples folder is also a good source of documentation.

License

Quill is licensed under the MIT License

Quill depends on third party libraries with separate copyright notices and license terms. Your use of the source code for these subcomponents is subject to the terms and conditions of the following licenses.

(MIT License) {fmt} (https://github.com/fmtlib/fmt/blob/master/LICENSE.rst)
(MIT License) invoke.hpp (https://github.com/BlackMATov/invoke.hpp/blob/master/LICENSE.md)
(MIT License) doctest (https://github.com/onqtam/doctest/blob/master/LICENSE.txt)

Comments

Thread Sanitizer Reporting data race

Running my program with -fsanitize=thread produces a data race warning with quill. Is this potentially because some other thread is accessing a value passed to a quill log statement that has not yet completed?

It looks like thread T2, created by quill, is reading from the same data that thread T8 is writing to. Thread T8 was created by taskflow.

If it's expected behavior that the threaded nature of quill could potentially cause a data race if some other thread is modifying the same data that was passed to quill, then this may not be a quill bug and I need to be more careful with my usage of quill.

From the summary at the bottom of the thread trace:

SUMMARY: ThreadSanitizer: data race /home/xander/dev/vcpkg/installed/x64-linux/include/quill/detail/sps
c_queue/UnboundedSPSCEventQueue.h:140:27 in quill::detail::UnboundedSPSCEventQueue<quill::detail::BaseE
vent>::try_pop()

Thanks for this fantastic open source library. Grateful for any guidance on usage in a multi-threaded environment.

Full thread trace:

==================                                                                                     
WARNING: ThreadSanitizer: data race (pid=18389)
  Read of size 8 at 0x7b4800090100 by thread T2:                                                           
#0 quill::detail::UnboundedSPSCEventQueue<quill::detail::BaseEvent>::try_pop() /home/xander/dev/vc$
kg/installed/x64-linux/include/quill/detail/spsc_queue/UnboundedSPSCEventQueue.h:140:27 (MyDataPre$rocessor+0x42df57)
    #1 quill::detail::BackendWorker::_read_event_queue(quill::detail::ThreadContext*) 
/home/xander/dev$vcpkg/installed/x64-linux/include/quill/detail/backend/BackendWorker.h:352:37 (MyDataPreProcessor+$
x42df57)
    #2 quill::detail::BackendWorker::_populate_priority_queue(std::vector<quill::detail::ThreadContext$
, quill::detail::CacheAlignedAllocator<quill::detail::ThreadContext*> > const&) /home/xander/dev/vcpkg$
installed/x64-linux/include/quill/detail/backend/BackendWorker.h:335:5 (MyDataPreProcessor+0x42d3a$
)
    #3 quill::detail::BackendWorker::_main_loop() /home/xander/dev/vcpkg/installed/x64-linux/include/q$
ill/detail/backend/BackendWorker.h:548:3 (MyDataPreProcessor+0x42d3a5)
    #4 quill::detail::BackendWorker::run()::'lambda'()::operator()() const::'lambda'()::operator()() c$
nst /home/xander/dev/vcpkg/installed/x64-linux/include/quill/detail/backend/BackendWorker.h:296:21 (Pl$
tusDataPreProcessor+0x51a6c7)
    #5 void std::__invoke_impl<void, quill::detail::BackendWorker::run()::'lambda'()::operator()() con$
t::'lambda'()>(std::__invoke_other, quill::detail::BackendWorker::run()::'lambda'()::operator()() cons$
::'lambda'()&&) /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/invoke.h:60:14
(MyDataPreProcessor+0x51a559)
    #6 std::__invoke_result<quill::detail::BackendWorker::run()::'lambda'()::operator()() const::'lamb$
a'()>::type std::__invoke<quill::detail::BackendWorker::run()::'lambda'()::operator()() const::'lambda$
()>(quill::detail::BackendWorker::run()::'lambda'()::operator()() const::'lambda'()&&) /usr/bin/../lib$
gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/invoke.h:95:14 (MyDataPreProcessor+0x51a55$
)
    #7 void std::thread::_Invoker<std::tuple<quill::detail::BackendWorker::run()::'lambda'()::operator$
)() const::'lambda'()> >::_M_invoke<0ul>(std::_Index_tuple<0ul>) /usr/bin/../lib/gcc/x86_64-linux-gnu/$
0/../../../../include/c++/10/thread:264:13 (MyDataPreProcessor+0x51a559)
    #8 std::thread::_Invoker<std::tuple<quill::detail::BackendWorker::run()::'lambda'()::operator()() $
onst::'lambda'()> >::operator()() /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/t$
read:271:11 (MyDataPreProcessor+0x51a559)
    #9 std::thread::_State_impl<std::thread::_Invoker<std::tuple<quill::detail::BackendWorker::run()::$
lambda'()::operator()() const::'lambda'()> > >::_M_run() /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../..$
../../include/c++/10/thread:215:13 (MyDataPreProcessor+0x51a559)
    #10 execute_native_thread_routine /home/conda/feedstock_root/build_artifacts/ctng-compilers_161823$
181388/work/.build/x86_64-conda-linux-gnu/src/gcc/libstdc++-v3/src/c++11/thread.cc:80:18 (libstdc++.so$
6+0xc9066)

  Previous write of size 8 at 0x7b4800090100 by thread T8:
    #0 posix_memalign <null> (MyDataPreProcessor+0x435c98)                                             
#1 quill::detail::aligned_alloc(unsigned long, unsigned long) /home/xander/dev/vcpkg/buildtrees/qu$
ll/src/v1.6.2-b72390cb62.clean/quill/src/detail/misc/Os.cpp:254:28 (MyDataPreProcessor+0x5979cb)
    #2 ccMydata::preprocess_record_batch(std::shared_ptr<arrow::RecordBatch>, junction::Concurrent$
ap_Leapfrog<long, ccMydata::Book*, junction::DefaultKeyTraits<long>, junction::DefaultValu$
Traits<ccMydata::Book*> >&, folly::fbvector<std::shared_ptr<ccMydata::Book>, s$
d::allocator<std::shared_ptr<ccMydata::Book> > >&, long&, std::unordered_map<long, folly::$
bvector<std::shared_ptr<ccMydata::BookItem>, std::allocator<std::shared_ptr<ccMydata::Or$
erBookLevel> > >, std::hash<long>, std::equal_to<long>, std::allocator<std::pair<long const, folly::fb$
ector<std::shared_ptr<ccMydata::BookItem>, std::allocator<std::shared_ptr<ccMydata::Orde$
BookLevel> > > > > >&, std::unordered_set<long, std::hash<long>, std::equal_to<long>, std::allocator<l$
ng> >&, long&, tf::Executor&, tf::Taskflow&, folly::fbvector<std::shared_ptr<std::deque<folly::fbvecto$
<std::shared_ptr<ccMydata::BookItem>, std::allocator<std::shared_ptr<ccMydata::BookItem> > >, std::allocator<folly::fbvector<std::shared_ptr<ccMydata::BookItem>, std::allocator
<std::shared_ptr<ccMydata::BookItem> > > > > >, std::allocator<std::shared_ptr<std::deque<fol
ly::fbvector<std::shared_ptr<ccMydata::BookItem>, std::allocator<std::shared_ptr<ccMydata
::BookItem> > >, std::allocator<folly::fbvector<std::shared_ptr<ccMydata::BookItem>, st
d::allocator<std::shared_ptr<ccMydata::BookItem> > > > > > > >&, unsigned short&)::$_2::opera
tor()() const /home/xander/dev/My_model/pyMydata/pyMydata/PreprocessData.cc:152:41 (MyD
ataPreProcessor+0x4dd52a)
    #3 void std::__invoke_impl<void, ccMydata::preprocess_record_batch(std::shared_ptr<arrow::Recor
dBatch>, junction::ConcurrentMap_Leapfrog<long, ccMydata::Book*, junction::DefaultKeyTraits
<long>, junction::DefaultValueTraits<ccMydata::Book*> >&, folly::fbvector<std::shared_ptr<c
cMydata::Book>, std::allocator<std::shared_ptr<ccMydata::Book> > >&, long&, std
::unordered_map<long, folly::fbvector<std::shared_ptr<ccMydata::BookItem>, std::allocator<std
::shared_ptr<ccMydata::BookItem> > >, std::hash<long>, std::equal_to<long>, std::allocator<st
d::pair<long const, folly::fbvector<std::shared_ptr<ccMydata::BookItem>, std::allocator<std::
shared_ptr<ccMydata::BookItem> > > > > >&, std::unordered_set<long, std::hash<long>, std::equ
al_to<long>, std::allocator<long> >&, long&, tf::Executor&, tf::Taskflow&, folly::fbvector<std::shared_
ptr<std::deque<folly::fbvector<std::shared_ptr<ccMydata::BookItem>, std::allocator<std::share
d_ptr<ccMydata::BookItem> > >, std::allocator<folly::fbvector<std::shared_ptr<ccMydata::O
rderBookLevel>, std::allocator<std::shared_ptr<ccMydata::BookItem> > > > > >, std::allocator<
std::shared_ptr<std::deque<folly::fbvector<std::shared_ptr<ccMydata::BookItem>, std::allocato
r<std::shared_ptr<ccMydata::BookItem> > >, std::allocator<folly::fbvector<std::shared_ptr<ccp
lutusdata::BookItem>, std::allocator<std::shared_ptr<ccMydata::BookItem> > > > > > > >&
, unsigned short&)::$_2&>(std::__invoke_other, ccMydata::preprocess_record_batch(std::shared_ptr<ar
row::RecordBatch>, junction::ConcurrentMap_Leapfrog<long, ccMydata::Book*, junction::Defaul
tKeyTraits<long>, junction::DefaultValueTraits<ccMydata::Book*> >&, folly::fbvector<std::sh
ared_ptr<ccMydata::Book>, std::allocator<std::shared_ptr<ccMydata::Book> > >&,
long&, std::unordered_map<long, folly::fbvector<std::shared_ptr<ccMydata::BookItem>, std::all
ocator<std::shared_ptr<ccMydata::BookItem> > >, std::hash<long>, std::equal_to<long>, std::al
locator<std::pair<long const, folly::fbvector<std::shared_ptr<ccMydata::BookItem>, std::alloc
ator<std::shared_ptr<ccMydata::BookItem> > > > > >&, std::unordered_set<long, std::hash<long>
, std::equal_to<long>, std::allocator<long> >&, long&, tf::Executor&, tf::Taskflow&, folly::fbvector<st
d::shared_ptr<std::deque<folly::fbvector<std::shared_ptr<ccMydata::BookItem>, std::allocator<
std::shared_ptr<ccMydata::BookItem> > >, std::allocator<folly::fbvector<std::shared_ptr<ccplu
tusdata::BookItem>, std::allocator<std::shared_ptr<ccMydata::BookItem> > > > > >, std::
allocator<std::shared_ptr<std::deque<folly::fbvector<std::shared_ptr<ccMydata::BookItem>, std
::allocator<std::shared_ptr<ccMydata::BookItem> > >, std::allocator<folly::fbvector<std::shar
ed_ptr<ccMydata::BookItem>, std::allocator<std::shared_ptr<ccMydata::BookItem> > >
> > > > >&, unsigned short&)::$_2&) /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/
bits/invoke.h:60:14 (MyDataPreProcessor+0x4dd52a)

#4 std::enable_if<is_invocable_r_v<void, ccMydata::preprocess_record_batch(std::shared_ptr<arro
w::RecordBatch>, junction::ConcurrentMap_Leapfrog<long, ccMydata::Book*, junction::DefaultK
eyTraits<long>, junction::DefaultValueTraits<ccMydata::Book*> >&, folly::fbvector<std::shar
ed_ptr<ccMydata::Book>, std::allocator<std::shared_ptr<ccMydata::Book> > >&, lo
ng&, std::unordered_map<long, folly::fbvector<std::shared_ptr<ccMydata::BookItem>, std::alloc
ator<std::shared_ptr<ccMydata::BookItem> > >, std::hash<long>, std::equal_to<long>, std::allo
cator<std::pair<long const, folly::fbvector<std::shared_ptr<ccMydata::BookItem>, std::allocat
or<std::shared_ptr<ccMydata::BookItem> > > > > >&, std::unordered_set<long, std::hash<long>,
std::equal_to<long>, std::allocator<long> >&, long&, tf::Executor&, tf::Taskflow&, folly::fbvector<std:
:shared_ptr<std::deque<folly::fbvector<std::shared_ptr<ccMydata::BookItem>, std::allocator<st
d::shared_ptr<ccMydata::BookItem> > >, std::allocator<folly::fbvector<std::shared_ptr<ccplutu
sdata::BookItem>, std::allocator<std::shared_ptr<ccMydata::BookItem> > > > > >, std::al
locator<std::shared_ptr<std::deque<folly::fbvector<std::shared_ptr<ccMydata::BookItem>, std::
allocator<std::shared_ptr<ccMydata::BookItem> > >, std::allocator<folly::fbvector<std::shared
_ptr<ccMydata::BookItem>, std::allocator<std::shared_ptr<ccMydata::BookItem> > > >
> > > >&, unsigned short&)::$_2&>, void>::type std::__invoke_r<void, ccMydata::preprocess_record_ba
tch(std::shared_ptr<arrow::RecordBatch>, junction::ConcurrentMap_Leapfrog<long, ccMydata::Book*, junction::DefaultKeyTraits<long>, junction::DefaultValueTraits<ccMydata::Book*> >&, f
olly::fbvector<std::shared_ptr<ccMydata::Book>, std::allocator<std::shared_ptr<ccMydata
::Book> > >&, long&, std::unordered_map<long, folly::fbvector<std::shared_ptr<ccMydata::Ord
erBookLevel>, std::allocator<std::shared_ptr<ccMydata::BookItem> > >, std::hash<long>, std::e
qual_to<long>, std::allocator<std::pair<long const, folly::fbvector<std::shared_ptr<ccMydata::Order
BookLevel>, std::allocator<std::shared_ptr<ccMydata::BookItem> > > > > >&, std::unordered_set
<long, std::hash<long>, std::equal_to<long>, std::allocator<long> >&, long&, tf::Executor&, tf::Taskflo
w&, folly::fbvector<std::shared_ptr<std::deque<folly::fbvector<std::shared_ptr<ccMydata::BookItem>, std::allocator<std::shared_ptr<ccMydata::BookItem> > >, std::allocator<folly::fbvector
<std::shared_ptr<ccMydata::BookItem>, std::allocator<std::shared_ptr<ccMydata::BookItem> > > > > >, std::allocator<std::shared_ptr<std::deque<folly::fbvector<std::shared_ptr<ccMydata
::BookItem>, std::allocator<std::shared_ptr<ccMydata::BookItem> > >, std::allocator<fol
ly::fbvector<std::shared_ptr<ccMydata::BookItem>, std::allocator<std::shared_ptr<ccMydata
::BookItem> > > > > > > >&, unsigned short&)::$_2&>(ccMydata::preprocess_record_batch(std::sh
ared_ptr<arrow::RecordBatch>, junction::ConcurrentMap_Leapfrog<long, ccMydata::Book*, junct
ion::DefaultKeyTraits<long>, junction::DefaultValueTraits<ccMydata::Book*> >&, folly::fbvec
tor<std::shared_ptr<ccMydata::Book>, std::allocator<std::shared_ptr<ccMydata::Book> > >&, long&, std::unordered_map<long, folly::fbvector<std::shared_ptr<ccMydata::BookItem
>, std::allocator<std::shared_ptr<ccMydata::BookItem> > >, std::hash<long>, std::equal_to<lon
g>, std::allocator<std::pair<long const, folly::fbvector<std::shared_ptr<ccMydata::BookItem>,
 std::allocator<std::shared_ptr<ccMydata::BookItem> > > > > >&, std::unordered_set<long, std:
:hash<long>, std::equal_to<long>, std::allocator<long> >&, long&, tf::Executor&, tf::Taskflow&, folly::
fbvector<std::shared_ptr<std::deque<folly::fbvector<std::shared_ptr<ccMydata::BookItem>, std:
:allocator<std::shared_ptr<ccMydata::BookItem> > >, std::allocator<folly::fbvector<std::share
d_ptr<ccMydata::BookItem>, std::allocator<std::shared_ptr<ccMydata::BookItem> > > >
 > >, std::allocator<std::shared_ptr<std::deque<folly::fbvector<std::shared_ptr<ccMydata::BookItem>, std::allocator<std::shared_ptr<ccMydata::BookItem> > >, std::allocator<folly::fbvecto
r<std::shared_ptr<ccMydata::BookItem>, std::allocator<std::shared_ptr<ccMydata::BookItem> > > > > > > >&, unsigned short&)::$_2&) /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../incl
ude/c++/10/bits/invoke.h:110:2 (MyDataPreProcessor+0x4dd52a)

#5 std::_Function_handler<void (), ccMydata::preprocess_record_batch(std::shared_ptr<arrow::Rec
ordBatch>, junction::ConcurrentMap_Leapfrog<long, ccMydata::Book*, junction::DefaultKeyTrai
ts<long>, junction::DefaultValueTraits<ccMydata::Book*> >&, folly::fbvector<std::shared_ptr
<ccMydata::Book>, std::allocator<std::shared_ptr<ccMydata::Book> > >&, long&, s
td::unordered_map<long, folly::fbvector<std::shared_ptr<ccMydata::BookItem>, std::allocator<s
td::shared_ptr<ccMydata::BookItem> > >, std::hash<long>, std::equal_to<long>, std::allocator<
std::pair<long const, folly::fbvector<std::shared_ptr<ccMydata::BookItem>, std::allocator<std
::shared_ptr<ccMydata::BookItem> > > > > >&, std::unordered_set<long, std::hash<long>, std::e
qual_to<long>, std::allocator<long> >&, long&, tf::Executor&, tf::Taskflow&, folly::fbvector<std::share
d_ptr<std::deque<folly::fbvector<std::shared_ptr<ccMydata::BookItem>, std::allocator<std::sha
red_ptr<ccMydata::BookItem> > >, std::allocator<folly::fbvector<std::shared_ptr<ccMydata:
:BookItem>, std::allocator<std::shared_ptr<ccMydata::BookItem> > > > > >, std::allocato
r<std::shared_ptr<std::deque<folly::fbvector<std::shared_ptr<ccMydata::BookItem>, std::alloca
tor<std::shared_ptr<ccMydata::BookItem> > >, std::allocator<folly::fbvector<std::shared_ptr<c
cMydata::BookItem>, std::allocator<std::shared_ptr<ccMydata::BookItem> > > > > > >
>&, unsigned short&)::$_2>::_M_invoke(std::_Any_data const&) /usr/bin/../lib/gcc/x86_64-linux-gnu/10/..
/../../../include/c++/10/bits/std_function.h:291:9 (MyDataPreProcessor+0x4dd52a)
    #6 std::function<void ()>::operator()() const /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../i
nclude/c++/10/bits/std_function.h:622:14 (MyDataPreProcessor+0x50ffd8)
    #7 tf::Executor::_invoke_static_task(tf::Worker&, tf::Node*) /home/xander/dev/vcpkg/installed/x64-l
inux/include/taskflow/core/algorithm/../executor.hpp:818:3 (MyDataPreProcessor+0x50ffd8)
    #8 tf::Executor::_invoke(tf::Worker&, tf::Node*) /home/xander/dev/vcpkg/installed/x64-linux/include
/taskflow/core/algorithm/../executor.hpp:666:7 (MyDataPreProcessor+0x50f3a1)
    #9 tf::Executor::_exploit_task(tf::Worker&, tf::Node*&) /home/xander/dev/vcpkg/installed/x64-linux/
include/taskflow/core/algorithm/../executor.hpp:462:7 (MyDataPreProcessor+0x50e957)
    #10 tf::Executor::_spawn(unsigned long)::'lambda'(tf::Worker&)::operator()(tf::Worker&) const /home
/xander/dev/vcpkg/installed/x64-linux/include/taskflow/core/algorithm/../executor.hpp:392:9 (MyData
PreProcessor+0x50e957)
    #11 void std::__invoke_impl<void, tf::Executor::_spawn(unsigned long)::'lambda'(tf::Worker&), std::
reference_wrapper<tf::Worker> >(std::__invoke_other, tf::Executor::_spawn(unsigned long)::'lambda'(tf::
Worker&)&&, std::reference_wrapper<tf::Worker>&&) /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../i
nclude/c++/10/bits/invoke.h:60:14 (MyDataPreProcessor+0x50e809)
    #12 std::__invoke_result<tf::Executor::_spawn(unsigned long)::'lambda'(tf::Worker&), std::reference
_wrapper<tf::Worker> >::type std::__invoke<tf::Executor::_spawn(unsigned long)::'lambda'(tf::Worker&),
std::reference_wrapper<tf::Worker> >(tf::Executor::_spawn(unsigned long)::'lambda'(tf::Worker&)&&, std:
:reference_wrapper<tf::Worker>&&) /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bi
ts/invoke.h:95:14 (MyDataPreProcessor+0x50e809)
    #13 void std::thread::_Invoker<std::tuple<tf::Executor::_spawn(unsigned long)::'lambda'(tf::Worker&
), std::reference_wrapper<tf::Worker> > >::_M_invoke<0ul, 1ul>(std::_Index_tuple<0ul, 1ul>) /usr/bin/..
/lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/thread:264:13 (MyDataPreProcessor+0x50e809)
    #14 std::thread::_Invoker<std::tuple<tf::Executor::_spawn(unsigned long)::'lambda'(tf::Worker&), st
d::reference_wrapper<tf::Worker> > >::operator()() /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../
include/c++/10/thread:271:11 (MyDataPreProcessor+0x50e809)
    #15 std::thread::_State_impl<std::thread::_Invoker<std::tuple<tf::Executor::_spawn(unsigned long)::
'lambda'(tf::Worker&), std::reference_wrapper<tf::Worker> > > >::_M_run() /usr/bin/../lib/gcc/x86_64-li
nux-gnu/10/../../../../include/c++/10/thread:215:13 (MyDataPreProcessor+0x50e809)
    #16 execute_native_thread_routine /home/conda/feedstock_root/build_artifacts/ctng-compilers_1618239
181388/work/.build/x86_64-conda-linux-gnu/src/gcc/libstdc++-v3/src/c++11/thread.cc:80:18 (libstdc++.so.
6+0xc9066)

  As if synchronized via sleep:
    #0 nanosleep <null> (MyDataPreProcessor+0x433b2d)
    #1 void std::this_thread::sleep_for<long, std::ratio<1l, 1000000000l> >(std::chrono::duration<long,
 std::ratio<1l, 1000000000l> > const&) /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/
10/thread:401:9 (MyDataPreProcessor+0x42d4ef)
    #2 quill::detail::BackendWorker::_main_loop() /home/xander/dev/vcpkg/installed/x64-linux/include/qu
ill/detail/backend/BackendWorker.h:570:5 (MyDataPreProcessor+0x42d4ef)
    #3 quill::detail::BackendWorker::run()::'lambda'()::operator()() const::'lambda'()::operator()() co
nst /home/xander/dev/vcpkg/installed/x64-linux/include/quill/detail/backend/BackendWorker.h:296:21 (Plu
tusDataPreProcessor+0x51a6c7)
    #4 void std::__invoke_impl<void, quill::detail::BackendWorker::run()::'lambda'()::operator()() cons
t::'lambda'()>(std::__invoke_other, quill::detail::BackendWorker::run()::'lambda'()::operator()() const
::'lambda'()&&) /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/invoke.h:60:14
(MyDataPreProcessor+0x51a559)
    #5 std::__invoke_result<quill::detail::BackendWorker::run()::'lambda'()::operator()() const::'lambd
a'()>::type std::__invoke<quill::detail::BackendWorker::run()::'lambda'()::operator()() const::'lambda'
()>(quill::detail::BackendWorker::run()::'lambda'()::operator()() const::'lambda'()&&) /usr/bin/../lib/
gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/invoke.h:95:14 (MyDataPreProcessor+0x51a559
)
    #6 void std::thread::_Invoker<std::tuple<quill::detail::BackendWorker::run()::'lambda'()::operator(
)() const::'lambda'()> >::_M_invoke<0ul>(std::_Index_tuple<0ul>) /usr/bin/../lib/gcc/x86_64-linux-gnu/1
0/../../../../include/c++/10/thread:264:13 (MyDataPreProcessor+0x51a559)
    #7 std::thread::_Invoker<std::tuple<quill::detail::BackendWorker::run()::'lambda'()::operator()() c
onst::'lambda'()> >::operator()() /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/th
read:271:11 (MyDataPreProcessor+0x51a559)
    #8 std::thread::_State_impl<std::thread::_Invoker<std::tuple<quill::detail::BackendWorker::run()::'
lambda'()::operator()() const::'lambda'()> > >::_M_run() /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../
../../include/c++/10/thread:215:13 (MyDataPreProcessor+0x51a559)
    #9 execute_native_thread_routine /home/conda/feedstock_root/build_artifacts/ctng-compilers_16182391
81388/work/.build/x86_64-conda-linux-gnu/src/gcc/libstdc++-v3/src/c++11/thread.cc:80:18 (libstdc++.so.6
+0xc9066)

Location is heap block of size 384 at 0x7b4800090000 allocated by thread T8:
    #0 posix_memalign <null> (MyDataPreProcessor+0x435c98)
    #1 quill::detail::aligned_alloc(unsigned long, unsigned long) /home/xander/dev/vcpkg/buildtrees/qui
ll/src/v1.6.2-b72390cb62.clean/quill/src/detail/misc/Os.cpp:254:28 (MyDataPreProcessor+0x5979cb)
    #2 ccMydata::preprocess_record_batch(std::shared_ptr<arrow::RecordBatch>, junction::ConcurrentM
ap_Leapfrog<long, ccMydata::Book*, junction::DefaultKeyTraits<long>, junction::DefaultValue
Traits<ccMydata::Book*> >&, folly::fbvector<std::shared_ptr<ccMydata::Book>, st
d::allocator<std::shared_ptr<ccMydata::Book> > >&, long&, std::unordered_map<long, folly::f
bvector<std::shared_ptr<ccMydata::BookItem>, std::allocator<std::shared_ptr<ccMydata::Ord
erBookLevel> > >, std::hash<long>, std::equal_to<long>, std::allocator<std::pair<long const, folly::fbv
ector<std::shared_ptr<ccMydata::BookItem>, std::allocator<std::shared_ptr<ccMydata::Order
BookLevel> > > > > >&, std::unordered_set<long, std::hash<long>, std::equal_to<long>, std::allocator<lo
ng> >&, long&, tf::Executor&, tf::Taskflow&, folly::fbvector<std::shared_ptr<std::deque<folly::fbvector
<std::shared_ptr<ccMydata::BookItem>, std::allocator<std::shared_ptr<ccMydata::BookItem> > >, std::allocator<folly::fbvector<std::shared_ptr<ccMydata::BookItem>, std::allocator
<std::shared_ptr<ccMydata::BookItem> > > > > >, std::allocator<std::shared_ptr<std::deque<fol
ly::fbvector<std::shared_ptr<ccMydata::BookItem>, std::allocator<std::shared_ptr<ccMydata
::BookItem> > >, std::allocator<folly::fbvector<std::shared_ptr<ccMydata::BookItem>, st
d::allocator<std::shared_ptr<ccMydata::BookItem> > > > > > > >&, unsigned short&)::$_2::opera
tor()() const /home/xander/dev/My_model/pyMydata/pyMydata/PreprocessData.cc:152:41 (MyD
ataPreProcessor+0x4dd52a)
    #3 void std::__invoke_impl<void, ccMydata::preprocess_record_batch(std::shared_ptr<arrow::Recor
dBatch>, junction::ConcurrentMap_Leapfrog<long, ccMydata::Book*, junction::DefaultKeyTraits
<long>, junction::DefaultValueTraits<ccMydata::Book*> >&, folly::fbvector<std::shared_ptr<c
cMydata::Book>, std::allocator<std::shared_ptr<ccMydata::Book> > >&, long&, std
::unordered_map<long, folly::fbvector<std::shared_ptr<ccMydata::BookItem>, std::allocator<std
::shared_ptr<ccMydata::BookItem> > >, std::hash<long>, std::equal_to<long>, std::allocator<st
d::pair<long const, folly::fbvector<std::shared_ptr<ccMydata::BookItem>, std::allocator<std::
shared_ptr<ccMydata::BookItem> > > > > >&, std::unordered_set<long, std::hash<long>, std::equ
al_to<long>, std::allocator<long> >&, long&, tf::Executor&, tf::Taskflow&, folly::fbvector<std::shared_
ptr<std::deque<folly::fbvector<std::shared_ptr<ccMydata::BookItem>, std::allocator<std::share
d_ptr<ccMydata::BookItem> > >, std::allocator<folly::fbvector<std::shared_ptr<ccMydata::O
rderBookLevel>, std::allocator<std::shared_ptr<ccMydata::BookItem> > > > > >, std::allocator<
std::shared_ptr<std::deque<folly::fbvector<std::shared_ptr<ccMydata::BookItem>, std::allocato
r<std::shared_ptr<ccMydata::BookItem> > >, std::allocator<folly::fbvector<std::shared_ptr<ccp
lutusdata::BookItem>, std::allocator<std::shared_ptr<ccMydata::BookItem> > > > > > > >&
, unsigned short&)::$_2&>(std::__invoke_other, ccMydata::preprocess_record_batch(std::shared_ptr<ar
row::RecordBatch>, junction::ConcurrentMap_Leapfrog<long, ccMydata::Book*, junction::Defaul
tKeyTraits<long>, junction::DefaultValueTraits<ccMydata::Book*> >&, folly::fbvector<std::sh
ared_ptr<ccMydata::Book>, std::allocator<std::shared_ptr<ccMydata::Book> > >&,
long&, std::unordered_map<long, folly::fbvector<std::shared_ptr<ccMydata::BookItem>, std::all
ocator<std::shared_ptr<ccMydata::BookItem> > >, std::hash<long>, std::equal_to<long>, std::al
locator<std::pair<long const, folly::fbvector<std::shared_ptr<ccMydata::BookItem>, std::alloc
ator<std::shared_ptr<ccMydata::BookItem> > > > > >&, std::unordered_set<long, std::hash<long>
, std::equal_to<long>, std::allocator<long> >&, long&, tf::Executor&, tf::Taskflow&, folly::fbvector<st
d::shared_ptr<std::deque<folly::fbvector<std::shared_ptr<ccMydata::BookItem>, std::allocator<
std::shared_ptr<ccMydata::BookItem> > >, std::allocator<folly::fbvector<std::shared_ptr<ccplu
tusdata::BookItem>, std::allocator<std::shared_ptr<ccMydata::BookItem> > > > > >, std::
allocator<std::shared_ptr<std::deque<folly::fbvector<std::shared_ptr<ccMydata::BookItem>, std
::allocator<std::shared_ptr<ccMydata::BookItem> > >, std::allocator<folly::fbvector<std::shar
ed_ptr<ccMydata::BookItem>, std::allocator<std::shared_ptr<ccMydata::BookItem> > >
> > > > >&, unsigned short&)::$_2&) /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/
bits/invoke.h:60:14 (MyDataPreProcessor+0x4dd52a)

#4 std::enable_if<is_invocable_r_v<void, ccMydata::preprocess_record_batch(std::shared_ptr<arro
w::RecordBatch>, junction::ConcurrentMap_Leapfrog<long, ccMydata::Book*, junction::DefaultK
eyTraits<long>, junction::DefaultValueTraits<ccMydata::Book*> >&, folly::fbvector<std::shar
ed_ptr<ccMydata::Book>, std::allocator<std::shared_ptr<ccMydata::Book> > >&, lo
ng&, std::unordered_map<long, folly::fbvector<std::shared_ptr<ccMydata::BookItem>, std::alloc
ator<std::shared_ptr<ccMydata::BookItem> > >, std::hash<long>, std::equal_to<long>, std::allo
cator<std::pair<long const, folly::fbvector<std::shared_ptr<ccMydata::BookItem>, std::allocat
or<std::shared_ptr<ccMydata::BookItem> > > > > >&, std::unordered_set<long, std::hash<long>,
std::equal_to<long>, std::allocator<long> >&, long&, tf::Executor&, tf::Taskflow&, folly::fbvector<std:
:shared_ptr<std::deque<folly::fbvector<std::shared_ptr<ccMydata::BookItem>, std::allocator<st
d::shared_ptr<ccMydata::BookItem> > >, std::allocator<folly::fbvector<std::shared_ptr<ccplutu
sdata::BookItem>, std::allocator<std::shared_ptr<ccMydata::BookItem> > > > > >, std::al
locator<std::shared_ptr<std::deque<folly::fbvector<std::shared_ptr<ccMydata::BookItem>, std::
allocator<std::shared_ptr<ccMydata::BookItem> > >, std::allocator<folly::fbvector<std::shared
_ptr<ccMydata::BookItem>, std::allocator<std::shared_ptr<ccMydata::BookItem> > > >
> > > >&, unsigned short&)::$_2&>, void>::type std::__invoke_r<void, ccMydata::preprocess_record_ba
tch(std::shared_ptr<arrow::RecordBatch>, junction::ConcurrentMap_Leapfrog<long, ccMydata::Book*, junction::DefaultKeyTraits<long>, junction::DefaultValueTraits<ccMydata::Book*> >&, f
olly::fbvector<std::shared_ptr<ccMydata::Book>, std::allocator<std::shared_ptr<ccMydata
::Book> > >&, long&, std::unordered_map<long, folly::fbvector<std::shared_ptr<ccMydata::Ord
erBookLevel>, std::allocator<std::shared_ptr<ccMydata::BookItem> > >, std::hash<long>, std::e
qual_to<long>, std::allocator<std::pair<long const, folly::fbvector<std::shared_ptr<ccMydata::Order
BookLevel>, std::allocator<std::shared_ptr<ccMydata::BookItem> > > > > >&, std::unordered_set
<long, std::hash<long>, std::equal_to<long>, std::allocator<long> >&, long&, tf::Executor&, tf::Taskflo
w&, folly::fbvector<std::shared_ptr<std::deque<folly::fbvector<std::shared_ptr<ccMydata::BookItem>, std::allocator<std::shared_ptr<ccMydata::BookItem> > >, std::allocator<folly::fbvector
<std::shared_ptr<ccMydata::BookItem>, std::allocator<std::shared_ptr<ccMydata::BookItem> > > > > >, std::allocator<std::shared_ptr<std::deque<folly::fbvector<std::shared_ptr<ccMydata
::BookItem>, std::allocator<std::shared_ptr<ccMydata::BookItem> > >, std::allocator<fol
ly::fbvector<std::shared_ptr<ccMydata::BookItem>, std::allocator<std::shared_ptr<ccMydata
::BookItem> > > > > > > >&, unsigned short&)::$_2&>(ccMydata::preprocess_record_batch(std::sh
ared_ptr<arrow::RecordBatch>, junction::ConcurrentMap_Leapfrog<long, ccMydata::Book*, junct
ion::DefaultKeyTraits<long>, junction::DefaultValueTraits<ccMydata::Book*> >&, folly::fbvec
tor<std::shared_ptr<ccMydata::Book>, std::allocator<std::shared_ptr<ccMydata::Book> > >&, long&, std::unordered_map<long, folly::fbvector<std::shared_ptr<ccMydata::BookItem
>, std::allocator<std::shared_ptr<ccMydata::BookItem> > >, std::hash<long>, std::equal_to<lon
g>, std::allocator<std::pair<long const, folly::fbvector<std::shared_ptr<ccMydata::BookItem>,
 std::allocator<std::shared_ptr<ccMydata::BookItem> > > > > >&, std::unordered_set<long, std:
:hash<long>, std::equal_to<long>, std::allocator<long> >&, long&, tf::Executor&, tf::Taskflow&, folly::
fbvector<std::shared_ptr<std::deque<folly::fbvector<std::shared_ptr<ccMydata::BookItem>, std:
:allocator<std::shared_ptr<ccMydata::BookItem> > >, std::allocator<folly::fbvector<std::share
d_ptr<ccMydata::BookItem>, std::allocator<std::shared_ptr<ccMydata::BookItem> > > >
 > >, std::allocator<std::shared_ptr<std::deque<folly::fbvector<std::shared_ptr<ccMydata::BookItem>, std::allocator<std::shared_ptr<ccMydata::BookItem> > >, std::allocator<folly::fbvecto
r<std::shared_ptr<ccMydata::BookItem>, std::allocator<std::shared_ptr<ccMydata::BookItem> > > > > > > >&, unsigned short&)::$_2&) /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../incl
ude/c++/10/bits/invoke.h:110:2 (MyDataPreProcessor+0x4dd52a)

#5 std::_Function_handler<void (), ccMydata::preprocess_record_batch(std::shared_ptr<arrow::Rec
ordBatch>, junction::ConcurrentMap_Leapfrog<long, ccMydata::Book*, junction::DefaultKeyTrai
ts<long>, junction::DefaultValueTraits<ccMydata::Book*> >&, folly::fbvector<std::shared_ptr
<ccMydata::Book>, std::allocator<std::shared_ptr<ccMydata::Book> > >&, long&, s
td::unordered_map<long, folly::fbvector<std::shared_ptr<ccMydata::BookItem>, std::allocator<s
td::shared_ptr<ccMydata::BookItem> > >, std::hash<long>, std::equal_to<long>, std::allocator<
std::pair<long const, folly::fbvector<std::shared_ptr<ccMydata::BookItem>, std::allocator<std
::shared_ptr<ccMydata::BookItem> > > > > >&, std::unordered_set<long, std::hash<long>, std::e
qual_to<long>, std::allocator<long> >&, long&, tf::Executor&, tf::Taskflow&, folly::fbvector<std::share
d_ptr<std::deque<folly::fbvector<std::shared_ptr<ccMydata::BookItem>, std::allocator<std::sha
red_ptr<ccMydata::BookItem> > >, std::allocator<folly::fbvector<std::shared_ptr<ccMydata:
:BookItem>, std::allocator<std::shared_ptr<ccMydata::BookItem> > > > > >, std::allocato
r<std::shared_ptr<std::deque<folly::fbvector<std::shared_ptr<ccMydata::BookItem>, std::alloca
tor<std::shared_ptr<ccMydata::BookItem> > >, std::allocator<folly::fbvector<std::shared_ptr<c
cMydata::BookItem>, std::allocator<std::shared_ptr<ccMydata::BookItem> > > > > > >
>&, unsigned short&)::$_2>::_M_invoke(std::_Any_data const&) /usr/bin/../lib/gcc/x86_64-linux-gnu/10/..
/../../../include/c++/10/bits/std_function.h:291:9 (MyDataPreProcessor+0x4dd52a)
    #6 std::function<void ()>::operator()() const /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../i
nclude/c++/10/bits/std_function.h:622:14 (MyDataPreProcessor+0x50ffd8)
    #7 tf::Executor::_invoke_static_task(tf::Worker&, tf::Node*) /home/xander/dev/vcpkg/installed/x64-l
inux/include/taskflow/core/algorithm/../executor.hpp:818:3 (MyDataPreProcessor+0x50ffd8)
    #8 tf::Executor::_invoke(tf::Worker&, tf::Node*) /home/xander/dev/vcpkg/installed/x64-linux/include
/taskflow/core/algorithm/../executor.hpp:666:7 (MyDataPreProcessor+0x50f3a1)
    #9 tf::Executor::_exploit_task(tf::Worker&, tf::Node*&) /home/xander/dev/vcpkg/installed/x64-linux/
include/taskflow/core/algorithm/../executor.hpp:462:7 (MyDataPreProcessor+0x50e957)
    #10 tf::Executor::_spawn(unsigned long)::'lambda'(tf::Worker&)::operator()(tf::Worker&) const /home
/xander/dev/vcpkg/installed/x64-linux/include/taskflow/core/algorithm/../executor.hpp:392:9 (MyData
PreProcessor+0x50e957)
    #11 void std::__invoke_impl<void, tf::Executor::_spawn(unsigned long)::'lambda'(tf::Worker&), std::
reference_wrapper<tf::Worker> >(std::__invoke_other, tf::Executor::_spawn(unsigned long)::'lambda'(tf::
Worker&)&&, std::reference_wrapper<tf::Worker>&&) /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../i
nclude/c++/10/bits/invoke.h:60:14 (MyDataPreProcessor+0x50e809)
    #12 std::__invoke_result<tf::Executor::_spawn(unsigned long)::'lambda'(tf::Worker&), std::reference
_wrapper<tf::Worker> >::type std::__invoke<tf::Executor::_spawn(unsigned long)::'lambda'(tf::Worker&),
std::reference_wrapper<tf::Worker> >(tf::Executor::_spawn(unsigned long)::'lambda'(tf::Worker&)&&, std:
:reference_wrapper<tf::Worker>&&) /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bi
ts/invoke.h:95:14 (MyDataPreProcessor+0x50e809)
    #13 void std::thread::_Invoker<std::tuple<tf::Executor::_spawn(unsigned long)::'lambda'(tf::Worker&
), std::reference_wrapper<tf::Worker> > >::_M_invoke<0ul, 1ul>(std::_Index_tuple<0ul, 1ul>) /usr/bin/..
/lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/thread:264:13 (MyDataPreProcessor+0x50e809)
    #14 std::thread::_Invoker<std::tuple<tf::Executor::_spawn(unsigned long)::'lambda'(tf::Worker&), st
d::reference_wrapper<tf::Worker> > >::operator()() /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../
include/c++/10/thread:271:11 (MyDataPreProcessor+0x50e809)
    #15 std::thread::_State_impl<std::thread::_Invoker<std::tuple<tf::Executor::_spawn(unsigned long)::
'lambda'(tf::Worker&), std::reference_wrapper<tf::Worker> > > >::_M_run() /usr/bin/../lib/gcc/x86_64-li
nux-gnu/10/../../../../include/c++/10/thread:215:13 (MyDataPreProcessor+0x50e809)
    #16 execute_native_thread_routine /home/conda/feedstock_root/build_artifacts/ctng-compilers_1618239
181388/work/.build/x86_64-conda-linux-gnu/src/gcc/libstdc++-v3/src/c++11/thread.cc:80:18 (libstdc++.so.
6+0xc9066)

 Thread T2 'Quill_Backend' (tid=18392, running) created by main thread at:
    #0 pthread_create <null> (MyDataPreProcessor+0x4361db)
    #1 __gthread_create /home/conda/feedstock_root/build_artifacts/ctng-compilers_1618239181388/work/.b
uild/x86_64-conda-linux-gnu/build/build-cc-gcc-final/x86_64-conda-linux-gnu/libstdc++-v3/include/x86_64
-conda-linux-gnu/bits/gthr-default.h:676:35 (libstdc++.so.6+0xc91f8)
    #2 std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::threa
d::_State> >, void (*)()) /home/conda/feedstock_root/build_artifacts/ctng-compilers_1618239181388/work/
.build/x86_64-conda-linux-gnu/src/gcc/libstdc++-v3/src/c++11/thread.cc:135:37 (libstdc++.so.6+0xc91f8)
    #3 void std::__invoke_impl<void, quill::detail::BackendWorker::run()::'lambda'()>(std::__invoke_oth
er, quill::detail::BackendWorker::run()::'lambda'()&&) /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../..
/../include/c++/10/bits/invoke.h:60:14 (MyDataPreProcessor+0x51a346)
    #4 std::__invoke_result<quill::detail::BackendWorker::run()::'lambda'()>::type std::__invoke<quill:
:detail::BackendWorker::run()::'lambda'()>(quill::detail::BackendWorker::run()::'lambda'()&&) /usr/bin/
../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/invoke.h:95:14 (MyDataPreProcessor+0
x51a346)
    #5 void std::call_once<quill::detail::BackendWorker::run()::'lambda'()>(std::once_flag&, quill::det
ail::BackendWorker::run()::'lambda'()&&)::'lambda'()::operator()() const /usr/bin/../lib/gcc/x86_64-lin
ux-gnu/10/../../../../include/c++/10/mutex:717:4 (MyDataPreProcessor+0x51a346)
    #6 void std::call_once<quill::detail::BackendWorker::run()::'lambda'()>(std::once_flag&, quill::det
ail::BackendWorker::run()::'lambda'()&&)::'lambda0'()::operator()() const /usr/bin/../lib/gcc/x86_64-li
nux-gnu/10/../../../../include/c++/10/mutex:722:25 (MyDataPreProcessor+0x51a346)
    #7 void std::call_once<quill::detail::BackendWorker::run()::'lambda'()>(std::once_flag&, quill::det
ail::BackendWorker::run()::'lambda'()&&)::'lambda0'()::__invoke() /usr/bin/../lib/gcc/x86_64-linux-gnu/
10/../../../../include/c++/10/mutex:722:21 (MyDataPreProcessor+0x51a346)
    #8 pthread_once <null> (MyDataPreProcessor+0x439595)
    #9 __gthread_once(int*, void (*)()) /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/x86
_64-linux-gnu/c++/10/bits/gthr-default.h:700:12 (MyDataPreProcessor+0x51a2e0)
    #10 void std::call_once<quill::detail::BackendWorker::run()::'lambda'()>(std::once_flag&, quill::de
tail::BackendWorker::run()::'lambda'()&&) /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c
++/10/mutex:729:17 (MyDataPreProcessor+0x51a2e0)
    #11 quill::detail::BackendWorker::run() /home/xander/dev/vcpkg/installed/x64-linux/include/quill/de
tail/backend/BackendWorker.h:249:3 (MyDataPreProcessor+0x42ad10)
    #12 quill::detail::LogManager::start_backend_worker(bool, std::initializer_list<int> const&) /home/
xander/dev/vcpkg/installed/x64-linux/include/quill/detail/LogManager.h:150:21 (MyDataPreProcessor+0
x42ad10)
    #13 quill::start(bool, std::initializer_list<int>) /home/xander/dev/vcpkg/installed/x64-linux/inclu
de/quill/Quill.h:60:57 (MyDataPreProcessor+0x42ac55)
    #14 initialize_logging() /home/xander/dev/My_model/pyMydata/pyMydata/PreprocessData.cc:
483:5 (MyDataPreProcessor+0x4dc771)
    #15 main /home/xander/dev/My_model/pyMydata/pyMydata/PreprocessData.cc:490:5 (MyDat
aPreProcessor+0x4dc832)

  Thread T8 (tid=18399, running) created by main thread at:
    #0 pthread_create <null> (MyDataPreProcessor+0x4361db)
    #1 __gthread_create /home/conda/feedstock_root/build_artifacts/ctng-compilers_1618239181388/work/.b
uild/x86_64-conda-linux-gnu/build/build-cc-gcc-final/x86_64-conda-linux-gnu/libstdc++-v3/include/x86_64
-conda-linux-gnu/bits/gthr-default.h:676:35 (libstdc++.so.6+0xc91f8)
    #2 std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::threa
d::_State> >, void (*)()) /home/conda/feedstock_root/build_artifacts/ctng-compilers_1618239181388/work/
.build/x86_64-conda-linux-gnu/src/gcc/libstdc++-v3/src/c++11/thread.cc:135:37 (libstdc++.so.6+0xc91f8)
    #3 tf::Executor::Executor(unsigned long) /home/xander/dev/vcpkg/installed/x64-linux/include/taskflo
w/core/algorithm/../executor.hpp:277:3 (MyDataPreProcessor+0x4e92cf)
    #4 ccMydata::process_incoming_batches(unsigned short&) /home/xander/dev/My_model/pyMyda
ta/pyMydata/PreprocessData.cc:294:22 (MyDataPreProcessor+0x4d54d3)
    #5 main /home/xander/dev/My_model/pyMydata/pyMydata/PreprocessData.cc:493:5 (MyData
PreProcessor+0x4dc858)

SUMMARY: ThreadSanitizer: data race /home/xander/dev/vcpkg/installed/x64-linux/include/quill/detail/sps
c_queue/UnboundedSPSCEventQueue.h:140:27 in quill::detail::UnboundedSPSCEventQueue<quill::detail::BaseE
vent>::try_pop()

opened by xanderdunn 14

Please support local timezone

Thank you for your great work! I love this library.

One thing to prevent using quill is supporting local time zone. Please support local timezone. Currently quill::PatternFormatter::_timezone_type is private and hard coded.
enhancement

opened by toge 14
to support compiling dynamic library

Hi Quill team,

We are currently using your quill logger for our C++ projects which works brilliantly. Thanks.

Now we would like to utilize quill logger for JAVA applications as well by loading it as a dynamic library. In this way, Quill need to dynamically depend on fmt lib as well instead of being bundled together, as we are also using fmt-cpp's dynamic library.

Is that possible you can help to add the support of generating dynamic library as well, together with existing support of static library for quill?

Thanks in advance, Xiong

opened by sandodo 13
Unable to compile with Visual Studio 2019 and vcpkg
Hello,

I don't know if this is the appropriate place for this issue since I don't know who is the maintainer of the vpckg quill package, let me know if I should make a similar issue directly on vcpkg.

Tools used

Visual Studio: v16.9.2

Windows 10: 20h2 (build 19042.746)

vcpkg: 2021-01-13-768d8f95c9e752603d2c5901c7a7c7fbdb08af35

Builds have been done on both x86 and x64, Debug and Release modes.

Code snippet

Nothing unusual here, I reduced the code and the dependencies to the minimum and the problem still persists.

#include <quill/Quill.h> int main() { quill::start(); quill::Logger* logger = quill::get_logger(); // Here is the IDE first error (C++ pointer to incomplete class type is not allowed) logger->set_log_level(quill::LogLevel::Info); LOG_INFO(logger, "A simple test."); return 0; }

What is the problem?

I was looking for a fast and lightweight logger, and quill appeared to me as the perfect library to use. However, I'm working with Visual Studio and there is an annoying issue with vcpkg concerning quill.

When I try to compile, I get a long list of errors, which seem to come from the bundled version of fmt inside of quill. Here is the output of the build (the actual error list of VS is composed of 32 errors, but they're all coming from the fmt integration):

Build started... 1>------ Build started: Project: QuillTest, Configuration: Debug Win32 ------ 1>main.cpp 1>C:\Users\Armillus\Programming\vcpkg\installed\x86-windows\include\quill\Fmt.h(20,12): fatal error C1083: Cannot open include file: 'quill/bundled/fmt/chrono.h': No such file or directory 1>Done building project "QuillTest.vcxproj" -- FAILED. ========== Build: 0 succeeded, 1 failed, 0 up-to-date, 0 skipped ==========

Of course, I can't compile nor run the program, but the IDE isn't recognizing properly the code after quill::Logger* logger = quill::get_logger();. On next line, it complains during static analysis about the logger object (C++ pointer to incomplete class type is not allowed).

What about CMake?

When it comes to CMake, everything is working well, the vcpkg integrations works nicely and I don't have any error. For the sake of accuracy and completeness, here is the CMakeLists.txt that I used.

cmake_minimum_required(VERSION 3.1.0) project(NAlamo) set(CMAKE_CXX_STANDARD 14) find_package(quill CONFIG REQUIRED) add_executable(NAlamo "main.cpp") target_link_libraries(NAlamo PRIVATE quill::quill)

Eventually, this error seems to come from the vcpkg package itself and maybe you'll have an idea about how to fix it. I can still compile with CMake, but I don't use it in the project where I plan to use quill and the IDE integration failure is quite disturbing.

Thank you in advance for your help!
bug
opened by Armillus 10
got an assert error `BaseEvent has a std::chrono timestamp, but the backend thread is using rdtsc timestamp`

Please help. Error message:

quill/detail/BackendWorker.h:396: std::chrono::nanoseconds quill::detail::BackendWorker::_get_real_timestamp(const quill::detail::BaseEvent*) const: Assertion `base_event->using_rdtsc() && "BaseEvent has a std::chrono timestamp, but the backend thread is using rdtsc timestamp"' failed.

opened by zhangzq 9

Cannot compile example_trivial.cpp in Linux

I try to compile example_trivial.cpp in Linux using clang++-9.0.1 and get below compilation errors. Do you have any suggestion?

Thanks, Hung

[ 50%] Building CXX object CMakeFiles/test_quill.dir/test_quill.cpp.o                                                                                                                                                                 
In file included from /home/hungptit/working/cppidioms/src/test_quill.cpp:1:                                                                                                                                                          
In file included from /home/hungptit/working/cppidioms/src/../3p/include/quill/Quill.h:8:                                                                                                                                             
/home/hungptit/working/cppidioms/src/../3p/include/quill/detail/LogMacros.h:39:62: error: token pasting of ',' and                                                                                                                    
      __VA_ARGS__ is a GNU extension [-Werror,-Wgnu-zero-variadic-macro-arguments]                                                                                                                                                    
    QUILL_LOGGER_CALL(logger, quill::LogLevel::TraceL3, fmt, ##__VA_ARGS__)                                                                                                                                                           
                                                             ^                                                                                                                                                                        
/home/hungptit/working/cppidioms/src/../3p/include/quill/detail/LogMacros.h:33:54: error: token pasting of ',' and                                                                                                                    
      __VA_ARGS__ is a GNU extension [-Werror,-Wgnu-zero-variadic-macro-arguments]                                                                                                                                                    
    logger->log<log_statement_level>(&log_line_info, ##__VA_ARGS__)...                                                                                                                                                                
                                                     ^                                                                                                                                                                                
/home/hungptit/working/cppidioms/src/../3p/include/quill/detail/LogMacros.h:46:62: error: token pasting of ',' and                                                                                                                    
      __VA_ARGS__ is a GNU extension [-Werror,-Wgnu-zero-variadic-macro-arguments]                                                                                                                                                    
    QUILL_LOGGER_CALL(logger, quill::LogLevel::TraceL2, fmt, ##__VA_ARGS__)                                                                                                                                                           
                                                             ^                                                                                                                                                                        
/home/hungptit/working/cppidioms/src/../3p/include/quill/detail/LogMacros.h:33:54: error: token pasting of ',' and                                                                                                                    
      __VA_ARGS__ is a GNU extension [-Werror,-Wgnu-zero-variadic-macro-arguments]                                                                                                                                                    
    logger->log<log_statement_level>(&log_line_info, ##__VA_ARGS__)...

And this is the compilation command

{                                                                                                                                                                                                                                     
  "directory": "/home/hungptit/working/cppidioms/src",                                                                                                                                                                                
  "command": "clang++   -I/home/hungptit/working/cppidioms/src/../3p/include -I/home/hungptit/working/cppidioms/src/../src    -O3  -march=native  -std=c++17  -Wall  -pedantic  -pedantic-errors  -fstrict-aliasing  -Wstrict-aliasing  -DFMT_HEADER_ONLY  -DCEREAL_RAPIDJSON_HAS_CXX11_RVALUE_REFS  -flto  -static -O3 -DNDEBUG   -o CMakeFiles/test_quill.dir/test_quill.cpp.o -c /home/hungptit/working/cppidioms/src/test_quill.cpp",                                  
  "file": "/home/hungptit/working/cppidioms/src/test_quill.cpp"                                                                                                                                                                       
},

question

opened by hungptit 9

Release build using nvcc crashes if dual queue mode is activated

I am observing a strange issue: When building a CUDA application using quill in Debug mode everything works fine. As soon as I switch to Release the application crashes with an access violation. The same application works fine, when not building with nvcc, but with MSVC.

It is hard to debug this issue, because the application does not crash in Debug or RelWithDebInfo. Any ideas?

Some more information: OS: Windows 10 MSVC: 19.28.29913.0 NVCC: 11.3.58

Stack when crashing: 0x00007ff67b4d567d 0x00007ff67b4d5045 0x00007ff67b4d366e _configthreadlocale 0x00007ffbc1591bb2 BaseThreadInitThunk 0x00007ffbc21b7034 RtlUserThreadStart 0x00007ffbc3e82651

Attached you find a minimal reproducer:

quill_nvcc.zip

opened by TillAlex 8
Quill::Logger crashes on Windows 2016 and below version of windows

Just started to use Quill:logger and when we are trying to run it on windows 2016 or below versions it wont start. Reason probably is that quill uses API that does not work in those systems.

See bug https://github.com/MicrosoftDocs/visualstudio-docs/issues/4124

Can make this use of SetThreadDescription conditional for those platform or use some other way?

opened by anluoma 8
file_handler/rotating_file_handler with MinGW-32

I'm having problems with the (rotating)_file_handler.

When trying quill::file_handler("my.log", "a") or quill::file_handler(L"my.log", "a") this error occur: Invalid parameter passed to C runtime function. terminate called after throwing an instance of 'quill::QuillError what(): fopen failed with error message errno: "22"

MinGW, 32bit, Windows10

opened by Schoppenglas 8

QUILL_USE_BOUNDED_QUEUE cause assertion error : "Object should always be cache aligned"

Thank you for fast development. I love using this library. When I try to use QUILL_USE_BOUNDED_QUEUE in In quill 1.3.1, I met the assertion error message:

.../include/quill/detail/BoundedSPSCQueue.h:362: quill::detail::BoundedSPSCQueue<TBaseObject, Capacity>::Handle quill::detail::BoundedSPSCQueue<TBaseObject, Capacity>::try_pop() [with TBaseObject = quill::detail::RecordBase; long unsigned int Capacity = 262144]: Assertion `(reinterpret_cast<uintptr_t>(buffer_pos) % CACHELINE_SIZE == 0) && "Object should always be cache aligned"' failed.

Please tell me workaround.

my environment : Fedora 32, gcc 10.1, glibc 2.31 quill version: 1.3.1

opened by toge 8

RotatingFileHandler::_rotate doesn't handle rotation if file is opened in append mode
I am using RotatingFileHandler with append mode (a). This causes the rotation to stop working. Exceptions are:

failed to close previous log file during rotation, with error message errno: "17" fwrite failed with error message errno: "9"

quill::Handler *file_handler = quill::rotating_file_handler(logDirFile, "a", maxFileSize, backupCount);

Problem seems to be that the rotate() function doesn't initialize the _currentIndex variable when booting up. This causes rotation always trying to rename function to logfile.1.log which already exists and rename operation fails.

See the code from rotate function and the comment // then we will always rename the base filename to 1

// if we have more than 2 files we need to start renaming recursively for (uint32_t i = _current_index; i >= 1; --i) { filename_t const previous_file = detail::file_utilities::append_index_to_filename(_filename, i); filename_t const new_file = detail::file_utilities::append_index_to_filename(_filename, i + 1); quill::detail::rename(previous_file, new_file); } // then we will always rename the base filename to 1 filename_t const previous_file = _filename; filename_t const new_file = detail::file_utilities::append_index_to_filename(_filename, 1); quill::detail::rename(previous_file, new_file);

If we open the file in 'w' mode the previous file gets overwritten and log writing is started ok.

Is this by design or can this be corrected?
bug
opened by anluoma 7
PVS Studio Warning

quill/detail/misc/Os.h 122 err V1076 Code contains invisible characters that may alter its logic. Consider enabling the display of invisible characters in the code editor.

This is issue is for your information. I am using v1.7.3

opened by alphanso 0
file event callbacks

Discussed in https://github.com/odygrd/quill/discussions/183

^{Originally posted by jmccabe July 12, 2022} I've just been having a brief look at quill, with the focus on whether it could replace spdlog in our code because there are a couple of things we want to be able to do that, to some extent, are more difficult than we'd hoped with spdlog.

The first thing is that, when using a rotating file handler (or "sink" in spdlog terms), we'd like to log a few lines of "header" info in. Until recently, doing that in spdlog was awkward as there was no indication of when the rotate occurred, and the rotating file sink was a final class, with a private "rotate" function, that meant just deriving and handling the rotate slightly differently wasn't possible. Spdlog's latest version adds support for callbacks that help this to be done. Unfortunately, though, quill (unless I'm missing something) seems to suffer from the same restriction; class rotating_file_handler is private and there are no indications that a rotate is about to happen :-(

The other thing would be to take the formatted text and encode/encrypt it before it gets written to a file. While it looks like it may be possible to work around this, somehow (possibly in the same way you need to with spdlog currently), it would be really nice to be able to configure everything 'normally' then, when you're sure it's all working, just be able to pass a callback in where the formatted log message was passed in, and an encoded version of it was returned which was then stored in the file, instead of the original formatted string.

Are either of these the sort of thing that could be considered for quill?

enhancement

opened by odygrd 1
Quill built with QUILL_FMT_EXTERNAL fails to find system Fmt includes
Hello, I encountered this issue in trying to package Quill for the Homebrew package manager. Right now we build Quill and use its bundled Fmt. Ideally we would use system/external Fmt, to reduce duplication of dependencies and make updating dependencies easier.

When building Quill with the QUILL_FMT_EXTERNAL option, the build and install succeeds. I see that the bundled Fmt is not installed. However, trying to include Quill headers results in:

In file included from /usr/local/Cellar/quill/1.6.3/include/quill/Logger.h:8: /usr/local/Cellar/quill/1.6.3/include/quill/Fmt.h:20:12: fatal error: 'quill/bundled/fmt/chrono.h' file not found #include "quill/bundled/fmt/chrono.h" ^~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1 error generated.

The issue is that when building with QUILL_FMT_EXTERNAL, Fmt headers are located somewhere else. But the Quill headers still expect Fmt to be present in the bundled location. A workaround for now that worked for me is to symlink the Fmt headers folder into the bundled location, but for distributing via package manager, this is not ideal.
opened by alebcay 2
[FR] define custom format with callbacks.
It's commonly used that we will use some callbacks to set the format of the logger. For example:

In some cases, we want to use the user passed timestamp rather than the system.

we want to pass some globally defined information to the logger.

enhancement
opened by sjyMystery 2
Test Issue: FileUtilities append_date_to_filename test expects local timezone to be GMT0

Looks like the FileUtilities test has a time zone issue that causes a false failure in some geo locations. The append_date_to_filename test constructs a system_clock::time_point from epoch seconds(1583376945), and then serializes it to a suffix string for a log file and compares it to an expected result (2020-03-05). The problem is that the input epoch timestamp translates to the date 2020-03-04 in some time zones and 2020-03-05 in others. The test should probably manually override the current timezone setting internally before running the convert and append function.

Nice library folks. I just started looking at it, but it's quite fancy.

opened by blakewoolbright 0
TimeRotatingFileHandler doesn't rotate when constructed

Previously used the DailyFileHandler, recently updated to TimeRotatingFileHandler but notice that our logs no longer rotate.

The application does not run 24/7 but we wanted to store the last 14 days of logs, the previous file handler would create a log file and name it based on the current day. The new one just seems to make a log file with the base name and then not rotate unless its active.

This would essentially render it useless for us in a lot of cases as we will have to manually name our logs per day, and check them ourselves to see if they need to be deleted.

If its intended to run 24/7 then that should be made clearer in the documentation. However it would be nice if when logs were created it also ran the same rotation checks.
enhancement

opened by adwodon 1

Releases(v2.5.1)

v2.5.1(Nov 25, 2022)
Improvements

Reduced the allocations performed by the backend worker thread as the same objects are now being reused rather than destroyed.

Complete summary of changes since v2.3.2

In version 2.3.2 when multiple threads performed heavy logging, the backend logging thread incorrectly gave priority to the logs of the same threads. That made logs from the remaining threads to appear much later or sometimes never in the log files.

There was a series of fixes and releases to address this.

Below is the summary of the changes from v2.3.2

Previously when multiple threads were logging, the backend logging thread would first try to read the log messages of the same thread until its queue was completely empty before reading the log messages of the next thread. When one of the threads was logging a lot, it could result in only displaying the log of that thread, hiding the logs of the other threads. This has now been fixed and all log messages from all threads are read fairly.

Optimise the backend logging thread to read all log messages from each queue. Ensure all queues from all active threads are fairly read.

fmt::basic_memory_buffer buffer stack size has been reduced. The backend thread shows better performance with a reduced stack size. This also reduces the risk of a stack overflow when too many log messages are cached.lllllllllll

Reduced the allocations performed by the backend worker thread as the same objects are now being reused rather than destroyed.

Added a config option backend_thread_strict_log_timestamp_order. This option enables an extra timestamp check on the backend logging thread when each message is popped from the queues. It prevents a rare situation where log messages from different threads could appear in the log file in the wrong order. This flag is now enabled by default.

Added a config option backend_thread_empty_all_queues_before_exit. This option makes the backend logging thread to wait until all the queues are empty before exiting. This ensures no log messages are lost when the application exists. This flag is now enabled by default

Source code(tar.gz)
Source code(zip)
v2.5.0(Nov 25, 2022)
Improvements

Performance improvements for the backend logging thread

Source code(tar.gz)
Source code(zip)
v2.4.2(Nov 20, 2022)
Fixes

Fixes an assertion that was triggered in debug mode due to changes in v2.4.1

Source code(tar.gz)
Source code(zip)
v2.4.1(Nov 20, 2022)
Improvements

Previously the backend worker thread would read all the log messages from the queue but not read the log messages when the buffer had wrapped around. It will now read all the messages.

Removed the min_available_bytes cache from the SPSC queue as an optimisation. It is not needed anymore as we now read all messages at once instead of reading message by message.

Source code(tar.gz)
Source code(zip)
v2.4.0(Nov 19, 2022)
Improvements

Added a config option backend_thread_strict_log_timestamp_order. This option enables an extra timestamp check on the backend logging thread when each message is popped from the queues. It prevents a rare situation where log messages from different threads could appear in the log file in the wrong order. This flag is now enabled by default.

Added a config option backend_thread_empty_all_queues_before_exit. This option makes the backend logging thread to wait until all the queues are empty before exiting. This ensures no log messages are lost when the application exists. This flag is now enabled by default.

Source code(tar.gz)
Source code(zip)
v2.3.4(Nov 19, 2022)
Improvements

Optimise the backend logging thread to read multiple log messages from the same queue, but still read each queue from all active threads.

Source code(tar.gz)
Source code(zip)
v2.3.3(Nov 18, 2022)
Fixes

Previously when multiple threads were loggin, Quill backend logging thread would first try reading the log messages of one thread until the queue was completely empty before reading the log messages of the next thread. When one of the threads was logging a lot, it could result in only displaying the log of that thread, hiding the logs of the other threads. This has now been fixed and all log messages from all threads are read fairly.

Source code(tar.gz)
Source code(zip)
v2.3.2(Oct 27, 2022)
Fixes

Fix code not compiling with treat warnings as errors set on Windows. (#198)

Source code(tar.gz)
Source code(zip)
v2.3.1(Oct 27, 2022)
Fixes

Optimise logging queue cache alignment of variables. It seems that v2.3.0 made the hot path slower by ~5 ns per message. This has been fixed in this version and the performance is now the same as in the previous versions.

Source code(tar.gz)
Source code(zip)

v2.3.0(Oct 26, 2022)

Improvements

Cache the available bytes for reading in the logging queue. This is meant to offer some minor performance improvement to the backend logging thread. #185
Fixed static code analysis and clang '-Wdocumentation' warnings.
The Handler.h API has changed in this version to support structured logs. If you have implemented your own custom Handler you will have to change it to follow the new API.
This version adds support for writing structured logs. Structured logs provide easier search through events. Structured logging is automatically enabled when named arguments are provided to the format string. Structured logs are only supported by the new quill::JsonFileHandler handler. The already existing FileHandler and ConsoleHandler are compatible with named arguments, but they will ignore them and output the log in its original format, as defined by the pattern formatter. Structured logs are not supported for wide characters at the moment. See example_json_structured_log.cpp

For example :

  quill::start();

  quill::Handler* json_handler =
    quill::create_handler<quill::JsonFileHandler>("json_output.log", "w");

  // create another logger tha logs e.g. to stdout and to the json file at the same time
  quill::Logger* logger = quill::create_logger("dual_logger", {quill::stdout_handler(), json_handler});
  for (int i = 2; i < 4; ++i)
  {
    LOG_INFO(logger, "{method} to {endpoint} took {elapsed} ms", "POST", "http://", 10 * i);
  }

Will write to stdout (stdout_handler) :

23:37:19.850432433 [11811] example_json_structured_log.cpp:39 LOG_INFO      dual_logger  - POST to http:// took 20 ms
23:37:19.850440154 [11811] example_json_structured_log.cpp:39 LOG_INFO      dual_logger  - POST to http:// took 30 ms

Will produce a JSON file (json_handler) :

{ "timestamp": "23:37:19.850432433", "file": "example_json_structured_log.cpp", "line": "39", "thread_id": "11811", "logger": "dual_logger", "level": "Info", "message": "{method} to {endpoint} took {elapsed} ms", "method": "POST", "endpoint": "http://", "elapsed": "20" }
{ "timestamp": "23:37:19.850440154", "file": "example_json_structured_log.cpp", "line": "39", "thread_id": "11811", "logger": "dual_logger", "level": "Info", "message": "{method} to {endpoint} took {elapsed} ms", "method": "POST", "endpoint": "http://", "elapsed": "30" }

Source code(tar.gz)
Source code(zip)

v2.2.0(Sep 23, 2022)
v2.2.0

Improvements

Previously storing the default root logger by calling quill::get_logger() followed by quill::configure(cfg) would invalidate the pointer to the default root logger returned by the former function. This has now been fixed and the obtained Logger* pointer is still valid.

Disable fmt::streamed(). (#189)

Update bundled fmt to 9.1.0

logger->should_log(level) is removed. A compile time check was added to logger->should_log<level>() . (#187)

Source code(tar.gz)
Source code(zip)
v2.1.0(Jul 8, 2022)
This version includes breaking changes to the API. Those changes are related to how quill is configured, before calling quill::start() to start the backend thread.

Check the updated examples.

Config.h - contains runtime configuration options

TweakMe.h - contains compile time configuration

For example quill::set_default_logger_handler(...) has been removed. To set a default filehandler :

// create a handler quill::Handler* file_handler = quill::file_handler("test.log", "w"); file_handler->set_pattern( "%(ascii_time) [%(thread)] %(fileline:<28) %(level_name) %(logger_name:<12) - %(message)", "%Y-%m-%d %H:%M:%S.%Qms", quill::Timezone::GmtTime); // set the handler as the default handler for any newly created logger in the config quill::Config cfg; cfg.default_handlers.emplace_back(file_handler); // configure must always be called prior to `start()` quill::configure(cfg); quill::start();

Removed some API functions from Quill.h that were previously used for configuration. Instead, quill::Config object has to be created. For example quill::config::set_backend_thread_cpu_affinity(1); has been removed and instead the following code is needed :

quill::Config cfg; cfg.backend_thread_cpu_affinity = 1; quill::configure(cfg);

QUILL_CHRONO_CLOCK has been moved from TweakMe.h to Config.h. It is now possible to switch between rdtsc and system clocks without re-compiling. See example_trivial_system_clock.cpp

QUILL_RDTSC_RESYNC_INTERVAL has been moved from TweakMe.h to Config.h.

It is now possible to log user timestamps rather than the system's. This feature is useful for time simulations. See example_custom_clock.cpp and example_custom_clock_advanced.cpp

Previously the logger names were limited to a maximum of 22 characters. This limitation has been removed.

Added support for gcc 7.5.0. (#178)

Updated bundled fmt to 9.0.0

Source code(tar.gz)
Source code(zip)
v2.0.2(May 31, 2022)
Fixes

Fix crash when a std::string containing null-terminated characters is passed to the logger. (#176)

Source code(tar.gz)
Source code(zip)
v2.0.1(May 21, 2022)
Improvements

Add a flag to RotatingFileHandler to disable removing the old files when w mode is used.

Source code(tar.gz)
Source code(zip)
v2.0.0(May 18, 2022)
From version v2 and onwards only c++17 is supported.

This version is a major refactor.

Fixes

RotatingFileHandler will now correctly rotate the files when append mode is used (#123)

Improvements

Reduced and simplified codebase.

Improved backend worker thread performance.

QUILL_DUAL_QUEUE_MODE has been removed. A single queue now handles every case.

QUILL_STRING has been removed. That macro is no longer required when passing a format string to the PatternFormatter.

Differences

v1.7 compiles with c++14, v2 only compiles for c++17.

v1.7 on Windows supports wide character logging, v2 has limited wide character support such as logging wchar_t, std::wstring, std::wstring_view. For example, logging std::vector<std::wstring> is not supported.

v1.7 on Windows requires the filepath used for the handlers as a wide strings, v2 supports only filenames as narrow strings.

Source code(tar.gz)
Source code(zip)
v1.7.3(May 15, 2022)
Fix crash on windows when a long wstring (>500 chars) is logged (#173)

Fix compiler error when trying to compile with -DQUILL_DISABLE_NON_PREFIXED_MACROS (#174)

Fix a compile warning in clang (#175)

Source code(tar.gz)
Source code(zip)
v1.7.2(Apr 25, 2022)

Improvements/Fixes

Fix compile error when C++20 is used on windows (https://github.com/odygrd/quill/issues/162)
Source code(tar.gz)
Source code(zip)
v1.7.1(Apr 23, 2022)
Improvements/Fixes

Fix support for wide characters on Windows (#168)

Fix compilation error when Quill::Logger* is stored as a class member in templated classes

Add FilenameAppend::DateTime as an option when creating a file handler

Source code(tar.gz)
Source code(zip)
v1.7.0(Apr 17, 2022)
New Features

Add a new function quill::get_all_loggers() that returns all the existing loggers. (#114)

Add %(level_id) to pattern formatter. (#136)

Users can now specialise copy_loggable<T> to mark user defined types as safe to copy. (#132)

Improvements

Update bundled fmt to 8.1.1

Fix initializations for C++17.

Fix compiler warning in check_format() function.

Replace QUILL_DUAL_QUEUE_MODE with QUILL_DISABLE_DUAL_QUEUE_MODE in TweakMe.h.

Minor performance and accuracy improvements to rdtsc clock used by the backend logging thread.

Fix compile error when C++20 is used. (#162)

Fix get_page_size() to only call sysconf once. (#160)

Previously if quill:start(true) was called more than once in the application, the signal handlers would get initialised again. Now any subsequent calls to quill:start(true) will now have no effect (#167)

Previously when the max limit of rotated files in RotatingFileHandler was reached, quill would stop rotating and instead keep logging everything into the last log file. Now when the maximum limit of files is reached, quill will now keep rotating by replacing the oldest logs. (#157)

Improve the backend logging thread responsiveness when variables are logged in loops without any delay from multiple threads. (#116)

Fix some undefined behaviour issues reported via the AddressSantizer on the backend logging thread. (#166)

Bug Fixes

Fix incorrect timestamps in the log file when the system clock is updated. (#127)

Source code(tar.gz)
Source code(zip)
v1.6.3(Aug 31, 2021)
Added support for %(thread_name) in PatternFormatter. (#97)

Added missing header needed for recent versions of fmt. (#95)

Force flush all active handlers on application exit.

Update bundled fmt to 8.0.1

Source code(tar.gz)
Source code(zip)
v1.6.2(Mar 28, 2021)
Fixes

Fix WIN32 compilation error when NOMINMAX is already defined.

Fix string to wstring MinGW conversion. (#92)

Log enums via the main queue. (#90)

Fix windows compiler error when min/max macros are defined. (#94)

Source code(tar.gz)
Source code(zip)
v1.6.1(Jan 1, 2021)
Fix windows C++20 build. (#83)

Fix ARM build on windows.

Fix example_backtrace and minor bug when destructing with empty backtrace.

Source code(tar.gz)
Source code(zip)
v1.6.0(Dec 3, 2020)
New Features

Dual queue mode offering even lower latencies on hot paths. See Dual Queue Mode.

Added a signal handler for linux and windows. The signal handler flushes the log when the app crashes or exits. (#1)

Added support for custom handlers. (#75)

Quill now compiles and runs on Cygwin.

Improvements/Fixes

The queue from the caller to the backend worker thread has been reworked. The new queue generates slightly better assembly than the previous one. Quill does no longer depend on mapping the same region of physical memory twice.

Replaced an assertion check that could trigger incorrectly. (#68)

Fixed build on ARM_ARCH < 6. (#78)

Fixed compile errors when QUILL_NOEXCEPTIONS, CMAKE_CXX_STANDARD 20, QUILL_USE_BOUNDED_QUEUE are set.

The unit tests have been moved to a separate binary with their own main(). This increased build times when building the tests, but the failures are now easier to debug on different CI platforms and the tests can also run faster in parallel.

Fixed minor compiler warnings on windows.

Upgraded bundled libfmt to 7.1.3

Updated readme benchmarks section for v.1.6.0

Note

If a custom queue capacity is defined using #define QUILL_QUEUE_CAPACITY after 1.6.0 the whole library needs to be recompiled.

Source code(tar.gz)
Source code(zip)
v1.5.2(Nov 7, 2020)
Removed the use of fmt::format() in FileUtilities.cpp as a workaround to the link errors in fmt v7. Use the header only version of libfmt when external libfmt is defiend is no longer required.

Source code(tar.gz)
Source code(zip)
v1.5.1(Nov 7, 2020)
When QUILL_FMT_EXTERNAL is defined, quill will use the header only version of libfmt. This is a workaround to the link errors after libftm v7

Source code(tar.gz)
Source code(zip)
v1.5.0(Nov 7, 2020)
Upgraded bundled libfmt to 7.1.2

Added Filters. The filter class can be used for filtering log records. Filters can be added to handler instances. See example_filters.cpp

It is now possible to set the log level severity on the handler objects. See example_filters.cpp (#49)

Timestamp formatting optimisation for the backend worker thread.

Free list allocator optimisation for the backend worker thread.

Fixed PatternFormatter ignoring a portion of the pattern was ignored, when no format specifiers were present. (#56)

When %(function_name) is used in PatternFormatter the namespace delimiter is replaced from :: to . (Windows only). (#61)

Arguments passed to the logger are no longer being evaluated when the log statement is not logged. (#67)

PatternFormatter enhancement. It is now possible to pass {fmt} string syntax to QUILL_STRING. The default PatternFormatter string has been changed to: "%(ascii_time) [%(thread)] %(fileline:<28) LOG_%(level_name) %(logger_name:<12) - %(message)". This results to the following log being properly aligned despite the different lengths of each filename and logger name.

22:31:07.995438465 [2666041] file1.h:11 LOG_INFO logger1 - Log from file. 22:31:07.995445699 [2666041] long_file2.h:11 LOG_INFO logger_fl2 - Log from other file. 22:31:07.995457144 [2666041] a_longer_file_3.hpp:11 LOG_INFO logger_fl2_l - Log from other file. 22:31:07.995462471 [2666041] example_trivial.cpp:30 LOG_TRACE_L3 root - This is a log trace l3 example 1
Source code(tar.gz)
Source code(zip)
v1.4.1(Sep 18, 2020)
Do not force quill to always build as static library in cmake.

Minor fix when quill is compiled with no exceptions.

Add the option to disable the non prefixed macro definitions if QUILL_DISABLE_NON_PREFIXED_MACROS is defined. (#40)

Source code(tar.gz)
Source code(zip)
v1.4.0(Aug 15, 2020)
Added support for printing colour codes in the terminal. See ConsoleHandler

RotatingFileHandler improvements and minor change in API. See RotatingFileHandler

DailyFileHandler is removed and replaced by TimeRotatingFileHandler. See TimeRotatingFileHandler

Added backtrace logging. Log messages can be stored in a buffer and flushed later on demand. See Backtrace Logging

Added bundled doctest 2.4.0

Migrated all tests from gtest to doctest.

Upgraded bundled libfmt to 7.0.3

Source code(tar.gz)
Source code(zip)
v1.3.3(Jul 31, 2020)
v1.3.3

Upgraded bundled libfmt to 7.0.2

Fixed compile error with libfmt versions > '7.0.0'

Source code(tar.gz)
Source code(zip)
v1.3.2(May 14, 2020)
Add a CMake option QUILL_USE_BOUNDED_QUEUE for bounded queue. (#30)

Fixed a clang 10 warning. (#29)

Fixed MinGw build on Windows. (#31)

Source code(tar.gz)
Source code(zip)