Configuru - Experimental config library for C++

Overview

Configuru

Configuru, an experimental JSON config library for C++, by Emil Ernerfeldt.

License

This software is in the public domain. Where that dedication is not recognized, you are granted a perpetual, irrevocable license to copy and modify this file as you see fit.

That being said, I would appreciate credit! If you find this library useful, send a tweet to @ernerfeldt or mail me at [email protected].

Documentation

You can read the documentation here: https://emilk.github.io/Configuru/html/index.html

Overview

Configuru is a JSON parser/writer for C++11. Configuru was written for human created/edited config files and therefore prioritizes helpful error messages over parse speed.

Goals

  • Debugable:
    • Find typos most config libs miss (like typos in keys).
    • Easily find source of typos with file name, line numbers and helpful error messages.
    • Cleverly help to point out mismatched braces in the right place.
  • Configurable:
    • Configure the format to allow relaxed and extended flavors of JSON.
    • Extensible with custom conversions.
    • Control how the configuru::Config behaves in your code via compile time constants:
      • Override CONFIGURU_ONERROR to add additional debug info (like stack traces) on errors.
      • Override CONFIGURU_ASSERT to use your own asserts.
      • Override CONFIGURU_ON_DANGLING to customize how non-referenced/dangling keys are reported.
      • Set CONFIGURU_IMPLICIT_CONVERSIONS to allow things like float f = some_config;
      • Set CONFIGURU_VALUE_SEMANTICS to have Config behave like a value type rather than a reference type.
  • Easy to use:
    • Smooth C++11 integration for reading and creating config values.
  • JSON compliant:
  • Beautiful output (pretty printing)
  • Reversible with comments:
    • If comments are turned on in FormatOptions they will be parsed and written together with the right key/value.

Non-goals

  • Low overhead

Error messages

Configuru prides itself on great error messages both for parse errors and for value errors (expecting one thing and getting another). All error messages come with file name a line number. When parsing from a string (rather than a file) the user can specify an identifier to be used in lieu of a file name.

Parse errors

equal_in_object.json:1:16: Expected : after object key
{ "is_this_ok" = true }
               ^

bad_escape.json:1:9: Unknown escape character 'x'
{"42":"\x42"}
        ^

no_quotes.json:2:22: Expected value
   "forgotten_quotes": here
                       ^

trucated_key.json:1:2: Unterminated string
{"X
 ^

single_line_comment.json:1:4: Single line comments forbidden.
{} // Blah-bla
   ^

unary_plus.json:1:1: Prefixing numbers with + is forbidden.
+42
^

Note how all errors mention follow the standard filename:line:column structure (most errors above happen on line 1 since they are from small unit tests).

Value errors

Similarly, using a parsed Config value in the wrong way produces nice error messages. Take the following file (config.json):

 1: {
 2: 	"pi":    3.14,
 3: 	"array": [ 1, 2, 3, 4 ],
 4: 	"obj":   {
 5: 		"nested_value": 42
 6: 	}
 7: }

Here's some use errors and their error messages:

auto b = (bool)config["pi"];

config.json:2: Expected bool, got float. Note that the file:line points to where the value is defined.

std::cout << config["obj"]["does_not_exist"];

config.json:4: Failed to find key 'does_not_exist'. Here the file and line of the owner ("obj") of the missing value is referenced.

std::cout << config["pi"][5];

config.json:2: Expected array, got float.

std::cout << config["array"][5];

config.json:3: Array index out of range

Config cfg;
cfg["hello"] = 42;

Expected object, got uninitialized. Did you forget to call Config::object()?. The first line should read Config cfg = Config::object();.

Config cfg;
cfg.push_back("hello");

Expected array, got uninitialized. Did you forget to call Config::array()?. The first line should read Config cfg = Config::array();.

Unused keys

Configuru has a novel mechanism for detecting subtle typos in object keys. Suppose you have a Config that looks like this:

{
	"colour": "red",
	...
}

Here's how it could be used:

auto cfg = configuru::parse_file("config.json", configuru::JSON);
auto color = cfg.get_or("color", DEFAULT_COLOR);
cfg.check_dangling();

The call to check_dangling will print a warning:

config.json:2: Key 'colour' never accessed

This is akin to a compiler warning about unused variables and it's an effective way of finding mistakes that would otherwise go undetected.

The call to check_dangling is recursive, so you only need to call it once for every config file. If you want to mute this warning for some key (which you may intentionally be ignoring, or saving for later) you can call cfg.mark_accessed(true). This will recursively mark each Config as having been accessed.

Usage

For using: #include <configuru.hpp>

And in one .cpp file:

#define CONFIGURU_IMPLEMENTATION 1
#include <configuru.hpp>

Usage (parsing)

Config cfg = configuru::parse_file("input.json", JSON);
float alpha = (float)cfg["alpha"];
std::cout << "alpha = " << alpha << std::endl;
if (cfg.has_key("beta")) {
	std::string beta = (std::string)cfg["beta"];
	std::cout << "beta = " << beta << std::endl;
}
float pi = cfg.get_or("pi", 3.14f);
std::cout << "pi = " << pi << std::endl;

if (cfg["array"].is_array()) {
	std::cout << "array:" << std::endl;
	for (const Config& element : cfg["array"].as_array()) {
		std::cout << "\t" << element << std::endl;
	}
}

std::cout << "object" << std::endl;
for (auto& p : cfg["object"].as_object()) {
	std::cout << "\tKey: " << p.key() << std::endl;
	std::cout << "\tValue: " << p.value() << std::endl;
	p.value() = "new value";
}

try {
	cfg.check_dangling(); // Make sure we haven't forgot reading a key!
} catch (const std::exception &e) {
	std::cerr << e.what() << std::endl;
}
// You can modify the read config:
cfg["message"] = "goodbye";

dump_file("output.json", cfg, JSON);

Usage (writing)

Config cfg = Config::object();
cfg["pi"]     = 3.14;
cfg["array"]  = Config::array{ 1, 2, 3 };
cfg["object"] = Config::object({
	{ "key1", "value1" },
	{ "key2", "value2" },
});

Alternative form:

Config cfg{
	{"pi",     3.14},
	{"array",  Config::array{ 1, 2, 3 }},
	{"object", {
		{ "key1", "value1" },
		{ "key2", "value2" },
	}},
};
std::string json = dump_string(cfg, JSON);
dump_file("output.json", cfg, JSON);

Usage (visit_struct.hpp)

If you include visit_struct.hpp from https://github.com/cbeck88/visit_struct before including configuru.hpp you will enable the following:

#include <visit_struct/visit_struct.hpp>
#include <configuru.hpp>

struct Foo
{
	float bar;
	std::string baz;
};
VISITABLE_STRUCT(Foo, bar, baz);

void error_reporter(std::string str)
{
	std::cerr << str << std::endl; // or throw or ignore
}

int main()
{
	Foo foo{42, "fortytwo"};
	configur::Config cfg = configuru::serialize(foo);
	// Save/load cfg
	configuru::deserialize(&foo, cfg, error_reporter);
}

The serialize/deserialize functions supports numbers, bool, std::string, std::vector and structs annotated with VISITABLE_STRUCT. It is recursive, so a struct can contain an std::vector of other structs if both types of structs are annotated with VISITABLE_STRUCT.

Reference semantics vs value semantics

By default, Config objects acts like reference types, e.g. like a std::shared_ptr:

Config cfg{{"message", "original"}};
auto shallow_copy = cfg;
cfg["message"] = "changed!";
std::cout << shallow_copy["message"]; // Will print "changed!";

auto deep_clone = cfg.deep_clone(); // Deep clones have to be explicit.

You can control this behavior with #define CONFIGURU_VALUE_SEMANTICS 1:

#define CONFIGURU_VALUE_SEMANTICS 1
#include <configuru.hpp>
...
Config cfg{{"message", "original"}};
auto deep_clone = cfg;
cfg["message"] = "changed!";
std::cout << deep_clone["message"]; // Will print "original";

Errors

The default behavior of Configuru is to throw an std::runtime_error on any error. You can change this behavior by overriding CONFIGURU_ONERROR.

CFG format

In addition to JSON, Configuru also has native support for a format I simply call CFG. CFG is a superset of JSON with some simplifications and extensions. Example file:

values: [1 2 3 4 5 6]
object: {
	nested_key: +inf
}
python_style: """This is a string
                 which spans many lines."""
"C# style": @"Also nice for \ and stuff"
  • Top-level can be key-value pairs (no need for {} surrounding entire document).
  • Keys need not be quoted if identifiers.
  • Commas optional for arrays and objects.
  • Trailing , allowed in arrays and objects.

""" starts a verbatim multi-line string

@" starts a C# style verbatim string which ends on next quote (except "" which is a single-quote).

Numbers can be represented in any common form: -42, 1e-32, 0xCAFE, 0b1010

+inf, -inf, +NaN are valid numbers.

Indentation is enforced, and must be done with tabs. Tabs anywhere else is not allowed.

You can also allow selective parts of the above extensions to create your own dialect of JSON. Look at the members of configuru::FormatOptions for details.

Beautiful output

One of the great things about JSON is that it is human readable (as opposed to XML). Configuru goes to great lengths to make the output as beautiful as possible. Here's an example structure (as defined in C++):

Config cfg = Config::object{
	{"float",       3.14f},
	{"double",      3.14},
	{"short_array", Config::array({1, 2, 3})},
	{"long_array",  Config::array({
		"one",
		Config::array({"two", "things"}),
		"three",
	})},
};

Here's how the output turns out in most JSON encoders (this one produced by the excellent nlohmann json library):

{
    "double": 3.14,
    "float": 3.14000010490417,
    "long_array": [
        "one",
        [
            "two",
            "things"
        ],
        "three"
    ],
    "short_array": [
        1,
        2,
        3
    ]
}

In contrast, here's how the output looks in Configuru:

{
	"float":       3.14,
	"double":      3.14,
	"short_array": [ 1, 2, 3 ],
	"long_array":  [
		"one",
		[ "two", "things" ],
		"three"
	]
}

Note how Configuru refrains from unnecessary line breaks on short arrays and does not write superfluous (and ugly!) trailing decimals. Configuru also writes the keys of the objects in the same order as it was given (unless the sort_keys option is explicitly set). The aligned values is just a preference of mine, inspired by how id software does it (turn off with object_align_values=false). Writing the same data in the CFG format makes it turn out like this:

float:       3.14
double:      3.14
short_array: [ 1 2 3 ]
long_array:  [
	"one"
	[ "two" "things" ]
	"three"
]
Issues
  • Disabled MSVC warnings (configuru.hpp)

    Disabled MSVC warnings (configuru.hpp)

    Managed to disable warnings from MSVC.

    Notes:

    1. Some warnings, like warning C4715 cannot be disabled by just setting warning level to 0.
    2. Done some static_casts in order to disable "possible loss of data" warnings to all C++ compilers

    (p.s: It is my first pull request ever.. be easy with me :) )

    opened by thesmallcreeper 2
  • No way to auto deserialize to a struct from disk

    No way to auto deserialize to a struct from disk

    When using visit_struct.hpp to enable auto serialise/deserialise of structs, there's no way to actually deserialise data from disk. If you consider the following:

    configuru::Config config = configuru::Config(rawJsonString); configuru::deserialize(&testStruct, config, errorReporter);

    It will fail with Failed to deserialize object: config is not an object. - since the instance of configuru::Config was constructed from a raw string. So unless I'm missing something, is it not possible to auto deserialise from a raw string?

    opened by lawmaestro 1
  • Compile on VS2015

    Compile on VS2015

    Big thanks for making this project & sharing it with the rest of us.

    I'm working on Windows and I've managed to make this compile with VS2015. The changes are actually very limited.

    First, VC++ doesn't recognize __attribute so __attribute__((noreturn)) cannot be used. The alternative is __declspec(noreturn) but it requires to be placed at the beginning for declaration like this:

    __declspec(noreturn) extern void fatal () {}  
    

    Second and isn't recognized as keyword so we need to use &&.

    And last there's min/max macro defined somewhere in windows.h and causing name collision.

    My commit is https://github.com/skyline75489/Configuru/commit/777795ee3bd54df268ab5aca758ef64ca43eaa7e

    If it looks OK to u I can send you a PR. 🥂

    opened by skyline75489 1
  • handling non existing gets to defaut zero cathing throw

    handling non existing gets to defaut zero cathing throw

    define getjs2(op,jsout){try{op=(decltype(op))( jsout );}catch(...){op=0;}}

    ex float op; getjs2(op,someconfiguru,["one"]["two"]);

    You could implement this better with an = method operator

    Cheers and thanks

    opened by superbem 1
  • huge parse_string memory leak

    huge parse_string memory leak

    Hi, I'm using Config cfgs = configuru::parse_string(str.c_str(), JSON,""); on a huge string and after I used it, it keeps on memory after exiting the stack. Is there a way to solve this? Thanks

    opened by vivision1 0
  • Feature Request: Enum Type Serialization/Deserialization

    Feature Request: Enum Type Serialization/Deserialization

    When using VISITABLE STRUCT, it would be nice to be able to serialize/deserialize enum types. Right now they are excluded from the serialization, so they are not available in the JSON.

    I think it's acceptable to just serialize these down to an integer type.

    Thoughts?

    opened by psyklopz 0
  • Move implementation into cpp

    Move implementation into cpp

    Addresses issue #9

    When Configuru is included in separate translation units, it can result in LNK2005 errors on MSVC. GCC seems to handle this fine, but it is still a violation of the One Definition Rule, where it appears Configuru is still built into multiple translation units.

    This commit is similar to that done on Loguru two years ago.

    I have shown this works successfully in two different projects that utilize Configuru in separate units. One uses two different configuration files. The other uses Configuru for a config file and then some simple JSON formatting for a web API.

    I don't use Boost and really didn't feel like messing with getting that to work. So I was able to build the test suite (using std::experimental::filesystem instead of boost::filesystem) and confirm the tests still all pass. I've not included this in the merge request, but if interested the code that works with std::experimental::filesystem can be seen on the separate_into_hpp_and_cpp branch of psyklopz/Configuru.

    opened by psyklopz 1
  • Saving to test file, file contents all NULL bytes

    Saving to test file, file contents all NULL bytes

    CMake Build System, targeting C++ 17, MSVC 2017 Community, building a 32-bit DLL, running on Windows 10

    I've observed this twice on production systems, and I know it sounds a little outlandish.

    I have a multi-threaded application which has a Config class. This class reads the configuration file into member variables and then exposes those to the rest of the application with get/set-style accessors. It also provides a method to save the configuration file. All methods are accessed from different threads, so thread safety is a concern. Because of this, I wrap all of the set accessors and the Save() method with a std::lock_guard, using a shared std::mutex. The get accessors are not mutex-protected. (The read of the config is part of the constructor, and happens in single-threaded context.) I think this is sufficient thread-protection.

    The system will sometimes (once every week) write out a config file of the expected length (4.7k, give or take). Except this config file is entirely NULL bytes (0x00). I can see this by viewing the file in a hex editor.

    I don't quite understand how this could happen, because Configuru converts the object to a std::string before writing that to a file. If that string contained only nulls, I would expect to get a zero-length file, because it would stop at the first null-terminator. However, that does not appear to be the case. https://stackoverflow.com/questions/2845769/can-a-stdstring-contain-embedded-nulls

    Any ideas? Have you heard of anything like this before?

    opened by psyklopz 2
  • The

    The "free" private method interferes with _CRTDBG_MAP_ALLOC

    Visual Studio allows for the _CRTDBG_MAP_ALLOC macro to be defined, which replaces malloc and free with Windows' own macros for detecting memory leaks. As the preprocessor attempts to replace the free word in void Config::free(), this results in an error. What I personally did was rename the method to void Config::do_free().

    opened by flriancu 0
Releases(v0.4.1)
Owner
Emil Ernerfeldt
Rust coder at Embark Studios. Previously worked with physics simulation, games, 3D scanning and cyber security. I am @ernerfeldt on Twitter
Emil Ernerfeldt
json file config for ESP8266 arduino framework

Description Every project I have is the same architecture and appconf is one of the components I use every time. This is an attempt at making a public

Garkusnko Nick 1 Nov 9, 2021
dwm config made with flexipatch

dwm Gruvbox One Dark eww Gruvbox One Dark some info dwm flexipatch - https://github.com/bakkeby/dwm-flexipatch st flexipatch - https://github.com/bakk

Nikita 26 May 23, 2022
Dotfiles for my qtile config

Dotfiles Here's the dotfiles for my Qtile config, they're a bit quick-and-dirty but I hope they'll do. Please tell me if i forgot anything, I'll try t

null 4 Jun 5, 2022
Config files for my GitHub profile.

Files to build the PicoMite. MMBasic running on the Raspberry Pi Pico NB: This should be built against pico-sdk version 1.3. Previous versions were bu

null 82 Jul 22, 2022
Config files for my GitHub profile.

#include <stdio.h> #include <math.h> #include <stdlib.h> int main() { // declaration of local variable op; int op, n1, n2; float res; char ch; do { //

null 1 Dec 22, 2021
Patterns - This is an experimental library that has evolved to P1371, proposed for C++23.

MPark.Patterns This is an experimental library that has evolved to P1371, being proposed for C++23. Introduction MPark.Patterns is an experimental pat

Michael Park 566 Aug 9, 2022
[WIP] Experimental C++14 multithreaded compile-time entity-component-system library.

ecst Experimental & work-in-progress C++14 multithreaded compile-time Entity-Component-System header-only library. Overview Successful development of

Vittorio Romeo 445 Jul 22, 2022
Experimental telegram client based on official Android sources

Catogram Experimental telegram client based on official Android sources Catogram features: Message translator TGX Style of context menu VKUI Icons and

null 179 Jul 28, 2022
An experimental tool to estimate the similarity between all pairs of contigs

This is an experimental tool to estimate the approximate distances between all pairs of unitigs. It takes a GFA or FASTA file as input and outputs a T

Heng Li 33 Mar 16, 2022
This is an experimental OS-from-scratch project. Just for demonstration, not useful at all.

OS Playground This is an experimental OS-from-scratch project. Just for demonstration, not useful at all. Different from OS in other projects, this OS

null 5 Dec 5, 2021
The Express LRS Handset project is first and foremost an experimental test bed for pushing the envelope with what can be done with the ELRS radio link

The Express LRS Handset project is first and foremost an experimental test bed for pushing the envelope with what can be done with the ELRS radio link

ExpressLRS 21 May 23, 2022
An experimental dynamic malware unpacker based on Intel Pin and PE-sieve

Pin'n'Sieve A dynamic malware unpacker based on Intel Pin and PE-sieve (deploys PE-sieve scan on specific triggers). Caution: during the process the m

hasherezade 50 Jun 10, 2022
codeless Android hook (experimental)

AppInspect Download app-inspect-v0.0.1.zip AppInspect-0.0.1.apk Install: install Riru module adb push app-inspect-v0.0.1.zip /data/local/tmp adb shel

null 54 Jun 21, 2022
An experimental operating system for x86 and ARM

Odyssey - an experimental operating system for x86 and ARM

Anuradha Weeraman 34 Jul 5, 2022
Implementation of the (not yet written) std::experimental::rational proposal.

Rational Implementation of the (not yet written) std::experimental::rational proposal. Getting started Copy include/std/experimental/rational.hpp to y

Ali Can Demiralp 9 Feb 6, 2022
Avian is an experimental digital currency that enables instant payments to anyone, anywhere in the world.

Avian Network [AVN] What is Avian? Avian is an experimental digital currency that enables instant payments to anyone, anywhere in the world. Avian use

null 41 Jul 29, 2022
Vaca - An experimental Win32 wrapper for C++ to develop GUI programs

Vaca Visual Application Components Abstraction Copyright (c) 2005-2012 David Capello All rights reserved. Vaca is a library to develop GUI application

David Capello 61 Jul 26, 2022
An experimental sprite rendering setup utilizing SSBO's, Threading, EnTT reactive systems, and array-textures based sprite caching.

entt-reactive An experimental sprite rendering setup utilizing pooled SSBO's, a multithreaded setup based on Even Todd's The Poor Man's Threading Arch

Jackie Fuchs 7 Apr 29, 2022
Experimental Worms Armageddon WormKit module that implements real-time online multiplayer for racing schemes

wkRealTime v.0.0.4d Experimental Worms Armageddon WormKit module that implements real-time online multiplayer for racing schemes. Installation Place w

null 15 Jul 12, 2022