Fast JSON serialization and parsing in C++

Overview

logo image

DAW JSON Link v2

Build Status Macos

Build Status Ubuntu

Build Status Windows - MSVC

Content

Intro

Top

The DAW JSON Link library provides multiple ways to serialization/deserialization JSON documents in C++. The primary one is parsing of JSON directly to your C++ data structures. This allows the known structure of the document to be exploited for greater checking and performance. Alternatively, there is an event passing(SAX) interface that can parse to generic types(double, string, bool,...) or can use the same type restricted parsers as the static parser previously mentioned. A generic DOM(lazy) based parser is provided that can be iterate over the document structure too, again it can use the generic parsers or the type based restricted versions. One can mix the three modes of parsing to form more complicated systems. For serialization, the first static mapping method is required, there is no json value type in the library. The library is, also, non-intrusive into your data structures and does not require member's to be declared/defined within them. This allows keeping the mapping in a separate header file from the data structures themselves.

The library is using the BSL licensed

When the structure of the JSON document is known, parsing is like the following:

MyThing thing = daw::json::from_json<MyThing>( json_string );

or for array documents, where the root of the document is an array, there is a helper method to make it easier and it can be parsed like the following:

std::vector<MyThing> things = daw::json::from_json_array<MyThing>( json_string2 );

If the structure of the JSON document is unknown, one can construct a json_value that acts as a container and allows iteration and parsing on demand. It is a lazy parser and will only parse when asked to. The following is an example of opening a json_value from JSON data:

json_value val = daw::json::json_value( json_string );

The from_json and to_json methods allow access most of the parsing needs.

The event based parser(SAX) can be called via daw::json::json_event_parser. It takes two arguments, a json document and an event handler. The event handler can opt into events by having the following members:

  • handle_on_value
  • handle_on_array_start
  • handle_on_array_end
  • handle_on_class_start
  • handle_on_class_end
  • handle_on_number
  • handle_on_bool
  • handle_on_string
  • handle_on_null
  • handle_on_error

Code Examples

Mapping of your classes to JSON documents is done by specializing the trait daw::json::json_data_contract. A class that is mapped does not need to be mapped again if it is a member of another mapped class. There are two parts to the trait json_data_contract, first is a type alias named type that maps the JSON members to our class's constructor. This gets around needing private access to the class, assuming that data we would serialize would also be needed to construct the class. For example:

struct Thing {
 int a;
 int b;	
};

The construct for Thing requires 2 integers and if we had the following JSON:

{
  "a": 42,
  "b": 1234
}

We could do the mapping like the following:

namespace daw::json {
  template<>
  struct json_data_contract<Thing> {
  	static constexpr char const a[] = "a";
  	static constexpr char const b[] = "b";
  	using type = json_member_list<
  	  json_number<a, int>,
  	  json_number<b, int>
  	>;
  };
}

This says that the JSON class will have at least two members "a", and "b" that will be numbers that are integers. They will be passed to the constructor of Thing when daw::json::from_json<Thing>( json_doc ); is called, or that another class has a json_class<MemberName, Thing> member mapping. The above is the C++17 mapping method for the names, it works in future C++ versions too. But, in C++20 and later the names can be inline in the mapping e.g. json_number<"a", int>. The above is all that is needed for parsing JSON, for serializing a static member function is needed in the trait. Taking the previous example and extending it we could serialize Thing with:

namespace daw::json {
  template<>
  struct json_data_contract<Thing> {
  	static constexpr char const a[] = "a";
  	static constexpr char const b[] = "b";
  	using type = json_member_list<
  	  json_number<a, int>,
  	  json_number<b, int>
  	>;
  };
  
  static auto to_json_data( Thing const & v ) {
    return std::forward_as_tuple( v.a, v.b );
  }
}

The ordering of the members returned as a tuple need to match the mapping in the type alias type. This allows for passing the result of accessor methods too, if the data members are not public.

  • Note: The return type of to_json_data does not have to return a tuple of references to the existing object members, but can return calculated values too.

The parsers work by constructing each argument in place in the call to the classes constructor. The individual argument parsers can be tuned for the specified circumstances of the data(e.g. floating point and integral numbers). Then with our type trait defining the arguments needed to construct the C++ class and their order we are able to look at each member in the JSON. Now we construct the value with the result of each parser; similar to T{ parse<0, json_string<"name">>( data ), parse<1, json_number<"age", unsigned>>( data ), parse<json_number<2, "number>>( data )}. For each member, the data stream will be moved forward until we find the member we need to parse, storing interested locations for later parsing. This process allows us to parse other classes as members too via the json_class<"member_name", Type> mapping type. So that each mapping trait only has to deal with it's specific members and not their details. general parsing flow

Default mapping of types

Top

In unnamed contexts, such as the root value, array elements, some key value types, and variant element lists where the name would be no_name, one can use some native C++ data types instead of the the JSON mapping types. This includes, integer, floating point, bool, std::string, std::string_view, and previously mapped classes.

For example, to map an array of string's.

template<>
struct daw::json::json_data_contract<MyType> {
  using type = json_member_list<json_array<"member_name", std::string>>;
};

Installing/Using

Top

Including in cmake project

To use daw_json_link in your cmake projects, adding the following should allow it to pull it in along with the dependencies:

include( FetchContent )
FetchContent_Declare(
        daw_json_link
        GIT_REPOSITORY https://github.com/beached/daw_json_link
				GIT_TAG release
)
FetchContent_MakeAvailable(daw_json_link)

Then in the targets that need it:

target_link_libraries( MyTarget daw::json_link )

Installing

On a system with bash, it is similar on other systems too, the following can install for the system

git clone https://github.com/beached/daw_json_link
cd daw_json_link
mkdir build
cd build
cmake ..
cmake --install . 

Testing

The following will build and run the tests.

git clone https://github.com/beached/daw_json_link
cd daw_json_link
mkdir build
cd build
cmake -DDAW_ENABLE_TESTING=On ..
cmake --build . 
ctest .

After the build there the individual examples can be tested too. city_test_bin requires the path to the cities JSON file.

./tests/city_test_bin ../test_data/cities.json

Performance considerations

Top

The order of the members in the data structures should generally match that of the JSON data. The parser is faster if it doesn't have to back track for values. Optional values, when missing in the JSON data, can slow down the parsing too. If possible have them sent as null. The parser does not allocate. The parsed to data types may and this allows one to use custom allocators or a mix as their data structures will do the allocation. The defaults for arrays is to use the std::vector and if this isn't desirable, you must supply the type.

Benchmarks

chart desribing kostya benmark results

Escaping/Unescaping of member names

Top

The library, currently, does not unescape/escape member names when serializing, they are expected to be valid and unescaped. This may be a future optional addition, as it does have a cost.

Differences between C++17 and C++20

Top

There are slight differences between C++17 and C++20

Naming of JSON members

namespace daw::json {
  template<>
  struct json_data_contract<MyType> {
    static constexpr char const member_name[] = "memberName";
    using type = json_member_list<json_number<member_name>>;
  };
}

C++ 20 Naming of JSON members

When compiled within C++20 compiler, in addition to passing a char const * as in C++17, the member names can be specified as string literals directly. C++20 compiler support is still really early and here be dragons. There are known issues with g++9.x and it's only tested with g++10. Here be dragons

namespace daw::json {
  template<>
  struct json_data_contract<MyType> {
    using type = json_member_list<json_number<"member_name">>;
  };
}

Using mapped data types

Top

Once a data type has been mapped with a json_data_contract, the library provides methods to parse JSON to them

MyClass my_class = from_json<MyClass>( json_str );

Alternatively, if the input is trusted, the less checked version can be faster

MyClass my_class = from_json<MyClass, NoCommentSkippingPolicyUnchecked>( json_str );

JSON documents with array root's use the from_json_array function to parse

std::vector<MyClass> my_data = from_json_array<MyClass>( json_str );

Alternatively, if the input is trusted, the less checked version can be faster

std::vector<MyClass> my_data = from_json_array<MyClass, std::vector<MyClass>, NoCommentSkippingPolicyUnchecked>( json_str );

If you want to work from JSON array data you can get an iterator and use the std algorithms to Iterating over array's in JSON data can be done via the json_array_iterator

using iterator_t = json_array_iterator<MyClass>;
auto pos = std::find( iterator_t( json_str ), iterator_t( ), MyClass( ... ) );

Alternatively, if the input is trusted you can called the less checked version

using iterator_t = daw::json::json_array_iterator_trusted<MyClass>;
auto pos = std::find( iterator_t( json_str ), iterator_t( ), MyClass( ... ) );

If you want to serialize to JSON

std::string my_json_data = to_json( MyClass{} );

Or serialize a collection of things

std::vector<MyClass> arry = ...;
std::string my_json_data = to_json_array( arry );

Error Handling

Parsing call

Top

Error checking can be modified on a per parse basis. the from_json/from_json_array calls can be supplied a Parser Policy. The current policies are

  • NoCommentSkippingPolicyChecked - No comments allowed, checks enabled
  • NoCommentSkippingPolicyUnchecked - No comments allowed, assumes perfect JSON
  • CppCommentSkippingPolicyChecked - C++ style comments /* commment */ and // comment until end of line, checks enabled
  • CppCommentSkippingPolicyUnchecked - C++ style comments /* commment */ and // comment until end of line, assumes perfect JSON
  • HashCommentSkippingPolicyChecked - Hash style comments # comment until end of line, checks enabled
  • HashCommentSkippingPolicyUnchecked - Hash style comments # comment until end of line, assumes perfect JSON

The unchecked variants can sometimes provide a 5-15% performance increase, but at great risk when the data isn't perfect.

Global

Top

There are two possible ways of handling errors. The default is to throw a daw::json::json_exception on an error in the data. json_exception has a member function std::string_view reason( ) const akin to std::exception's what( ). Second, calling std::terminate( ); on an error in data. If you want to disable exceptions in an environment that has them, you can defined DAW_JSON_DONT_USE_EXCEPTIONS to disable exception throwing by the library.

Deserializing/Parsing

Top

This can be accomplished by writing a function called json_data_contract_for with a single argument that is your type. The library is only concerned with it's return value. For example:

#include <daw/json/daw_json_link.h>

struct TestClass {
  int i = 0;
  double d = 0.0;
  bool b = false;
  daw::string_view s{};
  std::vector<int> y{};

  TestClass( int Int, double Double, bool Bool, daw::string_view S, std::vector<int> Y ) 
    : i( Int )
    , d( Double )
    , b( Bool )
    , s( S )
    , y( Y ) {}
};

namespace daw::json {
  template<>
  struct json_data_contract<TestClass> {
  using type = json_member_list<
    json_number<"i", int>,
    json_number<"d">,
    json_bool<"b">,
    json_string<"s", daw::string_view>,
    json_array<"y", int>
   >;
  };
}

int main( ) {
  std::string test_001_t_json_data = R"({
    "i":5,
    "d":2.2e4,
    "b":false,
    "s":"hello world",
    "y":[1,2,3,4] 
    })";
  std::string json_array_data = R"([{
    "i":5,
    "d":2.2e4,
    "b":false,
    "s":"hello world",
    "y":[1,2,3,4] 
    },{
    "i":4,
    "d":122e4,
    "b":true,
    "s":"goodbye world",
    "y":[4,3,1,4] 
    }])";

  TestClass test_class = daw::json::from_json<TestClass>( test_001_t_json_data );
  std::vector<TestClass> arry_of_test_class = daw::json::from_json_array<TestClass>( test_001_t_json_data );
}

Both aggregate and user constructors are supported. The description provides the values needed to construct your type and the order. The order specified is the order they are placed into the constructor. There are customization points to provide a way of constructing your type too(TODO discuss customization points) A class like:

#include <daw/json/daw_json_link.h>

struct AggClass {
  int a{};
  double b{};
};

namespace daw::json {
  template<>
  struct json_data_contract<AggClass> {
    using type = json_member_list<
      json_number<"a", int>,
      json_number<"b">
    >;
  };
}

Works too. Same but C++17

#include <daw/json/daw_json_link.h>

struct AggClass {
  int a{};
  double b{};
};

namespace daw::json {
  template<>
  struct json_data_contract<AggClass> {
    static inline constexpr char const a[] = "a";
    static inline constexpr char const b[] = "b";
    using type = json_member_list<
      json_number<a, int>,
      json_number<b>
    >;
  };
}

The class descriptions are recursive with their submembers. Using the previous AggClass one can include it as a member of another class

// See above for AggClass
struct MyClass {
  AggClass other;
  std::string_view some_name;
};

namespace daw::json {
  template<>
  struct json_data_contract<MyClass> {
    using type = json_member_list<
      json_class<"other", AggClass>,
      json_string<"id", std::string_view>
    >;
  };
}

The above maps a class MyClass that has another class that is described AggClass. Also, you can see that the member names of the C++ class do not have to match that of the mapped JSON names and that strings can use std::string_view as the result type. This is an important performance enhancement if you can guarantee the buffer containing the JSON file will exist as long as the class does.

Iterating over JSON arrays. The input iterator daw::json::json_array_iterator<JsonElement> allows one to iterator over the array of JSON elements. It is technically an input iterator but can be stored and reused like a forward iterator. It does not return a reference but a value.

#include <daw/json/daw_json_link.h>

struct AggClass {
  int a{};
  double b{};
};

namespace daw::json {
  template<>
  struct json_data_contract<AggClass> {
    using type = json_member_list<
      json_number<"a", int>,
      json_number<"b">
    >;
  };
}

int main( ) {
  std::string json_array_data = R"([
    {"a":5,"b":2.2},
    {"a":5,"b":3.14},
    {"a":5,"b":0.122e44},
    {"a":5334,"b":34342.2}
     ])";
  using iterator_t = daw::json::json_array_iterator<AggClass>;
  auto pos = std::find_if( iterator_t( json_array_data ), iterator_t( ),
    []( AggData const & element ) { return element.b > 1000.0; } );
  if( pos == iterator_t( ) ) {
    std::cout << "Not found\n";
  } else {
    std::cout << "Found\n";
  }
}

Member Paths

Parsing can begin at a specific member. An optional member path to from_json_array, from_json_array_unchecked, from_json_array, or from_json_array_unchecked can be specified. The format is a dot separated list of member names and optionally an array index such as member0.member1 or member0[5].member1.

Comments

Comments are supported when the parser policy for them is used. Currently there are two forms of comment policies. C++ style // and /* */. Comments can be placed anywhere there is whitespace allowed

  • Hash style
{ # This is a comment
    "a" #this is also a comment
      : "a's value"
}
  • C++ style
{ // This is a comment
    "a" /*this is also a comment*/: "a's value"
}

To change the parser policy, you add another argument to from_json and call like from_json<MyType, CppCommentParsePolicy>( json_data )

Serialization

Top

To enable serialization one must create an additional function in your specialization of json_data_contract called to_json_data( Thing const & ); It will provide a mapping from your type to the arguments provided in the class description. To serialize to a JSON string, one calls to_json( my_thing ); where value is a registered type or one of the fundamental types like string, bool, and numbers. The result of to_json_data( Thing const & ) is a tuple who's elements match order in json_data_contract's type alias type. Using the example above lets add that

#include <daw/json/daw_json_link.h>
#include <tuple>

struct AggClass {
  int a{};
  double b{};
};

namespace daw::json {
  template<>
  struct json_data_contract<AggClass> {
    using type = json_member_list<
      json_number<"a", int>,
      json_number<"b">
    >;

    static inline auto to_json_data( AggClass const & value ) {
      return std::forward_as_tuple( value.a, value.b );
    }
  };
}
//...
AggData value = //...;
std::string test_001_t_json_data = to_json( value );

// or
std::vector<AggData> values = //...;
std::string json_array_data = to_json_array( values );

Alternatively there is an optional iostreams interface. In you types json_data_constract add a type alias named opt_into_iostreams the type it aliases doesn't matter, and include daw_json_iostream.h . For example

#include <daw/json/daw_json_link.h>
#include <daw/json/daw_json_iostream.h>
#include <tuple>

struct AggClass {
  int a{};
  double b{};
};

namespace daw::json {
  template<>
  struct json_data_contract<AggClass> {
    using opt_into_iostreams = void;
    using type = json_member_list<
      json_number<"a", int>,
      json_number<"b">
    >;

    static inline auto to_json_data( AggClass const & value ) {
      return std::forward_as_tuple( value.a, value.b );
    }
  };
}
//...
AggData value = //...;
std::cout << value << '\n';

// or
std::vector<AggData> values = //...;
std::cout << values << '\n';

A working example can be found at daw_json_iostream_test.cpp

Build configuration points

There are a few defines that affect how JSON Link operates

  • DAW_JSON_DONT_USE_EXCEPTIONS - Controls if exceptions are allowed. If they are not, an std::terminate() on errors will occur
  • DAW_ALLOW_SSE42 - Allow experimental SSE3 mode
  • DAW_JSON_NO_CONST_EXPR - This can be used to allow classes without move/copy special members to be constructed from JSON data prior to C++ 20. This mode does not work in a constant expression prior to C++20 when this flag is no longer needed.

Requirements

Top
  • C++ 17 compiler
  • GCC(8/9)/Clang(7/8/9/10) have been tested.
  • MSVC 19.21 has been tested.

For building tests

  • git
  • cmake

Contact

Darrell Wright [email protected]

Limitations

  • When parsing classes, the first member with a mapped name will be used. If you want to parse a class that can have more than one of any member by name, either parse as a json_value see or as a json_key_value that is mapped to a std::multimap or a std::vector with a pair of key type(string) and value type(s). Cookbook Key Values demonstrates these methods. If a json_key_value is used and the mapped data type does not support duplicate keys, it will insert for each key. This may result in the last item being the value reflected after serializing. If the duplicate member is the tag type in a json_tagged_variant, it is undefined what the behaviour for parsing is.
Issues
  • Outdated(?) readme and missing checks in

    Outdated(?) readme and missing checks in "checked" parsing

    The Readme mentions a difference between checked (from_json) and unchecked (from_json_unchecked) calls https://github.com/beached/daw_json_link#using-mapped-data-types . The latter don't seem to exist (anymore?). Instead the unchecked version should probably be daw::json::from_json<MyClass, daw::json::NoCommentSkippingPolicyUnchecked>, right?

    When playing around with this, I was very surprised to find, that the "checked" version does not in fact check whether the input is valid json at all. Unused invalid numbers (even very blatantly wrong numbers {"a":1.0fsdf3, ...}) and especially unterminated json is not reported if all required keys have been found before the end of the string. (A behavior I expected for the trusted / "unchecked" version, but not for the "checked" version)

    Is there any way to add these checks or to have a FullyChecked version or something like that, that checks whether the input is valid json?

    Some context: When receiving json via websockets we occasionally receive faulty json strings that consist of a valid json appended to the incomplete beginning of another one (see example below). They are very obviously invalid - but relying on daw jsons parsing we might miss these issues and treat them as valid instead.

    Below code does not throw an exception:

    std::string test = R"({"e":"aggTrade","E":1610729466077,"s":"BTCUSDT","a":516119850,"p":"36062.5500{"e":"aggTrade","E":1610732218225,"s":"BTCUSDT","a":516218642,"p":"35943.45000000","q":"0.08800100","f":574701726,"l":574701727,"T":1610732218224,"m":false,"M":true})";
    std::cout << daw::json::from_json<int, daw::json::NoCommentSkippingPolicyChecked>(test, "a") << std::endl;
    
    bug question 
    opened by fuchsi-fuchs 19
  • Vcpkg port in progress

    Vcpkg port in progress

    I'm trying to get daw_json_link into vcpkg to make it easier to use. See https://github.com/microsoft/vcpkg/pull/18411

    If you are not familiar with vcpkg, it makes it almost rust-simple to include dependencies in your CMake build (you need to have a manifest file holding your program's dependencies and use the vcpkg build tooling) but it does have some quirks. One of the quirks is that the port names must be lower case and cannot have underscores. Another is that when trying your best to build across multiple architectures, you sometimes run into incompatibilities that you don't expect.

    So, the first change is the name of the package you use with CMake's find_package(). They are changed to daw::daw-header-libraries, daw::daw-utf-range, and daw::daw-json-link. I think I could have kept the same CMake package names but this was easier (because of some vcpkg-related automated CMake package fixup). Plus this style is more expected for vcpkg ports ;-)

    The way vpkg builds work, I had to actually make 3 ports, daw-header-libraries, daw-utf-range, and daw-json-link with the proper dependencies...The vcpkg automated build system didn't like CMake's FetchContent_Declare() on, I think, Linux. So, I patched those calls out of the build and used vcpkg's dependency mechanism to get the same results.

    The last issues I ran into was that utf_range includes source from the same place as the vcpkg utfcpp port. To ameliorate any conflicts that could arise from somebody wanting both daw-json-link and utfcpp, I moved the utf8 namespace daw_json_link uses under daw::utf8 and also moved the files under the include/daw directory.

    Vcpkg doesn't have dragonbox yet so I left that as it was but perhaps I shouldn't have...

    You can take a look at the patch files in the 3 port directories I linked in this message to see the changes.

    Thanks for a stellar JSON library - I cannot wait to see what you do with reflection :-)

    opened by mheyman 15
  • Update to_chars to use custom JSON specific one

    Update to_chars to use custom JSON specific one

    https://github.com/jk-jeon/dragonbox
    The benchmarks show better conversion times for the general (smaller number of digits) case compared to Ryu.

    I'm in the process of assisting moving it to CMake (https://github.com/jk-jeon/dragonbox/issues/4), so maybe this could be something to consider for v3.0

    enhancement 
    opened by friendlyanon 12
  • Regenerate project with cmake-init

    Regenerate project with cmake-init

    This makes the project use modern CMake, spellchecker and clang-format linting in CI, vcpkg for dependencies and cleans up the tests.

    Note that while vcpkg does have a port for nanobench, it does not properly install a CMake package, so an overlay port had to be written for that.
    Similarly, I created overlay ports for the other daw library dependencies, because I didn't want to wait for PRs to merge on those (also lazy). If you wish to make the overlay port as minimal as possible, you can use cmake-init to generate a project structure for those.

    For the tests, a structure similar to that of CMake's test suite was used by having variables affect the tests rather than arguments, which makes passing the same set of arguments to tests cleaner.

    The content section has been removed from the README, because GitHub already implements a Table of Contents feature for READMEs, so this serves no purpose in the document itself.

    Travis CI script has been nuked as they no longer provide CI services.

    The GitHub Actions workflow has been updated with a code coverage job. Follow the instructions in the comments to enable that job.


    Advices

    Nuke docs from version control

    You can automatically deploy documentation with every successful CI build. Doxygen generated docs just bloat the repository size unnecessarily. cmake-init generates projects with such a CI job, here is one working example.

    Nuke test_data from version control

    Or at least have it on an orphaned branch that is pulled at configure time. Same reason as above, just bloats the repository.

    Use the build system for the DAW_JSON_VER define

    If I remember correctly from your CppCast session, you don't intend to keep ABI stability, so you can easily do this:

    target_compile_definitions(daw_json_link INTERFACE "DAW_JSON_VER=\
    v${PROJECT_VERSION_MAJOR}_${PROJECT_VERSION_MINOR}_${PROJECT_VERSION_PATCH}")
    

    and now you have a ~~single~~ less places to change the version number: The lists file and vcpkg.json.


    Fixes: #238

    opened by friendlyanon 9
  • Use Modern CMake

    Use Modern CMake

    This is in relation to https://github.com/beached/daw_json_link/issues/141

    Things to do:

    • [ ] Fix CI scripts
    • [x] Fix and clean tests CMakeLists
    • [ ] Fix dependencies
    Inactive 
    opened by friendlyanon 8
  • tests: `make test` fails if `make check` was not run

    tests: `make test` fails if `make check` was not run

    Possibly a dependency-chain omission, but it was necessary to explicitly run make check in order for make test to work.

    To reproduce, it was enough to run make clean and then make test will fail, as it does not find the necessary binaries to run.

    bug 
    opened by rnsanchez 7
  • Modernize Projects CMake

    Modernize Projects CMake

    The first immediate error when trying to import this library is the use of CMAKE_SOURCE_DIR instead of CMAKE_CURRENT_SOURCE_DIR.

    Second, the CMakeLists.txt doesn't seem to export anything at all so there is nothing to include.

    As a sidenote, CMakeLists.txt seems to be in a very chaotic state, maybe it could use a clean-up anyway.

    enhancement 
    opened by friendlyanon 6
  • Can't link enum class for some reason

    Can't link enum class for some reason

    Here's the code:

    #include <iostream>
    #include <daw/json/daw_json_link.h>
    
    namespace api
    {
    
        struct request
        {
            const static inline std::string type = "request";
            enum class code
            {
                OK,
                ERROR
            };
            code c;
        };
    };
    
    namespace daw::json
    {
        template <>
        struct json_data_contract<api::request>
        {
            static constexpr char const name_type[] = "type";
            static constexpr char const name_code[] = "code";
    
            using type = json_member_list<
                json_string<name_type>,
                json_number<name_code>>;
    
            static inline auto to_json_data(const api::request &value)
            {
                return std::forward_as_tuple(
                    value.type,
                    value.c);
            }
        };
    } // namespace daw::json
    
    int main(int, char **)
    {
        auto data = "{ \"type\":\"request\", \"code\":0 }";
        auto req = daw::json::from_json<api::request>(data);
        std::cout << daw::json::to_json(req) << "\n";
    }
    
    

    Why can't I use enums with json_number? Making enum class something : int{} doesn't help either.

    It produces compile errors like that:

    Errors
    FAILED: CMakeFiles/example.dir/main.cpp.o 
    /bin/clang++-11  -I_deps/daw_json_link-src/include -I_deps/daw_utf_range-src/include -isystem _deps/daw_header_libraries-src/include -g -std=gnu++17 -MD -MT CMakeFiles/example.dir/main.cpp.o -MF CMakeFiles/example.dir/main.cpp.o.d -o CMakeFiles/example.dir/main.cpp.o -c ../main.cpp
    In file included from ../main.cpp:2:
    In file included from _deps/daw_json_link-src/include/daw/json/daw_json_link.h:10:
    In file included from _deps/daw_json_link-src/include/daw/json/daw_from_json.h:11:
    In file included from _deps/daw_json_link-src/include/daw/json/daw_from_json_fwd.h:11:
    In file included from _deps/daw_json_link-src/include/daw/json/impl/daw_json_link_types_fwd.h:12:
    In file included from _deps/daw_json_link-src/include/daw/json/impl/daw_json_link_impl.h:12:
    In file included from _deps/daw_json_link-src/include/daw/json/impl/daw_json_assert.h:11:
    In file included from _deps/daw_json_link-src/include/daw/json/daw_json_exception.h:12:
    In file included from _deps/daw_header_libraries-src/include/daw/daw_string_view.h:11:
    In file included from _deps/daw_header_libraries-src/include/daw/iterator/../daw_algorithm.h:11:
    In file included from _deps/daw_header_libraries-src/include/daw/iterator/../cpp_17.h:19:
    /bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/tuple:1727:14: error: no matching function for call to '__invoke'
        return std::__invoke(std::forward<_Fn>(__f),
               ^~~~~~~~~~~~~
    /bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/tuple:1738:19: note: in instantiation of function template specialization 'std::__apply_impl<const daw::json::default_constructor<api::request> &, std::tuple<std::__cxx11::basic_string<char> &&, double &&>, 0, 1>' requested here
        return std::__apply_impl(std::forward<_Fn>(__f),
                    ^
    _deps/daw_json_link-src/include/daw/json/impl/daw_json_link_impl.h:488:29: note: in instantiation of function template specialization 'std::apply<const daw::json::default_constructor<api::request> &, std::tuple<std::__cxx11::basic_string<char> &&, double &&>>' requested here
                                  JsonClass result = std::apply(
                                                          ^
    _deps/daw_json_link-src/include/daw/json/daw_json_link_types.h:70:25: note: in instantiation of function template specialization 'daw::json::json_details::parse_json_class<api::request, daw::json::json_string<&daw::json::json_data_contract<api::request>::name_type, std::__cxx11::basic_string<char>, daw::json::default_constructor<std::__cxx11::basic_string<char>>, daw::json::JsonNullable::Never, daw::json::EightBitModes::AllowFull, daw::json::JsonNullable::Never>, daw::json::json_number<&daw::json::json_data_contract<api::request>::name_code, double, daw::json::LiteralAsStringOpt::Never, daw::json::default_constructor<double>, daw::json::JsonRangeCheck::Never, daw::json::JsonNullable::Never>, 0, 1, daw::json::BasicParsePolicy<false, daw::json::BasicNoCommentSkippingPolicy<false>, daw::json::constexpr_exec_tag, false, daw::json::json_details::NoAllocator>>' requested here
                          return json_details::parse_json_class<T, JsonMembers...>(
                                               ^
    _deps/daw_json_link-src/include/daw/json/impl/daw_json_parse_value.h:450:28: note: in instantiation of function template specialization 'daw::json::json_member_list<daw::json::json_string<&daw::json::json_data_contract<api::request>::name_type, std::__cxx11::basic_string<char>, daw::json::default_constructor<std::__cxx11::basic_string<char>>, daw::json::JsonNullable::Never, daw::json::EightBitModes::AllowFull, daw::json::JsonNullable::Never>, daw::json::json_number<&daw::json::json_data_contract<api::request>::name_code, double, daw::json::LiteralAsStringOpt::Never, daw::json::default_constructor<double>, daw::json::JsonRangeCheck::Never, daw::json::JsonNullable::Never>>::parse_to_class<api::request, daw::json::BasicParsePolicy<false, daw::json::BasicNoCommentSkippingPolicy<false>, daw::json::constexpr_exec_tag, false, daw::json::json_details::NoAllocator>>' requested here
                                    element_t>::template parse_to_class<element_t>( rng );
                                                         ^
    _deps/daw_json_link-src/include/daw/json/daw_from_json.h:42:24: note: in instantiation of function template specialization 'daw::json::json_details::parse_value<daw::json::json_class<&daw::json::no_name, api::request, daw::json::default_constructor<api::request>, daw::json::JsonNullable::Never>, false, daw::json::BasicParsePolicy<false, daw::json::BasicNoCommentSkippingPolicy<false>, daw::json::constexpr_exec_tag, false, daw::json::json_details::NoAllocator>>' requested here
                  return json_details::parse_value<json_member, KnownBounds>(
                                       ^
    ../main.cpp:43:27: note: in instantiation of function template specialization 'daw::json::from_json<api::request, daw::json::BasicParsePolicy<false, daw::json::BasicNoCommentSkippingPolicy<false>, daw::json::constexpr_exec_tag, false, daw::json::json_details::NoAllocator>, false, api::request>' requested here
      auto req = daw::json::from_json<api::request>(data);
                            ^
    /bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/invoke.h:89:5: note: candidate template ignored: substitution failure [with _Callable = const daw::json::default_constructor<api::request> &, _Args = <std::__cxx11::basic_string<char>, double>]: no type named 'type' in 'std::__invoke_result<const daw::json::default_constructor<api::request> &, std::__cxx11::basic_string<char>, double>'
      __invoke(_Callable&& __fn, _Args&&... __args)
      ^
    In file included from ../main.cpp:2:
    In file included from _deps/daw_json_link-src/include/daw/json/daw_json_link.h:10:
    In file included from _deps/daw_json_link-src/include/daw/json/daw_from_json.h:11:
    In file included from _deps/daw_json_link-src/include/daw/json/daw_from_json_fwd.h:11:
    In file included from _deps/daw_json_link-src/include/daw/json/impl/daw_json_link_types_fwd.h:12:
    In file included from _deps/daw_json_link-src/include/daw/json/impl/daw_json_link_impl.h:16:
    In file included from _deps/daw_json_link-src/include/daw/json/impl/daw_json_serialize_impl.h:10:
    _deps/daw_json_link-src/include/daw/json/impl/daw_json_to_string.h:501:3: error: static_assert failed due to requirement 'std::is_convertible_v<api::request::code, double>' "value must be convertible to specified type in class contract"
                  static_assert(
                  ^
    _deps/daw_json_link-src/include/daw/json/impl/daw_json_to_string.h:996:10: note: in instantiation of function template specialization 'daw::json::json_details::to_string<daw::json::json_number<&daw::json::json_data_contract<api::request>::name_code, double, daw::json::LiteralAsStringOpt::Never, daw::json::default_constructor<double>, daw::json::JsonRangeCheck::Never, daw::json::JsonNullable::Never>, std::back_insert_iterator<std::__cxx11::basic_string<char>>, api::request::code>' requested here
                  return to_string<JsonMember>( ParseTag<JsonMember::expected_type>{ },
                         ^
    _deps/daw_json_link-src/include/daw/json/impl/daw_json_to_string.h:506:8: error: no matching function for call to 'isnan'
                          if( std::isnan( value ) ) {
                              ^~~~~~~~~~
    /bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/cmath:611:3: note: candidate function not viable: no known conversion from 'const api::request::code' to 'float' for 1st argument
    isnan(float __x)
    ^
    /bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/cmath:619:3: note: candidate function not viable: no known conversion from 'const api::request::code' to 'double' for 1st argument
    isnan(double __x)
    ^
    /bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/cmath:624:3: note: candidate function not viable: no known conversion from 'const api::request::code' to 'long double' for 1st argument
    isnan(long double __x)
    ^
    /bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/cmath:632:5: note: candidate template ignored: substitution failure [with _Tp = api::request::code]: no type named '__type' in '__gnu_cxx::__enable_if<false, bool>'
      isnan(_Tp __x)
      ^
    In file included from ../main.cpp:2:
    In file included from _deps/daw_json_link-src/include/daw/json/daw_json_link.h:10:
    In file included from _deps/daw_json_link-src/include/daw/json/daw_from_json.h:11:
    In file included from _deps/daw_json_link-src/include/daw/json/daw_from_json_fwd.h:11:
    In file included from _deps/daw_json_link-src/include/daw/json/impl/daw_json_link_types_fwd.h:12:
    In file included from _deps/daw_json_link-src/include/daw/json/impl/daw_json_link_impl.h:16:
    In file included from _deps/daw_json_link-src/include/daw/json/impl/daw_json_serialize_impl.h:10:
    _deps/daw_json_link-src/include/daw/json/impl/daw_json_to_string.h:516:15: error: no matching function for call to 'isinf'
                          } else if( std::isinf( value ) ) {
                                     ^~~~~~~~~~
    /bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/cmath:584:3: note: candidate function not viable: no known conversion from 'const api::request::code' to 'float' for 1st argument
    isinf(float __x)
    ^
    /bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/cmath:592:3: note: candidate function not viable: no known conversion from 'const api::request::code' to 'double' for 1st argument
    isinf(double __x)
    ^
    /bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/cmath:597:3: note: candidate function not viable: no known conversion from 'const api::request::code' to 'long double' for 1st argument
    isinf(long double __x)
    ^
    /bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/cmath:605:5: note: candidate template ignored: substitution failure [with _Tp = api::request::code]: no type named '__type' in '__gnu_cxx::__enable_if<false, bool>'
      isinf(_Tp __x)
      ^
    In file included from ../main.cpp:2:
    In file included from _deps/daw_json_link-src/include/daw/json/daw_json_link.h:10:
    In file included from _deps/daw_json_link-src/include/daw/json/daw_from_json.h:11:
    In file included from _deps/daw_json_link-src/include/daw/json/daw_from_json_fwd.h:11:
    In file included from _deps/daw_json_link-src/include/daw/json/impl/daw_json_link_types_fwd.h:12:
    In file included from _deps/daw_json_link-src/include/daw/json/impl/daw_json_link_impl.h:16:
    In file included from _deps/daw_json_link-src/include/daw/json/impl/daw_json_serialize_impl.h:10:
    _deps/daw_json_link-src/include/daw/json/impl/daw_json_to_string.h:547:38: error: no matching function for call to 'to_string'
                          it = utils::copy_to_iterator( it, to_string( value ) );
                                                            ^~~~~~~~~
    /bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/basic_string.h:6597:3: note: candidate function not viable: no known conversion from 'const api::request::code' to 'int' for 1st argument
    to_string(int __val)
    ^
    /bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/basic_string.h:6608:3: note: candidate function not viable: no known conversion from 'const api::request::code' to 'unsigned int' for 1st argument
    to_string(unsigned __val)
    ^
    /bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/basic_string.h:6616:3: note: candidate function not viable: no known conversion from 'const api::request::code' to 'long' for 1st argument
    to_string(long __val)
    ^
    /bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/basic_string.h:6627:3: note: candidate function not viable: no known conversion from 'const api::request::code' to 'unsigned long' for 1st argument
    to_string(unsigned long __val)
    ^
    /bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/basic_string.h:6635:3: note: candidate function not viable: no known conversion from 'const api::request::code' to 'long long' for 1st argument
    to_string(long long __val)
    ^
    /bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/11/bits/basic_string.h:6647:3: note: candidate function not viable: no known conversion from 'const api::request::code' to 'unsigned long long' for 1st argument
    to_string(unsigned long long __val)
    ^
    /bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/basic_string.h:6658:3: note: candidate function not viable: no known conversion from 'const api::request::code' to 'float' for 1st argument
    to_string(float __val)
    ^
    /bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/basic_string.h:6667:3: note: candidate function not viable: no known conversion from 'const api::request::code' to 'double' for 1st argument
    to_string(double __val)
    ^
    /bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/basic_string.h:6676:3: note: candidate function not viable: no known conversion from 'const api::request::code' to 'long double' for 1st argument
    to_string(long double __val)
    ^
    _deps/daw_json_link-src/include/daw/json/impl/daw_json_to_string.h:77:35: note: candidate template ignored: could not match 'optional<type-parameter-0-0>' against 'const api::request::code'
          [[nodiscard, maybe_unused]] auto to_string( std::optional<T> const &v )
                                           ^
    5 errors generated.
    ninja: build stopped: subcommand failed.
    Build finished with exit code 1
    
    opened by 0xBYTESHIFT 5
  •  ‘const daw::json::json_name<1>’ is not derived from ‘const std::shared_ptr<_Tp>’

    ‘const daw::json::json_name<1>’ is not derived from ‘const std::shared_ptr<_Tp>’

    Hi again,

    when trying to build my code, i got a really verbose error that I can't past entirely, it told me first some about pedantic (which i disabled), then i got a much bigger one that i paste here:

    /usr/include/c++/9/bits/shared_ptr.h:388:5: note: template argument deduction/substitution failed: In file included from /home/tom/Projects/cproject/build/_deps/daw_json_link-src/include/daw/json/impl/daw_json_parse_policy.h:12, from /home/tom/Projects/cproject/build/_deps/daw_json_link-src/include/daw/json/impl/daw_json_iterator_range.h:11, from /home/tom/Projects/cproject/build/_deps/daw_json_link-src/include/daw/json/daw_json_link.h:11, from /home/tom/Projects/cproject/app/main.cpp:29: /home/tom/Projects/cproject/build/_deps/daw_json_link-src/include/daw/json/impl/daw_json_parse_common.h:265:56: note: ‘const string_view’ {aka ‘const daw::basic_string_view<char>’} is not derived from ‘const std::shared_ptr<_Tp>’ 265 | inline constexpr bool is_no_name = ( JsonMember::name == no_name ); | ~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~ In file included from /usr/include/c++/9/memory:81, from /usr/local/include/boost/system/detail/std_interoperability.hpp:12, from /usr/local/include/boost/system/error_code.hpp:963, from /usr/local/include/boost/beast/core/error.hpp:14, from /usr/local/include/boost/beast/core/detail/bind_handler.hpp:13, from /usr/local/include/boost/beast/core/bind_handler.hpp:14, from /usr/local/include/boost/beast/core/async_base.hpp:14, from /usr/local/include/boost/beast/core.hpp:15, from /home/tom/Projects/cproject/app/main.cpp:19: /usr/include/c++/9/bits/shared_ptr.h:393:5: note: candidate: ‘template<class _Tp> bool std::operator==(std::nullptr_t, const std::shared_ptr<_Tp>&)’ 393 | operator==(nullptr_t, const shared_ptr<_Tp>& __a) noexcept | ^~~~~~~~

    any idea what is happening here?

    (here is my code) ` struct WeatherUpdate { string type; // e

    };

    namespace daw::json { template <> struct json_data_contract { using type = json_member_list< json_string<"e">>; }; } `

    opened by ThomasP1988 5
  • Possible optimization in parsing whitespaces

    Possible optimization in parsing whitespaces

    for this: https://github.com/kostya/benchmarks#json, i increased file size by 5 times and test with and without whitespaced json.

    with whitespaces: |parser|time|memory| |-|-|-| |C++/g++ (simdjson On-Demand)|0.788|1756 Mb| |C++/g++ (DAW JSON Link)|0.929|1123 Mb|

    without whitespaces: |parser|time|memory| |-|-|-| |C++/g++ (DAW JSON Link)|0.609|607 Mb| |C++/g++ (simdjson On-Demand)|0.725|1239 Mb|

    looks like whitespaces process not effective.

    help wanted 
    opened by kostya 4
  • reordering a json may lead to

    reordering a json may lead to "A value of known size was accessed past the end"

    There seems to be some issue with json_custom types - or I am missing something. I wanted to try daw_json for the public data from binance.com . It seemed to work fine at first - but when I implemented the custom type Fixed8 (replaced by double in the code below) the order of the fields "a" and "b" in the json suddenly seemed to make a difference. When the order in the json matches the oder in the json_data_contract declaration it works fine, if the two arrays are swapped an exception is thrown instead.

    The order of keys in the input are not meant to make a difference, right?

    const std::string testJson = R"({"e":"depthUpdate","E":1609884707320,"s":"BTCBUSD","U":2544556159,"u":2544556219,"a":[["34198.19000000","0.00000000"],["34198.23000000","0.00000000"],["34198.25000000","0.00000000"],["34198.27000000","0.00000000"],["34198.30000000","0.00000000"],["34198.32000000","0.00958500"],["34198.40000000","0.01232200"],["34198.41000000","0.01000000"],["34198.87000000","0.00000000"],["34199.12000000","0.00000000"],["34199.16000000","0.00000000"],["34199.42000000","0.00000000"],["34200.25000000","0.00000000"],["34200.71000000","0.03199900"],["34201.27000000","0.03100000"],["34201.62000000","0.00000000"],["34202.58000000","0.00000000"],["34204.45000000","0.00952700"],["34207.64000000","0.00000000"],["34207.74000000","0.00000000"],["34209.77000000","0.00000000"],["34209.81000000","0.20400000"],["34225.94000000","0.20200000"],["34226.60000000","0.91050000"],["34236.08000000","0.30000000"]],"b":[["34198.31000000","0.00000000"],["34196.54000000","0.00453200"],["34193.34000000","0.00000000"],["34189.89000000","0.00000000"],["34188.82000000","0.00000000"],["34185.32000000","0.00000000"],["34184.84000000","0.06350200"],["34184.83000000","0.20000000"],["34180.61000000","0.08622700"],["34180.60000000","0.00000000"],["34180.59000000","0.19200000"],["34180.02000000","0.00000000"],["34180.01000000","0.00000000"],["34176.88000000","0.00000000"],["34166.48000000","0.00000000"],["34166.47000000","0.00000000"],["34159.85000000","0.03317500"],["34159.24000000","0.09394900"],["34158.29000000","1.00000000"],["34154.86000000","0.00000000"]]})";
    
    struct Fixed8JsonConverter {
    	double operator( )( std::string_view sv ) const {
    		return stod(std::string(sv));
    	}
    	
    	template<typename OutputIterator>
    	constexpr OutputIterator operator()(OutputIterator _it, double _f) const {
    		return daw::json::utils::copy_to_iterator(_it, std::to_string(_f));
    	}
    };
    
    template<JSONNAMETYPE name>
    using json_fixed8 = daw::json::json_custom<name, double, Fixed8JsonConverter, Fixed8JsonConverter>;
    
    struct Change {
    	double rate;
    	double amount;
    };
    
    namespace daw::json {
    	template<>
    	struct json_data_contract<Change> {
    		using type = json_ordered_member_list<
    						json_fixed8<no_name>, 
    						json_fixed8<no_name>
    					>;
    	};
    }
    
    class DepthUpdateJson {
    public:
    	int64 time; // E
    	std::string pairName; // s
    	int64 idTo; // u
    	int64 idFrom; // U
    	std::vector<Change> bid; // b
    	std::vector<Change> ask; // a
    };
    
    namespace daw::json {
    	template<>
    	struct json_data_contract<DepthUpdateJson> {
    		using type = json_member_list<
    			json_number<"E", int64>,
    			json_string<"s">,
    			json_number<"u", int64>,
    			json_number<"U", int64>,
    			json_array<"b", Change>,
    			json_array<"a", Change>
    		>;
    	};
    }
    
    int main (const int argC, char* argV[] ) {
    	try {
    		auto parsed = daw::json::from_json<DepthUpdateJson>(testJson);
    	} catch (daw::json::json_exception e) {
    		std::cout << "daw error: " << e.reason() << " near: '" << e.parse_location() << "'"  << std::endl;
    	}
    }
    

    The code produces the output

    daw error: A value of known size was accessed past the end near: '["34198.23000000","0.00000000"],["34198.25000000","0.00000000"],["34198.27000000","0.00000000"],["34198.30000000","0.00000000"],["34198.32000000","0.00958500"],["34198.40000000","0.01232200"],["34198.41000000","0.01000000"],["34198.87000000","0.00000000"],["34199.12000000","0.00000000"],["34199.16000000","0.00000000"],["34199.42000000","0.00000000"],["34200.25000000","0.00000000"],["34200.71000000","0.03199900"],["34201.27000000","0.03100000"],["34201.62000000","0.00000000"],["34202.58000000","0.00000000"],["34204.45000000","0.00952700"],["34207.64000000","0.00000000"],["34207.74000000","0.00000000"],["34209.77000000","0.00000000"],["34209.81000000","0.20400000"],["34225.94000000","0.20200000"],["34226.60000000","0.91050000"],["34236.08000000","0.30000000"]],"b":[["34198.31000000","0.00000000"],["34196.54000000","0.00453200"],["34193.34000000","0.00000000"],["34189.89000000","0.00000000"],["34188.82000000","0.00000000"],["34185.32000000","0.00000000"],["34184.84000000","0.06350200"],["34184.83000000","0.20000000"],["34180.61000000","0.08622700"],["34180.60000000","0.00000000"],["34180.59000000","0.19200000"],["34180.02000000","0.00000000"],["34180.01000000","0.00000000"],["34176.88000000","0.00000000"],["34166.48000000","0.00000000"],["34166.47000000","0.00000000"],["34159.85000000","0.03317500"],["34159.24000000","0.09394900"],["34158.29000000","1.00000000"],["34154.86000000","0.00000000"]]}'
    
    bug 
    opened by fuchsi-fuchs 4
  • Investigate a full JSON query impl

    Investigate a full JSON query impl

    Allow querying and returning a range of elements as either json_value or parsed. This could then be extended to returning a vector<json_value> or vector<T>.

    opened by beached 0
  • Investigate implementation of a catch-all json member

    Investigate implementation of a catch-all json member

    Name of mapping : TBD Purpose: To provide a json_value like structure over the unmapped members in a class. Because we know the mapped members and their locations, that location_info structure can be used as a filter list and is of fixed size.

    example usage:

    {
      "a": 1,
      "b": 2,
      "c": 3 
    }
    

    With a mapping like

    using type = json_member_list<json_link<"b", int>, json_catch_all>;
    

    Would require a class that is constructible from an int, json_catch_all_value<1>

    The 1 is because the size of the filter list is 1 element.

    opened by beached 0
  • Investigate parsing sub-members as member

    Investigate parsing sub-members as member

    Investigate allowing the parsing of submembers. e.g

    { "a": { "b": 5 } }
    

    The transformation would be potentially lossy

    Possible mapping syntax

    json_submember<"a", "b", json_number_no_name<int>>
    

    Where "b" can use the path syntax, e.g "b.c" would mean a class like

    { "a": { "b": { "c": 5 } } }
    
    opened by beached 0
  • Investigate adding optimized paths for arrays of number, bool, string

    Investigate adding optimized paths for arrays of number, bool, string

    Investigate adding optimized paths for arrays of number, bool, string. This should be doable as seen in other JSON libraries that SIMDify these things globally. But because we know what it should it, we should be able to exploit the known types.

    opened by beached 0
  • Allow piecemeal serialization of arrays.

    Allow piecemeal serialization of arrays.

    Provide a mechanism to output arrays 1 element at a time without creating an iterator. Maybe something like

    auto hnd = daw::json::array_start( output_Iterator );
    hnd.to_json( value0 );
    hnd.to_json( value1 );
    hnd.to_json( value2 );
    hnd.close( ); // or let go out of scope```
    
    This can work with heterogenous types too.
    opened by beached 0
Releases(v3.0.0)
  • v3.0.0(Jun 28, 2022)

    This is the next significant release of DAW JSON Link with lots of goodies and help to make the job of mapping JSON to C++ data structures easier along with a much improved json_value that allows on demand parsing via a DOM like interface without the storage. Together they allow for parsing pretty much any JSON entity.

    What's Changed

    • type deduction in mappings. This allows integration with Reflection and reflection like libraries(e.g. Boost.Describe)

    • Pretty printing/output formatting options

    • json_nullable/readable trait to separate nullability from mappers

    • json_raw/json_custom for things that don't fit

    • Json Schema generation via to_json_schema

    • Minimized symbol names

    • Using a single Options value for member mappings, serialization options, and parse options

    • Using better testing in CI

    • added json_type_alias mapping type to allow classes to be parsed as other types e.g json_string/json_number/... ... and more

    • Updated json_value to allow for better exploration by @beached in https://github.com/beached/daw_json_link/pull/284

    • Fixed some issues with path parsing by @beached in https://github.com/beached/daw_json_link/pull/285

    • Added JSON Lines Support by @beached in https://github.com/beached/daw_json_link/pull/286

    • Made json exception inherit from std::exception by @beached in https://github.com/beached/daw_json_link/pull/287

    • Added json lines partitioning helper by @beached in https://github.com/beached/daw_json_link/pull/288

    • Documentation and -fno-exceptions update by @beached in https://github.com/beached/daw_json_link/pull/292

    • Added WritableOuputType support by @beached in https://github.com/beached/daw_json_link/pull/293

    • to_json simplifications by @beached in https://github.com/beached/daw_json_link/pull/294

    • Fix wrong link in cookbook/readme.md by @jk-jeon in https://github.com/beached/daw_json_link/pull/296

    • Cleanup of from JSON, Container concept by @beached in https://github.com/beached/daw_json_link/pull/297

    • benchmark and json_value conversion enhancements by @beached in https://github.com/beached/daw_json_link/pull/298

    • Documentation updates by @beached in https://github.com/beached/daw_json_link/pull/301

    • Updated editorconfig by @beached in https://github.com/beached/daw_json_link/pull/302

    • documentation fixes by @beached in https://github.com/beached/daw_json_link/pull/303

    • Formatting cleanup by @beached in https://github.com/beached/daw_json_link/pull/304

    • clang-9 and clang-10 bugfix and renamed Readable concept to Nullable by @beached in https://github.com/beached/daw_json_link/pull/305

    • Fixed code that wasn't updated to new mapping by @beached in https://github.com/beached/daw_json_link/pull/306

    • excluding clang-9/Debug/cpp17 from CI due to compiler ICE by @beached in https://github.com/beached/daw_json_link/pull/307

    • Updated code links in documents and pined version of extern deps by @beached in https://github.com/beached/daw_json_link/pull/308

    • Fix: from_json of json_value dropped class bounds and allocator by @beached in https://github.com/beached/daw_json_link/pull/309

    • Updated version to 3_0_0 by @beached in https://github.com/beached/daw_json_link/pull/310

    New Contributors

    • @jk-jeon made their first contribution in https://github.com/beached/daw_json_link/pull/296

    Full Changelog: https://github.com/beached/daw_json_link/compare/v2.10.2...v3.0.0

    Source code(tar.gz)
    Source code(zip)
  • v3.0.0_rc2(Jun 26, 2022)

    What's Changed

    • Updated editorconfig by @beached in https://github.com/beached/daw_json_link/pull/302
    • documentation fixes by @beached in https://github.com/beached/daw_json_link/pull/303
    • Formatting cleanup by @beached in https://github.com/beached/daw_json_link/pull/304
    • clang-9 and clang-10 bugfix and renamed Readable concept to Nullable by @beached in https://github.com/beached/daw_json_link/pull/305
    • Fixed code that wasn't updated to new mapping by @beached in https://github.com/beached/daw_json_link/pull/306

    Full Changelog: https://github.com/beached/daw_json_link/compare/v3.0.0_rc1...v3.0.0_rc2

    Source code(tar.gz)
    Source code(zip)
  • v3.0.0_rc1(Jun 26, 2022)

    What's Changed

    • benchmark and json_value conversion enhancements by @beached in https://github.com/beached/daw_json_link/pull/298
    • Documentation updates by @beached in https://github.com/beached/daw_json_link/pull/301

    Full Changelog: https://github.com/beached/daw_json_link/compare/v3.0.0_beta07...v3.0.0_beta08

    Source code(tar.gz)
    Source code(zip)
  • v3.0.0_beta07(Jun 25, 2022)

    What's Changed

    • Documentation and -fno-exceptions update by @beached in https://github.com/beached/daw_json_link/pull/292
    • Added WritableOuputType support by @beached in https://github.com/beached/daw_json_link/pull/293
    • to_json simplifications by @beached in https://github.com/beached/daw_json_link/pull/294
    • Fix wrong link in cookbook/readme.md by @jk-jeon in https://github.com/beached/daw_json_link/pull/296
    • Cleanup of from JSON, Container concept by @beached in https://github.com/beached/daw_json_link/pull/297

    New Contributors

    • @jk-jeon made their first contribution in https://github.com/beached/daw_json_link/pull/296

    Full Changelog: https://github.com/beached/daw_json_link/compare/v3.0.0_beta06...v3.0.0_beta07

    Source code(tar.gz)
    Source code(zip)
  • v3.0.0_beta06(Jun 2, 2022)

    Highlights

    • Updated json_value to allow for better exploration by @beached in https://github.com/beached/daw_json_link/pull/284
    • Fixed some issues with path parsing by @beached in https://github.com/beached/daw_json_link/pull/285
    • Added JSON Lines Support by @beached in https://github.com/beached/daw_json_link/pull/286
    • Made json exception inherit from std::exception by @beached in https://github.com/beached/daw_json_link/pull/287
    • Added json lines partitioning helper by @beached in https://github.com/beached/daw_json_link/pull/288

    Full Changelog: https://github.com/beached/daw_json_link/compare/v3.0.0_beta04...v3.0.0_beta06

    Source code(tar.gz)
    Source code(zip)
  • v3.0.0_beta05(May 26, 2022)

    What's Changed

    • type deduction in mappings. This allows integration with Reflection and reflection like libraries(e.g. Boost.Describe)
    • Pretty printing/output formatting options
    • json_nullable/readable trait to separate nullability from mappers
    • json_raw/json_custom for things that don't fit
    • Json Schema generation via to_json_schema
    • Minimized symbol names
    • Using a single Options value for member mappings, serialization options, and parse options
    • Using better testing in CI
    • added json_type_alias mapping type to allow classes to be parsed as other types e.g json_string/json_number/... ... and more
    Source code(tar.gz)
    Source code(zip)
  • v2.15.3(May 26, 2022)

    Maintenance Release

    What's Changed

    • Catching up to main for v2 by @beached in https://github.com/beached/daw_json_link/pull/274

    Full Changelog: https://github.com/beached/daw_json_link/compare/v2.15.2...v2.15.3

    Source code(tar.gz)
    Source code(zip)
  • v2.15.2(May 10, 2022)

  • v2.15.1(May 10, 2022)

  • v3.0.0_beta04(May 9, 2022)

    • Bugfixes
    • Cleanup of customization around optional like types(e.g. pointers/shared ptr/unique ptr)
    • Fixed deduction bugs(e.g. for json_link types)
    • Added example of deduction of type mapping concepts https://github.com/beached/daw_json_link_describe
    Source code(tar.gz)
    Source code(zip)
  • v2.15.0(May 9, 2022)

    Corrected some perf issues when DAW_JSON_DONT_USE_EXCEPTIONS was set

    • std::uncaught_exceptions was showing how in some profiling. It shouldn't be needed if there are no exceptions.
      • This can potentially result in termination if multiple exceptions are throwing in some cases.
    Source code(tar.gz)
    Source code(zip)
  • v2.14.0(May 4, 2022)

  • v2.13.0(Apr 30, 2022)

  • v2.12.9(Mar 24, 2022)

  • v2.12.8(Mar 24, 2022)

  • v2.12.7(Mar 24, 2022)

  • v2.12.6(Mar 24, 2022)

  • v2.12.4(Jan 22, 2022)

  • v2.12.1(Jan 22, 2022)

  • v2.12.0(Jan 22, 2022)

  • v2.11.2(Jan 5, 2022)

  • v2.11.0(Dec 18, 2021)

  • v3.0.0_beta03(Oct 7, 2021)

    • Readme updates and json_link_version( ) method
    • Readme updates and added json_link_version( )
    • More readme updates, fixed namespace dependency
    • Readme updates and cleaned up iostream example/test
    • Fixed typo
    • Added fuzzing and artifact generation
    • Updated readme, fixed link
    • BUGFIX: CI found a 0 being passed to __builtin_ctzl, added check
    • Added some optional consteval support in select places
    • Excluded MSVC from consteval due to codegen bug and __builtin_array_init_helper
    • Added customization optons DAW_JSON_USE_STDEXCEPT and DAW_JSON_VER_OVERRIDE
    • Allow overriding inline namespace for API version via DAW_JSON_VER_OVERRIDE
    • Exception fixes
    • Fixed exception handling/disbling in tests
    • Made constexpr only method only throw if exceptions are used
    • Fixed two typos
    • made all ci builds use the full target
    • Using ninja for building ubuntu
    • Added constexpr to_json for doubles.
    • slow paths umul128 on MSVC when IS_CONSTANT_EVALUATED isn't avail
    • Excluded compilers without constexpr bitcast from using constexpr to_decimal
    • Fixed issue with throwing, using daw_json_error
    • Adjusted daw_json_link_test to report, but not abort, on ulp diff
    • fixed issue with MSVC bug and constexpr in test

    And more

    Source code(tar.gz)
    Source code(zip)
  • v3.0.0_beta02(Aug 26, 2021)

  • v3.0.0_beta01(Aug 26, 2021)

  • v2.10.2(Aug 25, 2021)

  • v2.10.1(Aug 25, 2021)

  • v2.10.0(Aug 25, 2021)

  • v2.9.11(Jul 24, 2021)

  • v2.9.9(Jul 23, 2021)

    Moved the thirdparty/dragonbox code into the daw namespace to prevent id collisions with codebases that use the upstream library. jkj::dragonbox -> daw::jkj::dragonbox

    Source code(tar.gz)
    Source code(zip)
Owner
Darrell Wright
Darrell Wright
Fast JSON serialization and parsing in C++

DAW JSON Link v2 Content Intro Default Mapping of Types API Documentation - Member mapping classes and methods Cookbook Get cooking and putting it all

Darrell Wright 276 Jun 22, 2022
json-cpp is a C++11 JSON serialization library.

JSON parser and generator for C++ Version 0.1 alpha json-cpp is a C++11 JSON serialization library. Example #include <json-cpp.hpp> struct Foo {

Anatoly Scheglov 4 Dec 31, 2019
A convenience C++ wrapper library for JSON-Glib providing friendly syntactic sugar for parsing JSON

This library is a wrapper for the json-glib library that aims to provide the user with a trivial alternative API to the API provided by the base json-

Rob J Meijer 16 May 10, 2022
An easy-to-use and competitively fast JSON parsing library for C++17, forked from Bitcoin Cash Node's own UniValue library.

UniValue JSON Library for C++17 (and above) An easy-to-use and competitively fast JSON parsing library for C++17, forked from Bitcoin Cash Node's own

Calin Culianu 10 Aug 31, 2021
A fast streaming JSON parsing library in C.

********************************************************************** This is YAJL 2. For the legacy version of YAJL see https

Lloyd Hilaiel 2k Jun 17, 2022
Very fast Python JSON parsing library

cysimdjson Fast JSON parsing library for Python, 7-12 times faster than standard Python JSON parser. It is Python bindings for the simdjson using Cyth

TeskaLabs 205 Jun 22, 2022
Immediate Mode JSON Serialization Library in C

JIM Immediate Mode JSON Serialization Library in C. Similar to imgui but for generating JSON. Example example.c: #include <stdio.h> #define JIM_IMPLE

Tsoding 27 May 6, 2022
C++ Serialization library for JSON

ThorsSerializer Support for Json Yaml Bson NEW Benchmark Results Conformance mac linux Performance max linux For details see: JsonBenchmark Yet anothe

Loki Astari 273 Jun 15, 2022
A simple class for parsing JSON data into a QVariant hierarchy and vice versa.

The qt-json project is a simple collection of functions for parsing and serializing JSON data to and from QVariant hierarchies. NOTE: Qt5 introduced a

null 299 Jun 17, 2022
json_struct is a single header only C++ library for parsing JSON directly to C++ structs and vice versa

Structurize your JSON json_struct is a single header only library that parses JSON to C++ structs/classes and serializing structs/classes to JSON. It

Jørgen Lind 229 Jun 30, 2022
Jvar - JS inspired Variants and JSON parsing for C++

jvar jvar tries to capture some of the expressiveness of JavaScript and bring it into C++ In particular, it implements a Variant type which is similar

Yasser Asmi 23 Jun 15, 2022
A simple class for parsing JSON data into a QVariant hierarchy and vice versa.

The qt-json project is a simple collection of functions for parsing and serializing JSON data to and from QVariant hierarchies. NOTE: Qt5 introduced a

null 299 Jun 17, 2022
jstruct is an automatic C code generation tool for generating JSON parsing and stringifying code.

jstruct is an automatic C code generation tool for generating JSON parsing and stringifying code. The C code generated by this tool needs to depend on the cJSON library for execution.

acoinfo 27 Apr 18, 2022
Parsing gigabytes of JSON per second

simdjson : Parsing gigabytes of JSON per second JSON is everywhere on the Internet. Servers spend a *lot* of time parsing it. We need a fresh approach

null 15.8k Jul 1, 2022
Parsing gigabytes of JSON per second

simdjson : Parsing gigabytes of JSON per second JSON is everywhere on the Internet. Servers spend a *lot* of time parsing it. We need a fresh approach

null 15.8k Jun 27, 2022
This is a JSON C++ library. It can write and read JSON files with ease and speed.

Json Box JSON (JavaScript Object Notation) is a lightweight data-interchange format. Json Box is a C++ library used to read and write JSON with ease a

Anhero inc. 109 Apr 24, 2022
json-build is a zero-allocation JSON serializer compatible with C89

json-build is a zero-allocation JSON serializer compatible with C89. It is inspired by jsmn, a minimalistic JSON tokenizer.

Lucas Müller 24 Jun 20, 2022
A fast JSON parser/generator for C++ with both SAX/DOM style API

A fast JSON parser/generator for C++ with both SAX/DOM style API Tencent is pleased to support the open source community by making RapidJSON available

Tencent 12.1k Jun 24, 2022
A Haskell library for fast decoding of JSON documents using the simdjson C++ library

hermes A Haskell interface over the simdjson C++ library for decoding JSON documents. Hermes, messenger of the gods, was the maternal great-grandfathe

Josh Miller 35 May 26, 2022