advanced, flexible JSON manipulation in C

Related tags

JSON wjelement
Overview

WJElement - JSON manipulation in C

WJElement is a very flexible JSON library developed by Messaging Architects. It was created for MA's "WARP" webserver ("Warp Json Elements"), and is built on top of the lower-level WJReader and WJWriter libraries (also included).

See the wiki for more information, example code and full API reference.

WJReader and WJWriter are optimized for speed and memory-efficiency. WJElement focuses on flexibility and handy features, allowing C code to manipulate JSON documents with as few statements (fewer, sometimes!) as JavaScript itself. WJElement is also capable of json-schema validation.

WJElement has grown into a generally-useful library, and is used across Messaging Architects' netmail and related projects. It is loved enough by MA's developers that we desire to use it elsewhere too, and we think others will enjoy it as well. So, here it is, ready to be consumed in any project, open or closed, as outlined by the GNU LGPL (any version). Include it as-is and link to it from your code, massage it into your own statically-linked package, or use it in ways we haven't thought of. Read the docs/headers, have fun, and if you use it for something awesome, let us know about it! :^)

Issues
  • Graceful malloc failure handling and/or memory optimization for constrained environments

    Graceful malloc failure handling and/or memory optimization for constrained environments

    Thank you for the library. It may be just what we need for a project on an (embedded) AVR platform. We're specifically interested in the schema validation feature for validating some relatively simple JSON blobs.

    I'm having trouble figuring out what I presume is a memory allocation problem.

    We have a pretty capable build environment in which we can run unit tests on our production AVR code natively (OS X and Linux) and build some code libraries for all three platforms. I was able to strip down wjelement (removed 64 bit support, etc.) and add missing functionality (strndup()) such that the library would build both for AVR release and our native testing environments.

    I have been successful in using the slimmed down library to validate JSON in our native test environment. Now I'm working to see if wjelement will run on our AVR chip. At first everything died when I put validation calls in our existing code. I soon figured out that it was because of memory allocation issues.

    The default internal memory allocation is for 4k RAM in WJROpenDocument(), but we only have 8k RAM total!

    I tweaked _WJEParse() to take a buffer allocation size value as a parameter. With that change and smaller memory allocation calls to _WJEParse() for a test JSON blob and a schema string both parsing calls now succeed. Unfortunately, the firmware crashes upon trying to validate the JSON blob against the schema. I assume this is more memory allocation issues.

    1. What kind of memory optimization might I be able to do in the guts of WJESchemaValidate()? Is it even worth trying? I'm hoping there's more memory blocks whose default sizes I can cut down.
    2. Could you direct me on how to change the code to more gracefully error out if memory allocation fails so I can track down the issue and robustify validation for our use case?

    Anything you can say is very much appreciated.

    opened by mkarlesky 17
  • test memory ,found Coredump

    test memory ,found Coredump

    Segmentation fault

    Whe using the following code to test the lib,I found "Segmentation fault": for(int i =0; i<10000;i++){ char json[] = "{"name":"Serenity","class":"firefly","crew":[{"name":"Malcolm Reynolds","job":"captain","born":2486},{"name":"Kaywinnet Lee Fry","job":"mechanic","born":2494},{"name":"Jayne Cobb","job":"public relations","born":2485}]}"; WJReader jsonReader =WJROpenMemDocument(json,NULL,0); WJElement jsonDoc = WJEOpenDocument(jsonReader, NULL, NULL, NULL); WJEDump(jsonDoc); WJECloseDocument(jsonDoc); }

    when i >6000

    opened by zhoubonjour 15
  • [Question] Validate against just a portion of a schema file (an all in one validation file)

    [Question] Validate against just a portion of a schema file (an all in one validation file)

    I've tried a lot of things stepping thru code and such, but how do I do schema validation against just a portion of a schema file. I tried using the "where" parameter of the WJEOpenDocument, but wasn't quite sure if that was right or what to pass in.

    Sample schema: { "properties":{}, "validateAgainstThis": { "type": "object", "properties": { } } }

    opened by ken-reister 14
  • Completion of error handling

    Completion of error handling

    I have looked at a few source files for your current software. I have noticed that some checks for return codes are missing.

    Would you like to add more error handling for return values from functions like the following?

    opened by elfring 9
  • Non-linear parsing performance

    Non-linear parsing performance

    When using the following code: WJReader r = WJROpenMemDocument(buffer, NULL, 0); handle = WJEOpenDocument(r,NULL,NULL,NULL);

    I've noted that the performance degrade as a function of n.

    Unfortunately it's degrading in a non-linear way: json parsing of 100000 items took : 0.204 per-element time : 0.000002 json parsing of 200000 items took : 0.697 per-element time : 0.000003 json parsing of 300000 items took : 1.465 per-element time : 0.000005 json parsing of 400000 items took : 2.639 per-element time : 0.000007 json parsing of 500000 items took : 4.190 per-element time : 0.000008 json parsing of 600000 items took : 6.559 per-element time : 0.000011 json parsing of 700000 items took : 9.934 per-element time : 0.000014 json parsing of 800000 items took : 14.148 per-element time : 0.000018 json parsing of 900000 items took : 18.535 per-element time : 0.000021

    Any suggestions would be appreciated.

    opened by tsachiherman 8
  • json object comparision in c using CompareJson API

    json object comparision in c using CompareJson API

    Hi, I want to compare to json objects , could you suggest any API. Can I use Comparejson API by passing two json object for comparision in c language.

    Thanks, Mohan

    opened by primohan01 8
  • Exponential memory growth in schema validation

    Exponential memory growth in schema validation

    Consider a simple schema like: { "$schema": "http://json-schema.org/draft-03/schema#", "type": "array", "items": { "type": "object", "properties": { "headers": { "type": "object", "required": true, "properties": { "messageId": { "type": "string", "required": true } } }, "properties": { "type": "object", "required": false }, "body": { "type": "string", "required": true } } } }

    If the "body" element is ~600 byte string, it takes 12K RAM to validate. If the "body" element is ~308K byte string, it takes 17.6M RAM to validate. If the "body" element is ~9.1M byte string, it takes 16.9G (yes, giga bytes) RAM to validate.

    opened by JayCase 7
  • Schema pattern matching always comes back valid

    Schema pattern matching always comes back valid

    Schema:

    {
        "$schema": "http://json-schema.org/draft-03/schema#",
        "type": "object",
        "properties": {
          "key": {
            "type": "string",
            "pattern": "ABC"
          }
        }
    }
    

    JSON String to validate

    {
      "key": "123"
    }
    

    Example

       std::string jsonString = "{\n  \"key\": \"123\"\n}";
       WJReader reader = WJROpenMemDocument((void*)jsonString.c_str(), NULL, 0);
       WJElement doc = WJEOpenDocument(reader, NULL, NULL, NULL);
    
       std::string schemaString = "{\n    \"$schema\": \"http://json-schema.org/draft-03/schema#\",\n    \"type\": \"object\",\n    \"properties\": {\n      \"key\": {\n        \"type\": \"string\",\n        \"pattern\": \"ABC\"\n      }\n    }\n}";
       WJReader SchemaReader = WJROpenMemDocument((void*)schemaString.c_str(), NULL, 0);
       WJElement SchemaDoc = WJEOpenDocument(SchemaReader, NULL, NULL, NULL);
    
       if(WJESchemaValidate(
          SchemaDoc, doc,
          NULL, NULL, NULL, NULL))
       {
          std::cout << "schema matches\n";
       }
       else
       {
          std::cout << "schema does not match\n";
       }
    
    

    This prints "schema matches\n", as does any other setting of the pattern.

    opened by mehagar 7
  • reserved identifier violation

    reserved identifier violation

    I would like to point out that identifiers like "__WJEBool" and "_WJROpenDocument" do not fit to the expected naming convention of the C language standard. Would you like to adjust your selection for unique names?

    opened by elfring 7
  • v4

    v4 "required" assertion applied for instance that is not object

    Section 3.2.1 of JSON Schema Validation specification states:

    Most validation assertions only constrain values within a certain primitive type. When the type of the instance is not of the type targeted by the keyword, the instance is considered to conform to the assertion.

    If the required keyword is used with an array value, per v4 syntax, it is asserted even if the instance being tested is not an object.

    For example: {"required":["name1"]} - Schema {"name1":null} - Instance conforms {} - Instance does not conform null - Instance conforms but WJESchemaValidate() claims it does not

    Below is code for an example console application, expected output and actual output.

    #include <stdarg.h>
    #include "stdafx.h"
    #include "wjelement.h"
    
    void ErrCB(void *client, const char *format, ...)
    {
    	va_list args;
    	va_start(args, format);
    	vprintf(format, args);
    	va_end(args);
    	printf("\n");
    }
    
    void RunTest(int index, WJElement schema, char* json)
    {
    	if (WJElement document = WJEParse(json)) {
    		printf("Test %d\n", index);
    		if (WJESchemaValidate(schema, document, &ErrCB, NULL, NULL, NULL)) {
    			printf("OK\n");
    		}
    		WJECloseDocument(document);
    		printf("\n");
    	}
    }
    
    int main()
    {
    	XplBool valid;
    	if (WJElement schema = WJEParse("{\"required\":[\"name1\"]}")) {
    		RunTest(1, schema, "{\"name1\":null}");
    		RunTest(2, schema, "{}");
    		RunTest(3, schema, "null");
    		WJECloseDocument(schema);
    	}
    	return 0;
    }
    

    The expected output is as follows:

    Test 1 OK

    Test 2 (root): required member 'name1' not found.

    Test 3 OK

    The actual output is as follows:

    Test 1 OK

    Test 2 (root): required member 'name1' not found.

    Test 3 (root): required member 'name1' not found.

    opened by PaulenAndrew 5
  • Schema validation unexpectedly succeeds when key doesn't match patternProperties and additionalProperties=false

    Schema validation unexpectedly succeeds when key doesn't match patternProperties and additionalProperties=false

    With this schema…

    {
    	"$schema": "http://json-schema.org/draft-04/schema#",
    	"type": "object",
    	"patternProperties": {
    		"^string[0-9]+$": {
    			"type": "string"
    		}
    	},
    	"additionalProperties": false
    }
    

    and this document…

    {"foo":"bar"}
    

    …since the key doesn't match patternProperties, and additionalProperties is false, I'd expect the document to fail schema validation. But, with wjelement-1.2, it passes:

    echo 'validate patternProperties-noAdditional.jsonschema' | ./wjecli test.json
    Schema validation: PASS
    
    opened by smokris 5
  • WJWNull returns

    WJWNull returns "FALSE" when succeeds

    If WJWNull is called with non-null "name" argument, it returns "FALSE", though name is created:

    result = WJWNull("error", wjWriter);

    Looks like error is caused by line 894: return(6 == WJWrite(doc, ":null", 5));

    Argument ":null" is 5 bytes long, WJWrite returns 5, but it is compared with "6".

    opened by GlebPlekhotko 0
  • WJE_IGNORE_CASE not working for conditions separated with ;

    WJE_IGNORE_CASE not working for conditions separated with ;

    If the flag is set on a WJEAction, it is not used by the following WJEGet call in search.c:

    static int WJECheckCondition(WJElement e, char **condition, WJEAction action) { .... if (';' == *cond) { if (WJEGet(e, value, NULL)) { if (condition) *condition = NULL;

    opened by ChadGoulding 0
  • Read JSON document from socket and file descriptor

    Read JSON document from socket and file descriptor

    Hello, I really like the library. Is there any safe way to load a document from a socket having its descriptor? How do I deal with framing and merging two different documents?

    opened by Lucas286 3
  • wjecli validation -- suboptimal results

    wjecli validation -- suboptimal results

    Hi, I've just tried schema-validation, the results were suboptimal:

    echo 'validate json-schema-draft-04.json' | wjecli json-schema-draft-04.json
    definitions: extra property 'schemaArray' found.
    definitions: extra property 'positiveInteger' found.
    definitions: extra property 'positiveIntegerDefault0' found.
    definitions: extra property 'simpleTypes' found.
    definitions: extra property 'stringArray' found.
    definitions failed validation
    properties: extra property 'id' found.
    properties: extra property '$schema' found.
    ...
    

    As a start, the schema should be valid against itself, but even if it isn't, the error message should at least contain a line-number...

    enhancement 
    opened by lzsiga 4
  • Inconsistency: on a mismatch in type array error cb may be called or not

    Inconsistency: on a mismatch in type array error cb may be called or not

    If I validate

    1
    

    against

    {
        "type": [
            {
                "type": "string"
            },
            "integer"
        ]
    }
    

    My error callback gets called upon the string mismatch. However if I validate against

    {
        "type": [
            "string",
            "integer"
        ]
    }
    

    it is not called.

    opened by RelayrMA 0
  • _WJEGet takes path as char*

    _WJEGet takes path as char*

    Since the path is char* and not const char*, uses of constant strings cause compiler warnings. Yet all _WJEGet does is pass the parameter to _WJESearch which takes a const pointer anyway.

    opened by RelayrMA 0
Releases(v1.3)
Owner
netmail
Open source projects created by netmail, Inc
netmail
A C++, header-only library for constructing JSON and JSON-like data formats, with JSON Pointer, JSON Patch, JSON Schema, JSONPath, JMESPath, CSV, MessagePack, CBOR, BSON, UBJSON

JSONCONS jsoncons is a C++, header-only library for constructing JSON and JSON-like data formats such as CBOR. For each supported data format, it enab

Daniel Parker 510 Aug 1, 2022
jq is a lightweight and flexible command-line JSON processor.

jq is a lightweight and flexible command-line JSON processor.

Stephen Dolan 22.7k Aug 2, 2022
json-cpp is a C++11 JSON serialization library.

JSON parser and generator for C++ Version 0.1 alpha json-cpp is a C++11 JSON serialization library. Example #include <json-cpp.hpp> struct Foo {

Anatoly Scheglov 4 Dec 31, 2019
This is a JSON C++ library. It can write and read JSON files with ease and speed.

Json Box JSON (JavaScript Object Notation) is a lightweight data-interchange format. Json Box is a C++ library used to read and write JSON with ease a

Anhero inc. 108 Jul 7, 2022
A convenience C++ wrapper library for JSON-Glib providing friendly syntactic sugar for parsing JSON

This library is a wrapper for the json-glib library that aims to provide the user with a trivial alternative API to the API provided by the base json-

Rob J Meijer 16 May 10, 2022
json-build is a zero-allocation JSON serializer compatible with C89

json-build is a zero-allocation JSON serializer compatible with C89. It is inspired by jsmn, a minimalistic JSON tokenizer.

Lucas Müller 27 Jul 26, 2022
Ultralightweight JSON parser in ANSI C

cJSON Ultralightweight JSON parser in ANSI C. Table of contents License Usage Welcome to cJSON Building Copying the source CMake Makefile Vcpkg Includ

Dave Gamble 7.7k Aug 5, 2022
JSON parser and generator for C/C++ with scanf/printf like interface. Targeting embedded systems.

JSON parser and emitter for C/C++ Features ISO C and ISO C++ compliant portable code Very small footprint No dependencies json_scanf() scans a string

Cesanta Software 611 Aug 5, 2022
C library for encoding, decoding and manipulating JSON data

Jansson README Jansson is a C library for encoding, decoding and manipulating JSON data. Its main features and design principles are: Simple and intui

Petri Lehtinen 2.7k Aug 4, 2022
JSON & BSON parser/writer

jbson is a library for building & iterating BSON data, and JSON documents in C++14. \tableofcontents Features # {#features} Header only. Boost license

Chris Manning 39 May 12, 2022
A very sane (header only) C++14 JSON library

JeayeSON - a very sane C++14 JSON library JeayeSON was designed out of frustration that there aren't many template-based approaches to handling JSON i

Jeaye Wilkerson 128 Jun 7, 2022
Jsmn is a world fastest JSON parser/tokenizer. This is the official repo replacing the old one at Bitbucket

JSMN jsmn (pronounced like 'jasmine') is a minimalistic JSON parser in C. It can be easily integrated into resource-limited or embedded projects. You

Serge Zaitsev 3k Jul 29, 2022
JSON for Modern C++

Design goals Sponsors Integration CMake Package Managers Pkg-config Examples JSON as first-class data type Serialization / Deserialization STL-like ac

Niels Lohmann 31.1k Jul 31, 2022
A JSON parser in C++

JSON++ Introduction JSON++ is a light-weight JSON parser, writer and reader written in C++. JSON++ can also convert JSON documents into lossless XML d

Hong Jiang 485 Jul 23, 2022
🗄️ single header json parser for C and C++

??️ json.h A simple single header solution to parsing JSON in C and C++. JSON is parsed into a read-only, single allocation buffer. The current suppor

Neil Henning 506 Aug 2, 2022
A C++ library for interacting with JSON.

JsonCpp JSON is a lightweight data-interchange format. It can represent numbers, strings, ordered sequences of values, and collections of name/value p

null 6.6k Aug 4, 2022
Very low footprint JSON parser written in portable ANSI C

Very low footprint JSON parser written in portable ANSI C. BSD licensed with no dependencies (i.e. just drop the C file into your project) Never recur

James McLaughlin 1.2k Jul 28, 2022
A tiny JSON library for C++11.

json11 json11 is a tiny JSON library for C++11, providing JSON parsing and serialization. The core object provided by the library is json11::Json. A J

Dropbox 2.4k Aug 6, 2022
A killer modern C++ library for interacting with JSON.

JSON Voorhees Yet another JSON library for C++. This one touts new C++11 features for developer-friendliness, an extremely slow-speed parser and no de

Travis Gockel 125 Mar 4, 2022