Thank you for the library. It may be just what we need for a project on an (embedded) AVR platform. We're specifically interested in the schema validation feature for validating some relatively simple JSON blobs.

I'm having trouble figuring out what I presume is a memory allocation problem.

We have a pretty capable build environment in which we can run unit tests on our production AVR code natively (OS X and Linux) and build some code libraries for all three platforms. I was able to strip down wjelement (removed 64 bit support, etc.) and add missing functionality (strndup()) such that the library would build both for AVR release and our native testing environments.

I have been successful in using the slimmed down library to validate JSON in our native test environment. Now I'm working to see if wjelement will run on our AVR chip. At first everything died when I put validation calls in our existing code. I soon figured out that it was because of memory allocation issues.

The default internal memory allocation is for 4k RAM in WJROpenDocument(), but we only have 8k RAM total!

I tweaked _WJEParse() to take a buffer allocation size value as a parameter. With that change and smaller memory allocation calls to _WJEParse() for a test JSON blob and a schema string both parsing calls now succeed. Unfortunately, the firmware crashes upon trying to validate the JSON blob against the schema. I assume this is more memory allocation issues.

1. What kind of memory optimization might I be able to do in the guts of WJESchemaValidate()? Is it even worth trying? I'm hoping there's more memory blocks whose default sizes I can cut down.
2. Could you direct me on how to change the code to more gracefully error out if memory allocation fails so I can track down the issue and robustify validation for our use case?

Anything you can say is very much appreciated.

Segmentation fault

Whe using the following code to test the lib,I found "Segmentation fault": for(int i =0; i<10000;i++){ char json[] = "{"name":"Serenity","class":"firefly","crew":[{"name":"Malcolm Reynolds","job":"captain","born":2486},{"name":"Kaywinnet Lee Fry","job":"mechanic","born":2494},{"name":"Jayne Cobb","job":"public relations","born":2485}]}"; WJReader jsonReader =WJROpenMemDocument(json,NULL,0); WJElement jsonDoc = WJEOpenDocument(jsonReader, NULL, NULL, NULL); WJEDump(jsonDoc); WJECloseDocument(jsonDoc); }

when i >6000

I've tried a lot of things stepping thru code and such, but how do I do schema validation against just a portion of a schema file. I tried using the "where" parameter of the WJEOpenDocument, but wasn't quite sure if that was right or what to pass in.

Sample schema: { "properties":{}, "validateAgainstThis": { "type": "object", "properties": { } } }

I have looked at a few source files for your current software. I have noticed that some checks for return codes are missing.

Would you like to add more error handling for return values from functions like the following?

• MemStrdup ⇒ _WJECopy
• printf ⇒ usage

When using the following code: WJReader r = WJROpenMemDocument(buffer, NULL, 0); handle = WJEOpenDocument(r,NULL,NULL,NULL);

I've noted that the performance degrade as a function of n.

Unfortunately it's degrading in a non-linear way: json parsing of 100000 items took : 0.204 per-element time : 0.000002 json parsing of 200000 items took : 0.697 per-element time : 0.000003 json parsing of 300000 items took : 1.465 per-element time : 0.000005 json parsing of 400000 items took : 2.639 per-element time : 0.000007 json parsing of 500000 items took : 4.190 per-element time : 0.000008 json parsing of 600000 items took : 6.559 per-element time : 0.000011 json parsing of 700000 items took : 9.934 per-element time : 0.000014 json parsing of 800000 items took : 14.148 per-element time : 0.000018 json parsing of 900000 items took : 18.535 per-element time : 0.000021

Any suggestions would be appreciated.

Hi, I want to compare to json objects , could you suggest any API. Can I use Comparejson API by passing two json object for comparision in c language.

Thanks, Mohan

Consider a simple schema like: { "$schema": "http://json-schema.org/draft-03/schema#", "type": "array", "items": { "type": "object", "properties": { "headers": { "type": "object", "required": true, "properties": { "messageId": { "type": "string", "required": true } } }, "properties": { "type": "object", "required": false }, "body": { "type": "string", "required": true } } } }

If the "body" element is ~600 byte string, it takes 12K RAM to validate. If the "body" element is ~308K byte string, it takes 17.6M RAM to validate. If the "body" element is ~9.1M byte string, it takes 16.9G (yes, giga bytes) RAM to validate.

Schema:

{
    "$schema": "http://json-schema.org/draft-03/schema#",
    "type": "object",
    "properties": {
      "key": {
        "type": "string",
        "pattern": "ABC"
      }
    }
}

JSON String to validate

{
  "key": "123"
}

Example

   std::string jsonString = "{\n  \"key\": \"123\"\n}";
   WJReader reader = WJROpenMemDocument((void*)jsonString.c_str(), NULL, 0);
   WJElement doc = WJEOpenDocument(reader, NULL, NULL, NULL);

   std::string schemaString = "{\n    \"$schema\": \"http://json-schema.org/draft-03/schema#\",\n    \"type\": \"object\",\n    \"properties\": {\n      \"key\": {\n        \"type\": \"string\",\n        \"pattern\": \"ABC\"\n      }\n    }\n}";
   WJReader SchemaReader = WJROpenMemDocument((void*)schemaString.c_str(), NULL, 0);
   WJElement SchemaDoc = WJEOpenDocument(SchemaReader, NULL, NULL, NULL);

   if(WJESchemaValidate(
      SchemaDoc, doc,
      NULL, NULL, NULL, NULL))
   {
      std::cout << "schema matches\n";
   }
   else
   {
      std::cout << "schema does not match\n";
   }

This prints "schema matches\n", as does any other setting of the pattern.

I would like to point out that identifiers like "__WJEBool" and "_WJROpenDocument" do not fit to the expected naming convention of the C language standard. Would you like to adjust your selection for unique names?

Section 3.2.1 of JSON Schema Validation specification states:

Most validation assertions only constrain values within a certain primitive type. When the type of the instance is not of the type targeted by the keyword, the instance is considered to conform to the assertion.

If the required keyword is used with an array value, per v4 syntax, it is asserted even if the instance being tested is not an object.

For example: {"required":["name1"]} - Schema {"name1":null} - Instance conforms {} - Instance does not conform null - Instance conforms but WJESchemaValidate() claims it does not

Below is code for an example console application, expected output and actual output.

#include <stdarg.h>
#include "stdafx.h"
#include "wjelement.h"

void ErrCB(void *client, const char *format, ...)
{
	va_list args;
	va_start(args, format);
	vprintf(format, args);
	va_end(args);
	printf("\n");
}

void RunTest(int index, WJElement schema, char* json)
{
	if (WJElement document = WJEParse(json)) {
		printf("Test %d\n", index);
		if (WJESchemaValidate(schema, document, &ErrCB, NULL, NULL, NULL)) {
			printf("OK\n");
		}
		WJECloseDocument(document);
		printf("\n");
	}
}

int main()
{
	XplBool valid;
	if (WJElement schema = WJEParse("{\"required\":[\"name1\"]}")) {
		RunTest(1, schema, "{\"name1\":null}");
		RunTest(2, schema, "{}");
		RunTest(3, schema, "null");
		WJECloseDocument(schema);
	}
	return 0;
}

The expected output is as follows:

Test 1 OK

Test 2 (root): required member 'name1' not found.

Test 3 OK

The actual output is as follows:

Test 1 OK

Test 2 (root): required member 'name1' not found.

Test 3 (root): required member 'name1' not found.

With this schema…

{
	"$schema": "http://json-schema.org/draft-04/schema#",
	"type": "object",
	"patternProperties": {
		"^string[0-9]+$": {
			"type": "string"
		}
	},
	"additionalProperties": false
}

and this document…

{"foo":"bar"}

…since the key doesn't match patternProperties, and additionalProperties is false, I'd expect the document to fail schema validation. But, with wjelement-1.2, it passes:

echo 'validate patternProperties-noAdditional.jsonschema' | ./wjecli test.json
Schema validation: PASS

Our code

#include <iterator>
#include <list>
#include <vector>
#include "element.h"

On AIX

-bash-4.4# cat /usr/include/sys/inttypes.h | grep "int32;"
typedef signed int              int32;
typedef unsigned int            u_int32;

Compile output

wjelement-1.2/include/xpl.h:140:21: error: conflicting declaration 'typedef long int int32'
 typedef signed long int32;
                     ^
/usr/include/sys/inttypes.h:620:21: error: 'int32' has a previous declaration as 'typedef int int32'
 typedef signed int  int32;

<iterator> had a whole chain that leads to inttypes.h. inttypes.h does #include <stdint.h> which defines this:

-bash-4.4# cat /usr/include/sys/stdint.h | grep INT32_MAX
#define INT32_MAX       (2147483647)

I will note that I've solved this one already for us as we were not using <iterator>, <list>, or <vector> in that file, so I just removed them.

In xpl.h EXPORT is being defined. This conflicts with a Microsoft lib

...\windows kits\10\include\10.0.15063.0\um\sqltypes.h

Perhaps a namespace could be added or just a rename of the symbol?

If WJWNull is called with non-null "name" argument, it returns "FALSE", though name is created:

result = WJWNull("error", wjWriter);

Looks like error is caused by line 894: return(6 == WJWrite(doc, ":null", 5));

Argument ":null" is 5 bytes long, WJWrite returns 5, but it is compared with "6".

If the flag is set on a WJEAction, it is not used by the following WJEGet call in search.c:

static int WJECheckCondition(WJElement e, char **condition, WJEAction action) { .... if (';' == *cond) { if (WJEGet(e, value, NULL)) { if (condition) *condition = NULL;

Hello, I really like the library. Is there any safe way to load a document from a socket having its descriptor? How do I deal with framing and merging two different documents?

Hi, I've just tried schema-validation, the results were suboptimal:

echo 'validate json-schema-draft-04.json' | wjecli json-schema-draft-04.json
definitions: extra property 'schemaArray' found.
definitions: extra property 'positiveInteger' found.
definitions: extra property 'positiveIntegerDefault0' found.
definitions: extra property 'simpleTypes' found.
definitions: extra property 'stringArray' found.
definitions failed validation
properties: extra property 'id' found.
properties: extra property '$schema' found.
...

As a start, the schema should be valid against itself, but even if it isn't, the error message should at least contain a line-number...