Very low footprint JSON parser written in portable ANSI C

Fails with 1:104: Unexpected . in object

{ "status": "ok", "results": [ { "recordings": [ { "id": "889ec8e0-b8a6-4ff1-a104-5512ea49fe87" } ], "score": 0.879051, "id": "45047cb1-3d3f-477e-a3dc-f14e8254e78d" } ] }

VS2013 flagged a couple of possible issues and I'm not sure it it's right or not. Line 411:

else if (!state.settings.settings & json_relaxed_commas)

parses as ((!state.settings.settings) & json_relaxed_commas) and I think maybe it should be (!(state.settings.settings & json_relaxed_commas))

And likewise line 572:

if (flags & flag_need_comma && (!state.settings.settings & json_relaxed_commas))

A couple of lines trigger strict-aliasing warnings on gcc (Ubuntu)

error: dereferencing type-punned pointer will break strict-aliasing rules [-Werror=strict-aliasing]

Output of gcc --version

gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0

This PR addresses those two warnings

Commit d7b67db02aaa17fcc9bfbc8b60f41c3b677bd5a8 also simplifies the pointer notation by removing a reference and a dereference that seem to cancel each other out. Let's say that value->u.object.values is a pointer in memory location A and points to location B. Then | expression | type | pointer location | pointed location | |---|---|---|---| | value->u.object.values | json_object_entry * | A | B | | &value->u.object.values | json_object_entry ** | | A | | (char **) &value->u.object.values | char ** | | A | | *(char **) &value->u.object.values | char * | | B |

The last expression can be simplified in (char *) value->u.object.values

From https://github.com/udp/json-parser/blob/e6426ae/json.c#L49-L55. That value can be calculated mathematically using 2**((n * 8) - 1) - 1. Which can also be done using the following bit shift:

static const json_int_t JSON_INT_MAX = (1L << ((8 * sizeof(json_int_t) - 1))) - 1

Hi,

The parser fails when the following JSON is provided: {"myArray": [] }

In code: static char str[] = "{ "myArray": [] }"; json_value * root = json_parse(str, strlen(str));

This is because malloc is called with a size of 0, due to the empty array. However some implementations of malloc will return NULL when asked to allocate 0 bytes and in this case the parser stops. I think this is wrong. In my opinion the parser should not try to allocate 0 bytes. A simple check in the newValue method for the json_array case can prevent this:

In code: if (value->u.array.length == 0) { break; }

With these three lines of code the parser can also handle the above input.

Regards, Matthijs

The compiler on my system is 64 bit and sizeof(void *) is 8.

json-parser\json.c: In function 'new_value':
json-parser\json.c:135:42: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]
                   (state, values_size + ((unsigned long) value->u.object.values), 0)) )
                                          ^

https://github.com/udp/json-parser/blob/master/json.c#L141 In context, I'm not sure what this is supposed to be doing, so I don't have a proposed solution.

[100%] Building C object AudioAgent/external/CMakeFiles/JsonParser.dir/json-parser/json.c.o cc: warning: argument unused during compilation: '-no-cpp-precomp' /Users/johank/safe/audio/trunk/AudioAgent/external/json-parser/json.c:42:51: error: missing field 'type' initializer [-Werror,-Wmissing-field-initializers] const struct _json_value json_value_none = { 0 }; ^ I would suggest changing to

const struct _json_value json_value_none;

since static variables should anyway be zero initialized by default.

Hey

I just tried compiling this with Raisonance's RC51 compiler, and it threw this error:

*** ERROR C203 IN LINE 638 OF json.c : '(XDATA  CODE)': pointer: different mspace

on the first lines of e_failed.

With those lines commented out it worked fine.

The C standard library consistently uses size_t for variables related to memory size. See for example malloc(), memset(), sizeof().

The maximum value of an object of size_t type is:

• SIZE_MAX, defined in <limits.h> starting with C99,
• ((size_t)(-1)), if SIZE_MAX is not defined.

Note that we cannot change unsigned longto size_t everywhere without modifying the API, so we will probably have to live with this exception:

typedef struct
{
   unsigned long max_memory;
   [...]
} json_settings;

Perhaps it's time we change:

 * Copyright (C) 2012, 2013, 2014, 2016, 2018, 2019, 2021 James McLaughlin et al.  All rights reserved.

to:

 * Copyright (C) 2012-2021 James McLaughlin et al.  All rights reserved.

json.h: <inttypes.h> → <stdint.h> int64_t is defined in <stdint.h>, no need for the full <inttypes.h>

json.h: <stdlib.h> → <stddef.h> <stddef.h> is enough to define size_t

json.c: <stdlib.h> Now that <stdlib.h> has been removed from json.h, add it to json.c for malloc()/calloc() and free()

I came across your library while investigating security of a widely deployed product using it. I found the overflow checks to be very lacking.

The calculations performed under https://github.com/json-parser/json-parser/blob/94f66d8f83c1d84ccccd7540ec2f51cf8325d272/json.c#L154 and https://github.com/json-parser/json-parser/blob/94f66d8f83c1d84ccccd7540ec2f51cf8325d272/json.c#L167 can overflow even if length stays below UINT_MAX. For 32 bit applications both size_t and unsigned int are 32 bit values. As a result, this code can handle at most 0x40000000 array entries and at most 0x13B13B13 object entries before the allocation size overflows resulting in far too little memory being allocated. Once array/object data is written into this buffer, it overflows which could result in code execution.

Luckily, this allocation only happens in the second pass. Your code will allocate all the json_value instances in the first pass however, resulting in an out of memory error before this allocation can be reached. So with the current logic this buffer overflow vulnerability doesn’t seem exploitable. It might turn exploitable however if you change the allocation pattern or implement a streaming mode. So I sincerely recommend fixing this ASAP.

I’m not sure what the best course of action is here. gcc and clang compilers allow you to perform arithmetic operations with overflow checking. I suspect that you cannot expect these builtins to exist however. So in case where you don’t have them, maybe https://www.fefe.de/intof.html is of help to construct similar functionality.

It might be a good idea to also drop the current overflow checks which rely on sizes never growing by more than 8 at once. Instead, whenever you increase a length you should use addition with overflow checking.

The description says that this library never allocates more memory than it needs. That's nice but what is needed? Other JSON parser libs make explicit statements about whether they allocate any memory for the parsed data. This one looks like it needs to allocate memory for all parsed values, including all strings (providing a zero-terminated copy) and also those object keys that I'm not interested in. I'm looking for a JSON parser to use on an ESP32 microcontroller in conjunction with MQTT and WiFi. There is some RAM available but I'd like to keep its use for this task to a minimum.

I don't need the original JSON string anymore after parsing, so destroying it and providing pointers into it would be fine with me (adding terminating zero and shifting backslash sequences in-place).

If that's not supported, would I be able to prevent running out of memory by limiting the accepted JSON string length so something reasonable for my application?

CodeQL Action v1 will be deprecated on December 7th, 2022.

For more information, see: https://github.blog/changelog/2022-04-27-code-scanning-deprecation-of-codeql-action-v1/

To reproduce, I built the code 'examples' with the following command on my mac:

cd examples
clang test_json.c ../json.c -I ../ -fsanitize=undefined

Then when I run:

./a.out ../tests/valid-0000.json

I get this runtime warning:

../json.c:437:34: runtime error: applying non-zero offset 8 to null pointer

It seems to work if I replace in the code the line:

json_car **char = (json_char **) &top->u.object.values;

by:

intptr_t *chars = (intptr_t *) &top->u.object.values;

Looking on https://github.com/json-parser/json-parser/compare/v1.1.0...master I think that it would be good to flush all those commits and make new release.

Pkgconfig file needs to be installed in $(libdir) rather than $(datadir), otherwise it is not possible to support multilib systems properly, e.g. 32 and 64 bits library on one system, where location must differ /usr/lib/pkgconfig for 32bit and /usr/lib64/pkgconfig for 64bit, for example.