mirror of
https://github.com/nlohmann/json.git
synced 2026-11-14 03:35:31 +00:00
deploy: 81888ff9f0
This commit is contained in:
nlohmann
@@ -58,7 +58,9 @@ is not UTF-8 encoded and `error_handler` is set to `strict`
|
||||
|
||||
## Complexity
|
||||
|
||||
Linear.
|
||||
Linear in the size of the result. With an `indent`, every line starts with up to `indent` characters per nesting level,
|
||||
so the result can be larger than the value by a factor of its nesting depth (see
|
||||
[Parsing untrusted input](../../features/parsing/untrusted_input.md#nesting-depth)).
|
||||
|
||||
## Notes
|
||||
|
||||
|
||||
File diff suppressed because one or more lines are too long
@@ -39,7 +39,7 @@ See the [FAQ](https://json.nlohmann.me/home/faq/#serializing-untrusted-or-invali
|
||||
|
||||
## Complexity
|
||||
|
||||
Linear.
|
||||
Linear in the size of the result. With an `indent`, every line starts with up to `indent` characters per nesting level, so the result can be larger than the value by a factor of its nesting depth (see [Parsing untrusted input](https://json.nlohmann.me/features/parsing/untrusted_input/#nesting-depth)).
|
||||
|
||||
## Notes
|
||||
|
||||
|
||||
@@ -18,7 +18,8 @@ Strong exception safety: if an exception occurs, the original value stays intact
|
||||
|
||||
## Complexity
|
||||
|
||||
Linear in the size of the JSON value.
|
||||
Linear in the size of the result. Every key repeats the path of its value, so the keys can be larger than the value by a
|
||||
factor of its nesting depth (see [Parsing untrusted input](../../features/parsing/untrusted_input.md#nesting-depth)).
|
||||
|
||||
## Notes
|
||||
|
||||
|
||||
File diff suppressed because one or more lines are too long
@@ -16,7 +16,7 @@ Strong exception safety: if an exception occurs, the original value stays intact
|
||||
|
||||
## Complexity
|
||||
|
||||
Linear in the size of the JSON value.
|
||||
Linear in the size of the result. Every key repeats the path of its value, so the keys can be larger than the value by a factor of its nesting depth (see [Parsing untrusted input](https://json.nlohmann.me/features/parsing/untrusted_input/#nesting-depth)).
|
||||
|
||||
## Notes
|
||||
|
||||
|
||||
@@ -69,8 +69,10 @@ Strong guarantee: if an exception is thrown, there are no changes in the JSON va
|
||||
|
||||
## Exceptions
|
||||
|
||||
- Throws [parse_error.110](../../home/exceptions.md#jsonexceptionparse_error110) if the given input ends prematurely or
|
||||
the end of the file was not reached when `strict` was set to true
|
||||
- Throws [parse_error.110](../../home/exceptions.md#jsonexceptionparse_error110) if the given input ends prematurely,
|
||||
if the end of the file was not reached when `strict` was set to true, or if a string at the end of the value is not
|
||||
terminated by 0xFF although a byte follows it (also when `strict` is false: the byte that ends the string is
|
||||
taken from the input, so the next value would be lost)
|
||||
- Throws [parse_error.112](../../home/exceptions.md#jsonexceptionparse_error112) if a parse error occurs, for instance
|
||||
an invalid byte, a string that is not valid UTF-8, or an object key that is not a string
|
||||
|
||||
|
||||
File diff suppressed because one or more lines are too long
@@ -64,7 +64,7 @@ Strong guarantee: if an exception is thrown, there are no changes in the JSON va
|
||||
|
||||
## Exceptions
|
||||
|
||||
- Throws [parse_error.110](https://json.nlohmann.me/home/exceptions/#jsonexceptionparse_error110) if the given input ends prematurely or the end of the file was not reached when `strict` was set to true
|
||||
- Throws [parse_error.110](https://json.nlohmann.me/home/exceptions/#jsonexceptionparse_error110) if the given input ends prematurely, if the end of the file was not reached when `strict` was set to true, or if a string at the end of the value is not terminated by 0xFF although a byte follows it (also when `strict` is false: the byte that ends the string is taken from the input, so the next value would be lost)
|
||||
- Throws [parse_error.112](https://json.nlohmann.me/home/exceptions/#jsonexceptionparse_error112) if a parse error occurs, for instance an invalid byte, a string that is not valid UTF-8, or an object key that is not a string
|
||||
|
||||
The UTF-8 validity of strings is always checked, independent of [`JSON_STRICT_BINARY_UTF8`](https://json.nlohmann.me/api/macros/json_strict_binary_utf8/index.md), because the end of a BON8 string is determined from its encoding. Duplicate object keys are accepted; the last value wins (see [BON8](https://json.nlohmann.me/features/binary_formats/bon8/index.md)).
|
||||
|
||||
@@ -27,7 +27,7 @@ comparison of two [`json_view`](../basic_json_view/index.md) documents has a fix
|
||||
and does not depend on it either.
|
||||
|
||||
The call stack that these operations need grows with the limit: roughly *a few kilobytes plus a cost per level times the
|
||||
limit*, where the cost per level is up to about 2.3 KiB in an unoptimized and up to about 0.5 KiB in an optimized build
|
||||
limit*, where the cost per level is up to about 1.8 KiB in an unoptimized and up to about 0.5 KiB in an optimized build
|
||||
(see [Stack needed](#stack-needed)). The default of `128` levels is tuned for threads with ordinary stacks of a few
|
||||
hundred kilobytes or more. Set a lower limit if your threads have small stacks (RTOS tasks, fibers, threads of the musl
|
||||
C
|
||||
@@ -67,7 +67,7 @@ When the macro is not defined, the library defines it to its default value.
|
||||
the levels that are descended into are counted, so the count never exceeds the limit, not even when user code (a
|
||||
copy constructor of a custom base class, say) copies a value while a copy is going on. A byte holds `255` at
|
||||
most, and the limit stays below that, so that the count cannot wrap around. (More levels would hardly be
|
||||
useful: `254` levels need up to 576 KiB of stack in an unoptimized build already, twice as much as the
|
||||
useful: `254` levels need up to 448 KiB of stack in an unoptimized build already, twice as much as the
|
||||
default.)
|
||||
|
||||
### Stack needed
|
||||
@@ -83,7 +83,7 @@ Unoptimized (`-O0`):
|
||||
|
||||
| Operation | limit 16 | limit 32 | limit 64 | limit 128 | limit 254 |
|
||||
|---|---:|---:|---:|---:|---:|
|
||||
| `dump()` and `dump(indent)`, `operator<<` | 48 / 32 | 80 / 64 | 144 / 128 | 288 / 256 | 576 / 512 |
|
||||
| `dump()` and `dump(indent)`, `operator<<` | 16 / 16 | 16 / 16 | 32 / 32 | 48 / 48 | 96 / 96 |
|
||||
| copy construction and assignment | 32 / 16 | 48 / 32 | 112 / 80 | 224 / 144 | 432 / 304 |
|
||||
| `==` | 16 / 16 | 16 / 16 | 16 / 16 | 48 / 32 | 96 / 48 |
|
||||
| `<=>` (C++20) | 16 / 16 | 16 / 16 | 32 / 32 | 64 / 64 | 144 / 128 |
|
||||
@@ -123,23 +123,24 @@ The rule of thumb is
|
||||
|
||||
> stack needed ≈ a few KiB + cost per level × limit
|
||||
|
||||
with a cost per level of up to 2.3 KiB (`dump` at `-O0` with Clang; 2.1 KiB with GCC), 1.7 KiB for copying and merging
|
||||
(1.3 KiB with GCC), and 0.1 to 0.5 KiB for most other operations at `-O0`; and 0.5 KiB or less at `-O2`. The numbers
|
||||
come from the difference between the limits `128` and `254` in the tables, so they are rough: other compilers,
|
||||
other targets (the frames of x86-64 code are about as large), other standard libraries, `JSON_DIAGNOSTICS`, and
|
||||
sanitizers change them. Measure with your own tool chain (see the example below) rather than relying on the tables.
|
||||
The stack that your own code and the operating system need comes on top of these numbers.
|
||||
with a cost per level of up to 1.8 KiB (conversion to another `basic_json` type at `-O0` with Clang; 1.3 KiB with
|
||||
GCC), 1.7 KiB for copying and merging (1.3 KiB with GCC), 0.4 KiB for `dump`, and 0.1 to 0.5 KiB for most other
|
||||
operations at `-O0`; and 0.5 KiB or less at `-O2`. The numbers come from the difference between the limits `128` and
|
||||
`254` in the tables, so they are rough: other compilers, other targets (the frames of x86-64 code are about as large),
|
||||
other standard libraries, `JSON_DIAGNOSTICS`, and sanitizers change them. Measure with your own tool chain (see the
|
||||
example below) rather than relying on the tables. The stack that your own code and the operating system need comes on
|
||||
top of these numbers.
|
||||
|
||||
Which limit to choose, for the stack that you can spare for the library (the largest limit of those measured
|
||||
for which the slowest operation needs at most that much):
|
||||
|
||||
| Stack for the library | optimized (`-O2`) | unoptimized (`-O0`) |
|
||||
|---|---:|---:|
|
||||
| 32 KiB | 64 | 0 |
|
||||
| 64 KiB | 128 | 16 |
|
||||
| 128 KiB (the default stack of threads on musl) | 254 | 32 |
|
||||
| 256 KiB | 254 | 64 |
|
||||
| 512 KiB | 254 | 128 |
|
||||
| 32 KiB | 64 | 16 |
|
||||
| 64 KiB | 128 | 32 |
|
||||
| 128 KiB (the default stack of threads on musl) | 254 | 64 |
|
||||
| 256 KiB | 254 | 128 |
|
||||
| 512 KiB | 254 | 254 |
|
||||
| 1 MiB or more | 254 | 254 |
|
||||
|
||||
### Speed
|
||||
|
||||
File diff suppressed because one or more lines are too long
@@ -8,7 +8,7 @@ This macro sets how many nesting levels the operations that walk a value recurse
|
||||
|
||||
The operations are copying a value, comparing two values, serializing with [`dump`](https://json.nlohmann.me/api/basic_json/dump/index.md) or [`operator<<`](https://json.nlohmann.me/api/operator_ltlt/index.md), hashing with `std::hash`, [`merge_patch`](https://json.nlohmann.me/api/basic_json/merge_patch/index.md), [`update`](https://json.nlohmann.me/api/basic_json/update/index.md) with `merge_objects`, [`diff`](https://json.nlohmann.me/api/basic_json/diff/index.md), [`merge_diff`](https://json.nlohmann.me/api/basic_json/merge_diff/index.md), converting a value to another specialization of `basic_json`, destroying a value, and the binary writers [`to_cbor`](https://json.nlohmann.me/api/basic_json/to_cbor/index.md), [`to_msgpack`](https://json.nlohmann.me/api/basic_json/to_msgpack/index.md), [`to_ubjson`](https://json.nlohmann.me/api/basic_json/to_ubjson/index.md), [`to_bjdata`](https://json.nlohmann.me/api/basic_json/to_bjdata/index.md), and [`to_bon8`](https://json.nlohmann.me/api/basic_json/to_bon8/index.md). They share this one limit; none has a limit of its own. Recursing is the fastest way to walk a value, but a value nested deeply enough would exhaust the call stack, so the library recurses only as far as the limit and finishes the levels below with an explicit stack. This is what makes all of them safe for untrusted input (see [Nesting depth](https://json.nlohmann.me/features/parsing/untrusted_input/#nesting-depth)). Parsing, the binary readers, [`flatten`](https://json.nlohmann.me/api/basic_json/flatten/index.md), [`unflatten`](https://json.nlohmann.me/api/basic_json/unflatten/index.md), and [`to_bson`](https://json.nlohmann.me/api/basic_json/to_bson/index.md) always use an explicit stack and do not depend on the macro. The lockstep comparison of two [`json_view`](https://json.nlohmann.me/api/basic_json_view/index.md) documents has a fixed bound of its own, 48 levels, and does not depend on it either.
|
||||
|
||||
The call stack that these operations need grows with the limit: roughly *a few kilobytes plus a cost per level times the limit*, where the cost per level is up to about 2.3 KiB in an unoptimized and up to about 0.5 KiB in an optimized build (see [Stack needed](#stack-needed)). The default of `128` levels is tuned for threads with ordinary stacks of a few hundred kilobytes or more. Set a lower limit if your threads have small stacks (RTOS tasks, fibers, threads of the musl C library, which has a default stack of 128 KiB), or a higher limit to trade stack for speed on deeply nested values.
|
||||
The call stack that these operations need grows with the limit: roughly *a few kilobytes plus a cost per level times the limit*, where the cost per level is up to about 1.8 KiB in an unoptimized and up to about 0.5 KiB in an optimized build (see [Stack needed](#stack-needed)). The default of `128` levels is tuned for threads with ordinary stacks of a few hundred kilobytes or more. Set a lower limit if your threads have small stacks (RTOS tasks, fibers, threads of the musl C library, which has a default stack of 128 KiB), or a higher limit to trade stack for speed on deeply nested values.
|
||||
|
||||
The macro changes only how much stack is used and how fast the operations are. It never changes a result: every operation returns the same value for every setting of the limit, for values nested less deeply than the limit as well as for values nested more deeply.
|
||||
|
||||
@@ -29,7 +29,7 @@ Allowed values
|
||||
The value must be an integer constant expression from `0` to `254`; any other value (a negative number, a number above `254`, a floating-point number such as `254.0`, or `true` or `false`) is rejected at compile time with an error message that names the macro. A character of any character type (`'a'`, `L'a'`, `u8'a'`, ...) is rejected, too, although it is an integer.
|
||||
|
||||
- `0` is allowed, and means that every operation uses its explicit stack from the first level on and no operation recurses over the nesting of a value at all. That needs a constant amount of call stack (at most 16 KiB in the measurements below, for a value nested 20,000 levels deep) and is the setting for tiny stacks, at the price of speed (see [Speed](#speed)).
|
||||
- `254` is the largest allowed value. Copying a value and comparing two values (with `<=>`, or with the operators on objects and arrays of a custom type) cannot pass the depth down as an argument, because the signatures of the copy constructor and the operators are fixed. They count the levels they have descended into in a variable of type `std::uint8_t` that exists once per thread (see [`JSON_NO_THREAD_LOCAL`](https://json.nlohmann.me/api/macros/json_no_thread_local/index.md)). Only the levels that are descended into are counted, so the count never exceeds the limit, not even when user code (a copy constructor of a custom base class, say) copies a value while a copy is going on. A byte holds `255` at most, and the limit stays below that, so that the count cannot wrap around. (More levels would hardly be useful: `254` levels need up to 576 KiB of stack in an unoptimized build already, twice as much as the default.)
|
||||
- `254` is the largest allowed value. Copying a value and comparing two values (with `<=>`, or with the operators on objects and arrays of a custom type) cannot pass the depth down as an argument, because the signatures of the copy constructor and the operators are fixed. They count the levels they have descended into in a variable of type `std::uint8_t` that exists once per thread (see [`JSON_NO_THREAD_LOCAL`](https://json.nlohmann.me/api/macros/json_no_thread_local/index.md)). Only the levels that are descended into are counted, so the count never exceeds the limit, not even when user code (a copy constructor of a custom base class, say) copies a value while a copy is going on. A byte holds `255` at most, and the limit stays below that, so that the count cannot wrap around. (More levels would hardly be useful: `254` levels need up to 448 KiB of stack in an unoptimized build already, twice as much as the default.)
|
||||
|
||||
### Stack needed
|
||||
|
||||
@@ -37,21 +37,21 @@ The tables show the smallest stack that an operation needs, in KiB, at several l
|
||||
|
||||
Unoptimized (`-O0`):
|
||||
|
||||
| Operation | limit 16 | limit 32 | limit 64 | limit 128 | limit 254 |
|
||||
| ----------------------------------------- | -------- | -------- | --------- | --------- | --------- |
|
||||
| `dump()` and `dump(indent)`, `operator<<` | 48 / 32 | 80 / 64 | 144 / 128 | 288 / 256 | 576 / 512 |
|
||||
| copy construction and assignment | 32 / 16 | 48 / 32 | 112 / 80 | 224 / 144 | 432 / 304 |
|
||||
| `==` | 16 / 16 | 16 / 16 | 16 / 16 | 48 / 32 | 96 / 48 |
|
||||
| `<=>` (C++20) | 16 / 16 | 16 / 16 | 32 / 32 | 64 / 64 | 144 / 128 |
|
||||
| `std::hash` | 16 / 16 | 16 / 16 | 32 / 32 | 80 / 64 | 160 / 128 |
|
||||
| `merge_patch`, `update(j, true)`, `patch` | 32 / 16 | 48 / 32 | 112 / 80 | 224 / 144 | 432 / 304 |
|
||||
| `diff` | 16 / 16 | 16 / 16 | 32 / 32 | 64 / 48 | 144 / 96 |
|
||||
| `merge_diff` | 16 / 16 | 48 / 32 | 80 / 48 | 176 / 112 | 352 / 224 |
|
||||
| `to_cbor`, `to_msgpack` | 16 / 16 | 32 / 16 | 64 / 32 | 112 / 80 | 224 / 160 |
|
||||
| `to_ubjson`, `to_bjdata` | 16 / 16 | 16 / 16 | 48 / 32 | 96 / 64 | 176 / 112 |
|
||||
| `to_bon8` | 16 / 16 | 16 / 16 | 16 / 16 | 32 / 16 | 64 / 48 |
|
||||
| conversion to another `basic_json` type | 32 / 16 | 64 / 32 | 112 / 80 | 224 / 144 | 448 / 304 |
|
||||
| destruction | 16 / 16 | 16 / 16 | 16 / 16 | 16 / 16 | 32 / 32 |
|
||||
| Operation | limit 16 | limit 32 | limit 64 | limit 128 | limit 254 |
|
||||
| ----------------------------------------- | -------- | -------- | -------- | --------- | --------- |
|
||||
| `dump()` and `dump(indent)`, `operator<<` | 16 / 16 | 16 / 16 | 32 / 32 | 48 / 48 | 96 / 96 |
|
||||
| copy construction and assignment | 32 / 16 | 48 / 32 | 112 / 80 | 224 / 144 | 432 / 304 |
|
||||
| `==` | 16 / 16 | 16 / 16 | 16 / 16 | 48 / 32 | 96 / 48 |
|
||||
| `<=>` (C++20) | 16 / 16 | 16 / 16 | 32 / 32 | 64 / 64 | 144 / 128 |
|
||||
| `std::hash` | 16 / 16 | 16 / 16 | 32 / 32 | 80 / 64 | 160 / 128 |
|
||||
| `merge_patch`, `update(j, true)`, `patch` | 32 / 16 | 48 / 32 | 112 / 80 | 224 / 144 | 432 / 304 |
|
||||
| `diff` | 16 / 16 | 16 / 16 | 32 / 32 | 64 / 48 | 144 / 96 |
|
||||
| `merge_diff` | 16 / 16 | 48 / 32 | 80 / 48 | 176 / 112 | 352 / 224 |
|
||||
| `to_cbor`, `to_msgpack` | 16 / 16 | 32 / 16 | 64 / 32 | 112 / 80 | 224 / 160 |
|
||||
| `to_ubjson`, `to_bjdata` | 16 / 16 | 16 / 16 | 48 / 32 | 96 / 64 | 176 / 112 |
|
||||
| `to_bon8` | 16 / 16 | 16 / 16 | 16 / 16 | 32 / 16 | 64 / 48 |
|
||||
| conversion to another `basic_json` type | 32 / 16 | 64 / 32 | 112 / 80 | 224 / 144 | 448 / 304 |
|
||||
| destruction | 16 / 16 | 16 / 16 | 16 / 16 | 16 / 16 | 32 / 32 |
|
||||
|
||||
Optimized (`-O2`):
|
||||
|
||||
@@ -77,17 +77,17 @@ The rule of thumb is
|
||||
|
||||
> stack needed ≈ a few KiB + cost per level × limit
|
||||
|
||||
with a cost per level of up to 2.3 KiB (`dump` at `-O0` with Clang; 2.1 KiB with GCC), 1.7 KiB for copying and merging (1.3 KiB with GCC), and 0.1 to 0.5 KiB for most other operations at `-O0`; and 0.5 KiB or less at `-O2`. The numbers come from the difference between the limits `128` and `254` in the tables, so they are rough: other compilers, other targets (the frames of x86-64 code are about as large), other standard libraries, `JSON_DIAGNOSTICS`, and sanitizers change them. Measure with your own tool chain (see the example below) rather than relying on the tables. The stack that your own code and the operating system need comes on top of these numbers.
|
||||
with a cost per level of up to 1.8 KiB (conversion to another `basic_json` type at `-O0` with Clang; 1.3 KiB with GCC), 1.7 KiB for copying and merging (1.3 KiB with GCC), 0.4 KiB for `dump`, and 0.1 to 0.5 KiB for most other operations at `-O0`; and 0.5 KiB or less at `-O2`. The numbers come from the difference between the limits `128` and `254` in the tables, so they are rough: other compilers, other targets (the frames of x86-64 code are about as large), other standard libraries, `JSON_DIAGNOSTICS`, and sanitizers change them. Measure with your own tool chain (see the example below) rather than relying on the tables. The stack that your own code and the operating system need comes on top of these numbers.
|
||||
|
||||
Which limit to choose, for the stack that you can spare for the library (the largest limit of those measured for which the slowest operation needs at most that much):
|
||||
|
||||
| Stack for the library | optimized (`-O2`) | unoptimized (`-O0`) |
|
||||
| ---------------------------------------------- | ----------------- | ------------------- |
|
||||
| 32 KiB | 64 | 0 |
|
||||
| 64 KiB | 128 | 16 |
|
||||
| 128 KiB (the default stack of threads on musl) | 254 | 32 |
|
||||
| 256 KiB | 254 | 64 |
|
||||
| 512 KiB | 254 | 128 |
|
||||
| 32 KiB | 64 | 16 |
|
||||
| 64 KiB | 128 | 32 |
|
||||
| 128 KiB (the default stack of threads on musl) | 254 | 64 |
|
||||
| 256 KiB | 254 | 128 |
|
||||
| 512 KiB | 254 | 254 |
|
||||
| 1 MiB or more | 254 | 254 |
|
||||
|
||||
### Speed
|
||||
|
||||
+12
-10
@@ -89,19 +89,21 @@ This differs from `#!cpp std::map`, where the same operations are O(log n).
|
||||
!!! warning "Quadratic cost of building large objects"
|
||||
|
||||
Because every insertion scans all elements inserted so far, building an object of `n` distinct keys costs
|
||||
**O(n²)** in total. This applies to filling an [`ordered_json`](ordered_json.md) object key by key as well as to
|
||||
parsing one, since the parser inserts each key as it is read.
|
||||
**O(n²)** in total. This applies to filling an [`ordered_json`](ordered_json.md) object key by key (with
|
||||
`operator[]`, `emplace`, or `insert`) and, for now, to reading one with a binary reader such as
|
||||
[`from_cbor`](basic_json/from_cbor.md) ([#6104](https://github.com/nlohmann/json/issues/6104)). Parsing JSON text
|
||||
is not affected: the parser finds the keys of an object with many members in a hash table and takes linear time.
|
||||
|
||||
The cost is negligible for the object sizes typically found in configuration files or API payloads, but it grows
|
||||
steeply for machine-generated objects with many thousands of keys. Measured with `-O2 -DNDEBUG` for parsing a flat
|
||||
object of `n` keys, relative to `#!cpp nlohmann::json` (which uses `#!cpp std::map`):
|
||||
steeply for machine-generated objects with many thousands of keys. Measured with Apple clang 21, `-O2 -DNDEBUG`,
|
||||
arm64, for a flat object of `n` keys (best of seven runs):
|
||||
|
||||
| `n` | `json` | `ordered_json` | factor |
|
||||
|--------|--------|----------------|--------|
|
||||
| 2000 | 0.7 ms | 3.6 ms | 5× |
|
||||
| 4000 | 0.8 ms | 14.0 ms | 19× |
|
||||
| 8000 | 1.6 ms | 67.8 ms | 43× |
|
||||
| 16 000 | 3.3 ms | 181.6 ms | 54× |
|
||||
| `n` | `json::parse` | `ordered_json::parse` | `ordered_json`, key by key | `ordered_json::from_cbor` |
|
||||
|--------|---------------|-----------------------|----------------------------|---------------------------|
|
||||
| 2000 | 0.3 ms | 0.1 ms | 4.5 ms | 4.4 ms |
|
||||
| 4000 | 0.5 ms | 0.2 ms | 17.1 ms | 16.8 ms |
|
||||
| 8000 | 0.9 ms | 0.3 ms | 67.0 ms | 66.8 ms |
|
||||
| 16 000 | 2.0 ms | 0.7 ms | 282.4 ms | 282.4 ms |
|
||||
|
||||
If key order matters for objects of that size, consider a container with a lookup index, such as
|
||||
[`nlohmann::fifo_map`](https://github.com/nlohmann/fifo_map)
|
||||
|
||||
File diff suppressed because one or more lines are too long
@@ -84,16 +84,16 @@ This differs from `std::map`, where the same operations are O(log n).
|
||||
|
||||
Quadratic cost of building large objects
|
||||
|
||||
Because every insertion scans all elements inserted so far, building an object of `n` distinct keys costs **O(n²)** in total. This applies to filling an [`ordered_json`](https://json.nlohmann.me/api/ordered_json/index.md) object key by key as well as to parsing one, since the parser inserts each key as it is read.
|
||||
Because every insertion scans all elements inserted so far, building an object of `n` distinct keys costs **O(n²)** in total. This applies to filling an [`ordered_json`](https://json.nlohmann.me/api/ordered_json/index.md) object key by key (with `operator[]`, `emplace`, or `insert`) and, for now, to reading one with a binary reader such as [`from_cbor`](https://json.nlohmann.me/api/basic_json/from_cbor/index.md) ([#6104](https://github.com/nlohmann/json/issues/6104)). Parsing JSON text is not affected: the parser finds the keys of an object with many members in a hash table and takes linear time.
|
||||
|
||||
The cost is negligible for the object sizes typically found in configuration files or API payloads, but it grows steeply for machine-generated objects with many thousands of keys. Measured with `-O2 -DNDEBUG` for parsing a flat object of `n` keys, relative to `nlohmann::json` (which uses `std::map`):
|
||||
The cost is negligible for the object sizes typically found in configuration files or API payloads, but it grows steeply for machine-generated objects with many thousands of keys. Measured with Apple clang 21, `-O2 -DNDEBUG`, arm64, for a flat object of `n` keys (best of seven runs):
|
||||
|
||||
| `n` | `json` | `ordered_json` | factor |
|
||||
| ------ | ------ | -------------- | ------ |
|
||||
| 2000 | 0.7 ms | 3.6 ms | 5× |
|
||||
| 4000 | 0.8 ms | 14.0 ms | 19× |
|
||||
| 8000 | 1.6 ms | 67.8 ms | 43× |
|
||||
| 16 000 | 3.3 ms | 181.6 ms | 54× |
|
||||
| `n` | `json::parse` | `ordered_json::parse` | `ordered_json`, key by key | `ordered_json::from_cbor` |
|
||||
| ------ | ------------- | --------------------- | -------------------------- | ------------------------- |
|
||||
| 2000 | 0.3 ms | 0.1 ms | 4.5 ms | 4.4 ms |
|
||||
| 4000 | 0.5 ms | 0.2 ms | 17.1 ms | 16.8 ms |
|
||||
| 8000 | 0.9 ms | 0.3 ms | 67.0 ms | 66.8 ms |
|
||||
| 16 000 | 2.0 ms | 0.7 ms | 282.4 ms | 282.4 ms |
|
||||
|
||||
If key order matters for objects of that size, consider a container with a lookup index, such as [`nlohmann::fifo_map`](https://github.com/nlohmann/fifo_map) ([integration](https://github.com/nlohmann/json/issues/485#issuecomment-333652309)), as the object type — see [object order](https://json.nlohmann.me/features/object_order/index.md).
|
||||
|
||||
|
||||
@@ -204,7 +204,10 @@ of the message are reported as [parse_error.110](../../home/exceptions.md#jsonex
|
||||
arrays and objects with up to four elements that are terminated by 0xFE, unsorted object keys, or a 0xFF after a
|
||||
string that would also end without it, are accepted. A second 0xFF is not a terminator but an empty string.
|
||||
|
||||
Strings must be valid UTF-8, and a string at the very end of a message must be terminated by 0xFF.
|
||||
Strings must be valid UTF-8, and a string at the very end of a message must be terminated by 0xFF. This also
|
||||
holds when `strict` is false and more data follows the value: the byte that would end the string belongs to the
|
||||
next value, and reading it would remove it from the input, so the missing 0xFF is reported as
|
||||
[parse_error.110](../../home/exceptions.md#jsonexceptionparse_error110).
|
||||
|
||||
Duplicate object keys are accepted; the last value wins (`ordered_json` keeps the position of the first occurrence).
|
||||
The specification does not say how to treat them; the HikoGUI reference decoder keeps the first value.
|
||||
|
||||
File diff suppressed because one or more lines are too long
@@ -179,7 +179,7 @@ Non-canonical input
|
||||
|
||||
Values that do not use the canonical representation, such as integers with a longer encoding than necessary, arrays and objects with up to four elements that are terminated by 0xFE, unsorted object keys, or a 0xFF after a string that would also end without it, are accepted. A second 0xFF is not a terminator but an empty string.
|
||||
|
||||
Strings must be valid UTF-8, and a string at the very end of a message must be terminated by 0xFF.
|
||||
Strings must be valid UTF-8, and a string at the very end of a message must be terminated by 0xFF. This also holds when `strict` is false and more data follows the value: the byte that would end the string belongs to the next value, and reading it would remove it from the input, so the missing 0xFF is reported as [parse_error.110](https://json.nlohmann.me/home/exceptions/#jsonexceptionparse_error110).
|
||||
|
||||
Duplicate object keys are accepted; the last value wins (`ordered_json` keeps the position of the first occurrence). The specification does not say how to treat them; the HikoGUI reference decoder keeps the first value.
|
||||
|
||||
|
||||
@@ -64,13 +64,13 @@ stack for the nesting.)
|
||||
|
||||
The call stack still needs room for the levels that are recursed into, and the default limit assumes a stack of ordinary
|
||||
size. The stack that an operation needs is roughly proportional to the limit; with the default of 128 levels, the most
|
||||
demanding operation (`dump` in an unoptimized build) needed up to 288 KiB, and an optimized build needed 48 KiB or less,
|
||||
in measurements on macOS with arm64. The most demanding operation needed this much stack at the limits `0`, `32`,
|
||||
`128`, and `254`:
|
||||
demanding operations (copying a value and converting it to another `basic_json` type, in an unoptimized build) needed
|
||||
up to 224 KiB, and an optimized build needed 48 KiB or less, in measurements on macOS with arm64. The most demanding
|
||||
operation needed this much stack at the limits `0`, `32`, `128`, and `254`:
|
||||
|
||||
| Stack needed, in KiB | limit 0 | limit 32 | limit 128 (default) | limit 254 |
|
||||
|---|---:|---:|---:|---:|
|
||||
| unoptimized build (`-O0`) | 16 or less | 80 | 288 | 576 |
|
||||
| unoptimized build (`-O0`) | 16 or less | 64 | 224 | 448 |
|
||||
| optimized build (`-O2`) | 16 or less | 16 or less | 48 | 112 |
|
||||
|
||||
A thread with a smaller stack (a task of a real-time operating system, a fiber, or a thread of the musl C library, whose
|
||||
@@ -81,6 +81,20 @@ result. See the [documentation of the macro](../../api/macros/json_recursion_dep
|
||||
the speed, and the rules for choosing a value. Destroying a value (its destructor) recurses over the same number of
|
||||
levels and then continues without the call stack, and needs 32 KiB or less at every limit.
|
||||
|
||||
!!! warning "Indented output and flattened keys grow with the depth"
|
||||
|
||||
Some results are larger than the value by a factor of its nesting depth.
|
||||
[`dump`](../../api/basic_json/dump.md) with an `indent` of `n` (and likewise `operator<<` with `std::setw(n)` and
|
||||
[`basic_json_view::dump`](../../api/basic_json_view/dump.md)) writes up to `n` characters per nesting level in
|
||||
front of every line, and [`flatten`](../../api/basic_json/flatten.md) repeats the path of every value in its key.
|
||||
For the 80,001-byte array `[1,[1,...[1,1]...]]` with 20,000 levels, `dump(1)` returns 600 MB, `dump(4)` 2.4 GB,
|
||||
and the keys of `flatten()` hold 400 MB. A compact `dump()` without indentation stays the size of the value. If you
|
||||
pretty-print, log, or flatten untrusted input, limit its nesting depth while parsing, for example with a
|
||||
[parser callback](parser_callbacks.md#recipe-max-nesting-depth-via-a-callback): with a limit of `d`, every line has
|
||||
at most `n * d` indentation characters, and every flattened key has at most `d` reference tokens.
|
||||
[`JSON_RECURSION_DEPTH_LIMIT`](../../api/macros/json_recursion_depth_limit.md) does not help here: it changes only
|
||||
how much of the call stack a walk uses, never its result, and it does not limit the depth that `parse()` accepts.
|
||||
|
||||
## Input size
|
||||
|
||||
The library does not limit the overall size of a JSON text; a value nested or wide enough will use memory
|
||||
@@ -140,6 +154,20 @@ values for a repeated key ends up in the parsed object. If your application must
|
||||
silently resolving them one way or another, see the
|
||||
[parser callback recipe for rejecting duplicate keys](parser_callbacks.md#recipe-rejecting-duplicate-object-keys).
|
||||
|
||||
## Objects with many keys in `ordered_json`
|
||||
|
||||
[`ordered_json`](../../api/ordered_json.md) keeps the members of an object in a vector in insertion order
|
||||
([`ordered_map`](../../api/ordered_map.md)) and finds a key by searching all members. Parsing JSON text into an
|
||||
`ordered_json` avoids that search for objects with many keys and takes linear time. The binary readers
|
||||
([`from_cbor`](../../api/basic_json/from_cbor.md), [`from_msgpack`](../../api/basic_json/from_msgpack.md),
|
||||
[`from_ubjson`](../../api/basic_json/from_ubjson.md), [`from_bjdata`](../../api/basic_json/from_bjdata.md),
|
||||
[`from_bson`](../../api/basic_json/from_bson.md), and [`from_bon8`](../../api/basic_json/from_bon8.md)) do not yet:
|
||||
reading an object of `n` keys into an `ordered_json` costs time proportional to `n²`. A flat object of 40,000 keys
|
||||
(about 300 KB of CBOR) takes about 1.3 s, against a few milliseconds as JSON text or into a `json`
|
||||
([#6104](https://github.com/nlohmann/json/issues/6104)). `json` is not affected. If you read untrusted binary input
|
||||
into `ordered_json`, bound the input size more tightly, read it into a `json`, or reject objects with many keys in a
|
||||
[SAX handler](sax_interface.md).
|
||||
|
||||
## Numbers
|
||||
|
||||
A number whose value cannot be represented — for instance `1E1000`, which overflows `double` — is rejected while
|
||||
@@ -171,9 +199,11 @@ indices (or the pointer as a whole) before using them, or use [`at`](../../api/b
|
||||
|
||||
- Wrap parsing in a `#!cpp try`/`#!cpp catch` block, or use `allow_exceptions=false`/`accept()` if your environment
|
||||
cannot use exceptions; never let `JSON_NOEXCEPTION`'s `abort()` be the first time you think about error handling.
|
||||
- If the input's nesting depth matters to you, enforce your own limit with a
|
||||
[parser callback](parser_callbacks.md#recipe-max-nesting-depth-via-a-callback) or a
|
||||
- If the input's nesting depth matters to you (for example because you pretty-print, log, or flatten it), enforce your
|
||||
own limit with a [parser callback](parser_callbacks.md#recipe-max-nesting-depth-via-a-callback) or a
|
||||
[SAX handler](sax_interface.md); the library bounds the call stack but not memory use.
|
||||
- If you read untrusted binary input into `ordered_json`, mind that objects with many keys are
|
||||
[slow to read](#objects-with-many-keys-in-ordered_json).
|
||||
- Bound the size of the input itself before parsing, independent of the library's own bounded allocations for binary
|
||||
format lengths.
|
||||
- Decide up front how you want a string with invalid UTF-8 — from any source, not only parsing — to be serialized
|
||||
|
||||
File diff suppressed because one or more lines are too long
@@ -24,15 +24,19 @@ Neither the parser nor the binary readers impose a limit on how deep the input m
|
||||
|
||||
Once a value has been parsed, operations that walk it recursively — serializing it with [`dump`](https://json.nlohmann.me/api/basic_json/dump/index.md), hashing it, copying it, comparing two values with `==`, `<`, or (in C++20) `<=>`, merging with [`update`](https://json.nlohmann.me/api/basic_json/update/index.md), applying a [`merge_patch`](https://json.nlohmann.me/api/basic_json/merge_patch/index.md), creating a patch with [`diff`](https://json.nlohmann.me/api/basic_json/diff/index.md) or [`merge_diff`](https://json.nlohmann.me/api/basic_json/merge_diff/index.md), converting it to another `basic_json` type, destroying it, and the binary writers (`to_cbor`, `to_msgpack`, ...) — descend at most [`JSON_RECURSION_DEPTH_LIMIT`](https://json.nlohmann.me/api/macros/json_recursion_depth_limit/index.md) levels on the call stack, 128 by default, and continue below that with an explicit stack on the heap. So the call stack does not grow with the nesting depth of the value, however deeply it is nested. ([`flatten`](https://json.nlohmann.me/api/basic_json/flatten/index.md), [`unflatten`](https://json.nlohmann.me/api/basic_json/unflatten/index.md), and [`to_bson`](https://json.nlohmann.me/api/basic_json/to_bson/index.md) never use the call stack for the nesting.)
|
||||
|
||||
The call stack still needs room for the levels that are recursed into, and the default limit assumes a stack of ordinary size. The stack that an operation needs is roughly proportional to the limit; with the default of 128 levels, the most demanding operation (`dump` in an unoptimized build) needed up to 288 KiB, and an optimized build needed 48 KiB or less, in measurements on macOS with arm64. The most demanding operation needed this much stack at the limits `0`, `32`, `128`, and `254`:
|
||||
The call stack still needs room for the levels that are recursed into, and the default limit assumes a stack of ordinary size. The stack that an operation needs is roughly proportional to the limit; with the default of 128 levels, the most demanding operations (copying a value and converting it to another `basic_json` type, in an unoptimized build) needed up to 224 KiB, and an optimized build needed 48 KiB or less, in measurements on macOS with arm64. The most demanding operation needed this much stack at the limits `0`, `32`, `128`, and `254`:
|
||||
|
||||
| Stack needed, in KiB | limit 0 | limit 32 | limit 128 (default) | limit 254 |
|
||||
| ------------------------- | ---------- | ---------- | ------------------- | --------- |
|
||||
| unoptimized build (`-O0`) | 16 or less | 80 | 288 | 576 |
|
||||
| unoptimized build (`-O0`) | 16 or less | 64 | 224 | 448 |
|
||||
| optimized build (`-O2`) | 16 or less | 16 or less | 48 | 112 |
|
||||
|
||||
A thread with a smaller stack (a task of a real-time operating system, a fiber, or a thread of the musl C library, whose default stack is 128 KiB) can still overflow its stack on deeply nested input, which is why the limit can be lowered: define `JSON_RECURSION_DEPTH_LIMIT` to a smaller value, down to `0`, before including the library. Such a limit changes only the stack that is needed and the speed (values nested more deeply than the limit are walked more slowly), never a result. See the [documentation of the macro](https://json.nlohmann.me/api/macros/json_recursion_depth_limit/index.md) for the complete tables, the speed, and the rules for choosing a value. Destroying a value (its destructor) recurses over the same number of levels and then continues without the call stack, and needs 32 KiB or less at every limit.
|
||||
|
||||
Indented output and flattened keys grow with the depth
|
||||
|
||||
Some results are larger than the value by a factor of its nesting depth. [`dump`](https://json.nlohmann.me/api/basic_json/dump/index.md) with an `indent` of `n` (and likewise `operator<<` with `std::setw(n)` and [`basic_json_view::dump`](https://json.nlohmann.me/api/basic_json_view/dump/index.md)) writes up to `n` characters per nesting level in front of every line, and [`flatten`](https://json.nlohmann.me/api/basic_json/flatten/index.md) repeats the path of every value in its key. For the 80,001-byte array `[1,[1,...[1,1]...]]` with 20,000 levels, `dump(1)` returns 600 MB, `dump(4)` 2.4 GB, and the keys of `flatten()` hold 400 MB. A compact `dump()` without indentation stays the size of the value. If you pretty-print, log, or flatten untrusted input, limit its nesting depth while parsing, for example with a [parser callback](https://json.nlohmann.me/features/parsing/parser_callbacks/#recipe-max-nesting-depth-via-a-callback): with a limit of `d`, every line has at most `n * d` indentation characters, and every flattened key has at most `d` reference tokens. [`JSON_RECURSION_DEPTH_LIMIT`](https://json.nlohmann.me/api/macros/json_recursion_depth_limit/index.md) does not help here: it changes only how much of the call stack a walk uses, never its result, and it does not limit the depth that `parse()` accepts.
|
||||
|
||||
## Input size
|
||||
|
||||
The library does not limit the overall size of a JSON text; a value nested or wide enough will use memory proportional to the input. If you parse untrusted input of unbounded size, check the size of the file or stream yourself before — or while — handing it to `parse()`.
|
||||
@@ -60,6 +64,10 @@ Parsing is not the only place invalid UTF-8 matters: a string that reached a `js
|
||||
|
||||
The JSON specification leaves the handling of repeated keys in an object up to the implementation, and this library does too: as described in [`object_t`](https://json.nlohmann.me/api/basic_json/object_t/#behavior), it is unspecified which of the values for a repeated key ends up in the parsed object. If your application must reject duplicate keys instead of silently resolving them one way or another, see the [parser callback recipe for rejecting duplicate keys](https://json.nlohmann.me/features/parsing/parser_callbacks/#recipe-rejecting-duplicate-object-keys).
|
||||
|
||||
## Objects with many keys in `ordered_json`
|
||||
|
||||
[`ordered_json`](https://json.nlohmann.me/api/ordered_json/index.md) keeps the members of an object in a vector in insertion order ([`ordered_map`](https://json.nlohmann.me/api/ordered_map/index.md)) and finds a key by searching all members. Parsing JSON text into an `ordered_json` avoids that search for objects with many keys and takes linear time. The binary readers ([`from_cbor`](https://json.nlohmann.me/api/basic_json/from_cbor/index.md), [`from_msgpack`](https://json.nlohmann.me/api/basic_json/from_msgpack/index.md), [`from_ubjson`](https://json.nlohmann.me/api/basic_json/from_ubjson/index.md), [`from_bjdata`](https://json.nlohmann.me/api/basic_json/from_bjdata/index.md), [`from_bson`](https://json.nlohmann.me/api/basic_json/from_bson/index.md), and [`from_bon8`](https://json.nlohmann.me/api/basic_json/from_bon8/index.md)) do not yet: reading an object of `n` keys into an `ordered_json` costs time proportional to `n²`. A flat object of 40,000 keys (about 300 KB of CBOR) takes about 1.3 s, against a few milliseconds as JSON text or into a `json` ([#6104](https://github.com/nlohmann/json/issues/6104)). `json` is not affected. If you read untrusted binary input into `ordered_json`, bound the input size more tightly, read it into a `json`, or reject objects with many keys in a [SAX handler](https://json.nlohmann.me/features/parsing/sax_interface/index.md).
|
||||
|
||||
## Numbers
|
||||
|
||||
A number whose value cannot be represented — for instance `1E1000`, which overflows `double` — is rejected while parsing as [`out_of_range.406`](https://json.nlohmann.me/home/exceptions/#jsonexceptionout_of_range406) rather than silently becoming infinity. An integer that is syntactically valid but does not fit the 64-bit integer types is not rejected; it is instead stored as a `double`, which may lose precision for very large values. See [number limits](https://json.nlohmann.me/features/types/number_handling/#number-limits) for the exact ranges and an example.
|
||||
@@ -75,7 +83,8 @@ A [JSON Pointer](https://json.nlohmann.me/features/json_pointer/index.md) can co
|
||||
## Checklist
|
||||
|
||||
- Wrap parsing in a `try`/`catch` block, or use `allow_exceptions=false`/`accept()` if your environment cannot use exceptions; never let `JSON_NOEXCEPTION`'s `abort()` be the first time you think about error handling.
|
||||
- If the input's nesting depth matters to you, enforce your own limit with a [parser callback](https://json.nlohmann.me/features/parsing/parser_callbacks/#recipe-max-nesting-depth-via-a-callback) or a [SAX handler](https://json.nlohmann.me/features/parsing/sax_interface/index.md); the library bounds the call stack but not memory use.
|
||||
- If the input's nesting depth matters to you (for example because you pretty-print, log, or flatten it), enforce your own limit with a [parser callback](https://json.nlohmann.me/features/parsing/parser_callbacks/#recipe-max-nesting-depth-via-a-callback) or a [SAX handler](https://json.nlohmann.me/features/parsing/sax_interface/index.md); the library bounds the call stack but not memory use.
|
||||
- If you read untrusted binary input into `ordered_json`, mind that objects with many keys are [slow to read](#objects-with-many-keys-in-ordered_json).
|
||||
- Bound the size of the input itself before parsing, independent of the library's own bounded allocations for binary format lengths.
|
||||
- Decide up front how you want a string with invalid UTF-8 — from any source, not only parsing — to be serialized (`strict`, `replace`, or `ignore`), rather than discovering it from an uncaught `type_error.316`.
|
||||
- For untrusted input in a binary format, pass `error_handler_t::strict` to the `from_*` function if the strings must be valid UTF-8; by default they are not checked (BON8 always checks).
|
||||
|
||||
File diff suppressed because one or more lines are too long
Reference in New Issue
Block a user