* Add BON8 support Add to_bon8/from_bon8 and input_format_t::bon8 for BON8, a binary format that uses the byte values that cannot begin a UTF-8 character as type markers, so strings need no length prefix. It is the most compact of the supported binary formats on the benchmark files. The reader is non-recursive like the other binary readers. A string ends at the first byte that cannot continue it, so the reader hands the one or two bytes it reads past a string back to the value that follows. The writer produces the canonical representation of the specification, except for NFC normalization; its output is identical to that of the reference implementation (HikoGUI) on all files of the test data. The round-trip tests need the .bon8 files of json_test_data 3.2.0. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Address review comments - Reuse detail::validate_one_utf8 to check strings in to_bon8; the error now names the first byte of the invalid sequence. - Document that to_bon8 leaves bytes in the output adapter on an exception, and that string_open is only an output of write_bon8_marker. - Explain why the pushback buffer of the BON8 reader cannot overflow. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Select the BON8 float prefix by type get_bon8_float_prefix only depends on the type of its argument, so make the type a template parameter instead of passing an unused value. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Rename a test variable that Flawfinder mistakes for read() Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Fix the BON8 CI failures - compare the float in write_bon8_float with number_float_t constants, so GCC does not warn about a float-to-double conversion - mark check_bon8_utf8's context as used when exceptions are disabled - choose the compact float prefix in a helper rather than with nested conditional operators (clang-tidy) - use auto for the cast in the BON8 integer reader (clang-tidy) - write the int32 minimum test values as long long literals (MSVC C4146) Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Amalgamate Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Read BON8 strings in bulk from contiguous input - copy the valid UTF-8 of a string in one step when the input is contiguous (twitter.json is read in 1.68 instead of 2.52 ms, jeopardy.json in 196 instead of 297 ms, close to CBOR and MessagePack) - share the new valid_utf8_prefix() with the writer's UTF-8 check, which now skips ASCII 8 bytes at a time - let the fuzzer check that contiguous and stream input give the same value or error, and test both paths in the unit tests - clarify that a second 0xFF after a string is an empty string Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Link the BON8 functions from the other binary format pages Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Name the bulk scan flag after the input, not BON8 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Read BSON keys in bulk from contiguous input BSON keys (and array indices) are C-style strings, which were read byte by byte. For contiguous input they are now read up to their \x00-byte in one step, using the same bulk_scan flag as BON8 strings: twitter.json is read in 1.46 instead of 2.01 ms, citm_catalog.json in 2.93 instead of 3.33 ms, jeopardy.json in 182 instead of 207 ms. canada.json, whose keys are almost all one-digit array indices, takes 2 % longer. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Fix the BON8 CI failures of the bulk-read tests - skip the contiguous-versus-stream tests of BON8 strings and BSON keys when exceptions are disabled: they catch the parse errors of invalid input, and without exceptions the library aborts instead - use static_cast for the int64 test value (google-readability-casting) Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Move the explicit basic_json instantiation into its own test file Linking test-regression3_cpp20 with clang and MinGW failed with "relocation truncated to fit: IMAGE_REL_AMD64_REL32 against `.rdata'", as test-regression2 did before #5511. The explicit instantiation of basic_json<> for #4825 compiles every member function, including the BON8 reader and writer, into that object, and it was already close to the limit (2,226,104 bytes on develop, 2,234,960 with BON8; clang -O1, C++20). Give the instantiation a file of its own: unit-regression3 is now 1,594,736 bytes and unit-explicit_instantiation 1,095,064. The new file mentions JSON_HAS_CPP_17 and JSON_HAS_CPP_20 so it keeps being built for the C++17 standard the regression was about. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Convert the bytes of the BON8 test strings explicitly The str() helper constructed a std::string from a byte range, which converts each unsigned char implicitly; -fsanitize=integer reports that for bytes of 0x80 and above (ci_test_clang_sanitizer). Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me>
5.3 KiB
Serialization
Serialization is the process of turning a JSON value back into JSON text. It is the counterpart to
parsing. The central function is dump, which returns the JSON text as
a string.
json j = {{"pi", 3.141}, {"happy", true}};
std::string s = j.dump(); // {"happy":true,"pi":3.141}
To write a value directly to a stream (for example, a file or #!cpp std::cout), the
operator<< is provided:
std::cout << j << std::endl;
!!! note "String, not raw value"
`dump` always returns a **JSON text**. Serializing a JSON string therefore includes the surrounding quotes and
escapes special characters. To obtain the *contained* string value without quotes, use
[`get<std::string>()`](conversions.md) instead of `dump`. See the [converting values](conversions.md) page.
Pretty-printing
By default, dump produces the most compact representation without any superfluous whitespace. Passing a non-negative
indent argument pretty-prints the output with the given number of spaces per level:
??? example
```cpp
--8<-- "examples/dump.cpp"
```
Output:
```json
--8<-- "examples/dump.output"
```
The indentation character can be changed with the second argument (e.g., a tab #!cpp '\t'). An indent of 0 inserts
newlines but no leading spaces, and the default of #!cpp -1 selects the compact single-line form.
Non-ASCII characters
Strings are stored and serialized as UTF-8 (see types). By default, dump copies valid
non-ASCII characters as-is. Setting the third argument ensure_ascii to #!cpp true escapes all non-ASCII characters
with \uXXXX sequences, so that the output contains only ASCII characters:
json j = "苹果";
j.dump(); // "苹果"
j.dump(-1, ' ', true); // "苹果"
Handling invalid UTF-8
If a string contains invalid UTF-8 sequences (for example, because it holds data in another encoding such as Latin-1),
serialization fails by default. The fourth argument of dump selects an
error_handler:
strict(default) — throw atype_error.316exception.replace— replace invalid bytes with the Unicode replacement character U+FFFD (<EFBFBD>).ignore— silently drop invalid bytes.
??? example
```cpp
--8<-- "examples/error_handler_t.cpp"
```
Output:
```json
--8<-- "examples/error_handler_t.output"
```
!!! tip "Avoiding invalid UTF-8"
The best fix is to ensure that all strings are UTF-8 encoded before storing them. See the
[FAQ on non-ASCII characters](../home/faq.md#parse-errors-reading-non-ascii-characters) for how to convert wide or
Latin-1 strings.
Numbers, NaN, and binary values
- Numbers are serialized with enough precision to round-trip; see number serialization.
- NaN and infinity cannot be represented in JSON and are serialized as
#!json null; see NaN handling. The binary formats can preserve them. - Binary values have no JSON representation and are serialized as a helper object for debugging only; see binary values.
Using std::format, std::print, and fmt
Since version 3.12.0, JSON values can be formatted directly with C++20's
std::format whenever the standard library provides the
<format> header (controlled by JSON_HAS_STD_FORMAT). This is enabled by the
std::formatter<basic_json> specialization, which also makes JSON values work with
std::format_to and with C++23's std::print/std::println:
std::print("{}", j); // compact, like j.dump()
std::print("{:2}", j); // pretty-printed with indent 2 (like j.dump(2))
std::println("{:#}", j); // pretty-printed with the default indent
The format spec mirrors the dump parameters: #!cpp "{:#}" pretty-prints, a width such as #!cpp "{:2}" sets the
indent, and a fill-and-align prefix such as #!cpp "{:.>#}" sets the indent character.
For the {fmt} library, the library ships a
format_as helper. Note its behavior depends on the fmt version; see the
FAQ entry for the details and a recipe for a full
fmt::formatter specialization.
Serializing to other formats
Besides JSON text, a value can also be serialized to the more compact binary formats (BJData, BON8, BSON, CBOR, MessagePack, UBJSON).
See also
dump- serialize to a JSON-formatted stringoperator<<- serialize to a streamto_string- user-defined-conversion helperstd::formatter<basic_json>- use JSON values withstd::formatandstd::printformat_as- use JSON values with the {fmt} library- Parsing - the reverse operation