Scarfski/json
1
0
Fork 0
mirror of https://github.com/nlohmann/json.git synced 2026-09-25 19:45:49 +00:00
Code Issues Packages Projects Releases Wiki Activity

Check the fuzzers' UBJSON/BJData round-trip invariants in the unit tests (#5569)

Browse Source 
* Check the fuzzers' UBJSON/BJData round-trip invariants in the unit tests

The strongest correctness checks for the UBJSON and BJData writers lived
only in the OSS-Fuzz drivers: anything from_ubjson()/from_bjdata()
returns must serialize with every option combination, parse back, and
re-serialize stably. Those checks only run at OSS-Fuzz, so regressions
surfaced days later as external reports - the same BJData assert pair
was reported five times over three years, and #5494's harness change
was followed by OSS-Fuzz 563659413 within a day.

Add "UBJSON round-trip invariants" and "BJData round-trip invariants"
test cases that run the drivers' checks on a fixed, deterministic corpus
(tests/src/round_trip_corpus.hpp): integer and float boundaries,
non-finite numbers, strings, binary values, optimized containers, deep
nesting, the JData annotated-array matrix, and seeded random containers.
They also check two properties the drivers do not: the first round trip
preserves the value, and re-serializing reproduces the exact bytes. For
BJData both exclude values containing a binary value, which is read back
as an array of integers unless it was written as a Draft 3 optimized
binary array; this carve-out is now documented in bjdata.md. Run against
the headers before #5542, the BJData test fails, including on the shape
from OSS-Fuzz 563659413.

Also document how OSS-Fuzz reports are handled (reference them as
"OSS-Fuzz: <id>", turn the reproducer into a unit test, keep drivers and
unit tests in sync) in tests/fuzzing.md, and link it from the PR
template and the quality assurance page.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Add the OSS-Fuzz reproducers for 474400817 and 474480402 as unit tests

Following the convention added to tests/fuzzing.md, the reproducers of
the two BJData fuzzer asserts tracked since January are now unit tests:

- 474400817 (assert(false)): an empty object _ArraySize_ was written as
  the ND-array header length, which from_bjdata() could not read back.
  Fixed by #5455.

- 474480402 (to_bjdata(j2, false, false) == vec2): a one-byte Draft 3
  binary array is written in Draft 2 mode as a uint8 array and then
  re-serialized with the int8 marker. This is the documented exception to
  byte stability, not a library bug; OSS-Fuzz closed it after #5494
  relaxed the harness to value stability. The test pins the exact bytes
  so the exception stays deliberate.

The 563659413 reproducer is already a unit test (#5542). A comment also
ties the existing UBJSON excessive-count test to the timeout OSS-Fuzz
reported for that shape (testcase 6347769435193344).

OSS-Fuzz: 474400817
OSS-Fuzz: 474480402

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix GCC -Weffc++ and -Wuseless-cast warnings in the round-trip corpus

Initialize the atoms in the member initialization list, and drop the cast of
the generator's result, which already is std::size_t on 64-bit Linux.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
This commit is contained in:
Niels Lohmann
2026-09-25 08:29:02 +02:00
committed by GitHub
parent 80bf54a5a2
commit cc472af13f
9 changed files with 408 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
.github/PULL_REQUEST_TEMPLATE.md vendored
View File

@@ -2,6 +2,7 @@
- [ ] The changes are described in detail, both the what and why.
- [ ] If applicable, an [existing issue](https://github.com/nlohmann/json/issues) is referenced.
- [ ] If applicable, a fixed [OSS-Fuzz](https://issues.oss-fuzz.com) issue is referenced as `OSS-Fuzz: <id>` (see [fuzz testing](https://github.com/nlohmann/json/blob/develop/tests/fuzzing.md#handling-oss-fuzz-reports)).
- [ ] The [Code coverage](https://coveralls.io/github/nlohmann/json) remained at 100%. A test case for every new line of code.
- [ ] If applicable, the [documentation](https://json.nlohmann.me) is updated.
- [ ] The source code is amalgamated by running `make amalgamate`.

3
docs/mkdocs/docs/community/quality_assurance.md
View File

@@ -164,6 +164,9 @@ Note: Some modern features (like C++20 ranges or filesystem support) may be disa
- [x] The parser is tested against extensive correctness suites for JSON compliance.
- [x] In addition, the library is continuously fuzz-tested at [OSS-Fuzz](https://google.github.io/oss-fuzz/) where the
library is checked against billions of inputs.
- [x] Every crash reported by OSS-Fuzz is fixed together with a unit test that reproduces it, and the fix references
the OSS-Fuzz issue. The round-trip checks of the fuzzer drivers are also part of the unit tests. See the
[fuzz testing documentation](https://github.com/nlohmann/json/blob/develop/tests/fuzzing.md#handling-oss-fuzz-reports).
## Static analysis

10
docs/mkdocs/docs/features/binary_formats/bjdata.md
View File

@@ -208,6 +208,16 @@ The library maps BJData types to JSON value types as follows:
The mapping is **complete** in the sense that any BJData value can be converted to a JSON value.
!!! info "Round trips"
A value returned by [`from_bjdata`](../../api/basic_json/from_bjdata.md) can be serialized with
[`to_bjdata`](../../api/basic_json/to_bjdata.md) using any combination of options and parsed back into an equal
value, and serializing that value again with the same options produces the same bytes. The exception is binary
values: they are only written as an optimized binary array (`[$B`) if Draft 3 is enabled and both `use_size` and
`use_type` are set. Otherwise, they are written as arrays of integers and parsed back as such (see the notes on
binary values above), and serializing such an array again may choose different, but equally valid, type markers.
The bytes can then differ, but parsing them again yields the same value.
??? example
```cpp

23
tests/fuzzing.md
View File

@@ -79,3 +79,26 @@ the same `fuzzers` target as above and also relies on the `FUZZER_ENGINE` variab
[build script](https://github.com/google/oss-fuzz/blob/master/projects/json/build.sh) for more information.
In case the build at OSS-Fuzz fails, an issue will be created automatically.
### Handling OSS-Fuzz reports
OSS-Fuzz files the crashes it finds in its own [issue tracker](https://issues.oss-fuzz.com), not on GitHub. So that
each report can be traced to the change that fixed it, and each fix to the report it answers, fixes follow these
conventions:
- **Reference the OSS-Fuzz issue in the pull request**, next to any GitHub issue it closes, as `OSS-Fuzz: <id>` (for
example, `OSS-Fuzz: 563659413`), and in the commit message. The ID alone does not disclose the crash. If the report
was triaged into a GitHub issue, link the OSS-Fuzz issue there too.
- **Turn the reproducer into a unit test.** Download the testcase from the OSS-Fuzz report, reduce it if possible, and
add it as a regression test to the unit test of the affected format (e.g., `tests/src/unit-bjdata.cpp`), with a
comment naming the OSS-Fuzz issue. This way the input is checked by every CI run rather than only by OSS-Fuzz, and
it stays covered even if OSS-Fuzz later closes the report as not reproducible.
- **Keep the fuzzer drivers and the unit tests in sync.** The round-trip checks of the UBJSON and BJData drivers are
also run on a fixed corpus in the unit tests (see `tests/src/round_trip_corpus.hpp` and the "round-trip invariants"
test cases), so a regression shows up in CI first. When a driver's checks change, change the unit tests with them.
- **Record in the report whether the bug shipped.** OSS-Fuzz asks whether a crash was a short-lived regression or
affects a released version; answer it when the fix is merged, as it decides whether the fix needs a release note or
a security advisory (see the [security policy](../.github/SECURITY.md)).
After the fix is merged, OSS-Fuzz re-runs the reproducer on its next build and marks the report as verified and
closed. If it does not, the fix is incomplete.

3
tests/src/fuzzer-parse_bjdata.cpp
View File

@@ -42,6 +42,9 @@ dump() serializes any non-finite double the same deterministic way (as JSON
`null`, since JSON itself cannot represent NaN/Infinity), so comparing
dumps is stable under exactly the same values that break operator==.
The unit tests run the same checks on a fixed corpus (see the "BJData round-trip
invariants" test case), so keep both in sync.
The provided function `LLVMFuzzerTestOneInput` can be used in different fuzzer
drivers.
*/

3
tests/src/fuzzer-parse_ubjson.cpp
View File

@@ -21,6 +21,9 @@ array data, it performs the following steps:
- j4 = from_ubjson(vec3)
- assert(j1 == j4)
The unit tests run the same checks on a fixed corpus (see the "UBJSON round-trip
invariants" test case), so keep both in sync.
The provided function `LLVMFuzzerTestOneInput` can be used in different fuzzer
drivers.
*/

213
tests/src/round_trip_corpus.hpp Normal file
View File

@@ -0,0 +1,213 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++ (supporting code)
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
#pragma once
#include <cmath> // nan
#include <cstddef> // size_t
#include <cstdint> // int32_t, int64_t, uint32_t, uint64_t
#include <limits> // numeric_limits
#include <random> // mt19937
#include <string> // string, to_string
#include <utility> // move
#include <vector> // vector
#include <nlohmann/json.hpp>
// Values for the round-trip property tests of the UBJSON and BJData writers.
//
// The fuzzer drivers (tests/src/fuzzer-parse_ubjson.cpp and
// fuzzer-parse_bjdata.cpp) check that anything the library parses can be
// serialized, parsed back, and serialized again without loss. Those checks
// only run at OSS-Fuzz, so a regression used to surface days later as an
// external report. The unit tests run the same checks on this corpus in CI.
//
// The corpus is deterministic: std::mt19937's output sequence is fixed by
// the standard, and it is used directly rather than through a distribution
// (whose results are implementation-defined).
namespace utils
{
class round_trip_corpus
{
public:
using json = nlohmann::json;
static std::vector<json> values()
{
round_trip_corpus corpus;
return corpus.build();
}
// whether a value contains a binary value, which a BJData or UBJSON round
// trip may turn into an array of integers
static bool contains_binary(const json& j)
{
if (j.is_binary())
{
return true;
}
if (j.is_structured())
{
for (const auto& element : j)
{
if (contains_binary(element))
{
return true;
}
}
}
return false;
}
private:
std::vector<json> atoms;
// a fixed seed is the point: the corpus must be the same in every run
std::mt19937 generator{42}; // NOLINT(cert-msc32-c,cert-msc51-cpp,bugprone-random-generator-seed)
round_trip_corpus()
: atoms
{
nullptr, true, false,
// integers at the boundaries of every UBJSON/BJData integer type
0, 1, -1, 127, 128, 255, 256, -128, -129,
32767, 32768, 65535, 65536, -32768, -32769,
(std::numeric_limits<std::int32_t>::min)(), (std::numeric_limits<std::int32_t>::max)(),
(std::numeric_limits<std::uint32_t>::max)(),
(std::numeric_limits<std::int64_t>::min)(), (std::numeric_limits<std::int64_t>::max)(),
static_cast<std::uint64_t>((std::numeric_limits<std::int64_t>::max)()) + 1u,
(std::numeric_limits<std::uint64_t>::max)(),
// floating-point numbers, including non-finite ones
0.0, -0.0, 1.5, -2.25, 3.4e38, (std::numeric_limits<double>::max)(),
std::nan(""), std::numeric_limits<double>::infinity(), -std::numeric_limits<double>::infinity(),
// strings, including a non-ASCII one and one longer than 255 bytes
"", "a", "\xC3\xA4", std::string(300, 'x'),
// binary values with and without subtype
json::binary({}), json::binary({1, 2, 255}), json::binary({0x80, 0x7F}, 42), json::binary({1}, 0)
}
{}
std::vector<json> build()
{
std::vector<json> result = atoms;
// each atom inside containers, including homogeneous ones that the
// writers encode as optimized (typed) containers
result.emplace_back(json::array());
result.emplace_back(json::object());
for (const auto& atom : atoms)
{
result.push_back(json::array({atom}));
result.push_back(json::array({atom, atom, atom}));
result.push_back(json::array({json::array({atom})}));
result.push_back(json::object({{"key", atom}}));
}
result.push_back(json::array({1, 1.5}));
result.push_back(json::array({-1, 255}));
result.push_back(json::array({"a", "b"}));
// deep, but well below any recursion or depth limit
json nested_array = 1;
json nested_object = 1;
for (int i = 0; i < 300; ++i)
{
nested_array = json::array({nested_array});
nested_object = json::object({{"key", nested_object}});
}
result.push_back(nested_array);
result.push_back(nested_object);
add_annotated_arrays(result);
add_random_values(result);
return result;
}
// objects in the JData annotated array format, which the BJData writer
// encodes as ND-arrays when the annotation describes a packed array, and
// as plain objects otherwise (see #5398, #5399, #5403, #5404, and #5542)
static void add_annotated_arrays(std::vector<json>& result)
{
const std::vector<json> types =
{
"uint8", "int8", "uint16", "int16", "uint32", "int32", "uint64", "int64",
"single", "double", "char", "byte", "bool", "unknown", 5, nullptr
};
const std::vector<json> sizes =
{
json::array(), {3}, {1, 3}, {3, 1}, {2, 3}, {2, 0}, {0, 2}, {2, 2, 2}, {-1, 2}, {2, 1.5},
"3", 3, nullptr, json::binary({})
};
const std::vector<json> data =
{
nullptr, 5, "s", json::object({{"a", 1}}), json::array(),
{1, 2, 3}, {1, 2, 3, 4, 5, 6}, {1, 2, 3, 4, 5, 6, 7, 8},
{1.5, 2.5, 3.5, 4.5, 5.5, 6.5}, {300, -300, 70000, -70000, 1, 2},
{"a", "b", "c", "d", "e", "f"}, {json::array({1, 2, 3}), json::array({4, 5, 6})}
};
for (const auto& type : types)
{
for (const auto& size : sizes)
{
for (const auto& d : data)
{
result.push_back({{"_ArrayType_", type}, {"_ArraySize_", size}, {"_ArrayData_", d}});
}
}
}
// incomplete annotations and annotations with an extra key
result.push_back({{"_ArraySize_", {2, 3}}, {"_ArrayData_", {1, 2, 3, 4, 5, 6}}});
result.push_back({{"_ArrayType_", "uint8"}, {"_ArrayData_", {1, 2, 3, 4, 5, 6}}});
result.push_back({{"_ArrayType_", "uint8"}, {"_ArraySize_", {2, 3}}});
result.push_back({{"_ArrayType_", "uint8"}, {"_ArraySize_", {2, 3}}, {"_ArrayData_", {1, 2, 3, 4, 5, 6}}, {"extra", 1}});
}
// random containers of atoms, both homogeneous and mixed
void add_random_values(std::vector<json>& result)
{
for (int i = 0; i < 1000; ++i)
{
result.push_back(random_value(0));
}
}
std::size_t random_below(std::size_t bound)
{
return generator() % bound;
}
json random_value(int depth)
{
const auto kind = random_below(10);
if (depth > 3 || kind < 5)
{
return atoms[random_below(atoms.size())];
}
json result = kind < 8 ? json::array() : json::object();
const auto count = random_below(5);
const bool homogeneous = random_below(2) == 0;
const json fixed = atoms[random_below(atoms.size())];
for (std::size_t i = 0; i < count; ++i)
{
json element = homogeneous ? fixed : random_value(depth + 1);
if (result.is_array())
{
result.push_back(std::move(element));
}
else
{
result[std::to_string(i)] = std::move(element);
}
}
return result;
}
};
} // namespace utils

103
tests/src/unit-bjdata.cpp
View File

@@ -19,6 +19,7 @@ using nlohmann::json;
#include <fstream>
#include <set>
#include "make_test_data_available.hpp"
#include "round_trip_corpus.hpp"
#include "test_utils.hpp"
namespace
@@ -2867,6 +2868,21 @@ TEST_CASE("BJData")
const auto out_num = json::to_bjdata(j_num);
CHECK(out_num.at(0) == '{');
CHECK(json::from_bjdata(out_num) == j_num);
// OSS-Fuzz issue 474400817: an empty object _ArraySize_ was
// written as the ND-array header length, which from_bjdata()
// could not read back
const std::vector<uint8_t> input =
{
'[', '{', 'U', 11, '_', 'A', 'r', 'r', 'a', 'y', 'D', 'a', 't', 'a', '_', 'Z',
'U', 11, '_', 'A', 'r', 'r', 'a', 'y', 'T', 'y', 'p', 'e', '_', 'S', 'i', 5, 'i', 'n', 't', '1', '6',
'U', 11, '_', 'A', 'r', 'r', 'a', 'y', 'S', 'i', 'z', 'e', '_', '{', '}', '}', ']'
};
const json j1 = json::from_bjdata(input);
CHECK(j1 == json::parse(R"([{"_ArrayType_":"int16","_ArraySize_":{},"_ArrayData_":null}])"));
json j2;
CHECK_NOTHROW(j2 = json::from_bjdata(json::to_bjdata(j1, false, false)));
CHECK(j2 == j1);
}
SECTION("ndarray with out-of-range _ArrayData_ elements stays as object")
@@ -4273,6 +4289,93 @@ TEST_CASE("BJData use_type requires use_size")
}
}
TEST_CASE("BJData round-trip invariants")
{
// This checks what the parse_bjdata_fuzzer driver checks (see
// tests/src/fuzzer-parse_bjdata.cpp), so that a regression shows up in CI
// rather than as an OSS-Fuzz report: every value from_bjdata() returns
// (j1) can be serialized with any combination of options, the result can
// be parsed back (j2), and serializing j2 again with the same options
// yields a value-equal result.
//
// Beyond the driver, this also checks that j2 equals j1 and that
// serializing j2 reproduces the exact bytes, both except for values that
// contain a binary value: a binary value is only written as a binary
// value with Draft 3's optimized binary array, and otherwise read back as
// an array of integers, for which the writer may choose different (but
// equally valid) type markers when it is serialized again (see #5494).
//
// Values are compared with dump() rather than operator==, because a NaN
// never compares equal to itself.
struct options
{
bool use_size;
bool use_type;
json::bjdata_version_t version;
};
const std::vector<options> all_options =
{
{false, false, json::bjdata_version_t::draft2},
{true, false, json::bjdata_version_t::draft2},
{true, true, json::bjdata_version_t::draft2},
{false, false, json::bjdata_version_t::draft3},
{true, false, json::bjdata_version_t::draft3},
{true, true, json::bjdata_version_t::draft3},
};
for (const auto& j0 : utils::round_trip_corpus::values())
{
// turn the corpus value into a value as from_bjdata() returns it
for (const auto& initial : all_options)
{
const json j1 = json::from_bjdata(json::to_bjdata(j0, initial.use_size, initial.use_type, initial.version));
const bool has_binary = utils::round_trip_corpus::contains_binary(j1);
for (const auto& o : all_options)
{
INFO("j1 = " << j1.dump() << ", use_size = " << o.use_size << ", use_type = " << o.use_type
<< ", draft3 = " << (o.version == json::bjdata_version_t::draft3));
const std::vector<std::uint8_t> vec = json::to_bjdata(j1, o.use_size, o.use_type, o.version);
json j2;
// anything the library writes must be parsable by the library
REQUIRE_NOTHROW(j2 = json::from_bjdata(vec));
const std::vector<std::uint8_t> vec2 = json::to_bjdata(j2, o.use_size, o.use_type, o.version);
CHECK(json::from_bjdata(vec2).dump() == j2.dump());
if (!has_binary)
{
CHECK(j2.dump() == j1.dump());
CHECK(vec2 == vec);
}
}
}
}
}
TEST_CASE("BJData round trip of a binary value is value-stable, not byte-stable")
{
// OSS-Fuzz issue 474480402: a Draft 3 optimized binary array is read as a
// binary value, which to_bjdata() writes in the default Draft 2 mode as a
// plain array of uint8 numbers. That is read back as an array of numbers,
// for which the writer then picks the smallest type marker, int8 ('i'),
// so re-serializing changes the bytes, but not the value. This is the
// exception described in the "Round trips" note of the BJData
// documentation, and why the fuzzer checks value stability (see #5494).
const std::vector<uint8_t> input = {'[', '$', 'B', '#', 'U', 1, 0x20};
const json j1 = json::from_bjdata(input);
CHECK(j1 == json::binary({0x20}));
const std::vector<uint8_t> vec = json::to_bjdata(j1, false, false);
CHECK(vec == std::vector<uint8_t>({'[', 'U', 0x20, ']'}));
const json j2 = json::from_bjdata(vec);
CHECK(j2 == json::array({0x20}));
const std::vector<uint8_t> vec2 = json::to_bjdata(j2, false, false);
CHECK(vec2 == std::vector<uint8_t>({'[', 'i', 0x20, ']'}));
CHECK(json::from_bjdata(vec2) == j2);
}
TEST_CASE("BJData roundtrips" * doctest::skip())
{
SECTION("input from self-generated BJData files")

50
tests/src/unit-ubjson.cpp
View File

@@ -15,6 +15,7 @@ using nlohmann::json;
#include <fstream>
#include <set>
#include "make_test_data_available.hpp"
#include "round_trip_corpus.hpp"
#include "test_utils.hpp"
namespace
@@ -2265,7 +2266,9 @@ TEST_CASE("UBJSON optimized arrays of a valueless type are bounded")
SECTION("an excessive count is rejected")
{
// 'l' is a big-endian int32: 0x7FFFFFFF elements, about 34 GB of value
// 'l' is a big-endian int32: 0x7FFFFFFF elements, about 34 GB of value;
// OSS-Fuzz reported this shape as a parse_ubjson_fuzzer timeout
// (testcase 6347769435193344, no issue filed)
for (const auto marker :
{'Z', 'T', 'F'
})
@@ -2817,6 +2820,51 @@ TEST_CASE("UBJSON use_type requires use_size")
}
}
TEST_CASE("UBJSON round-trip invariants")
{
// This checks what the parse_ubjson_fuzzer driver checks (see
// tests/src/fuzzer-parse_ubjson.cpp), so that a regression shows up in CI
// rather than as an OSS-Fuzz report: every value from_ubjson() returns
// (j1) can be serialized with any combination of options, the result can
// be parsed back (j2), and serializing j2 again with the same options
// reproduces the exact bytes. Beyond the driver, this also checks that j2
// equals j1. Values are compared with dump() rather than operator==,
// because a NaN never compares equal to itself.
struct options
{
bool use_size;
bool use_type;
};
const std::vector<options> all_options =
{
{false, false},
{true, false},
{true, true},
};
for (const auto& j0 : utils::round_trip_corpus::values())
{
// turn the corpus value into a value as from_ubjson() returns it; this
// has no binary values, as UBJSON writes them as arrays of integers
for (const auto& initial : all_options)
{
const json j1 = json::from_ubjson(json::to_ubjson(j0, initial.use_size, initial.use_type));
for (const auto& o : all_options)
{
INFO("j1 = " << j1.dump() << ", use_size = " << o.use_size << ", use_type = " << o.use_type);
const std::vector<std::uint8_t> vec = json::to_ubjson(j1, o.use_size, o.use_type);
json j2;
// anything the library writes must be parsable by the library
REQUIRE_NOTHROW(j2 = json::from_ubjson(vec));
CHECK(j2.dump() == j1.dump());
CHECK(json::to_ubjson(j2, o.use_size, o.use_type) == vec);
}
}
}
}
TEST_CASE("UBJSON roundtrips" * doctest::skip())
{
SECTION("input from self-generated UBJSON files")