mirror of
https://github.com/nlohmann/json.git
synced 2026-10-01 22:45:17 +00:00
b54ed188e6
* Run the README test case in JSON_FastTests jobs The "README" test case was marked doctest::skip() when the tests moved from Catch to doctest in 2019, where it replaced Catch's hidden tag. It is not slow (17 assertions, about 0.00 s), but cmake/test.cmake only passes --no-skip when JSON_FastTests is off, so the per-compiler ci_test_*_cxxNN matrix, macOS, Windows Release/ARM, icpc, icpx and nvhpc compiled the README examples without running them. Drop the skip decorator so every job runs the case. Test-only change. Part of #5713 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Remove stale clang ranges guards in unit-iterators2.cpp The "algorithms" and "views" sections were guarded by clang/libstdc++ checks written for a clang 15 (04/2022) bug. The first guard's condition contradicts its own comment: it skips clang+libc++ and keeps clang+libstdc++. Both sections already sit inside `#if JSON_HAS_RANGES`, which macro_scope.hpp excludes for the toolchains these guards targeted, so the inner guards never let the sections run on the platforms they meant to protect and are redundant on the rest. Verified locally with Apple clang 21/libc++ and clang 16.0.6/libstdc++ 12 (Docker): both pass all 1355 assertions with the guards removed. Part of #5713 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Fix copy-pasted CBOR half-float checks; enable stale encode checks In the RFC 8949 Appendix A test case, the decode checks for 5.960464477539063e-8 (0xf9 0x00 0x01) and 0.00006103515625 (0xf9 0x04 0x00) were copy-pasted from the neighboring -4.0 example, so those two half-float byte sequences were never actually decoded and checked, and -4.0 was checked three times instead. The two float32 encode checks for 100000.0 and 3.4028234663852886e+38 were commented out before the writer supported emitting float32 and are now verified to match byte for byte, so they are enabled. The remaining commented-out half-precision to_cbor checks are collapsed into a single explanatory comment, since the writer never emits half-precision floats. Part of #5713 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Assert on the result of STL container conversions in tests The "object-like STL containers" and "array-like STL containers" sections converted json values into std::map, unordered_map, multimap, unordered_multimap, list, forward_list, array, valarray, vector, deque, set and unordered_set and discarded the result, so these ~60 conversions only proved that the code compiles and does not throw; a conversion that dropped or reordered elements would still pass. Bind each result and compare it against the expected container. Also fix a copy-paste slip in the deque section (`j2.get<std::deque<double>>()` instead of j3, so j3's doubles were never converted to a deque), and remove the dead `// CHECK(m5["one"] == "eins")` comments that referred to a variable that did not exist by asserting the equivalent through the bound result. Part of #5713 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Deduplicate SaxCountdown and other test helpers across formats SaxCountdown was copied byte-for-byte into six binary-format test files (unit-cbor.cpp, unit-msgpack.cpp, unit-ubjson.cpp, unit-bjdata.cpp, unit-bon8.cpp, unit-bson.cpp), about 370 redundant lines. Move it into tests/src/sax_countdown.hpp (namespace utils, alongside test_utils.hpp and round_trip_corpus.hpp) and include it from all six. trait_test_arg and the "value_in_range_of trait" TEST_CASE_TEMPLATE_DEFINE were duplicated between unit-32bit.cpp and unit-bjdata.cpp; the trait is a detail/meta trait, not specific to either file. Move it into tests/src/value_in_range_of_test.hpp; unit-32bit.cpp keeps its own include, since JSON_32bitTest=ONLY builds only that file. Each file keeps its own TEST_CASE_TEMPLATE_INVOKE list. sax_no_exception and the "issue #2824" section were duplicated in unit-regression2.cpp and unit-disabled_exceptions.cpp. Drop the copy from unit-regression2.cpp; unit-disabled_exceptions.cpp already covers the no-exceptions case that #2824 was about, and ci_test_noexceptions reruns it. No behavior change. Verified by building and running unit-cbor, unit-msgpack, unit-ubjson, unit-bjdata, unit-bon8, unit-bson, unit-32bit, unit-regression2 and unit-disabled_exceptions against include/ (clang++ -std=c++11, ASan/UBSan where applicable); assertion counts are unchanged from before the refactor. Part of #5714 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Remove the unreferenced vendored libFuzzer tests/thirdparty/Fuzzer (155 files, ~776 KB of vendored Apache-2.0 LLVM code from the 2016 OSS-Fuzz import) is not referenced by any CMakeLists, Makefile or workflow: the fuzz drivers link against -fsanitize=fuzzer or the repo's own tests/src/fuzzer-driver_afl.cpp. Its vendored README only points at llvm.org's own libFuzzer docs. Being dead code, it also adds noise to the flawfinder code-scanning workflow, which scans the whole tree. Remove the directory and its .reuse/dep5 entry. Part of #5714 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Remove unreferenced 2016 benchmark and fuzz reports tests/reports (1.6 MB) holds AFL status pages and plots from 2016-08-29 and 2016-10-02, and a nativejson-benchmark snapshot from 2016 with links to rawgit.com, which shut down in 2019. Nothing references this directory: no doc, README section, script or workflow points at it, and it describes a ten-years-old, pre-2.0 snapshot of the library. Part of #5714 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Run the CBOR, MessagePack, BSON and BON8 round-trip invariants in CI tests/src/round_trip_corpus.hpp exists so that the byte-stability invariant the fuzzer drivers check also runs on a fixed corpus in CI, instead of only at OSS-Fuzz. So far only the UBJSON and BJData drivers had a matching unit test; the CBOR, MessagePack, BSON and BON8 drivers assert the same invariant (assert(to_X(j2) == vec)) but nothing ran it outside OSS-Fuzz. Add "<FORMAT> round-trip invariants" test cases to unit-cbor.cpp, unit-msgpack.cpp, unit-bson.cpp and unit-bon8.cpp, modeled on the UBJSON case: seed j1 from the corpus (skipping values that do not survive the format's own round trip, as the fuzzer drivers only ever see values from_X() actually produced), then require from_X(to_X(j1)) not to throw and check to_X(j2) == to_X(j1). BSON only serializes objects, so non-object corpus values are skipped. Update the comments in round_trip_corpus.hpp and tests/fuzzing.md to name all six formats. The stream-versus-contiguous check in the BON8 driver is left out, as #5601 reworks it. A local probe confirms no violations on the current corpus (CBOR 3849 checked, MessagePack 3909, BSON 2958, BON8 3841 - matching the counts already recorded for this probe in the issue). Closes #5714 item 1. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Fix fuzzer driver step lists to match the checks the code performs The header comment of six of the seven binary-format fuzzer drivers listed an invariant the code does not check: CBOR, MessagePack, BSON and BON8 said "assert(j1 == j2)", but the code checks byte stability, assert(to_X(j2) == vec). UBJSON and BJData still described the old "assert(j1 == j2/j3/j4)" byte-exact check from before PR #5494 replaced it with a use_size/use_type-aware round trip (UBJSON) and a value-stability check (BJData); BJData's added paragraph already explained the new check, but the step list above it did not. Also remove a dead branch in fuzzer-parse_bson.cpp: from_bson() is called with allow_exceptions = true, so it throws instead of returning a discarded value, and the "if (j1.is_discarded()) return 0;" guard could never trigger. Drop the unused <iostream> include from all seven drivers and <sstream> from all but fuzzer-parse_bon8.cpp, which is the only one that uses std::istringstream. Overlaps #5601, which edits all seven drivers in the same hunks. Closes #5714 item 4. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Silence the CMP0169 deprecation in cmake_fetch_content, fix stale guards tests/cmake_fetch_content/project calls the single-argument FetchContent_Populate(json) after FetchContent_Declare(), which CMake 3.30 deprecated as CMP0169. Since the project declares cmake_minimum_required(VERSION 3.11...3.14), the policy stays unset, so every configure with a current CMake prints the deprecation warning. The test is kept on purpose: it is the only coverage of the FetchContent_Populate + add_subdirectory pattern for CMake 3.11-3.13 users, which the docs still describe as supported. Explicitly set CMP0169 to OLD, with a comment explaining why. Also fix two stale version guards: - tests/cmake_fetch_content/CMakeLists.txt guarded the test with VERSION_GREATER "3.11.0", which is dead now that tests/CMakeLists.txt requires CMake 3.13. - tests/cmake_fetch_content2/CMakeLists.txt guarded with VERSION_GREATER "3.14.0", which skips exactly 3.14.0, the first version with FetchContent_MakeAvailable. Change it to VERSION_GREATER_EQUAL "3.14". Verified locally: `ctest -R cmake_fetch_content` passes with CMake 4.1, and the CMP0169 deprecation warning that appeared before this change is gone. Closes #5714 item 5. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Make the CMake integration-test wrappers consistent The six tests/cmake_* integration-test wrappers had drifted: - Only cmake_import and cmake_import_minver forwarded -A "${CMAKE_GENERATOR_PLATFORM}" to the inner configure, and none forwarded -T "${CMAKE_GENERATOR_TOOLSET}". The Windows workflow configures the outer build with -A Win32 -T ClangCL, so without forwarding, the inner projects of cmake_add_subdirectory, cmake_fetch_content, cmake_fetch_content2 and cmake_target_include_directories built with the generator defaults instead of matching the outer build's platform and toolset. Forward both consistently from all six wrappers. - cmake_fetch_content and cmake_fetch_content2 passed -Dnlohmann_json_source to their inner projects, which never read it (CMake warns "manually-specified variables were not used"); the inner projects fetch their own copy of the library instead. Drop it. - tests/CMakeLists.txt set JSON_FORCED_GLOBAL_COMPILE_OPTIONS from the matching environment variable but never read the cache variable again; the lines right below it read $ENV{JSON_FORCED_GLOBAL_COMPILE_OPTIONS} directly, like the LINK_OPTIONS counterpart already does. Remove the dead set(). This changes which platform and toolset the Win32 and ClangCL CI jobs build the four newly-forwarding wrappers' inner projects with, which may surface new failures there; CI has to confirm those jobs. Verified locally with Ninja (empty -A ""/-T "" is accepted): all 12 cmake_* tests still pass, and the inner fetch_content configures no longer warn about the unused variable. Closes #5714 item 6. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Turn the #972 fifo_map regression test into a real test The #972 regression test in unit-regression1.cpp only built a my_json array from a string literal (the original crash) and had no CHECK, so the fifo_map object type it exists to demonstrate was never exercised. Meanwhile the docs recommend fifo_map for keeping object keys in insertion order (object_order.md, template_parameters.md), and nothing tested that recommendation. Extend the section: after the original array assignment, parse an object with my_json::parse() (not via the "..."_json UDL, which returns a plain nlohmann::json and would exercise the cross-basic_json conversion constructor instead of the parser's own key insertion - and, as tried locally, does not keep fifo order for this stateful comparator) and check that dump() keeps insertion order, and that it survives erase() and inserting a new key. Also narrow thirdparty/fifo_map off the include path of every other test-* target: it was a PUBLIC include directory of test_main, even though unit-regression1.cpp is its only user. Add a small fifo_map_include INTERFACE library with that include directory and attach it to test-regression1 only via json_test_set_test_options(). Verified locally (test-regression1_cpp11, default build and -fsanitize=address,undefined): the new checks pass; `git grep fifo_map tests` still only finds unit-regression1.cpp and the vendored header. Closes #5714 item 7. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Document the vendored doctest.h patch; fix stale doctest_compatibility.h comments tests/thirdparty/doctest/doctest.h is doctest 2.4.12, imported in #4771. Two weeks later, #4801 hand-edited translateActiveException() to declare "String res;" inside the translator loop instead of before it, so a translator that does not match does not leave a previous translator's result in "res" for the next iteration to see. Nothing recorded this, so re-vendoring doctest.h from upstream would silently drop the fix. Add a comment at the patched site naming the version, the PR and the reason, so a future re-vendor knows to re-apply it. Also fix two stale comments in doctest_compatibility.h: - The DOCTEST_THREAD_LOCAL comment referenced Xcode 6/7, which is no longer supported; reword it to explain why the define must stay regardless (it keeps doctest's own thread_local usage out of the way of the same Clang/MinGW crash that JSON_NO_THREAD_LOCAL works around in the library, see ci_test_no_thread_local). - The <iosfwd> include's comment justified it with tests that define "private" as "public"; no test under tests/src does that any more (removed by #2352). Reword the comment instead of dropping the include, since confirming it is safe to drop needs the full CI matrix including MSVC 2015+. Verified locally that tests/src/unit-readme.cpp still builds and passes 17/17 with these headers. Closes #5714 item 9. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Stop compiling unit-wstring.cpp out entirely on classic ICC tests/src/unit-wstring.cpp wrapped the whole file in #ifndef __INTEL_COMPILER, with the comment "ICPC errors out on multibyte character sequences in source files". The ci_icpc job (intel/oneapi-hpckit:2023.2.1) still exists, so that job ran none of the wstring/u16string/u32string input adapter tests, including the malformed-input checks #5704 (open) extends. Only 9 lines contained non-ASCII bytes: the three *_is_utf16()/ *_is_utf32() probe functions, and three std::wstring/u16string/ u32string literals plus their narrow-string dump() expectations. Rewrite all of them with \u/\U escapes in the wide/u16/u32 literals and \x escapes (split into separate string-literal tokens so a following byte is never read as part of the same hex escape, e.g. "\xE1\x83\x85" "a") in the narrow ones. Remove the #ifndef __INTEL_COMPILER/#endif guard along with it. The *_is_utf16()/*_is_utf32() probes compared a raw multibyte literal against an escape-based one to detect a compiler that misreads the source file's encoding; with no raw literals left to misread, the comparison is now tautological, so drop the probes and the "if" guards around each SECTION's body instead of leaving them in as dead checks. The same non-ASCII-in-source-and-in-a-narrow-comparison pattern existed once more in unit-deserialization.cpp's "Using _json with char8_t literals #4945" test: a raw emoji character in a u8R"(...)" literal, guarded by a check_utf8() that returned false for ICC (same reason) and for Windows without the active UTF-8 code page. Rewrite the literal with a \U escape and compare it against a \x-escaped expectation instead of a second raw literal, and drop check_utf8() and the now-unused <windows.h> include along with the guard. Verified locally (clang, -std=c++11 and -std=c++20, -fsanitize=address,undefined, and a plain build): test-wstring keeps 18/18 assertions and unit-deserialization keeps 466/466 (c++11) and 477/477 (c++20) assertions, matching this branch before the change exactly - no coverage was gained or lost, only the source-encoding dependency was removed. ci_icpc has to confirm classic ICC actually builds and passes test-wstring now; if it does not, that is a real finding, not a reason to restore the guard. Overlaps #5704 (open), which edits unit-wstring.cpp inside the previously-guarded region (an include near the top, checks in the invalid-string sections, and a new section at the end). Closes #5713 item 5. Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me>
3182 lines
141 KiB
C++
3182 lines
141 KiB
C++
// __ _____ _____ _____
|
|
// __| | __| | | | JSON for Modern C++ (supporting code)
|
|
// | | |__ | | | | | | version 3.12.0
|
|
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
|
|
//
|
|
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
|
|
// SPDX-License-Identifier: MIT
|
|
|
|
#include "doctest_compatibility.h"
|
|
|
|
#include <nlohmann/json.hpp>
|
|
using nlohmann::json;
|
|
|
|
#include <iostream>
|
|
#include <fstream>
|
|
#include <set>
|
|
#include "make_test_data_available.hpp"
|
|
#include "round_trip_corpus.hpp"
|
|
#include "test_utils.hpp"
|
|
#include "sax_countdown.hpp"
|
|
using utils::SaxCountdown;
|
|
|
|
|
|
TEST_CASE("UBJSON")
|
|
{
|
|
SECTION("individual values")
|
|
{
|
|
SECTION("discarded")
|
|
{
|
|
// discarded values are not serialized
|
|
json const j = json::value_t::discarded;
|
|
const auto result = json::to_ubjson(j);
|
|
CHECK(result.empty());
|
|
}
|
|
|
|
SECTION("null")
|
|
{
|
|
json const j = nullptr;
|
|
std::vector<uint8_t> expected = {'Z'};
|
|
const auto result = json::to_ubjson(j);
|
|
CHECK(result == expected);
|
|
|
|
// roundtrip
|
|
CHECK(json::from_ubjson(result) == j);
|
|
CHECK(json::from_ubjson(result, true, false) == j);
|
|
}
|
|
|
|
SECTION("boolean")
|
|
{
|
|
SECTION("true")
|
|
{
|
|
json const j = true;
|
|
std::vector<uint8_t> const expected = {'T'};
|
|
const auto result = json::to_ubjson(j);
|
|
CHECK(result == expected);
|
|
|
|
// roundtrip
|
|
CHECK(json::from_ubjson(result) == j);
|
|
CHECK(json::from_ubjson(result, true, false) == j);
|
|
}
|
|
|
|
SECTION("false")
|
|
{
|
|
json const j = false;
|
|
std::vector<uint8_t> const expected = {'F'};
|
|
const auto result = json::to_ubjson(j);
|
|
CHECK(result == expected);
|
|
|
|
// roundtrip
|
|
CHECK(json::from_ubjson(result) == j);
|
|
CHECK(json::from_ubjson(result, true, false) == j);
|
|
}
|
|
}
|
|
|
|
SECTION("number")
|
|
{
|
|
SECTION("signed")
|
|
{
|
|
SECTION("-9223372036854775808..-2147483649 (int64)")
|
|
{
|
|
std::vector<int64_t> const numbers
|
|
{
|
|
(std::numeric_limits<int64_t>::min)(),
|
|
-1000000000000000000LL,
|
|
-100000000000000000LL,
|
|
-10000000000000000LL,
|
|
-1000000000000000LL,
|
|
-100000000000000LL,
|
|
-10000000000000LL,
|
|
-1000000000000LL,
|
|
-100000000000LL,
|
|
-10000000000LL,
|
|
-2147483649LL,
|
|
};
|
|
for (auto i : numbers)
|
|
{
|
|
CAPTURE(i)
|
|
|
|
// create JSON value with integer number
|
|
json const j = i;
|
|
|
|
// check type
|
|
CHECK(j.is_number_integer());
|
|
|
|
// create expected byte vector
|
|
std::vector<uint8_t> const expected
|
|
{
|
|
static_cast<uint8_t>('L'),
|
|
static_cast<uint8_t>((i >> 56) & 0xff),
|
|
static_cast<uint8_t>((i >> 48) & 0xff),
|
|
static_cast<uint8_t>((i >> 40) & 0xff),
|
|
static_cast<uint8_t>((i >> 32) & 0xff),
|
|
static_cast<uint8_t>((i >> 24) & 0xff),
|
|
static_cast<uint8_t>((i >> 16) & 0xff),
|
|
static_cast<uint8_t>((i >> 8) & 0xff),
|
|
static_cast<uint8_t>(i & 0xff),
|
|
};
|
|
|
|
// compare result + size
|
|
const auto result = json::to_ubjson(j);
|
|
CHECK(result == expected);
|
|
CHECK(result.size() == 9);
|
|
|
|
// check individual bytes
|
|
CHECK(result[0] == 'L');
|
|
int64_t const restored = (static_cast<int64_t>(result[1]) << 070) +
|
|
(static_cast<int64_t>(result[2]) << 060) +
|
|
(static_cast<int64_t>(result[3]) << 050) +
|
|
(static_cast<int64_t>(result[4]) << 040) +
|
|
(static_cast<int64_t>(result[5]) << 030) +
|
|
(static_cast<int64_t>(result[6]) << 020) +
|
|
(static_cast<int64_t>(result[7]) << 010) +
|
|
static_cast<int64_t>(result[8]);
|
|
CHECK(restored == i);
|
|
|
|
// roundtrip
|
|
CHECK(json::from_ubjson(result) == j);
|
|
CHECK(json::from_ubjson(result, true, false) == j);
|
|
}
|
|
}
|
|
|
|
SECTION("-2147483648..-32769 (int32)")
|
|
{
|
|
std::vector<int32_t> numbers;
|
|
numbers.push_back(-32769);
|
|
numbers.push_back(-100000);
|
|
numbers.push_back(-1000000);
|
|
numbers.push_back(-10000000);
|
|
numbers.push_back(-100000000);
|
|
numbers.push_back(-1000000000);
|
|
numbers.push_back(-2147483647 - 1); // https://stackoverflow.com/a/29356002/266378
|
|
for (auto i : numbers)
|
|
{
|
|
CAPTURE(i)
|
|
|
|
// create JSON value with integer number
|
|
json const j = i;
|
|
|
|
// check type
|
|
CHECK(j.is_number_integer());
|
|
|
|
// create expected byte vector
|
|
std::vector<uint8_t> const expected
|
|
{
|
|
static_cast<uint8_t>('l'),
|
|
static_cast<uint8_t>((i >> 24) & 0xff),
|
|
static_cast<uint8_t>((i >> 16) & 0xff),
|
|
static_cast<uint8_t>((i >> 8) & 0xff),
|
|
static_cast<uint8_t>(i & 0xff),
|
|
};
|
|
|
|
// compare result + size
|
|
const auto result = json::to_ubjson(j);
|
|
CHECK(result == expected);
|
|
CHECK(result.size() == 5);
|
|
|
|
// check individual bytes
|
|
CHECK(result[0] == 'l');
|
|
int32_t const restored = (static_cast<int32_t>(result[1]) << 030) +
|
|
(static_cast<int32_t>(result[2]) << 020) +
|
|
(static_cast<int32_t>(result[3]) << 010) +
|
|
static_cast<int32_t>(result[4]);
|
|
CHECK(restored == i);
|
|
|
|
// roundtrip
|
|
CHECK(json::from_ubjson(result) == j);
|
|
CHECK(json::from_ubjson(result, true, false) == j);
|
|
}
|
|
}
|
|
|
|
SECTION("-32768..-129 (int16)")
|
|
{
|
|
for (int32_t i = -32768; i <= -129; i = utils::next_integer_sample(i, -129, 7))
|
|
{
|
|
CAPTURE(i)
|
|
|
|
// create JSON value with integer number
|
|
json const j = i;
|
|
|
|
// check type
|
|
CHECK(j.is_number_integer());
|
|
|
|
// create expected byte vector
|
|
std::vector<uint8_t> const expected
|
|
{
|
|
static_cast<uint8_t>('I'),
|
|
static_cast<uint8_t>((i >> 8) & 0xff),
|
|
static_cast<uint8_t>(i & 0xff),
|
|
};
|
|
|
|
// compare result + size
|
|
const auto result = json::to_ubjson(j);
|
|
CHECK(result == expected);
|
|
CHECK(result.size() == 3);
|
|
|
|
// check individual bytes
|
|
CHECK(result[0] == 'I');
|
|
auto const restored = static_cast<int16_t>(((result[1] << 8) + result[2]));
|
|
CHECK(restored == i);
|
|
|
|
// roundtrip
|
|
CHECK(json::from_ubjson(result) == j);
|
|
CHECK(json::from_ubjson(result, true, false) == j);
|
|
}
|
|
}
|
|
|
|
SECTION("-9263 (int16)")
|
|
{
|
|
json const j = -9263;
|
|
std::vector<uint8_t> expected = {'I', 0xdb, 0xd1};
|
|
|
|
// compare result + size
|
|
const auto result = json::to_ubjson(j);
|
|
CHECK(result == expected);
|
|
CHECK(result.size() == 3);
|
|
|
|
// check individual bytes
|
|
CHECK(result[0] == 'I');
|
|
auto const restored = static_cast<int16_t>(((result[1] << 8) + result[2]));
|
|
CHECK(restored == -9263);
|
|
|
|
// roundtrip
|
|
CHECK(json::from_ubjson(result) == j);
|
|
CHECK(json::from_ubjson(result, true, false) == j);
|
|
}
|
|
|
|
SECTION("-128..-1 (int8)")
|
|
{
|
|
for (auto i = -128; i <= -1; ++i)
|
|
{
|
|
CAPTURE(i)
|
|
|
|
// create JSON value with integer number
|
|
json const j = i;
|
|
|
|
// check type
|
|
CHECK(j.is_number_integer());
|
|
|
|
// create expected byte vector
|
|
std::vector<uint8_t> const expected
|
|
{
|
|
'i',
|
|
static_cast<uint8_t>(i),
|
|
};
|
|
|
|
// compare result + size
|
|
const auto result = json::to_ubjson(j);
|
|
CHECK(result == expected);
|
|
CHECK(result.size() == 2);
|
|
|
|
// check individual bytes
|
|
CHECK(result[0] == 'i');
|
|
CHECK(static_cast<int8_t>(result[1]) == i);
|
|
|
|
// roundtrip
|
|
CHECK(json::from_ubjson(result) == j);
|
|
CHECK(json::from_ubjson(result, true, false) == j);
|
|
}
|
|
}
|
|
|
|
SECTION("0..127 (int8)")
|
|
{
|
|
for (size_t i = 0; i <= 127; ++i)
|
|
{
|
|
CAPTURE(i)
|
|
|
|
// create JSON value with integer number
|
|
json j = -1;
|
|
j.get_ref<json::number_integer_t&>() = static_cast<json::number_integer_t>(i);
|
|
|
|
// check type
|
|
CHECK(j.is_number_integer());
|
|
|
|
// create expected byte vector
|
|
std::vector<uint8_t> const expected
|
|
{
|
|
static_cast<uint8_t>('i'),
|
|
static_cast<uint8_t>(i),
|
|
};
|
|
|
|
// compare result + size
|
|
const auto result = json::to_ubjson(j);
|
|
CHECK(result == expected);
|
|
CHECK(result.size() == 2);
|
|
|
|
// check individual bytes
|
|
CHECK(result[0] == 'i');
|
|
CHECK(result[1] == i);
|
|
|
|
// roundtrip
|
|
CHECK(json::from_ubjson(result) == j);
|
|
CHECK(json::from_ubjson(result, true, false) == j);
|
|
}
|
|
}
|
|
|
|
SECTION("128..255 (uint8)")
|
|
{
|
|
for (size_t i = 128; i <= 255; ++i)
|
|
{
|
|
CAPTURE(i)
|
|
|
|
// create JSON value with integer number
|
|
json j = -1;
|
|
j.get_ref<json::number_integer_t&>() = static_cast<json::number_integer_t>(i);
|
|
|
|
// check type
|
|
CHECK(j.is_number_integer());
|
|
|
|
// create expected byte vector
|
|
std::vector<uint8_t> const expected
|
|
{
|
|
static_cast<uint8_t>('U'),
|
|
static_cast<uint8_t>(i),
|
|
};
|
|
|
|
// compare result + size
|
|
const auto result = json::to_ubjson(j);
|
|
CHECK(result == expected);
|
|
CHECK(result.size() == 2);
|
|
|
|
// check individual bytes
|
|
CHECK(result[0] == 'U');
|
|
CHECK(result[1] == i);
|
|
|
|
// roundtrip
|
|
CHECK(json::from_ubjson(result) == j);
|
|
CHECK(json::from_ubjson(result, true, false) == j);
|
|
}
|
|
}
|
|
|
|
SECTION("256..32767 (int16)")
|
|
{
|
|
for (size_t i = 256; i <= 32767; i = utils::next_integer_sample(i, static_cast<size_t>(32767), static_cast<size_t>(7)))
|
|
{
|
|
CAPTURE(i)
|
|
|
|
// create JSON value with integer number
|
|
json j = -1;
|
|
j.get_ref<json::number_integer_t&>() = static_cast<json::number_integer_t>(i);
|
|
|
|
// check type
|
|
CHECK(j.is_number_integer());
|
|
|
|
// create expected byte vector
|
|
std::vector<uint8_t> const expected
|
|
{
|
|
static_cast<uint8_t>('I'),
|
|
static_cast<uint8_t>((i >> 8) & 0xff),
|
|
static_cast<uint8_t>(i & 0xff),
|
|
};
|
|
|
|
// compare result + size
|
|
const auto result = json::to_ubjson(j);
|
|
CHECK(result == expected);
|
|
CHECK(result.size() == 3);
|
|
|
|
// check individual bytes
|
|
CHECK(result[0] == 'I');
|
|
auto const restored = static_cast<uint16_t>((static_cast<uint8_t>(result[1]) * 256) + static_cast<uint8_t>(result[2]));
|
|
CHECK(restored == i);
|
|
|
|
// roundtrip
|
|
CHECK(json::from_ubjson(result) == j);
|
|
CHECK(json::from_ubjson(result, true, false) == j);
|
|
}
|
|
}
|
|
|
|
SECTION("65536..2147483647 (int32)")
|
|
{
|
|
for (uint32_t i :
|
|
{
|
|
65536u, 77777u, 1048576u
|
|
})
|
|
{
|
|
CAPTURE(i)
|
|
|
|
// create JSON value with integer number
|
|
json j = -1;
|
|
j.get_ref<json::number_integer_t&>() = static_cast<json::number_integer_t>(i);
|
|
|
|
// check type
|
|
CHECK(j.is_number_integer());
|
|
|
|
// create expected byte vector
|
|
std::vector<uint8_t> const expected
|
|
{
|
|
'l',
|
|
static_cast<uint8_t>((i >> 24) & 0xff),
|
|
static_cast<uint8_t>((i >> 16) & 0xff),
|
|
static_cast<uint8_t>((i >> 8) & 0xff),
|
|
static_cast<uint8_t>(i & 0xff),
|
|
};
|
|
|
|
// compare result + size
|
|
const auto result = json::to_ubjson(j);
|
|
CHECK(result == expected);
|
|
CHECK(result.size() == 5);
|
|
|
|
// check individual bytes
|
|
CHECK(result[0] == 'l');
|
|
uint32_t const restored = (static_cast<uint32_t>(result[1]) << 030) +
|
|
(static_cast<uint32_t>(result[2]) << 020) +
|
|
(static_cast<uint32_t>(result[3]) << 010) +
|
|
static_cast<uint32_t>(result[4]);
|
|
CHECK(restored == i);
|
|
|
|
// roundtrip
|
|
CHECK(json::from_ubjson(result) == j);
|
|
CHECK(json::from_ubjson(result, true, false) == j);
|
|
}
|
|
}
|
|
|
|
SECTION("2147483648..9223372036854775807 (int64)")
|
|
{
|
|
std::vector<uint64_t> const v = {2147483648ul, 9223372036854775807ul};
|
|
for (uint64_t i : v)
|
|
{
|
|
CAPTURE(i)
|
|
|
|
// create JSON value with integer number
|
|
json j = -1;
|
|
j.get_ref<json::number_integer_t&>() = static_cast<json::number_integer_t>(i);
|
|
|
|
// check type
|
|
CHECK(j.is_number_integer());
|
|
|
|
// create expected byte vector
|
|
std::vector<uint8_t> const expected
|
|
{
|
|
'L',
|
|
static_cast<uint8_t>((i >> 070) & 0xff),
|
|
static_cast<uint8_t>((i >> 060) & 0xff),
|
|
static_cast<uint8_t>((i >> 050) & 0xff),
|
|
static_cast<uint8_t>((i >> 040) & 0xff),
|
|
static_cast<uint8_t>((i >> 030) & 0xff),
|
|
static_cast<uint8_t>((i >> 020) & 0xff),
|
|
static_cast<uint8_t>((i >> 010) & 0xff),
|
|
static_cast<uint8_t>(i & 0xff),
|
|
};
|
|
|
|
// compare result + size
|
|
const auto result = json::to_ubjson(j);
|
|
CHECK(result == expected);
|
|
CHECK(result.size() == 9);
|
|
|
|
// check individual bytes
|
|
CHECK(result[0] == 'L');
|
|
uint64_t const restored = (static_cast<uint64_t>(result[1]) << 070) +
|
|
(static_cast<uint64_t>(result[2]) << 060) +
|
|
(static_cast<uint64_t>(result[3]) << 050) +
|
|
(static_cast<uint64_t>(result[4]) << 040) +
|
|
(static_cast<uint64_t>(result[5]) << 030) +
|
|
(static_cast<uint64_t>(result[6]) << 020) +
|
|
(static_cast<uint64_t>(result[7]) << 010) +
|
|
static_cast<uint64_t>(result[8]);
|
|
CHECK(restored == i);
|
|
|
|
// roundtrip
|
|
CHECK(json::from_ubjson(result) == j);
|
|
CHECK(json::from_ubjson(result, true, false) == j);
|
|
}
|
|
}
|
|
}
|
|
|
|
SECTION("unsigned")
|
|
{
|
|
SECTION("0..127 (int8)")
|
|
{
|
|
for (size_t i = 0; i <= 127; ++i)
|
|
{
|
|
CAPTURE(i)
|
|
|
|
// create JSON value with unsigned integer number
|
|
json const j = i;
|
|
|
|
// check type
|
|
CHECK(j.is_number_unsigned());
|
|
|
|
// create expected byte vector
|
|
std::vector<uint8_t> const expected
|
|
{
|
|
'i',
|
|
static_cast<uint8_t>(i),
|
|
};
|
|
|
|
// compare result + size
|
|
const auto result = json::to_ubjson(j);
|
|
CHECK(result == expected);
|
|
CHECK(result.size() == 2);
|
|
|
|
// check individual bytes
|
|
CHECK(result[0] == 'i');
|
|
auto const restored = static_cast<uint8_t>(result[1]);
|
|
CHECK(restored == i);
|
|
|
|
// roundtrip
|
|
CHECK(json::from_ubjson(result) == j);
|
|
CHECK(json::from_ubjson(result, true, false) == j);
|
|
}
|
|
}
|
|
|
|
SECTION("128..255 (uint8)")
|
|
{
|
|
for (size_t i = 128; i <= 255; ++i)
|
|
{
|
|
CAPTURE(i)
|
|
|
|
// create JSON value with unsigned integer number
|
|
json const j = i;
|
|
|
|
// check type
|
|
CHECK(j.is_number_unsigned());
|
|
|
|
// create expected byte vector
|
|
std::vector<uint8_t> const expected
|
|
{
|
|
'U',
|
|
static_cast<uint8_t>(i),
|
|
};
|
|
|
|
// compare result + size
|
|
const auto result = json::to_ubjson(j);
|
|
CHECK(result == expected);
|
|
CHECK(result.size() == 2);
|
|
|
|
// check individual bytes
|
|
CHECK(result[0] == 'U');
|
|
auto const restored = static_cast<uint8_t>(result[1]);
|
|
CHECK(restored == i);
|
|
|
|
// roundtrip
|
|
CHECK(json::from_ubjson(result) == j);
|
|
CHECK(json::from_ubjson(result, true, false) == j);
|
|
}
|
|
}
|
|
|
|
SECTION("256..32767 (int16)")
|
|
{
|
|
for (size_t i = 256; i <= 32767; i = utils::next_integer_sample(i, static_cast<size_t>(32767), static_cast<size_t>(7)))
|
|
{
|
|
CAPTURE(i)
|
|
|
|
// create JSON value with unsigned integer number
|
|
json const j = i;
|
|
|
|
// check type
|
|
CHECK(j.is_number_unsigned());
|
|
|
|
// create expected byte vector
|
|
std::vector<uint8_t> const expected
|
|
{
|
|
'I',
|
|
static_cast<uint8_t>((i >> 8) & 0xff),
|
|
static_cast<uint8_t>(i & 0xff),
|
|
};
|
|
|
|
// compare result + size
|
|
const auto result = json::to_ubjson(j);
|
|
CHECK(result == expected);
|
|
CHECK(result.size() == 3);
|
|
|
|
// check individual bytes
|
|
CHECK(result[0] == 'I');
|
|
auto const restored = static_cast<uint16_t>((static_cast<uint8_t>(result[1]) * 256) + static_cast<uint8_t>(result[2]));
|
|
CHECK(restored == i);
|
|
|
|
// roundtrip
|
|
CHECK(json::from_ubjson(result) == j);
|
|
CHECK(json::from_ubjson(result, true, false) == j);
|
|
}
|
|
}
|
|
|
|
SECTION("65536..2147483647 (int32)")
|
|
{
|
|
for (uint32_t i :
|
|
{
|
|
65536u, 77777u, 1048576u
|
|
})
|
|
{
|
|
CAPTURE(i)
|
|
|
|
// create JSON value with unsigned integer number
|
|
json const j = i;
|
|
|
|
// check type
|
|
CHECK(j.is_number_unsigned());
|
|
|
|
// create expected byte vector
|
|
std::vector<uint8_t> const expected
|
|
{
|
|
'l',
|
|
static_cast<uint8_t>((i >> 24) & 0xff),
|
|
static_cast<uint8_t>((i >> 16) & 0xff),
|
|
static_cast<uint8_t>((i >> 8) & 0xff),
|
|
static_cast<uint8_t>(i & 0xff),
|
|
};
|
|
|
|
// compare result + size
|
|
const auto result = json::to_ubjson(j);
|
|
CHECK(result == expected);
|
|
CHECK(result.size() == 5);
|
|
|
|
// check individual bytes
|
|
CHECK(result[0] == 'l');
|
|
uint32_t const restored = (static_cast<uint32_t>(result[1]) << 030) +
|
|
(static_cast<uint32_t>(result[2]) << 020) +
|
|
(static_cast<uint32_t>(result[3]) << 010) +
|
|
static_cast<uint32_t>(result[4]);
|
|
CHECK(restored == i);
|
|
|
|
// roundtrip
|
|
CHECK(json::from_ubjson(result) == j);
|
|
CHECK(json::from_ubjson(result, true, false) == j);
|
|
}
|
|
}
|
|
|
|
SECTION("2147483648..9223372036854775807 (int64)")
|
|
{
|
|
std::vector<uint64_t> const v = {2147483648ul, 9223372036854775807ul};
|
|
for (uint64_t i : v)
|
|
{
|
|
CAPTURE(i)
|
|
|
|
// create JSON value with integer number
|
|
json const j = i;
|
|
|
|
// check type
|
|
CHECK(j.is_number_unsigned());
|
|
|
|
// create expected byte vector
|
|
std::vector<uint8_t> const expected
|
|
{
|
|
'L',
|
|
static_cast<uint8_t>((i >> 070) & 0xff),
|
|
static_cast<uint8_t>((i >> 060) & 0xff),
|
|
static_cast<uint8_t>((i >> 050) & 0xff),
|
|
static_cast<uint8_t>((i >> 040) & 0xff),
|
|
static_cast<uint8_t>((i >> 030) & 0xff),
|
|
static_cast<uint8_t>((i >> 020) & 0xff),
|
|
static_cast<uint8_t>((i >> 010) & 0xff),
|
|
static_cast<uint8_t>(i & 0xff),
|
|
};
|
|
|
|
// compare result + size
|
|
const auto result = json::to_ubjson(j);
|
|
CHECK(result == expected);
|
|
CHECK(result.size() == 9);
|
|
|
|
// check individual bytes
|
|
CHECK(result[0] == 'L');
|
|
uint64_t const restored = (static_cast<uint64_t>(result[1]) << 070) +
|
|
(static_cast<uint64_t>(result[2]) << 060) +
|
|
(static_cast<uint64_t>(result[3]) << 050) +
|
|
(static_cast<uint64_t>(result[4]) << 040) +
|
|
(static_cast<uint64_t>(result[5]) << 030) +
|
|
(static_cast<uint64_t>(result[6]) << 020) +
|
|
(static_cast<uint64_t>(result[7]) << 010) +
|
|
static_cast<uint64_t>(result[8]);
|
|
CHECK(restored == i);
|
|
|
|
// roundtrip
|
|
CHECK(json::from_ubjson(result) == j);
|
|
CHECK(json::from_ubjson(result, true, false) == j);
|
|
}
|
|
}
|
|
}
|
|
|
|
SECTION("float64")
|
|
{
|
|
SECTION("3.1415925")
|
|
{
|
|
double v = 3.1415925;
|
|
json const j = v;
|
|
std::vector<uint8_t> expected =
|
|
{
|
|
'D', 0x40, 0x09, 0x21, 0xfb, 0x3f, 0xa6, 0xde, 0xfc
|
|
};
|
|
const auto result = json::to_ubjson(j);
|
|
CHECK(result == expected);
|
|
|
|
// roundtrip
|
|
CHECK(json::from_ubjson(result) == j);
|
|
CHECK(json::from_ubjson(result) == v);
|
|
CHECK(json::from_ubjson(result, true, false) == j);
|
|
}
|
|
}
|
|
|
|
SECTION("high-precision number")
|
|
{
|
|
SECTION("unsigned integer number")
|
|
{
|
|
std::vector<uint8_t> const vec = {'H', 'i', 0x14, '1', '2', '3', '4', '5', '6', '7', '8', '9', '0', '1', '2', '3', '4', '5', '6', '7', '8', '9', '0'};
|
|
const auto j = json::from_ubjson(vec);
|
|
CHECK(j.is_number_unsigned());
|
|
CHECK(j.dump() == "12345678901234567890");
|
|
}
|
|
|
|
SECTION("signed integer number")
|
|
{
|
|
std::vector<uint8_t> const vec = {'H', 'i', 0x13, '-', '1', '2', '3', '4', '5', '6', '7', '8', '9', '0', '1', '2', '3', '4', '5', '6', '7', '8'};
|
|
const auto j = json::from_ubjson(vec);
|
|
CHECK(j.is_number_integer());
|
|
CHECK(j.dump() == "-123456789012345678");
|
|
}
|
|
|
|
SECTION("floating-point number")
|
|
{
|
|
std::vector<uint8_t> const vec = {'H', 'i', 0x16, '3', '.', '1', '4', '1', '5', '9', '2', '6', '5', '3', '5', '8', '9', '7', '9', '3', '2', '3', '8', '4', '6'};
|
|
const auto j = json::from_ubjson(vec);
|
|
CHECK(j.is_number_float());
|
|
CHECK(j.dump() == "3.141592653589793");
|
|
}
|
|
|
|
SECTION("errors")
|
|
{
|
|
// error while parsing length
|
|
std::vector<uint8_t> const vec0 = {'H', 'i'};
|
|
CHECK(json::from_ubjson(vec0, true, false).is_discarded());
|
|
// error while parsing string
|
|
std::vector<uint8_t> const vec1 = {'H', 'i', '1'};
|
|
CHECK(json::from_ubjson(vec1, true, false).is_discarded());
|
|
|
|
json _;
|
|
std::vector<uint8_t> const vec2 = {'H', 'i', 2, '1', 'A', '3'};
|
|
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(vec2), "[json.exception.parse_error.115] parse error at byte 5: syntax error while parsing UBJSON high-precision number: invalid number text: 1A", json::parse_error);
|
|
std::vector<uint8_t> const vec3 = {'H', 'i', 2, '1', '.'};
|
|
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(vec3), "[json.exception.parse_error.115] parse error at byte 5: syntax error while parsing UBJSON high-precision number: invalid number text: 1.", json::parse_error);
|
|
std::vector<uint8_t> const vec_overflow = {'H', 'i', 5, '1', 'e', '4', '0', '0'};
|
|
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(vec_overflow), "[json.exception.out_of_range.406] number overflow parsing '1e400'", json::out_of_range&);
|
|
std::vector<uint8_t> const vec4 = {'H', 2, '1', '0'};
|
|
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(vec4), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing UBJSON size: expected length type specification (U, i, I, l, L) after '#'; last byte: 0x02", json::parse_error);
|
|
}
|
|
|
|
SECTION("serialization")
|
|
{
|
|
// number that does not fit int64
|
|
json const j = 11111111111111111111ULL;
|
|
CHECK(j.is_number_unsigned());
|
|
|
|
// number will be serialized to high-precision number
|
|
const auto vec = json::to_ubjson(j);
|
|
std::vector<uint8_t> expected = {'H', 'i', 0x14, '1', '1', '1', '1', '1', '1', '1', '1', '1', '1', '1', '1', '1', '1', '1', '1', '1', '1', '1', '1'};
|
|
CHECK(vec == expected);
|
|
|
|
// roundtrip
|
|
CHECK(json::from_ubjson(vec) == j);
|
|
}
|
|
}
|
|
}
|
|
|
|
SECTION("string")
|
|
{
|
|
SECTION("N = 0..127")
|
|
{
|
|
for (size_t N = 0; N <= 127; ++N)
|
|
{
|
|
CAPTURE(N)
|
|
|
|
// create JSON value with string containing of N * 'x'
|
|
const auto s = std::string(N, 'x');
|
|
json const j = s;
|
|
|
|
// create expected byte vector
|
|
std::vector<uint8_t> expected;
|
|
expected.push_back('S');
|
|
expected.push_back('i');
|
|
expected.push_back(static_cast<uint8_t>(N));
|
|
for (size_t i = 0; i < N; ++i)
|
|
{
|
|
expected.push_back('x');
|
|
}
|
|
|
|
// compare result + size
|
|
const auto result = json::to_ubjson(j);
|
|
CHECK(result == expected);
|
|
CHECK(result.size() == N + 3);
|
|
// check that no null byte is appended
|
|
if (N > 0)
|
|
{
|
|
CHECK(result.back() != '\x00');
|
|
}
|
|
|
|
// roundtrip
|
|
CHECK(json::from_ubjson(result) == j);
|
|
CHECK(json::from_ubjson(result, true, false) == j);
|
|
}
|
|
}
|
|
|
|
SECTION("N = 128..255")
|
|
{
|
|
for (size_t N = 128; N <= 255; ++N)
|
|
{
|
|
CAPTURE(N)
|
|
|
|
// create JSON value with string containing of N * 'x'
|
|
const auto s = std::string(N, 'x');
|
|
json const j = s;
|
|
|
|
// create expected byte vector
|
|
std::vector<uint8_t> expected;
|
|
expected.push_back('S');
|
|
expected.push_back('U');
|
|
expected.push_back(static_cast<uint8_t>(N));
|
|
for (size_t i = 0; i < N; ++i)
|
|
{
|
|
expected.push_back('x');
|
|
}
|
|
|
|
// compare result + size
|
|
const auto result = json::to_ubjson(j);
|
|
CHECK(result == expected);
|
|
CHECK(result.size() == N + 3);
|
|
// check that no null byte is appended
|
|
CHECK(result.back() != '\x00');
|
|
|
|
// roundtrip
|
|
CHECK(json::from_ubjson(result) == j);
|
|
CHECK(json::from_ubjson(result, true, false) == j);
|
|
}
|
|
}
|
|
|
|
SECTION("N = 256..32767")
|
|
{
|
|
for (size_t N :
|
|
{
|
|
256u, 999u, 1025u, 3333u, 2048u, 32767u
|
|
})
|
|
{
|
|
CAPTURE(N)
|
|
|
|
// create JSON value with string containing of N * 'x'
|
|
const auto s = std::string(N, 'x');
|
|
json const j = s;
|
|
|
|
// create expected byte vector (hack: create string first)
|
|
std::vector<uint8_t> expected(N, 'x');
|
|
// reverse order of commands, because we insert at begin()
|
|
expected.insert(expected.begin(), static_cast<uint8_t>(N & 0xff));
|
|
expected.insert(expected.begin(), static_cast<uint8_t>((N >> 8) & 0xff));
|
|
expected.insert(expected.begin(), 'I');
|
|
expected.insert(expected.begin(), 'S');
|
|
|
|
// compare result + size
|
|
const auto result = json::to_ubjson(j);
|
|
CHECK(result == expected);
|
|
CHECK(result.size() == N + 4);
|
|
// check that no null byte is appended
|
|
CHECK(result.back() != '\x00');
|
|
|
|
// roundtrip
|
|
CHECK(json::from_ubjson(result) == j);
|
|
CHECK(json::from_ubjson(result, true, false) == j);
|
|
}
|
|
}
|
|
|
|
SECTION("N = 65536..2147483647")
|
|
{
|
|
for (size_t N :
|
|
{
|
|
65536u, 77777u, 1048576u
|
|
})
|
|
{
|
|
CAPTURE(N)
|
|
|
|
// create JSON value with string containing of N * 'x'
|
|
const auto s = std::string(N, 'x');
|
|
json const j = s;
|
|
|
|
// create expected byte vector (hack: create string first)
|
|
std::vector<uint8_t> expected(N, 'x');
|
|
// reverse order of commands, because we insert at begin()
|
|
expected.insert(expected.begin(), static_cast<uint8_t>(N & 0xff));
|
|
expected.insert(expected.begin(), static_cast<uint8_t>((N >> 8) & 0xff));
|
|
expected.insert(expected.begin(), static_cast<uint8_t>((N >> 16) & 0xff));
|
|
expected.insert(expected.begin(), static_cast<uint8_t>((N >> 24) & 0xff));
|
|
expected.insert(expected.begin(), 'l');
|
|
expected.insert(expected.begin(), 'S');
|
|
|
|
// compare result + size
|
|
const auto result = json::to_ubjson(j);
|
|
CHECK(result == expected);
|
|
CHECK(result.size() == N + 6);
|
|
// check that no null byte is appended
|
|
CHECK(result.back() != '\x00');
|
|
|
|
// roundtrip
|
|
CHECK(json::from_ubjson(result) == j);
|
|
CHECK(json::from_ubjson(result, true, false) == j);
|
|
}
|
|
}
|
|
}
|
|
|
|
SECTION("binary")
|
|
{
|
|
SECTION("N = 0..127")
|
|
{
|
|
for (std::size_t N = 0; N <= 127; ++N)
|
|
{
|
|
CAPTURE(N)
|
|
|
|
// create JSON value with byte array containing of N * 'x'
|
|
const auto s = std::vector<std::uint8_t>(N, 'x');
|
|
json const j = json::binary(s);
|
|
|
|
// create expected byte vector
|
|
std::vector<std::uint8_t> expected;
|
|
expected.push_back(static_cast<std::uint8_t>('['));
|
|
if (N != 0)
|
|
{
|
|
expected.push_back(static_cast<std::uint8_t>('$'));
|
|
expected.push_back(static_cast<std::uint8_t>('U'));
|
|
}
|
|
expected.push_back(static_cast<std::uint8_t>('#'));
|
|
expected.push_back(static_cast<std::uint8_t>('i'));
|
|
expected.push_back(static_cast<std::uint8_t>(N));
|
|
for (size_t i = 0; i < N; ++i)
|
|
{
|
|
expected.push_back(0x78);
|
|
}
|
|
|
|
// compare result + size
|
|
const auto result = json::to_ubjson(j, true, true);
|
|
CHECK(result == expected);
|
|
if (N == 0)
|
|
{
|
|
CHECK(result.size() == N + 4);
|
|
}
|
|
else
|
|
{
|
|
CHECK(result.size() == N + 6);
|
|
}
|
|
|
|
// check that no null byte is appended
|
|
if (N > 0)
|
|
{
|
|
CHECK(result.back() != '\x00');
|
|
}
|
|
|
|
// roundtrip only works to an array of numbers
|
|
json j_out = s;
|
|
CHECK(json::from_ubjson(result) == j_out);
|
|
CHECK(json::from_ubjson(result, true, false) == j_out);
|
|
}
|
|
}
|
|
|
|
SECTION("N = 128..255")
|
|
{
|
|
for (std::size_t N = 128; N <= 255; ++N)
|
|
{
|
|
CAPTURE(N)
|
|
|
|
// create JSON value with byte array containing of N * 'x'
|
|
const auto s = std::vector<std::uint8_t>(N, 'x');
|
|
json const j = json::binary(s);
|
|
|
|
// create expected byte vector
|
|
std::vector<uint8_t> expected;
|
|
expected.push_back(static_cast<std::uint8_t>('['));
|
|
expected.push_back(static_cast<std::uint8_t>('$'));
|
|
expected.push_back(static_cast<std::uint8_t>('U'));
|
|
expected.push_back(static_cast<std::uint8_t>('#'));
|
|
expected.push_back(static_cast<std::uint8_t>('U'));
|
|
expected.push_back(static_cast<std::uint8_t>(N));
|
|
for (size_t i = 0; i < N; ++i)
|
|
{
|
|
expected.push_back(0x78);
|
|
}
|
|
|
|
// compare result + size
|
|
const auto result = json::to_ubjson(j, true, true);
|
|
CHECK(result == expected);
|
|
CHECK(result.size() == N + 6);
|
|
// check that no null byte is appended
|
|
CHECK(result.back() != '\x00');
|
|
|
|
// roundtrip only works to an array of numbers
|
|
json j_out = s;
|
|
CHECK(json::from_ubjson(result) == j_out);
|
|
CHECK(json::from_ubjson(result, true, false) == j_out);
|
|
}
|
|
}
|
|
|
|
SECTION("N = 256..32767")
|
|
{
|
|
for (std::size_t N :
|
|
{
|
|
256u, 999u, 1025u, 3333u, 2048u, 32767u
|
|
})
|
|
{
|
|
CAPTURE(N)
|
|
|
|
// create JSON value with byte array containing of N * 'x'
|
|
const auto s = std::vector<std::uint8_t>(N, 'x');
|
|
json const j = json::binary(s);
|
|
|
|
// create expected byte vector
|
|
std::vector<std::uint8_t> expected(N + 7, 'x');
|
|
expected[0] = '[';
|
|
expected[1] = '$';
|
|
expected[2] = 'U';
|
|
expected[3] = '#';
|
|
expected[4] = 'I';
|
|
expected[5] = static_cast<std::uint8_t>((N >> 8) & 0xFF);
|
|
expected[6] = static_cast<std::uint8_t>(N & 0xFF);
|
|
|
|
// compare result + size
|
|
const auto result = json::to_ubjson(j, true, true);
|
|
CHECK(result == expected);
|
|
CHECK(result.size() == N + 7);
|
|
// check that no null byte is appended
|
|
CHECK(result.back() != '\x00');
|
|
|
|
// roundtrip only works to an array of numbers
|
|
json j_out = s;
|
|
CHECK(json::from_ubjson(result) == j_out);
|
|
CHECK(json::from_ubjson(result, true, false) == j_out);
|
|
}
|
|
}
|
|
|
|
SECTION("N = 32768..2147483647")
|
|
{
|
|
for (std::size_t N :
|
|
{
|
|
32768u, 77777u, 1048576u
|
|
})
|
|
{
|
|
CAPTURE(N)
|
|
|
|
// create JSON value with byte array containing of N * 'x'
|
|
const auto s = std::vector<std::uint8_t>(N, 'x');
|
|
json const j = json::binary(s);
|
|
|
|
// create expected byte vector
|
|
std::vector<std::uint8_t> expected(N + 9, 'x');
|
|
expected[0] = '[';
|
|
expected[1] = '$';
|
|
expected[2] = 'U';
|
|
expected[3] = '#';
|
|
expected[4] = 'l';
|
|
expected[5] = static_cast<std::uint8_t>((N >> 24) & 0xFF);
|
|
expected[6] = static_cast<std::uint8_t>((N >> 16) & 0xFF);
|
|
expected[7] = static_cast<std::uint8_t>((N >> 8) & 0xFF);
|
|
expected[8] = static_cast<std::uint8_t>(N & 0xFF);
|
|
|
|
// compare result + size
|
|
const auto result = json::to_ubjson(j, true, true);
|
|
CHECK(result == expected);
|
|
CHECK(result.size() == N + 9);
|
|
// check that no null byte is appended
|
|
CHECK(result.back() != '\x00');
|
|
|
|
// roundtrip only works to an array of numbers
|
|
json j_out = s;
|
|
CHECK(json::from_ubjson(result) == j_out);
|
|
CHECK(json::from_ubjson(result, true, false) == j_out);
|
|
}
|
|
}
|
|
|
|
SECTION("Other Serializations")
|
|
{
|
|
const std::size_t N = 10;
|
|
const auto s = std::vector<std::uint8_t>(N, 'x');
|
|
json const j = json::binary(s);
|
|
|
|
SECTION("No Count No Type")
|
|
{
|
|
std::vector<uint8_t> expected;
|
|
expected.push_back(static_cast<std::uint8_t>('['));
|
|
for (std::size_t i = 0; i < N; ++i)
|
|
{
|
|
expected.push_back(static_cast<std::uint8_t>('U'));
|
|
expected.push_back(static_cast<std::uint8_t>(0x78));
|
|
}
|
|
expected.push_back(static_cast<std::uint8_t>(']'));
|
|
|
|
// compare result + size
|
|
const auto result = json::to_ubjson(j, false, false);
|
|
CHECK(result == expected);
|
|
CHECK(result.size() == N + 12);
|
|
// check that no null byte is appended
|
|
CHECK(result.back() != '\x00');
|
|
|
|
// roundtrip only works to an array of numbers
|
|
json j_out = s;
|
|
CHECK(json::from_ubjson(result) == j_out);
|
|
CHECK(json::from_ubjson(result, true, false) == j_out);
|
|
}
|
|
|
|
SECTION("Yes Count No Type")
|
|
{
|
|
std::vector<std::uint8_t> expected;
|
|
expected.push_back(static_cast<std::uint8_t>('['));
|
|
expected.push_back(static_cast<std::uint8_t>('#'));
|
|
expected.push_back(static_cast<std::uint8_t>('i'));
|
|
expected.push_back(static_cast<std::uint8_t>(N));
|
|
|
|
for (size_t i = 0; i < N; ++i)
|
|
{
|
|
expected.push_back(static_cast<std::uint8_t>('U'));
|
|
expected.push_back(static_cast<std::uint8_t>(0x78));
|
|
}
|
|
|
|
// compare result + size
|
|
const auto result = json::to_ubjson(j, true, false);
|
|
CHECK(result == expected);
|
|
CHECK(result.size() == N + 14);
|
|
// check that no null byte is appended
|
|
CHECK(result.back() != '\x00');
|
|
|
|
// roundtrip only works to an array of numbers
|
|
json j_out = s;
|
|
CHECK(json::from_ubjson(result) == j_out);
|
|
CHECK(json::from_ubjson(result, true, false) == j_out);
|
|
}
|
|
}
|
|
}
|
|
|
|
SECTION("array")
|
|
{
|
|
SECTION("empty")
|
|
{
|
|
SECTION("size=false type=false")
|
|
{
|
|
json const j = json::array();
|
|
std::vector<uint8_t> expected = {'[', ']'};
|
|
const auto result = json::to_ubjson(j);
|
|
CHECK(result == expected);
|
|
|
|
// roundtrip
|
|
CHECK(json::from_ubjson(result) == j);
|
|
CHECK(json::from_ubjson(result, true, false) == j);
|
|
}
|
|
|
|
SECTION("size=true type=false")
|
|
{
|
|
json const j = json::array();
|
|
std::vector<uint8_t> expected = {'[', '#', 'i', 0};
|
|
const auto result = json::to_ubjson(j, true);
|
|
CHECK(result == expected);
|
|
|
|
// roundtrip
|
|
CHECK(json::from_ubjson(result) == j);
|
|
CHECK(json::from_ubjson(result, true, false) == j);
|
|
}
|
|
|
|
SECTION("size=true type=true")
|
|
{
|
|
json const j = json::array();
|
|
std::vector<uint8_t> expected = {'[', '#', 'i', 0};
|
|
const auto result = json::to_ubjson(j, true, true);
|
|
CHECK(result == expected);
|
|
|
|
// roundtrip
|
|
CHECK(json::from_ubjson(result) == j);
|
|
CHECK(json::from_ubjson(result, true, false) == j);
|
|
}
|
|
}
|
|
|
|
SECTION("[null]")
|
|
{
|
|
SECTION("size=false type=false")
|
|
{
|
|
json const j = {nullptr};
|
|
std::vector<uint8_t> expected = {'[', 'Z', ']'};
|
|
const auto result = json::to_ubjson(j);
|
|
CHECK(result == expected);
|
|
|
|
// roundtrip
|
|
CHECK(json::from_ubjson(result) == j);
|
|
CHECK(json::from_ubjson(result, true, false) == j);
|
|
}
|
|
|
|
SECTION("size=true type=false")
|
|
{
|
|
json const j = {nullptr};
|
|
std::vector<uint8_t> expected = {'[', '#', 'i', 1, 'Z'};
|
|
const auto result = json::to_ubjson(j, true);
|
|
CHECK(result == expected);
|
|
|
|
// roundtrip
|
|
CHECK(json::from_ubjson(result) == j);
|
|
CHECK(json::from_ubjson(result, true, false) == j);
|
|
}
|
|
|
|
SECTION("size=true type=true")
|
|
{
|
|
json const j = {nullptr};
|
|
std::vector<uint8_t> expected = {'[', '$', 'Z', '#', 'i', 1};
|
|
const auto result = json::to_ubjson(j, true, true);
|
|
CHECK(result == expected);
|
|
|
|
// roundtrip
|
|
CHECK(json::from_ubjson(result) == j);
|
|
CHECK(json::from_ubjson(result, true, false) == j);
|
|
}
|
|
}
|
|
|
|
SECTION("[1,2,3,4,5]")
|
|
{
|
|
SECTION("size=false type=false")
|
|
{
|
|
json const j = json::parse("[1,2,3,4,5]");
|
|
std::vector<uint8_t> expected = {'[', 'i', 1, 'i', 2, 'i', 3, 'i', 4, 'i', 5, ']'};
|
|
const auto result = json::to_ubjson(j);
|
|
CHECK(result == expected);
|
|
|
|
// roundtrip
|
|
CHECK(json::from_ubjson(result) == j);
|
|
CHECK(json::from_ubjson(result, true, false) == j);
|
|
}
|
|
|
|
SECTION("size=true type=false")
|
|
{
|
|
json const j = json::parse("[1,2,3,4,5]");
|
|
std::vector<uint8_t> expected = {'[', '#', 'i', 5, 'i', 1, 'i', 2, 'i', 3, 'i', 4, 'i', 5};
|
|
const auto result = json::to_ubjson(j, true);
|
|
CHECK(result == expected);
|
|
|
|
// roundtrip
|
|
CHECK(json::from_ubjson(result) == j);
|
|
CHECK(json::from_ubjson(result, true, false) == j);
|
|
}
|
|
|
|
SECTION("size=true type=true")
|
|
{
|
|
json const j = json::parse("[1,2,3,4,5]");
|
|
std::vector<uint8_t> expected = {'[', '$', 'i', '#', 'i', 5, 1, 2, 3, 4, 5};
|
|
const auto result = json::to_ubjson(j, true, true);
|
|
CHECK(result == expected);
|
|
|
|
// roundtrip
|
|
CHECK(json::from_ubjson(result) == j);
|
|
CHECK(json::from_ubjson(result, true, false) == j);
|
|
}
|
|
}
|
|
|
|
SECTION("[[[[]]]]")
|
|
{
|
|
SECTION("size=false type=false")
|
|
{
|
|
json const j = json::parse("[[[[]]]]");
|
|
std::vector<uint8_t> expected = {'[', '[', '[', '[', ']', ']', ']', ']'};
|
|
const auto result = json::to_ubjson(j);
|
|
CHECK(result == expected);
|
|
|
|
// roundtrip
|
|
CHECK(json::from_ubjson(result) == j);
|
|
CHECK(json::from_ubjson(result, true, false) == j);
|
|
}
|
|
|
|
SECTION("size=true type=false")
|
|
{
|
|
json const j = json::parse("[[[[]]]]");
|
|
std::vector<uint8_t> expected = {'[', '#', 'i', 1, '[', '#', 'i', 1, '[', '#', 'i', 1, '[', '#', 'i', 0};
|
|
const auto result = json::to_ubjson(j, true);
|
|
CHECK(result == expected);
|
|
|
|
// roundtrip
|
|
CHECK(json::from_ubjson(result) == j);
|
|
CHECK(json::from_ubjson(result, true, false) == j);
|
|
}
|
|
|
|
SECTION("size=true type=true")
|
|
{
|
|
json const j = json::parse("[[[[]]]]");
|
|
std::vector<uint8_t> expected = {'[', '$', '[', '#', 'i', 1, '$', '[', '#', 'i', 1, '$', '[', '#', 'i', 1, '#', 'i', 0};
|
|
const auto result = json::to_ubjson(j, true, true);
|
|
CHECK(result == expected);
|
|
|
|
// roundtrip
|
|
CHECK(json::from_ubjson(result) == j);
|
|
CHECK(json::from_ubjson(result, true, false) == j);
|
|
}
|
|
}
|
|
|
|
SECTION("array with uint16_t elements")
|
|
{
|
|
SECTION("size=false type=false")
|
|
{
|
|
json j(257, nullptr);
|
|
std::vector<uint8_t> expected(j.size() + 2, 'Z'); // all null
|
|
expected[0] = '['; // opening array
|
|
expected[258] = ']'; // closing array
|
|
const auto result = json::to_ubjson(j);
|
|
CHECK(result == expected);
|
|
|
|
// roundtrip
|
|
CHECK(json::from_ubjson(result) == j);
|
|
CHECK(json::from_ubjson(result, true, false) == j);
|
|
}
|
|
|
|
SECTION("size=true type=false")
|
|
{
|
|
json j(257, nullptr);
|
|
std::vector<uint8_t> expected(j.size() + 5, 'Z'); // all null
|
|
expected[0] = '['; // opening array
|
|
expected[1] = '#'; // array size
|
|
expected[2] = 'I'; // int16
|
|
expected[3] = 0x01; // 0x0101, first byte
|
|
expected[4] = 0x01; // 0x0101, second byte
|
|
const auto result = json::to_ubjson(j, true);
|
|
CHECK(result == expected);
|
|
|
|
// roundtrip
|
|
CHECK(json::from_ubjson(result) == j);
|
|
CHECK(json::from_ubjson(result, true, false) == j);
|
|
}
|
|
|
|
SECTION("size=true type=true")
|
|
{
|
|
json j(257, nullptr);
|
|
std::vector<uint8_t> expected = {'[', '$', 'Z', '#', 'I', 0x01, 0x01};
|
|
const auto result = json::to_ubjson(j, true, true);
|
|
CHECK(result == expected);
|
|
|
|
// roundtrip
|
|
CHECK(json::from_ubjson(result) == j);
|
|
CHECK(json::from_ubjson(result, true, false) == j);
|
|
}
|
|
}
|
|
|
|
SECTION("array with uint32_t elements")
|
|
{
|
|
SECTION("size=false type=false")
|
|
{
|
|
json j(65793, nullptr);
|
|
std::vector<uint8_t> expected(j.size() + 2, 'Z'); // all null
|
|
expected[0] = '['; // opening array
|
|
expected[65794] = ']'; // closing array
|
|
const auto result = json::to_ubjson(j);
|
|
CHECK(result == expected);
|
|
|
|
// roundtrip
|
|
CHECK(json::from_ubjson(result) == j);
|
|
CHECK(json::from_ubjson(result, true, false) == j);
|
|
}
|
|
|
|
SECTION("size=true type=false")
|
|
{
|
|
json j(65793, nullptr);
|
|
std::vector<uint8_t> expected(j.size() + 7, 'Z'); // all null
|
|
expected[0] = '['; // opening array
|
|
expected[1] = '#'; // array size
|
|
expected[2] = 'l'; // int32
|
|
expected[3] = 0x00; // 0x00010101, first byte
|
|
expected[4] = 0x01; // 0x00010101, second byte
|
|
expected[5] = 0x01; // 0x00010101, third byte
|
|
expected[6] = 0x01; // 0x00010101, fourth byte
|
|
const auto result = json::to_ubjson(j, true);
|
|
CHECK(result == expected);
|
|
|
|
// roundtrip
|
|
CHECK(json::from_ubjson(result) == j);
|
|
CHECK(json::from_ubjson(result, true, false) == j);
|
|
}
|
|
|
|
SECTION("size=true type=true")
|
|
{
|
|
json j(65793, nullptr);
|
|
std::vector<uint8_t> expected = {'[', '$', 'Z', '#', 'l', 0x00, 0x01, 0x01, 0x01};
|
|
const auto result = json::to_ubjson(j, true, true);
|
|
CHECK(result == expected);
|
|
|
|
// roundtrip
|
|
CHECK(json::from_ubjson(result) == j);
|
|
CHECK(json::from_ubjson(result, true, false) == j);
|
|
}
|
|
}
|
|
}
|
|
|
|
SECTION("object")
|
|
{
|
|
SECTION("empty")
|
|
{
|
|
SECTION("size=false type=false")
|
|
{
|
|
json const j = json::object();
|
|
std::vector<uint8_t> expected = {'{', '}'};
|
|
const auto result = json::to_ubjson(j);
|
|
CHECK(result == expected);
|
|
|
|
// roundtrip
|
|
CHECK(json::from_ubjson(result) == j);
|
|
CHECK(json::from_ubjson(result, true, false) == j);
|
|
}
|
|
|
|
SECTION("size=true type=false")
|
|
{
|
|
json const j = json::object();
|
|
std::vector<uint8_t> expected = {'{', '#', 'i', 0};
|
|
const auto result = json::to_ubjson(j, true);
|
|
CHECK(result == expected);
|
|
|
|
// roundtrip
|
|
CHECK(json::from_ubjson(result) == j);
|
|
CHECK(json::from_ubjson(result, true, false) == j);
|
|
}
|
|
|
|
SECTION("size=true type=true")
|
|
{
|
|
json const j = json::object();
|
|
std::vector<uint8_t> expected = {'{', '#', 'i', 0};
|
|
const auto result = json::to_ubjson(j, true, true);
|
|
CHECK(result == expected);
|
|
|
|
// roundtrip
|
|
CHECK(json::from_ubjson(result) == j);
|
|
CHECK(json::from_ubjson(result, true, false) == j);
|
|
}
|
|
}
|
|
|
|
SECTION("{\"\":null}")
|
|
{
|
|
SECTION("size=false type=false")
|
|
{
|
|
json const j = {{"", nullptr}};
|
|
std::vector<uint8_t> expected = {'{', 'i', 0, 'Z', '}'};
|
|
const auto result = json::to_ubjson(j);
|
|
CHECK(result == expected);
|
|
|
|
// roundtrip
|
|
CHECK(json::from_ubjson(result) == j);
|
|
CHECK(json::from_ubjson(result, true, false) == j);
|
|
}
|
|
|
|
SECTION("size=true type=false")
|
|
{
|
|
json const j = {{"", nullptr}};
|
|
std::vector<uint8_t> expected = {'{', '#', 'i', 1, 'i', 0, 'Z'};
|
|
const auto result = json::to_ubjson(j, true);
|
|
CHECK(result == expected);
|
|
|
|
// roundtrip
|
|
CHECK(json::from_ubjson(result) == j);
|
|
CHECK(json::from_ubjson(result, true, false) == j);
|
|
}
|
|
|
|
SECTION("size=true type=true")
|
|
{
|
|
json const j = {{"", nullptr}};
|
|
std::vector<uint8_t> expected = {'{', '$', 'Z', '#', 'i', 1, 'i', 0};
|
|
const auto result = json::to_ubjson(j, true, true);
|
|
CHECK(result == expected);
|
|
|
|
// roundtrip
|
|
CHECK(json::from_ubjson(result) == j);
|
|
CHECK(json::from_ubjson(result, true, false) == j);
|
|
}
|
|
}
|
|
|
|
SECTION("{\"a\": {\"b\": {\"c\": {}}}}")
|
|
{
|
|
SECTION("size=false type=false")
|
|
{
|
|
json const j = json::parse(R"({"a": {"b": {"c": {}}}})");
|
|
std::vector<uint8_t> expected =
|
|
{
|
|
'{', 'i', 1, 'a', '{', 'i', 1, 'b', '{', 'i', 1, 'c', '{', '}', '}', '}', '}'
|
|
};
|
|
const auto result = json::to_ubjson(j);
|
|
CHECK(result == expected);
|
|
|
|
// roundtrip
|
|
CHECK(json::from_ubjson(result) == j);
|
|
CHECK(json::from_ubjson(result, true, false) == j);
|
|
}
|
|
|
|
SECTION("size=true type=false")
|
|
{
|
|
json const j = json::parse(R"({"a": {"b": {"c": {}}}})");
|
|
std::vector<uint8_t> expected =
|
|
{
|
|
'{', '#', 'i', 1, 'i', 1, 'a', '{', '#', 'i', 1, 'i', 1, 'b', '{', '#', 'i', 1, 'i', 1, 'c', '{', '#', 'i', 0
|
|
};
|
|
const auto result = json::to_ubjson(j, true);
|
|
CHECK(result == expected);
|
|
|
|
// roundtrip
|
|
CHECK(json::from_ubjson(result) == j);
|
|
CHECK(json::from_ubjson(result, true, false) == j);
|
|
}
|
|
|
|
SECTION("size=true type=true")
|
|
{
|
|
json const j = json::parse(R"({"a": {"b": {"c": {}}}})");
|
|
std::vector<uint8_t> expected =
|
|
{
|
|
'{', '$', '{', '#', 'i', 1, 'i', 1, 'a', '$', '{', '#', 'i', 1, 'i', 1, 'b', '$', '{', '#', 'i', 1, 'i', 1, 'c', '#', 'i', 0
|
|
};
|
|
const auto result = json::to_ubjson(j, true, true);
|
|
CHECK(result == expected);
|
|
|
|
// roundtrip
|
|
CHECK(json::from_ubjson(result) == j);
|
|
CHECK(json::from_ubjson(result, true, false) == j);
|
|
}
|
|
}
|
|
}
|
|
}
|
|
|
|
SECTION("errors")
|
|
{
|
|
SECTION("strict mode")
|
|
{
|
|
std::vector<uint8_t> const vec = {'Z', 'Z'};
|
|
SECTION("non-strict mode")
|
|
{
|
|
const auto result = json::from_ubjson(vec, false);
|
|
CHECK(result == json());
|
|
}
|
|
|
|
SECTION("strict mode")
|
|
{
|
|
json _;
|
|
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(vec), "[json.exception.parse_error.110] parse error at byte 2: syntax error while parsing UBJSON value: expected end of input; last byte: 0x5A", json::parse_error&);
|
|
}
|
|
}
|
|
|
|
SECTION("excessive size")
|
|
{
|
|
SECTION("array")
|
|
{
|
|
std::vector<uint8_t> const v_ubjson = {'[', '$', 'Z', '#', 'L', 0x78, 0x28, 0x00, 0x68, 0x28, 0x69, 0x69, 0x17};
|
|
json _;
|
|
CHECK_THROWS_AS(_ = json::from_ubjson(v_ubjson), json::out_of_range&);
|
|
|
|
json j;
|
|
nlohmann::detail::json_sax_dom_callback_parser<json, decltype(nlohmann::detail::input_adapter(v_ubjson))> scp(j, [](int /*unused*/, json::parse_event_t /*unused*/, const json& /*unused*/) noexcept
|
|
{
|
|
return true;
|
|
});
|
|
CHECK_THROWS_AS(_ = json::sax_parse(v_ubjson, &scp, json::input_format_t::ubjson), json::out_of_range&);
|
|
}
|
|
|
|
SECTION("object")
|
|
{
|
|
std::vector<uint8_t> const v_ubjson = {'{', '$', 'Z', '#', 'L', 0x78, 0x28, 0x00, 0x68, 0x28, 0x69, 0x69, 0x17};
|
|
json _;
|
|
CHECK_THROWS_AS(_ = json::from_ubjson(v_ubjson), json::out_of_range&);
|
|
|
|
json j;
|
|
nlohmann::detail::json_sax_dom_callback_parser<json, decltype(nlohmann::detail::input_adapter(v_ubjson))> scp(j, [](int /*unused*/, json::parse_event_t /*unused*/, const json& /*unused*/) noexcept
|
|
{
|
|
return true;
|
|
});
|
|
CHECK_THROWS_AS(_ = json::sax_parse(v_ubjson, &scp, json::input_format_t::ubjson), json::out_of_range&);
|
|
}
|
|
|
|
SECTION("array with a known size, read with a callback")
|
|
{
|
|
// a sized array announces its length to start_array()
|
|
std::vector<uint8_t> const v_ubjson = {'[', '#', 'i', 2, 'i', 1, 'i', 2};
|
|
json j;
|
|
nlohmann::detail::json_sax_dom_callback_parser<json, decltype(nlohmann::detail::input_adapter(v_ubjson))> scp(j, [](int /*unused*/, json::parse_event_t /*unused*/, const json& /*unused*/) noexcept
|
|
{
|
|
return true;
|
|
});
|
|
CHECK(json::sax_parse(v_ubjson, &scp, json::input_format_t::ubjson));
|
|
CHECK(j == json({1, 2}));
|
|
|
|
// the readers reject a size this large before they announce
|
|
// it, so it can only reach start_array() directly (the largest
|
|
// value stands for an unknown size and is never checked)
|
|
json k;
|
|
nlohmann::detail::json_sax_dom_callback_parser<json, decltype(nlohmann::detail::input_adapter(v_ubjson))> scp2(k, [](int /*unused*/, json::parse_event_t /*unused*/, const json& /*unused*/) noexcept
|
|
{
|
|
return true;
|
|
});
|
|
CHECK_THROWS_AS(scp2.start_array((std::numeric_limits<std::size_t>::max)() - 1), json::out_of_range&);
|
|
}
|
|
}
|
|
}
|
|
|
|
SECTION("SAX aborts")
|
|
{
|
|
SECTION("start_array()")
|
|
{
|
|
std::vector<uint8_t> const v = {'[', 'T', 'F', ']'};
|
|
SaxCountdown scp(0);
|
|
CHECK(!json::sax_parse(v, &scp, json::input_format_t::ubjson));
|
|
}
|
|
|
|
SECTION("start_object()")
|
|
{
|
|
std::vector<uint8_t> const v = {'{', 'i', 3, 'f', 'o', 'o', 'F', '}'};
|
|
SaxCountdown scp(0);
|
|
CHECK(!json::sax_parse(v, &scp, json::input_format_t::ubjson));
|
|
}
|
|
|
|
SECTION("key() in object")
|
|
{
|
|
std::vector<uint8_t> const v = {'{', 'i', 3, 'f', 'o', 'o', 'F', '}'};
|
|
SaxCountdown scp(1);
|
|
CHECK(!json::sax_parse(v, &scp, json::input_format_t::ubjson));
|
|
}
|
|
|
|
SECTION("start_array(len)")
|
|
{
|
|
std::vector<uint8_t> const v = {'[', '#', 'i', '2', 'T', 'F'};
|
|
SaxCountdown scp(0);
|
|
CHECK(!json::sax_parse(v, &scp, json::input_format_t::ubjson));
|
|
}
|
|
|
|
SECTION("start_object(len)")
|
|
{
|
|
std::vector<uint8_t> const v = {'{', '#', 'i', '1', 3, 'f', 'o', 'o', 'F'};
|
|
SaxCountdown scp(0);
|
|
CHECK(!json::sax_parse(v, &scp, json::input_format_t::ubjson));
|
|
}
|
|
|
|
SECTION("key() in object with length")
|
|
{
|
|
std::vector<uint8_t> const v = {'{', 'i', 3, 'f', 'o', 'o', 'F', '}'};
|
|
SaxCountdown scp(1);
|
|
CHECK(!json::sax_parse(v, &scp, json::input_format_t::ubjson));
|
|
}
|
|
}
|
|
|
|
SECTION("parsing values")
|
|
{
|
|
SECTION("strings")
|
|
{
|
|
// create a single-character string for all number types
|
|
std::vector<uint8_t> s_i = {'S', 'i', 1, 'a'};
|
|
std::vector<uint8_t> const s_U = {'S', 'U', 1, 'a'};
|
|
std::vector<uint8_t> const s_I = {'S', 'I', 0, 1, 'a'};
|
|
std::vector<uint8_t> const s_l = {'S', 'l', 0, 0, 0, 1, 'a'};
|
|
std::vector<uint8_t> const s_L = {'S', 'L', 0, 0, 0, 0, 0, 0, 0, 1, 'a'};
|
|
|
|
// check if string is parsed correctly to "a"
|
|
CHECK(json::from_ubjson(s_i) == "a");
|
|
CHECK(json::from_ubjson(s_U) == "a");
|
|
CHECK(json::from_ubjson(s_I) == "a");
|
|
CHECK(json::from_ubjson(s_l) == "a");
|
|
CHECK(json::from_ubjson(s_L) == "a");
|
|
|
|
// roundtrip: output should be optimized
|
|
CHECK(json::to_ubjson(json::from_ubjson(s_i)) == s_i);
|
|
CHECK(json::to_ubjson(json::from_ubjson(s_U)) == s_i);
|
|
CHECK(json::to_ubjson(json::from_ubjson(s_I)) == s_i);
|
|
CHECK(json::to_ubjson(json::from_ubjson(s_l)) == s_i);
|
|
CHECK(json::to_ubjson(json::from_ubjson(s_L)) == s_i);
|
|
}
|
|
|
|
SECTION("no-op markers")
|
|
{
|
|
// A no-op ('N') is valid wherever a value may start; it is consumed
|
|
// by get_ignore_noop() before the value is read. It is not valid
|
|
// where a string length type specification is expected.
|
|
|
|
SECTION("accepted where a value may start")
|
|
{
|
|
// at top level, also repeated
|
|
CHECK(json::from_ubjson(std::vector<uint8_t>({'N', 'i', 1})) == json(1));
|
|
CHECK(json::from_ubjson(std::vector<uint8_t>({'N', 'N', 'N', 'i', 1})) == json(1));
|
|
|
|
// inside an array of unknown size, before and after an element
|
|
CHECK(json::from_ubjson(std::vector<uint8_t>({'[', 'N', 'i', 1, ']'})) == json({1}));
|
|
CHECK(json::from_ubjson(std::vector<uint8_t>({'[', 'i', 1, 'N', ']'})) == json({1}));
|
|
|
|
// inside an object of unknown size: before a key, between key
|
|
// and value, and before the closing '}'
|
|
CHECK(json::from_ubjson(std::vector<uint8_t>({'{', 'N', 'U', 1, 'a', 'i', 1, '}'})) == json({{"a", 1}}));
|
|
CHECK(json::from_ubjson(std::vector<uint8_t>({'{', 'U', 1, 'a', 'N', 'i', 1, '}'})) == json({{"a", 1}}));
|
|
CHECK(json::from_ubjson(std::vector<uint8_t>({'{', 'U', 1, 'a', 'i', 1, 'N', '}'})) == json({{"a", 1}}));
|
|
}
|
|
|
|
SECTION("rejected where a length type specification is expected")
|
|
{
|
|
json _;
|
|
|
|
// after the 'S' marker of a string value
|
|
std::vector<uint8_t> const v_S = {'S', 'N', 'U', 1, 'a'};
|
|
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(v_S), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing UBJSON string: expected length type specification (U, i, I, l, L); last byte: 0x4E", json::parse_error&);
|
|
|
|
// as the key length of an object with a known size, where
|
|
// no-ops are not permitted in the first place
|
|
std::vector<uint8_t> const v_key = {'{', '#', 'i', 1, 'N', 'U', 1, 'a', 'i', 1};
|
|
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(v_key), "[json.exception.parse_error.113] parse error at byte 5: syntax error while parsing UBJSON string: expected length type specification (U, i, I, l, L); last byte: 0x4E", json::parse_error&);
|
|
}
|
|
}
|
|
|
|
SECTION("number")
|
|
{
|
|
SECTION("float")
|
|
{
|
|
// float32
|
|
std::vector<uint8_t> const v_d = {'d', 0x40, 0x49, 0x0f, 0xd0};
|
|
CHECK(json::from_ubjson(v_d) == 3.14159f);
|
|
|
|
// float64
|
|
std::vector<uint8_t> const v_D = {'D', 0x40, 0x09, 0x21, 0xf9, 0xf0, 0x1b, 0x86, 0x6e};
|
|
CHECK(json::from_ubjson(v_D) == 3.14159);
|
|
|
|
// float32 is serialized as float64 as the library does not support float32
|
|
CHECK(json::to_ubjson(json::from_ubjson(v_d)) == json::to_ubjson(3.14159f));
|
|
}
|
|
}
|
|
|
|
SECTION("array")
|
|
{
|
|
SECTION("optimized version (length only)")
|
|
{
|
|
// create vector with two elements of the same type
|
|
std::vector<uint8_t> const v_TU = {'[', '#', 'U', 2, 'T', 'T'};
|
|
std::vector<uint8_t> const v_T = {'[', '#', 'i', 2, 'T', 'T'};
|
|
std::vector<uint8_t> const v_F = {'[', '#', 'i', 2, 'F', 'F'};
|
|
std::vector<uint8_t> const v_Z = {'[', '#', 'i', 2, 'Z', 'Z'};
|
|
std::vector<uint8_t> const v_i = {'[', '#', 'i', 2, 'i', 0x7F, 'i', 0x7F};
|
|
std::vector<uint8_t> const v_U = {'[', '#', 'i', 2, 'U', 0xFF, 'U', 0xFF};
|
|
std::vector<uint8_t> const v_I = {'[', '#', 'i', 2, 'I', 0x7F, 0xFF, 'I', 0x7F, 0xFF};
|
|
std::vector<uint8_t> const v_l = {'[', '#', 'i', 2, 'l', 0x7F, 0xFF, 0xFF, 0xFF, 'l', 0x7F, 0xFF, 0xFF, 0xFF};
|
|
std::vector<uint8_t> const v_L = {'[', '#', 'i', 2, 'L', 0x7F, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 'L', 0x7F, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF};
|
|
std::vector<uint8_t> const v_D = {'[', '#', 'i', 2, 'D', 0x40, 0x09, 0x21, 0xfb, 0x4d, 0x12, 0xd8, 0x4a, 'D', 0x40, 0x09, 0x21, 0xfb, 0x4d, 0x12, 0xd8, 0x4a};
|
|
std::vector<uint8_t> const v_S = {'[', '#', 'i', 2, 'S', 'i', 1, 'a', 'S', 'i', 1, 'a'};
|
|
std::vector<uint8_t> const v_C = {'[', '#', 'i', 2, 'C', 'a', 'C', 'a'};
|
|
|
|
// check if vector is parsed correctly
|
|
CHECK(json::from_ubjson(v_TU) == json({true, true}));
|
|
CHECK(json::from_ubjson(v_T) == json({true, true}));
|
|
CHECK(json::from_ubjson(v_F) == json({false, false}));
|
|
CHECK(json::from_ubjson(v_Z) == json({nullptr, nullptr}));
|
|
CHECK(json::from_ubjson(v_i) == json({127, 127}));
|
|
CHECK(json::from_ubjson(v_U) == json({255, 255}));
|
|
CHECK(json::from_ubjson(v_I) == json({32767, 32767}));
|
|
CHECK(json::from_ubjson(v_l) == json({2147483647, 2147483647}));
|
|
CHECK(json::from_ubjson(v_L) == json({9223372036854775807, 9223372036854775807}));
|
|
CHECK(json::from_ubjson(v_D) == json({3.1415926, 3.1415926}));
|
|
CHECK(json::from_ubjson(v_S) == json({"a", "a"}));
|
|
CHECK(json::from_ubjson(v_C) == json({"a", "a"}));
|
|
|
|
// roundtrip: output should be optimized
|
|
CHECK(json::to_ubjson(json::from_ubjson(v_T), true) == v_T);
|
|
CHECK(json::to_ubjson(json::from_ubjson(v_F), true) == v_F);
|
|
CHECK(json::to_ubjson(json::from_ubjson(v_Z), true) == v_Z);
|
|
CHECK(json::to_ubjson(json::from_ubjson(v_i), true) == v_i);
|
|
CHECK(json::to_ubjson(json::from_ubjson(v_U), true) == v_U);
|
|
CHECK(json::to_ubjson(json::from_ubjson(v_I), true) == v_I);
|
|
CHECK(json::to_ubjson(json::from_ubjson(v_l), true) == v_l);
|
|
CHECK(json::to_ubjson(json::from_ubjson(v_L), true) == v_L);
|
|
CHECK(json::to_ubjson(json::from_ubjson(v_D), true) == v_D);
|
|
CHECK(json::to_ubjson(json::from_ubjson(v_S), true) == v_S);
|
|
CHECK(json::to_ubjson(json::from_ubjson(v_C), true) == v_S); // char is serialized to string
|
|
}
|
|
|
|
SECTION("optimized version (type and length)")
|
|
{
|
|
// create vector with two elements of the same type
|
|
std::vector<uint8_t> const v_N = {'[', '$', 'N', '#', 'i', 2};
|
|
std::vector<uint8_t> const v_T = {'[', '$', 'T', '#', 'i', 2};
|
|
std::vector<uint8_t> const v_F = {'[', '$', 'F', '#', 'i', 2};
|
|
std::vector<uint8_t> const v_Z = {'[', '$', 'Z', '#', 'i', 2};
|
|
std::vector<uint8_t> const v_i = {'[', '$', 'i', '#', 'i', 2, 0x7F, 0x7F};
|
|
std::vector<uint8_t> const v_U = {'[', '$', 'U', '#', 'i', 2, 0xFF, 0xFF};
|
|
std::vector<uint8_t> const v_I = {'[', '$', 'I', '#', 'i', 2, 0x7F, 0xFF, 0x7F, 0xFF};
|
|
std::vector<uint8_t> const v_l = {'[', '$', 'l', '#', 'i', 2, 0x7F, 0xFF, 0xFF, 0xFF, 0x7F, 0xFF, 0xFF, 0xFF};
|
|
std::vector<uint8_t> const v_L = {'[', '$', 'L', '#', 'i', 2, 0x7F, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0x7F, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF};
|
|
std::vector<uint8_t> const v_D = {'[', '$', 'D', '#', 'i', 2, 0x40, 0x09, 0x21, 0xfb, 0x4d, 0x12, 0xd8, 0x4a, 0x40, 0x09, 0x21, 0xfb, 0x4d, 0x12, 0xd8, 0x4a};
|
|
std::vector<uint8_t> const v_S = {'[', '$', 'S', '#', 'i', 2, 'i', 1, 'a', 'i', 1, 'a'};
|
|
std::vector<uint8_t> const v_C = {'[', '$', 'C', '#', 'i', 2, 'a', 'a'};
|
|
|
|
// check if vector is parsed correctly
|
|
CHECK(json::from_ubjson(v_N) == json::array());
|
|
CHECK(json::from_ubjson(v_T) == json({true, true}));
|
|
CHECK(json::from_ubjson(v_F) == json({false, false}));
|
|
CHECK(json::from_ubjson(v_Z) == json({nullptr, nullptr}));
|
|
CHECK(json::from_ubjson(v_i) == json({127, 127}));
|
|
CHECK(json::from_ubjson(v_U) == json({255, 255}));
|
|
CHECK(json::from_ubjson(v_I) == json({32767, 32767}));
|
|
CHECK(json::from_ubjson(v_l) == json({2147483647, 2147483647}));
|
|
CHECK(json::from_ubjson(v_L) == json({9223372036854775807, 9223372036854775807}));
|
|
CHECK(json::from_ubjson(v_D) == json({3.1415926, 3.1415926}));
|
|
CHECK(json::from_ubjson(v_S) == json({"a", "a"}));
|
|
CHECK(json::from_ubjson(v_C) == json({"a", "a"}));
|
|
|
|
// roundtrip: output should be optimized
|
|
std::vector<uint8_t> const v_empty = {'[', '#', 'i', 0};
|
|
CHECK(json::to_ubjson(json::from_ubjson(v_N), true, true) == v_empty);
|
|
CHECK(json::to_ubjson(json::from_ubjson(v_T), true, true) == v_T);
|
|
CHECK(json::to_ubjson(json::from_ubjson(v_F), true, true) == v_F);
|
|
CHECK(json::to_ubjson(json::from_ubjson(v_Z), true, true) == v_Z);
|
|
CHECK(json::to_ubjson(json::from_ubjson(v_i), true, true) == v_i);
|
|
CHECK(json::to_ubjson(json::from_ubjson(v_U), true, true) == v_U);
|
|
CHECK(json::to_ubjson(json::from_ubjson(v_I), true, true) == v_I);
|
|
CHECK(json::to_ubjson(json::from_ubjson(v_l), true, true) == v_l);
|
|
CHECK(json::to_ubjson(json::from_ubjson(v_L), true, true) == v_L);
|
|
CHECK(json::to_ubjson(json::from_ubjson(v_D), true, true) == v_D);
|
|
CHECK(json::to_ubjson(json::from_ubjson(v_S), true, true) == v_S);
|
|
CHECK(json::to_ubjson(json::from_ubjson(v_C), true, true) == v_S); // char is serialized to string
|
|
}
|
|
}
|
|
}
|
|
|
|
SECTION("parse errors")
|
|
{
|
|
SECTION("empty byte vector")
|
|
{
|
|
json _;
|
|
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(std::vector<uint8_t>()), "[json.exception.parse_error.110] parse error at byte 1: syntax error while parsing UBJSON value: unexpected end of input", json::parse_error&);
|
|
}
|
|
|
|
SECTION("char")
|
|
{
|
|
SECTION("eof after C byte")
|
|
{
|
|
std::vector<uint8_t> const v = {'C'};
|
|
json _;
|
|
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(v), "[json.exception.parse_error.110] parse error at byte 2: syntax error while parsing UBJSON char: unexpected end of input", json::parse_error&);
|
|
}
|
|
|
|
SECTION("byte out of range")
|
|
{
|
|
std::vector<uint8_t> const v = {'C', 130};
|
|
json _;
|
|
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(v), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing UBJSON char: byte after 'C' must be in range 0x00..0x7F; last byte: 0x82", json::parse_error&);
|
|
}
|
|
}
|
|
|
|
SECTION("strings")
|
|
{
|
|
SECTION("eof after S byte")
|
|
{
|
|
std::vector<uint8_t> const v = {'S'};
|
|
json _;
|
|
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(v), "[json.exception.parse_error.110] parse error at byte 2: syntax error while parsing UBJSON value: unexpected end of input", json::parse_error&);
|
|
}
|
|
|
|
SECTION("invalid byte")
|
|
{
|
|
std::vector<uint8_t> const v = {'S', '1', 'a'};
|
|
json _;
|
|
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(v), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing UBJSON string: expected length type specification (U, i, I, l, L); last byte: 0x31", json::parse_error&);
|
|
}
|
|
|
|
SECTION("negative length")
|
|
{
|
|
json _;
|
|
|
|
std::vector<uint8_t> const vi = {'S', 'i', 0xFF};
|
|
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(vi), "[json.exception.parse_error.113] parse error at byte 3: syntax error while parsing UBJSON string: string length must not be negative", json::parse_error&);
|
|
CHECK(json::from_ubjson(vi, true, false).is_discarded());
|
|
|
|
std::vector<uint8_t> const vI = {'S', 'I', 0xFF, 0xFF};
|
|
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(vI), "[json.exception.parse_error.113] parse error at byte 4: syntax error while parsing UBJSON string: string length must not be negative", json::parse_error&);
|
|
CHECK(json::from_ubjson(vI, true, false).is_discarded());
|
|
|
|
std::vector<uint8_t> const vl = {'S', 'l', 0xFF, 0xFF, 0xFF, 0xFF};
|
|
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(vl), "[json.exception.parse_error.113] parse error at byte 6: syntax error while parsing UBJSON string: string length must not be negative", json::parse_error&);
|
|
CHECK(json::from_ubjson(vl, true, false).is_discarded());
|
|
|
|
std::vector<uint8_t> const vL = {'S', 'L', 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF};
|
|
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(vL), "[json.exception.parse_error.113] parse error at byte 10: syntax error while parsing UBJSON string: string length must not be negative", json::parse_error&);
|
|
CHECK(json::from_ubjson(vL, true, false).is_discarded());
|
|
|
|
// a length of zero remains valid and yields an empty string
|
|
std::vector<uint8_t> const v0 = {'S', 'i', 0};
|
|
CHECK(json::from_ubjson(v0) == json(""));
|
|
}
|
|
}
|
|
|
|
SECTION("array")
|
|
{
|
|
SECTION("optimized array: no size following type")
|
|
{
|
|
std::vector<uint8_t> const v = {'[', '$', 'i', 2};
|
|
json _;
|
|
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(v), "[json.exception.parse_error.112] parse error at byte 4: syntax error while parsing UBJSON size: expected '#' after type information; last byte: 0x02", json::parse_error&);
|
|
}
|
|
}
|
|
|
|
SECTION("strings")
|
|
{
|
|
std::vector<uint8_t> const vS = {'S'};
|
|
json _;
|
|
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(vS), "[json.exception.parse_error.110] parse error at byte 2: syntax error while parsing UBJSON value: unexpected end of input", json::parse_error&);
|
|
CHECK(json::from_ubjson(vS, true, false).is_discarded());
|
|
|
|
std::vector<uint8_t> const v = {'S', 'i', '2', 'a'};
|
|
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(v), "[json.exception.parse_error.110] parse error at byte 5: syntax error while parsing UBJSON string: unexpected end of input", json::parse_error&);
|
|
CHECK(json::from_ubjson(v, true, false).is_discarded());
|
|
|
|
std::vector<uint8_t> const vC = {'C'};
|
|
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(vC), "[json.exception.parse_error.110] parse error at byte 2: syntax error while parsing UBJSON char: unexpected end of input", json::parse_error&);
|
|
CHECK(json::from_ubjson(vC, true, false).is_discarded());
|
|
}
|
|
|
|
SECTION("sizes")
|
|
{
|
|
std::vector<uint8_t> const vU = {'[', '#', 'U'};
|
|
json _;
|
|
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(vU), "[json.exception.parse_error.110] parse error at byte 4: syntax error while parsing UBJSON number: unexpected end of input", json::parse_error&);
|
|
CHECK(json::from_ubjson(vU, true, false).is_discarded());
|
|
|
|
std::vector<uint8_t> const vi = {'[', '#', 'i'};
|
|
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(vi), "[json.exception.parse_error.110] parse error at byte 4: syntax error while parsing UBJSON number: unexpected end of input", json::parse_error&);
|
|
CHECK(json::from_ubjson(vi, true, false).is_discarded());
|
|
|
|
std::vector<uint8_t> const vI = {'[', '#', 'I'};
|
|
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(vI), "[json.exception.parse_error.110] parse error at byte 4: syntax error while parsing UBJSON number: unexpected end of input", json::parse_error&);
|
|
CHECK(json::from_ubjson(vI, true, false).is_discarded());
|
|
|
|
std::vector<uint8_t> const vl = {'[', '#', 'l'};
|
|
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(vl), "[json.exception.parse_error.110] parse error at byte 4: syntax error while parsing UBJSON number: unexpected end of input", json::parse_error&);
|
|
CHECK(json::from_ubjson(vl, true, false).is_discarded());
|
|
|
|
std::vector<uint8_t> const vL = {'[', '#', 'L'};
|
|
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(vL), "[json.exception.parse_error.110] parse error at byte 4: syntax error while parsing UBJSON number: unexpected end of input", json::parse_error&);
|
|
CHECK(json::from_ubjson(vL, true, false).is_discarded());
|
|
|
|
std::vector<uint8_t> const v0 = {'[', '#', 'T', ']'};
|
|
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(v0), "[json.exception.parse_error.113] parse error at byte 3: syntax error while parsing UBJSON size: expected length type specification (U, i, I, l, L) after '#'; last byte: 0x54", json::parse_error&);
|
|
CHECK(json::from_ubjson(v0, true, false).is_discarded());
|
|
}
|
|
|
|
SECTION("types")
|
|
{
|
|
std::vector<uint8_t> const v0 = {'[', '$'};
|
|
json _;
|
|
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(v0), "[json.exception.parse_error.110] parse error at byte 3: syntax error while parsing UBJSON type: unexpected end of input", json::parse_error&);
|
|
CHECK(json::from_ubjson(v0, true, false).is_discarded());
|
|
|
|
std::vector<uint8_t> const vi = {'[', '$', '#'};
|
|
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(vi), "[json.exception.parse_error.110] parse error at byte 4: syntax error while parsing UBJSON value: unexpected end of input", json::parse_error&);
|
|
CHECK(json::from_ubjson(vi, true, false).is_discarded());
|
|
|
|
std::vector<uint8_t> const vT = {'[', '$', 'T'};
|
|
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(vT), "[json.exception.parse_error.110] parse error at byte 4: syntax error while parsing UBJSON value: unexpected end of input", json::parse_error&);
|
|
CHECK(json::from_ubjson(vT, true, false).is_discarded());
|
|
}
|
|
|
|
SECTION("arrays")
|
|
{
|
|
std::vector<uint8_t> const vST = {'[', '$', 'i', '#', 'i', 2, 1};
|
|
json _;
|
|
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(vST), "[json.exception.parse_error.110] parse error at byte 8: syntax error while parsing UBJSON number: unexpected end of input", json::parse_error&);
|
|
CHECK(json::from_ubjson(vST, true, false).is_discarded());
|
|
|
|
std::vector<uint8_t> const vS = {'[', '#', 'i', 2, 'i', 1};
|
|
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(vS), "[json.exception.parse_error.110] parse error at byte 7: syntax error while parsing UBJSON value: unexpected end of input", json::parse_error&);
|
|
CHECK(json::from_ubjson(vS, true, false).is_discarded());
|
|
|
|
std::vector<uint8_t> const v = {'[', 'i', 2, 'i', 1};
|
|
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(v), "[json.exception.parse_error.110] parse error at byte 6: syntax error while parsing UBJSON value: unexpected end of input", json::parse_error&);
|
|
CHECK(json::from_ubjson(v, true, false).is_discarded());
|
|
}
|
|
|
|
SECTION("objects")
|
|
{
|
|
std::vector<uint8_t> const vST = {'{', '$', 'i', '#', 'i', 2, 'i', 1, 'a', 1};
|
|
json _;
|
|
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(vST), "[json.exception.parse_error.110] parse error at byte 11: syntax error while parsing UBJSON value: unexpected end of input", json::parse_error&);
|
|
CHECK(json::from_ubjson(vST, true, false).is_discarded());
|
|
|
|
std::vector<uint8_t> const vT = {'{', '$', 'i', 'i', 1, 'a', 1};
|
|
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(vT), "[json.exception.parse_error.112] parse error at byte 4: syntax error while parsing UBJSON size: expected '#' after type information; last byte: 0x69", json::parse_error&);
|
|
CHECK(json::from_ubjson(vT, true, false).is_discarded());
|
|
|
|
std::vector<uint8_t> const vS = {'{', '#', 'i', 2, 'i', 1, 'a', 'i', 1};
|
|
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(vS), "[json.exception.parse_error.110] parse error at byte 10: syntax error while parsing UBJSON value: unexpected end of input", json::parse_error&);
|
|
CHECK(json::from_ubjson(vS, true, false).is_discarded());
|
|
|
|
std::vector<uint8_t> const v = {'{', 'i', 1, 'a', 'i', 1};
|
|
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(v), "[json.exception.parse_error.110] parse error at byte 7: syntax error while parsing UBJSON value: unexpected end of input", json::parse_error&);
|
|
CHECK(json::from_ubjson(v, true, false).is_discarded());
|
|
|
|
std::vector<uint8_t> const v2 = {'{', 'i', 1, 'a', 'i', 1, 'i'};
|
|
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(v2), "[json.exception.parse_error.110] parse error at byte 8: syntax error while parsing UBJSON number: unexpected end of input", json::parse_error&);
|
|
CHECK(json::from_ubjson(v2, true, false).is_discarded());
|
|
|
|
std::vector<uint8_t> const v3 = {'{', 'i', 1, 'a'};
|
|
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(v3), "[json.exception.parse_error.110] parse error at byte 5: syntax error while parsing UBJSON value: unexpected end of input", json::parse_error&);
|
|
CHECK(json::from_ubjson(v3, true, false).is_discarded());
|
|
|
|
std::vector<uint8_t> const vST1 = {'{', '$', 'd', '#', 'i', 2, 'i', 1, 'a'};
|
|
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(vST1), "[json.exception.parse_error.110] parse error at byte 10: syntax error while parsing UBJSON number: unexpected end of input", json::parse_error&);
|
|
CHECK(json::from_ubjson(vST1, true, false).is_discarded());
|
|
|
|
std::vector<uint8_t> const vST2 = {'{', '#', 'i', 2, 'i', 1, 'a'};
|
|
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(vST2), "[json.exception.parse_error.110] parse error at byte 8: syntax error while parsing UBJSON value: unexpected end of input", json::parse_error&);
|
|
CHECK(json::from_ubjson(vST2, true, false).is_discarded());
|
|
}
|
|
}
|
|
|
|
SECTION("writing optimized values")
|
|
{
|
|
SECTION("integer")
|
|
{
|
|
SECTION("array of i")
|
|
{
|
|
json const j = {1, -1};
|
|
std::vector<uint8_t> expected = {'[', '$', 'i', '#', 'i', 2, 1, 0xff};
|
|
CHECK(json::to_ubjson(j, true, true) == expected);
|
|
}
|
|
|
|
SECTION("array of U")
|
|
{
|
|
json const j = {200, 201};
|
|
std::vector<uint8_t> expected = {'[', '$', 'U', '#', 'i', 2, 0xC8, 0xC9};
|
|
CHECK(json::to_ubjson(j, true, true) == expected);
|
|
}
|
|
|
|
SECTION("array of I")
|
|
{
|
|
json const j = {30000, -30000};
|
|
std::vector<uint8_t> expected = {'[', '$', 'I', '#', 'i', 2, 0x75, 0x30, 0x8a, 0xd0};
|
|
CHECK(json::to_ubjson(j, true, true) == expected);
|
|
}
|
|
|
|
SECTION("array of l")
|
|
{
|
|
json const j = {70000, -70000};
|
|
std::vector<uint8_t> expected = {'[', '$', 'l', '#', 'i', 2, 0x00, 0x01, 0x11, 0x70, 0xFF, 0xFE, 0xEE, 0x90};
|
|
CHECK(json::to_ubjson(j, true, true) == expected);
|
|
}
|
|
|
|
SECTION("array of L")
|
|
{
|
|
json const j = {5000000000, -5000000000};
|
|
std::vector<uint8_t> expected = {'[', '$', 'L', '#', 'i', 2, 0x00, 0x00, 0x00, 0x01, 0x2A, 0x05, 0xF2, 0x00, 0xFF, 0xFF, 0xFF, 0xFE, 0xD5, 0xFA, 0x0E, 0x00};
|
|
CHECK(json::to_ubjson(j, true, true) == expected);
|
|
}
|
|
}
|
|
|
|
SECTION("unsigned integer")
|
|
{
|
|
SECTION("array of i")
|
|
{
|
|
json const j = {1u, 2u};
|
|
std::vector<uint8_t> expected = {'[', '$', 'i', '#', 'i', 2, 1, 2};
|
|
std::vector<uint8_t> expected_size = {'[', '#', 'i', 2, 'i', 1, 'i', 2};
|
|
CHECK(json::to_ubjson(j, true, true) == expected);
|
|
CHECK(json::to_ubjson(j, true) == expected_size);
|
|
}
|
|
|
|
SECTION("array of U")
|
|
{
|
|
json const j = {200u, 201u};
|
|
std::vector<uint8_t> expected = {'[', '$', 'U', '#', 'i', 2, 0xC8, 0xC9};
|
|
std::vector<uint8_t> expected_size = {'[', '#', 'i', 2, 'U', 0xC8, 'U', 0xC9};
|
|
CHECK(json::to_ubjson(j, true, true) == expected);
|
|
CHECK(json::to_ubjson(j, true) == expected_size);
|
|
}
|
|
|
|
SECTION("array of I")
|
|
{
|
|
json const j = {30000u, 30001u};
|
|
std::vector<uint8_t> expected = {'[', '$', 'I', '#', 'i', 2, 0x75, 0x30, 0x75, 0x31};
|
|
std::vector<uint8_t> expected_size = {'[', '#', 'i', 2, 'I', 0x75, 0x30, 'I', 0x75, 0x31};
|
|
CHECK(json::to_ubjson(j, true, true) == expected);
|
|
CHECK(json::to_ubjson(j, true) == expected_size);
|
|
}
|
|
|
|
SECTION("array of l")
|
|
{
|
|
json const j = {70000u, 70001u};
|
|
std::vector<uint8_t> expected = {'[', '$', 'l', '#', 'i', 2, 0x00, 0x01, 0x11, 0x70, 0x00, 0x01, 0x11, 0x71};
|
|
std::vector<uint8_t> expected_size = {'[', '#', 'i', 2, 'l', 0x00, 0x01, 0x11, 0x70, 'l', 0x00, 0x01, 0x11, 0x71};
|
|
CHECK(json::to_ubjson(j, true, true) == expected);
|
|
CHECK(json::to_ubjson(j, true) == expected_size);
|
|
}
|
|
|
|
SECTION("array of L")
|
|
{
|
|
json const j = {5000000000u, 5000000001u};
|
|
std::vector<uint8_t> expected = {'[', '$', 'L', '#', 'i', 2, 0x00, 0x00, 0x00, 0x01, 0x2A, 0x05, 0xF2, 0x00, 0x00, 0x00, 0x00, 0x01, 0x2A, 0x05, 0xF2, 0x01};
|
|
std::vector<uint8_t> expected_size = {'[', '#', 'i', 2, 'L', 0x00, 0x00, 0x00, 0x01, 0x2A, 0x05, 0xF2, 0x00, 'L', 0x00, 0x00, 0x00, 0x01, 0x2A, 0x05, 0xF2, 0x01};
|
|
CHECK(json::to_ubjson(j, true, true) == expected);
|
|
CHECK(json::to_ubjson(j, true) == expected_size);
|
|
}
|
|
}
|
|
|
|
SECTION("discarded")
|
|
{
|
|
json const j = {json::value_t::discarded, json::value_t::discarded};
|
|
std::vector<uint8_t> expected = {'[', '$', 'N', '#', 'i', 2};
|
|
CHECK(json::to_ubjson(j, true, true) == expected);
|
|
}
|
|
}
|
|
}
|
|
|
|
TEST_CASE("UBJSON nesting does not consume the call stack")
|
|
{
|
|
// Containers used to be read by calling back into the value reader once
|
|
// per element, so the native call stack grew with the nesting depth of the
|
|
// input. '[' alone opens a container, so a payload of repeated '[' crashed
|
|
// the process (#5104), as did the optimized forms, which reach the same
|
|
// path through a type or size annotation. The containers are kept on a
|
|
// heap stack now.
|
|
//
|
|
// Deeply nested values must not be compared, copied or dumped here: those
|
|
// operations are still recursive and would reintroduce the crash.
|
|
json _;
|
|
|
|
SECTION("containers that end at a marker")
|
|
{
|
|
const std::vector<uint8_t> input(500000, '[');
|
|
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(input), "[json.exception.parse_error.110] parse error at byte 500001: syntax error while parsing UBJSON value: unexpected end of input", json::parse_error&);
|
|
CHECK(json::from_ubjson(input, true, false).is_discarded());
|
|
}
|
|
|
|
SECTION("containers with a size")
|
|
{
|
|
std::vector<uint8_t> input;
|
|
for (std::size_t i = 0; i < 100000; ++i)
|
|
{
|
|
input.push_back('[');
|
|
input.push_back('#');
|
|
input.push_back('i');
|
|
input.push_back(1);
|
|
}
|
|
CHECK_THROWS_AS(_ = json::from_ubjson(input), json::parse_error&);
|
|
CHECK(json::from_ubjson(input, true, false).is_discarded());
|
|
}
|
|
|
|
SECTION("containers with a type and a size")
|
|
{
|
|
// '[' is a permitted optimized type in UBJSON, so each element of such
|
|
// a container is itself a container, read without a marker of its own
|
|
std::vector<uint8_t> input;
|
|
for (std::size_t i = 0; i < 100000; ++i)
|
|
{
|
|
const std::vector<uint8_t> level = {'[', '$', '[', '#', 'i', 1};
|
|
input.insert(input.end(), level.begin(), level.end());
|
|
}
|
|
CHECK_THROWS_AS(_ = json::from_ubjson(input), json::parse_error&);
|
|
CHECK(json::from_ubjson(input, true, false).is_discarded());
|
|
}
|
|
|
|
SECTION("a well-formed deep value is read through the SAX interface")
|
|
{
|
|
std::vector<uint8_t> input(100000, '[');
|
|
input.insert(input.end(), 100000, ']');
|
|
|
|
SaxCountdown accept_all(1000000);
|
|
CHECK(json::sax_parse(input, &accept_all, json::input_format_t::ubjson));
|
|
}
|
|
|
|
SECTION("a well-formed deep value is read into a value")
|
|
{
|
|
const std::size_t depth = 10000;
|
|
std::vector<uint8_t> input(depth, '[');
|
|
input.insert(input.end(), depth, ']');
|
|
|
|
json j = json::from_ubjson(input);
|
|
|
|
std::size_t measured = 0;
|
|
const json* p = &j;
|
|
while (p->is_array() && !p->empty())
|
|
{
|
|
p = &p->front();
|
|
++measured;
|
|
}
|
|
// the innermost array is empty, so the descent stops one level short
|
|
CHECK(measured == depth - 1);
|
|
}
|
|
|
|
SECTION("containers are still read the same way")
|
|
{
|
|
CHECK(json::from_ubjson(std::vector<uint8_t>({'[', ']'})) == json::array());
|
|
CHECK(json::from_ubjson(std::vector<uint8_t>({'{', '}'})) == json::object());
|
|
CHECK(json::from_ubjson(std::vector<uint8_t>({'[', '#', 'i', 0})) == json::array());
|
|
CHECK(json::from_ubjson(std::vector<uint8_t>({'{', '#', 'i', 0})) == json::object());
|
|
CHECK(json::from_ubjson(std::vector<uint8_t>({'[', '$', 'i', '#', 'i', 2, 1, 2})) == json({1, 2}));
|
|
CHECK(json::from_ubjson(std::vector<uint8_t>({'[', '#', 'i', 2, 'i', 1, 'i', 2})) == json({1, 2}));
|
|
CHECK(json::from_ubjson(std::vector<uint8_t>({'{', '$', 'i', '#', 'i', 1, 'i', 1, 'a', 1})) == json({{"a", 1}}));
|
|
// a no-op is not a value, so a container of them holds none
|
|
CHECK(json::from_ubjson(std::vector<uint8_t>({'[', '$', 'N', '#', 'i', 2})) == json::array());
|
|
// sized and unsized forms nested inside one another
|
|
CHECK(json::from_ubjson(std::vector<uint8_t>({'[', '[', '#', 'i', 2, 'i', 1, 'i', 2, ']'})) == json({{1, 2}}));
|
|
CHECK(json::from_ubjson(std::vector<uint8_t>({'[', '#', 'i', 1, '[', 'i', 1, ']'})) == json({{1}}));
|
|
// an optimized container of containers
|
|
CHECK(json::from_ubjson(std::vector<uint8_t>({'[', '$', '[', '#', 'i', 2, 'i', 1, ']', 'i', 2, ']'})) == json({{1}, {2}}));
|
|
}
|
|
|
|
SECTION("BJData containers are still read the same way")
|
|
{
|
|
// the ND-array wrapper and the binary shortcut are complete values,
|
|
// not containers the reader descends into
|
|
CHECK(json::from_bjdata(std::vector<uint8_t>({'[', '$', 'U', '#', '[', '$', 'i', '#', 'i', 2, 2, 3, 1, 2, 3, 4, 5, 6})) ==
|
|
json({{"_ArrayType_", "uint8"}, {"_ArraySize_", {2, 3}}, {"_ArrayData_", {1, 2, 3, 4, 5, 6}}}));
|
|
CHECK(json::from_bjdata(std::vector<uint8_t>({'[', '$', 'i', '#', 'i', 2, 1, 2})) == json({1, 2}));
|
|
CHECK(json::from_bjdata(std::vector<uint8_t>({'[', '[', 'i', 1, ']', ']'})) == json({{1}}));
|
|
}
|
|
}
|
|
|
|
TEST_CASE("UBJSON input that cannot be read is discarded by every overload")
|
|
{
|
|
std::vector<std::uint8_t> input = json::to_ubjson(json({{"a", {1, 2}}}));
|
|
input.pop_back();
|
|
|
|
json _;
|
|
CHECK_THROWS_AS(_ = json::from_ubjson(input.begin(), input.end()), json::parse_error&);
|
|
CHECK(json::from_ubjson(input, true, false).is_discarded());
|
|
CHECK(json::from_ubjson(input.begin(), input.end(), true, false).is_discarded());
|
|
CHECK(json::from_ubjson(input.data(), input.size(), true, false).is_discarded());
|
|
CHECK(json::from_ubjson({input.data(), input.size()}, true, false).is_discarded());
|
|
}
|
|
|
|
TEST_CASE("UBJSON SAX parsing stops at every event")
|
|
{
|
|
// Containers are opened and closed by the loop that reads them; a SAX
|
|
// handler that rejects any event - including the end of a nested
|
|
// container - must stop the parse right there.
|
|
const auto count_events = [](const std::vector<std::uint8_t>& input)
|
|
{
|
|
int events = 0;
|
|
while (true)
|
|
{
|
|
SaxCountdown scp(events);
|
|
if (json::sax_parse(input, &scp, json::input_format_t::ubjson))
|
|
{
|
|
return events;
|
|
}
|
|
++events;
|
|
REQUIRE(events < 1000);
|
|
}
|
|
};
|
|
|
|
// 20 events: every container kind closes inside another one
|
|
const json j = json::parse(R"({"a": [1, {"b": []}], "c": {"d": [[2]]}})");
|
|
CHECK(count_events(json::to_ubjson(j)) == 20);
|
|
CHECK(count_events(json::to_ubjson(j, true)) == 20);
|
|
CHECK(count_events(json::to_ubjson(j, true, true)) == 20);
|
|
}
|
|
|
|
TEST_CASE("UBJSON optimized arrays of a valueless type are bounded")
|
|
{
|
|
// An element of type 'Z', 'T' or 'F' is encoded by its marker alone, so an
|
|
// optimized array of one of those has no payload and the declared count is
|
|
// the only thing deciding how much is allocated. Ten bytes used to produce
|
|
// billions of values (#2793); every other type costs at least one byte per
|
|
// element and is bounded by the end of the input.
|
|
json _;
|
|
|
|
SECTION("an excessive count is rejected")
|
|
{
|
|
// 'l' is a big-endian int32: 0x7FFFFFFF elements, about 34 GB of value;
|
|
// OSS-Fuzz reported this shape as a parse_ubjson_fuzzer timeout
|
|
// (testcase 6347769435193344, no issue filed)
|
|
for (const auto marker :
|
|
{'Z', 'T', 'F'
|
|
})
|
|
{
|
|
const std::vector<uint8_t> input = {'[', '$', static_cast<uint8_t>(marker), '#', 'l', 0x7F, 0xFF, 0xFF, 0xFF};
|
|
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(input), "[json.exception.out_of_range.408] syntax error while parsing UBJSON size: excessive array size", json::out_of_range&);
|
|
CHECK(json::from_ubjson(input, true, false).is_discarded());
|
|
}
|
|
}
|
|
|
|
SECTION("ordinary counts are unaffected")
|
|
{
|
|
CHECK(json::from_ubjson(std::vector<uint8_t>({'[', '$', 'Z', '#', 'i', 3})) == json({nullptr, nullptr, nullptr}));
|
|
CHECK(json::from_ubjson(std::vector<uint8_t>({'[', '$', 'T', '#', 'i', 2})) == json({true, true}));
|
|
CHECK(json::from_ubjson(std::vector<uint8_t>({'[', '$', 'F', '#', 'i', 2})) == json({false, false}));
|
|
// 'N' is a no-op rather than a value, and still yields an empty array
|
|
CHECK(json::from_ubjson(std::vector<uint8_t>({'[', '$', 'N', '#', 'i', 2})) == json::array());
|
|
}
|
|
|
|
SECTION("a type with a payload is unaffected")
|
|
{
|
|
// A count past the limit is not rejected for 'U', which costs a byte
|
|
// per element and is bounded by the end of the input instead. The
|
|
// count is kept just past the limit rather than made huge, because a
|
|
// count that also exceeds the array's max_size() is reported as
|
|
// out_of_range before the input runs out, and max_size() depends on
|
|
// the width of std::size_t.
|
|
const std::vector<uint8_t> input = {'[', '$', 'U', '#', 'l', 0x00, 0x10, 0x00, 0x01};
|
|
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(input), "[json.exception.parse_error.110] parse error at byte 10: syntax error while parsing UBJSON number: unexpected end of input", json::parse_error&);
|
|
CHECK(json::from_ubjson(input, true, false).is_discarded());
|
|
}
|
|
|
|
SECTION("the writer stays within what the reader accepts")
|
|
{
|
|
// below the limit the optimized form is used and is tiny; above it the
|
|
// writer falls back so that the result can still be read back
|
|
json const at_limit(1048576, nullptr);
|
|
const auto v_at_limit = json::to_ubjson(at_limit, true, true);
|
|
CHECK(v_at_limit.size() == 9);
|
|
CHECK(v_at_limit.at(1) == '$');
|
|
CHECK(json::from_ubjson(v_at_limit) == at_limit);
|
|
|
|
json const above_limit(1048577, nullptr);
|
|
const auto v_above_limit = json::to_ubjson(above_limit, true, true);
|
|
CHECK(v_above_limit.at(1) != '$');
|
|
CHECK(json::from_ubjson(v_above_limit) == above_limit);
|
|
}
|
|
}
|
|
|
|
TEST_CASE("issue #5405 - array reserve for definite-length UBJSON arrays")
|
|
{
|
|
#if !defined(JSON_NOEXCEPTION)
|
|
// this SECTION relies on catching a thrown exception to distinguish
|
|
// which of two acceptable, bounded rejections a hostile header took;
|
|
// under JSON_NOEXCEPTION, JSON_THROW never produces a catchable C++
|
|
// exception (it aborts instead), so this cannot be tested that way here
|
|
SECTION("a huge claimed length with no element data must not over-allocate")
|
|
{
|
|
// optimized form [$type#count: type 'i' (int8), count as a four-byte
|
|
// 'l' (int32) of 0x7FFFFFFF (2147483647), but no element data at all.
|
|
// max_size() for a std::vector is far larger than this count, so it
|
|
// does not reject the header outright; the (capped) reservation must
|
|
// not attempt to allocate space for billions of elements before the
|
|
// missing data is detected.
|
|
json _;
|
|
const std::vector<uint8_t> input = {'[', '$', 'i', '#', 'l', 0x7F, 0xFF, 0xFF, 0xFF};
|
|
// On a platform where std::vector<json>::max_size() is smaller than
|
|
// the claimed count (e.g. 32-bit, where max_size() is bounded by a
|
|
// 32-bit SIZE_MAX divided by sizeof(json)), the SAX consumer's own
|
|
// check rejects the header outright (out_of_range.408, with the
|
|
// claimed count in the message) instead of accepting it and only
|
|
// finding it short of data once the (capped) reservation looks for
|
|
// element bytes that were never provided (parse_error.110). Either
|
|
// is an acceptable, bounded rejection of the hostile header -- the
|
|
// property under test is that no path attempts to allocate space
|
|
// for billions of elements.
|
|
bool threw = false;
|
|
try
|
|
{
|
|
_ = json::from_ubjson(input);
|
|
}
|
|
catch (const json::parse_error& e)
|
|
{
|
|
threw = true;
|
|
CHECK(e.id == 110);
|
|
CHECK(std::string(e.what()) == "[json.exception.parse_error.110] parse error at byte 10: syntax error while parsing UBJSON number: unexpected end of input");
|
|
}
|
|
catch (const json::out_of_range& e)
|
|
{
|
|
threw = true;
|
|
CHECK(e.id == 408);
|
|
CHECK(std::string(e.what()).find("excessive array size") != std::string::npos);
|
|
}
|
|
CHECK(threw);
|
|
|
|
// json_sax_dom_parser::start_array()'s max_size() check (unlike the
|
|
// scanner's own parse_error path) throws unconditionally via
|
|
// JSON_THROW rather than going through sax->parse_error(), so it is
|
|
// not gated by allow_exceptions=false on a platform where this
|
|
// header hits that check (e.g. 32-bit, see above) -- allow either
|
|
// a discarded result or the same out_of_range it throws with
|
|
// exceptions enabled.
|
|
try
|
|
{
|
|
CHECK(json::from_ubjson(input, true, false).is_discarded());
|
|
}
|
|
catch (const json::out_of_range& e)
|
|
{
|
|
CHECK(e.id == 408);
|
|
}
|
|
}
|
|
#endif
|
|
|
|
SECTION("arrays of various sizes decode to the same value as before the reserve optimization")
|
|
{
|
|
for (const auto size :
|
|
{
|
|
std::size_t{0}, std::size_t{1}, std::size_t{5}, // small
|
|
std::size_t{16384}, // exactly at the reserve cap
|
|
std::size_t{20000} // above the reserve cap
|
|
})
|
|
{
|
|
CAPTURE(size)
|
|
json j = json::array();
|
|
for (std::size_t i = 0; i < size; ++i)
|
|
{
|
|
j.push_back(static_cast<int>(i % 1000));
|
|
}
|
|
|
|
// exercise both the plain and the optimized [$type#count encoding
|
|
const auto packed_plain = json::to_ubjson(j);
|
|
CHECK(json::from_ubjson(packed_plain) == j);
|
|
|
|
const auto packed_optimized = json::to_ubjson(j, true, true);
|
|
CHECK(json::from_ubjson(packed_optimized) == j);
|
|
}
|
|
}
|
|
|
|
SECTION("a user-defined SAX consumer is unaffected by the internal DOM reserve optimization")
|
|
{
|
|
// the reserve() call is local to json_sax_dom_parser / json_sax_dom_callback_parser;
|
|
// a custom SAX consumer that does not touch a DOM array sees identical events
|
|
json j = json::array();
|
|
for (int i = 0; i < 100; ++i)
|
|
{
|
|
j.push_back(i);
|
|
}
|
|
const auto packed = json::to_ubjson(j, true, true);
|
|
|
|
SaxCountdown scp(1000000); // large enough to never trigger an abort
|
|
CHECK(json::sax_parse(packed, &scp, json::input_format_t::ubjson));
|
|
}
|
|
}
|
|
|
|
|
|
TEST_CASE("Universal Binary JSON Specification Examples 1")
|
|
{
|
|
SECTION("Null Value")
|
|
{
|
|
json const j = {{"passcode", nullptr}};
|
|
std::vector<uint8_t> const v = {'{', 'i', 8, 'p', 'a', 's', 's', 'c', 'o', 'd', 'e', 'Z', '}'};
|
|
CHECK(json::to_ubjson(j) == v);
|
|
CHECK(json::from_ubjson(v) == j);
|
|
}
|
|
|
|
SECTION("No-Op Value")
|
|
{
|
|
json const j = {"foo", "bar", "baz"};
|
|
std::vector<uint8_t> const v = {'[', 'S', 'i', 3, 'f', 'o', 'o',
|
|
'S', 'i', 3, 'b', 'a', 'r',
|
|
'S', 'i', 3, 'b', 'a', 'z', ']'
|
|
};
|
|
std::vector<uint8_t> const v2 = {'[', 'S', 'i', 3, 'f', 'o', 'o', 'N',
|
|
'S', 'i', 3, 'b', 'a', 'r', 'N', 'N', 'N',
|
|
'S', 'i', 3, 'b', 'a', 'z', 'N', 'N', ']'
|
|
};
|
|
CHECK(json::to_ubjson(j) == v);
|
|
CHECK(json::from_ubjson(v) == j);
|
|
CHECK(json::from_ubjson(v2) == j);
|
|
}
|
|
|
|
SECTION("Boolean Types")
|
|
{
|
|
json const j = {{"authorized", true}, {"verified", false}};
|
|
std::vector<uint8_t> const v = {'{', 'i', 10, 'a', 'u', 't', 'h', 'o', 'r', 'i', 'z', 'e', 'd', 'T',
|
|
'i', 8, 'v', 'e', 'r', 'i', 'f', 'i', 'e', 'd', 'F', '}'
|
|
};
|
|
CHECK(json::to_ubjson(j) == v);
|
|
CHECK(json::from_ubjson(v) == j);
|
|
}
|
|
|
|
SECTION("Numeric Types")
|
|
{
|
|
json const j =
|
|
{
|
|
{"int8", 16},
|
|
{"uint8", 255},
|
|
{"int16", 32767},
|
|
{"int32", 2147483647},
|
|
{"int64", 9223372036854775807},
|
|
{"float64", 113243.7863123}
|
|
};
|
|
std::vector<uint8_t> const v = {'{',
|
|
'i', 7, 'f', 'l', 'o', 'a', 't', '6', '4', 'D', 0x40, 0xfb, 0xa5, 0xbc, 0x94, 0xbc, 0x34, 0xcf,
|
|
'i', 5, 'i', 'n', 't', '1', '6', 'I', 0x7f, 0xff,
|
|
'i', 5, 'i', 'n', 't', '3', '2', 'l', 0x7f, 0xff, 0xff, 0xff,
|
|
'i', 5, 'i', 'n', 't', '6', '4', 'L', 0x7f, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
|
|
'i', 4, 'i', 'n', 't', '8', 'i', 16,
|
|
'i', 5, 'u', 'i', 'n', 't', '8', 'U', 0xff,
|
|
'}'
|
|
};
|
|
CHECK(json::to_ubjson(j) == v);
|
|
CHECK(json::from_ubjson(v) == j);
|
|
}
|
|
|
|
SECTION("Char Type")
|
|
{
|
|
json const j = {{"rolecode", "a"}, {"delim", ";"}};
|
|
std::vector<uint8_t> const v = {'{', 'i', 5, 'd', 'e', 'l', 'i', 'm', 'C', ';', 'i', 8, 'r', 'o', 'l', 'e', 'c', 'o', 'd', 'e', 'C', 'a', '}'};
|
|
//CHECK(json::to_ubjson(j) == v);
|
|
CHECK(json::from_ubjson(v) == j);
|
|
}
|
|
|
|
SECTION("String Type")
|
|
{
|
|
SECTION("English")
|
|
{
|
|
json const j = "hello";
|
|
std::vector<uint8_t> const v = {'S', 'i', 5, 'h', 'e', 'l', 'l', 'o'};
|
|
CHECK(json::to_ubjson(j) == v);
|
|
CHECK(json::from_ubjson(v) == j);
|
|
}
|
|
|
|
SECTION("Russian")
|
|
{
|
|
json const j = "привет";
|
|
std::vector<uint8_t> const v = {'S', 'i', 12, 0xD0, 0xBF, 0xD1, 0x80, 0xD0, 0xB8, 0xD0, 0xB2, 0xD0, 0xB5, 0xD1, 0x82};
|
|
CHECK(json::to_ubjson(j) == v);
|
|
CHECK(json::from_ubjson(v) == j);
|
|
}
|
|
|
|
SECTION("Russian")
|
|
{
|
|
json const j = "مرحبا";
|
|
std::vector<uint8_t> const v = {'S', 'i', 10, 0xD9, 0x85, 0xD8, 0xB1, 0xD8, 0xAD, 0xD8, 0xA8, 0xD8, 0xA7};
|
|
CHECK(json::to_ubjson(j) == v);
|
|
CHECK(json::from_ubjson(v) == j);
|
|
}
|
|
}
|
|
|
|
SECTION("Array Type")
|
|
{
|
|
SECTION("size=false type=false")
|
|
{
|
|
// note the float has been replaced by a double
|
|
json const j = {nullptr, true, false, 4782345193, 153.132, "ham"};
|
|
std::vector<uint8_t> const v = {'[', 'Z', 'T', 'F', 'L', 0x00, 0x00, 0x00, 0x01, 0x1D, 0x0C, 0xCB, 0xE9, 'D', 0x40, 0x63, 0x24, 0x39, 0x58, 0x10, 0x62, 0x4e, 'S', 'i', 3, 'h', 'a', 'm', ']'};
|
|
CHECK(json::to_ubjson(j) == v);
|
|
CHECK(json::from_ubjson(v) == j);
|
|
}
|
|
|
|
SECTION("size=true type=false")
|
|
{
|
|
// note the float has been replaced by a double
|
|
json const j = {nullptr, true, false, 4782345193, 153.132, "ham"};
|
|
std::vector<uint8_t> const v = {'[', '#', 'i', 6, 'Z', 'T', 'F', 'L', 0x00, 0x00, 0x00, 0x01, 0x1D, 0x0C, 0xCB, 0xE9, 'D', 0x40, 0x63, 0x24, 0x39, 0x58, 0x10, 0x62, 0x4e, 'S', 'i', 3, 'h', 'a', 'm'};
|
|
CHECK(json::to_ubjson(j, true) == v);
|
|
CHECK(json::from_ubjson(v) == j);
|
|
}
|
|
|
|
SECTION("size=true type=true")
|
|
{
|
|
// note the float has been replaced by a double
|
|
json const j = {nullptr, true, false, 4782345193, 153.132, "ham"};
|
|
std::vector<uint8_t> const v = {'[', '#', 'i', 6, 'Z', 'T', 'F', 'L', 0x00, 0x00, 0x00, 0x01, 0x1D, 0x0C, 0xCB, 0xE9, 'D', 0x40, 0x63, 0x24, 0x39, 0x58, 0x10, 0x62, 0x4e, 'S', 'i', 3, 'h', 'a', 'm'};
|
|
CHECK(json::to_ubjson(j, true, true) == v);
|
|
CHECK(json::from_ubjson(v) == j);
|
|
}
|
|
}
|
|
|
|
SECTION("Object Type")
|
|
{
|
|
SECTION("size=false type=false")
|
|
{
|
|
json const j =
|
|
{
|
|
{
|
|
"post", {
|
|
{"id", 1137},
|
|
{"author", "rkalla"},
|
|
{"timestamp", 1364482090592},
|
|
{"body", "I totally agree!"}
|
|
}
|
|
}
|
|
};
|
|
std::vector<uint8_t> const v = {'{', 'i', 4, 'p', 'o', 's', 't', '{',
|
|
'i', 6, 'a', 'u', 't', 'h', 'o', 'r', 'S', 'i', 6, 'r', 'k', 'a', 'l', 'l', 'a',
|
|
'i', 4, 'b', 'o', 'd', 'y', 'S', 'i', 16, 'I', ' ', 't', 'o', 't', 'a', 'l', 'l', 'y', ' ', 'a', 'g', 'r', 'e', 'e', '!',
|
|
'i', 2, 'i', 'd', 'I', 0x04, 0x71,
|
|
'i', 9, 't', 'i', 'm', 'e', 's', 't', 'a', 'm', 'p', 'L', 0x00, 0x00, 0x01, 0x3D, 0xB1, 0x78, 0x66, 0x60,
|
|
'}', '}'
|
|
};
|
|
CHECK(json::to_ubjson(j) == v);
|
|
CHECK(json::from_ubjson(v) == j);
|
|
}
|
|
|
|
SECTION("size=true type=false")
|
|
{
|
|
json const j =
|
|
{
|
|
{
|
|
"post", {
|
|
{"id", 1137},
|
|
{"author", "rkalla"},
|
|
{"timestamp", 1364482090592},
|
|
{"body", "I totally agree!"}
|
|
}
|
|
}
|
|
};
|
|
std::vector<uint8_t> const v = {'{', '#', 'i', 1, 'i', 4, 'p', 'o', 's', 't', '{', '#', 'i', 4,
|
|
'i', 6, 'a', 'u', 't', 'h', 'o', 'r', 'S', 'i', 6, 'r', 'k', 'a', 'l', 'l', 'a',
|
|
'i', 4, 'b', 'o', 'd', 'y', 'S', 'i', 16, 'I', ' ', 't', 'o', 't', 'a', 'l', 'l', 'y', ' ', 'a', 'g', 'r', 'e', 'e', '!',
|
|
'i', 2, 'i', 'd', 'I', 0x04, 0x71,
|
|
'i', 9, 't', 'i', 'm', 'e', 's', 't', 'a', 'm', 'p', 'L', 0x00, 0x00, 0x01, 0x3D, 0xB1, 0x78, 0x66, 0x60
|
|
};
|
|
CHECK(json::to_ubjson(j, true) == v);
|
|
CHECK(json::from_ubjson(v) == j);
|
|
}
|
|
|
|
SECTION("size=true type=true")
|
|
{
|
|
json const j =
|
|
{
|
|
{
|
|
"post", {
|
|
{"id", 1137},
|
|
{"author", "rkalla"},
|
|
{"timestamp", 1364482090592},
|
|
{"body", "I totally agree!"}
|
|
}
|
|
}
|
|
};
|
|
std::vector<uint8_t> const v = {'{', '$', '{', '#', 'i', 1, 'i', 4, 'p', 'o', 's', 't', '#', 'i', 4,
|
|
'i', 6, 'a', 'u', 't', 'h', 'o', 'r', 'S', 'i', 6, 'r', 'k', 'a', 'l', 'l', 'a',
|
|
'i', 4, 'b', 'o', 'd', 'y', 'S', 'i', 16, 'I', ' ', 't', 'o', 't', 'a', 'l', 'l', 'y', ' ', 'a', 'g', 'r', 'e', 'e', '!',
|
|
'i', 2, 'i', 'd', 'I', 0x04, 0x71,
|
|
'i', 9, 't', 'i', 'm', 'e', 's', 't', 'a', 'm', 'p', 'L', 0x00, 0x00, 0x01, 0x3D, 0xB1, 0x78, 0x66, 0x60
|
|
};
|
|
CHECK(json::to_ubjson(j, true, true) == v);
|
|
CHECK(json::from_ubjson(v) == j);
|
|
}
|
|
}
|
|
|
|
SECTION("Optimized Format")
|
|
{
|
|
SECTION("Array Example")
|
|
{
|
|
SECTION("No Optimization")
|
|
{
|
|
// note the floats have been replaced by doubles
|
|
json const j = {29.97, 31.13, 67.0, 2.113, 23.888};
|
|
std::vector<uint8_t> const v = {'[',
|
|
'D', 0x40, 0x3d, 0xf8, 0x51, 0xeb, 0x85, 0x1e, 0xb8,
|
|
'D', 0x40, 0x3f, 0x21, 0x47, 0xae, 0x14, 0x7a, 0xe1,
|
|
'D', 0x40, 0x50, 0xc0, 0x00, 0x00, 0x00, 0x00, 0x00,
|
|
'D', 0x40, 0x00, 0xe7, 0x6c, 0x8b, 0x43, 0x95, 0x81,
|
|
'D', 0x40, 0x37, 0xe3, 0x53, 0xf7, 0xce, 0xd9, 0x17,
|
|
']'
|
|
};
|
|
CHECK(json::to_ubjson(j) == v);
|
|
CHECK(json::from_ubjson(v) == j);
|
|
}
|
|
|
|
SECTION("Optimized with count")
|
|
{
|
|
// note the floats have been replaced by doubles
|
|
json const j = {29.97, 31.13, 67.0, 2.113, 23.888};
|
|
std::vector<uint8_t> const v = {'[', '#', 'i', 5,
|
|
'D', 0x40, 0x3d, 0xf8, 0x51, 0xeb, 0x85, 0x1e, 0xb8,
|
|
'D', 0x40, 0x3f, 0x21, 0x47, 0xae, 0x14, 0x7a, 0xe1,
|
|
'D', 0x40, 0x50, 0xc0, 0x00, 0x00, 0x00, 0x00, 0x00,
|
|
'D', 0x40, 0x00, 0xe7, 0x6c, 0x8b, 0x43, 0x95, 0x81,
|
|
'D', 0x40, 0x37, 0xe3, 0x53, 0xf7, 0xce, 0xd9, 0x17
|
|
};
|
|
CHECK(json::to_ubjson(j, true) == v);
|
|
CHECK(json::from_ubjson(v) == j);
|
|
}
|
|
|
|
SECTION("Optimized with type & count")
|
|
{
|
|
// note the floats have been replaced by doubles
|
|
json const j = {29.97, 31.13, 67.0, 2.113, 23.888};
|
|
std::vector<uint8_t> const v = {'[', '$', 'D', '#', 'i', 5,
|
|
0x40, 0x3d, 0xf8, 0x51, 0xeb, 0x85, 0x1e, 0xb8,
|
|
0x40, 0x3f, 0x21, 0x47, 0xae, 0x14, 0x7a, 0xe1,
|
|
0x40, 0x50, 0xc0, 0x00, 0x00, 0x00, 0x00, 0x00,
|
|
0x40, 0x00, 0xe7, 0x6c, 0x8b, 0x43, 0x95, 0x81,
|
|
0x40, 0x37, 0xe3, 0x53, 0xf7, 0xce, 0xd9, 0x17
|
|
};
|
|
CHECK(json::to_ubjson(j, true, true) == v);
|
|
CHECK(json::from_ubjson(v) == j);
|
|
}
|
|
}
|
|
|
|
SECTION("Object Example")
|
|
{
|
|
SECTION("No Optimization")
|
|
{
|
|
// note the floats have been replaced by doubles
|
|
json const j = { {"lat", 29.976}, {"long", 31.131}, {"alt", 67.0} };
|
|
std::vector<uint8_t> const v = {'{',
|
|
'i', 3, 'a', 'l', 't', 'D', 0x40, 0x50, 0xc0, 0x00, 0x00, 0x00, 0x00, 0x00,
|
|
'i', 3, 'l', 'a', 't', 'D', 0x40, 0x3d, 0xf9, 0xdb, 0x22, 0xd0, 0xe5, 0x60,
|
|
'i', 4, 'l', 'o', 'n', 'g', 'D', 0x40, 0x3f, 0x21, 0x89, 0x37, 0x4b, 0xc6, 0xa8,
|
|
'}'
|
|
};
|
|
CHECK(json::to_ubjson(j) == v);
|
|
CHECK(json::from_ubjson(v) == j);
|
|
}
|
|
|
|
SECTION("Optimized with count")
|
|
{
|
|
// note the floats have been replaced by doubles
|
|
json const j = { {"lat", 29.976}, {"long", 31.131}, {"alt", 67.0} };
|
|
std::vector<uint8_t> const v = {'{', '#', 'i', 3,
|
|
'i', 3, 'a', 'l', 't', 'D', 0x40, 0x50, 0xc0, 0x00, 0x00, 0x00, 0x00, 0x00,
|
|
'i', 3, 'l', 'a', 't', 'D', 0x40, 0x3d, 0xf9, 0xdb, 0x22, 0xd0, 0xe5, 0x60,
|
|
'i', 4, 'l', 'o', 'n', 'g', 'D', 0x40, 0x3f, 0x21, 0x89, 0x37, 0x4b, 0xc6, 0xa8
|
|
};
|
|
CHECK(json::to_ubjson(j, true) == v);
|
|
CHECK(json::from_ubjson(v) == j);
|
|
}
|
|
|
|
SECTION("Optimized with type & count")
|
|
{
|
|
// note the floats have been replaced by doubles
|
|
json const j = { {"lat", 29.976}, {"long", 31.131}, {"alt", 67.0} };
|
|
std::vector<uint8_t> const v = {'{', '$', 'D', '#', 'i', 3,
|
|
'i', 3, 'a', 'l', 't', 0x40, 0x50, 0xc0, 0x00, 0x00, 0x00, 0x00, 0x00,
|
|
'i', 3, 'l', 'a', 't', 0x40, 0x3d, 0xf9, 0xdb, 0x22, 0xd0, 0xe5, 0x60,
|
|
'i', 4, 'l', 'o', 'n', 'g', 0x40, 0x3f, 0x21, 0x89, 0x37, 0x4b, 0xc6, 0xa8
|
|
};
|
|
CHECK(json::to_ubjson(j, true, true) == v);
|
|
CHECK(json::from_ubjson(v) == j);
|
|
}
|
|
}
|
|
|
|
SECTION("Special Cases (Null, No-Op and Boolean)")
|
|
{
|
|
SECTION("Array")
|
|
{
|
|
std::vector<uint8_t> const v = {'[', '$', 'N', '#', 'I', 0x02, 0x00};
|
|
CHECK(json::from_ubjson(v) == json::array());
|
|
}
|
|
|
|
SECTION("Object")
|
|
{
|
|
std::vector<uint8_t> const v = {'{', '$', 'Z', '#', 'i', 3, 'i', 4, 'n', 'a', 'm', 'e', 'i', 8, 'p', 'a', 's', 's', 'w', 'o', 'r', 'd', 'i', 5, 'e', 'm', 'a', 'i', 'l'};
|
|
CHECK(json::from_ubjson(v) == json({ {"name", nullptr}, {"password", nullptr}, {"email", nullptr} }));
|
|
}
|
|
}
|
|
}
|
|
}
|
|
|
|
TEST_CASE("Parse UBJSON directly from a file using iterator and sentinel")
|
|
{
|
|
std::string const filename = TEST_DATA_DIRECTORY "/json_testsuite/sample.json.ubjson";
|
|
std::ifstream file(filename, std::ios::binary);
|
|
const std::istreambuf_iterator<char> first(file);
|
|
const json parsed = json::from_ubjson(first, utils::istreambuf_sentinel{});
|
|
CHECK((parsed.is_object() || parsed.is_array()));
|
|
}
|
|
|
|
#if !defined(JSON_NOEXCEPTION)
|
|
TEST_CASE("all UBJSON first bytes")
|
|
{
|
|
// these bytes will fail immediately with exception parse_error.112
|
|
std::set<uint8_t> supported =
|
|
{
|
|
'T', 'F', 'Z', 'U', 'i', 'I', 'l', 'L', 'd', 'D', 'C', 'S', '[', '{', 'N', 'H'
|
|
};
|
|
|
|
for (auto i = 0; i < 256; ++i)
|
|
{
|
|
const auto byte = static_cast<uint8_t>(i);
|
|
CAPTURE(byte)
|
|
|
|
try
|
|
{
|
|
auto res = json::from_ubjson(std::vector<uint8_t>(1, byte));
|
|
}
|
|
catch (const json::parse_error& e)
|
|
{
|
|
// check that parse_error.112 is only thrown if the
|
|
// first byte is not in the supported set
|
|
INFO_WITH_TEMP(e.what());
|
|
if (supported.find(byte) == supported.end())
|
|
{
|
|
CHECK(e.id == 112);
|
|
}
|
|
else
|
|
{
|
|
CHECK(e.id != 112);
|
|
}
|
|
}
|
|
}
|
|
}
|
|
#endif
|
|
|
|
TEST_CASE("UBJSON use_type requires use_size")
|
|
{
|
|
SECTION("non-empty array throws other_error.502")
|
|
{
|
|
const json j = {1, 2, 3};
|
|
CHECK_THROWS_WITH_AS(json::to_ubjson(j, false, true),
|
|
"[json.exception.other_error.502] use_type requires use_size = true",
|
|
json::other_error&);
|
|
}
|
|
|
|
SECTION("non-empty object throws other_error.502")
|
|
{
|
|
const json j = {{"a", 1}, {"b", 2}};
|
|
CHECK_THROWS_WITH_AS(json::to_ubjson(j, false, true),
|
|
"[json.exception.other_error.502] use_type requires use_size = true",
|
|
json::other_error&);
|
|
}
|
|
|
|
SECTION("scalars do not throw with use_type=true, use_count=false")
|
|
{
|
|
CHECK_NOTHROW(json::to_ubjson(42, false, true));
|
|
CHECK_NOTHROW(json::to_ubjson(3.14, false, true));
|
|
CHECK_NOTHROW(json::to_ubjson("hello", false, true));
|
|
CHECK_NOTHROW(json::to_ubjson(true, false, true));
|
|
CHECK_NOTHROW(json::to_ubjson(nullptr, false, true));
|
|
}
|
|
|
|
SECTION("empty containers do not throw with use_type=true, use_count=false")
|
|
{
|
|
CHECK_NOTHROW(json::to_ubjson(json::array(), false, true));
|
|
CHECK_NOTHROW(json::to_ubjson(json::object(), false, true));
|
|
}
|
|
|
|
SECTION("valid combinations on non-empty containers")
|
|
{
|
|
const json j = {1, 2, 3};
|
|
CHECK_NOTHROW(json::to_ubjson(j, false, false));
|
|
CHECK_NOTHROW(json::to_ubjson(j, true, false));
|
|
CHECK_NOTHROW(json::to_ubjson(j, true, true));
|
|
}
|
|
}
|
|
|
|
TEST_CASE("UBJSON round-trip invariants")
|
|
{
|
|
// This checks what the parse_ubjson_fuzzer driver checks (see
|
|
// tests/src/fuzzer-parse_ubjson.cpp), so that a regression shows up in CI
|
|
// rather than as an OSS-Fuzz report: every value from_ubjson() returns
|
|
// (j1) can be serialized with any combination of options, the result can
|
|
// be parsed back (j2), and serializing j2 again with the same options
|
|
// reproduces the exact bytes. Beyond the driver, this also checks that j2
|
|
// equals j1. Values are compared with dump() rather than operator==,
|
|
// because a NaN never compares equal to itself.
|
|
struct options
|
|
{
|
|
bool use_size;
|
|
bool use_type;
|
|
};
|
|
const std::vector<options> all_options =
|
|
{
|
|
{false, false},
|
|
{true, false},
|
|
{true, true},
|
|
};
|
|
|
|
for (const auto& j0 : utils::round_trip_corpus::values())
|
|
{
|
|
// turn the corpus value into a value as from_ubjson() returns it; this
|
|
// has no binary values, as UBJSON writes them as arrays of integers
|
|
for (const auto& initial : all_options)
|
|
{
|
|
const json j1 = json::from_ubjson(json::to_ubjson(j0, initial.use_size, initial.use_type));
|
|
|
|
for (const auto& o : all_options)
|
|
{
|
|
INFO("j1 = " << j1.dump() << ", use_size = " << o.use_size << ", use_type = " << o.use_type);
|
|
|
|
const std::vector<std::uint8_t> vec = json::to_ubjson(j1, o.use_size, o.use_type);
|
|
json j2;
|
|
// anything the library writes must be parsable by the library
|
|
REQUIRE_NOTHROW(j2 = json::from_ubjson(vec));
|
|
CHECK(j2.dump() == j1.dump());
|
|
CHECK(json::to_ubjson(j2, o.use_size, o.use_type) == vec);
|
|
}
|
|
}
|
|
}
|
|
}
|
|
|
|
TEST_CASE("UBJSON roundtrips" * doctest::skip())
|
|
{
|
|
SECTION("input from self-generated UBJSON files")
|
|
{
|
|
for (std::string filename :
|
|
{
|
|
TEST_DATA_DIRECTORY "/json_nlohmann_tests/all_unicode.json",
|
|
TEST_DATA_DIRECTORY "/json.org/1.json",
|
|
TEST_DATA_DIRECTORY "/json.org/2.json",
|
|
TEST_DATA_DIRECTORY "/json.org/3.json",
|
|
TEST_DATA_DIRECTORY "/json.org/4.json",
|
|
TEST_DATA_DIRECTORY "/json.org/5.json",
|
|
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip01.json",
|
|
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip02.json",
|
|
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip03.json",
|
|
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip04.json",
|
|
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip05.json",
|
|
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip06.json",
|
|
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip07.json",
|
|
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip08.json",
|
|
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip09.json",
|
|
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip10.json",
|
|
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip11.json",
|
|
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip12.json",
|
|
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip13.json",
|
|
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip14.json",
|
|
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip15.json",
|
|
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip16.json",
|
|
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip17.json",
|
|
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip18.json",
|
|
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip19.json",
|
|
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip20.json",
|
|
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip21.json",
|
|
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip22.json",
|
|
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip23.json",
|
|
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip24.json",
|
|
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip25.json",
|
|
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip26.json",
|
|
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip27.json",
|
|
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip28.json",
|
|
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip29.json",
|
|
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip30.json",
|
|
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip31.json",
|
|
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip32.json",
|
|
TEST_DATA_DIRECTORY "/json_testsuite/sample.json",
|
|
TEST_DATA_DIRECTORY "/json_tests/pass1.json",
|
|
TEST_DATA_DIRECTORY "/json_tests/pass2.json",
|
|
TEST_DATA_DIRECTORY "/json_tests/pass3.json"
|
|
})
|
|
{
|
|
CAPTURE(filename)
|
|
|
|
std::ifstream f_json(filename);
|
|
json const j1 = json::parse(f_json);
|
|
auto const packed = utils::read_binary_file(filename + ".ubjson");
|
|
|
|
{
|
|
INFO_WITH_TEMP(filename + ": std::vector<uint8_t>");
|
|
json j2;
|
|
CHECK_NOTHROW(j2 = json::from_ubjson(packed));
|
|
CHECK(j1 == j2);
|
|
}
|
|
|
|
{
|
|
INFO_WITH_TEMP(filename + ": std::ifstream");
|
|
std::ifstream f_ubjson(filename + ".ubjson", std::ios::binary);
|
|
json j2;
|
|
CHECK_NOTHROW(j2 = json::from_ubjson(f_ubjson));
|
|
CHECK(j1 == j2);
|
|
}
|
|
|
|
{
|
|
INFO_WITH_TEMP(filename + ": uint8_t* and size");
|
|
json j2;
|
|
CHECK_NOTHROW(j2 = json::from_ubjson({packed.data(), packed.size()}));
|
|
CHECK(j1 == j2);
|
|
}
|
|
|
|
{
|
|
INFO_WITH_TEMP(filename + ": output to output adapters");
|
|
{
|
|
INFO_WITH_TEMP(filename + ": output adapters: std::vector<uint8_t>");
|
|
std::vector<uint8_t> vec;
|
|
json::to_ubjson(j1, vec);
|
|
CHECK(vec == packed);
|
|
}
|
|
}
|
|
}
|
|
}
|
|
}
|
|
|
|
TEST_CASE("UBJSON optimized array of unsigned integers beyond int64")
|
|
{
|
|
// UBJSON has no unsigned 64-bit type, so such values are written as
|
|
// high-precision numbers - also as the type of an optimized container
|
|
const json j = {18446744073709551615ULL, 9223372036854775808ULL};
|
|
const std::vector<std::uint8_t> expected =
|
|
{
|
|
'[', '$', 'H', '#', 'i', 2,
|
|
'i', 20, '1', '8', '4', '4', '6', '7', '4', '4', '0', '7', '3', '7', '0', '9', '5', '5', '1', '6', '1', '5',
|
|
'i', 19, '9', '2', '2', '3', '3', '7', '2', '0', '3', '6', '8', '5', '4', '7', '7', '5', '8', '0', '8'
|
|
};
|
|
CHECK(json::to_ubjson(j, true, true) == expected);
|
|
CHECK(json::from_ubjson(expected) == j);
|
|
}
|
|
|
|
namespace
|
|
{
|
|
// the bytes that follow the marker of an integer: the value in the width of
|
|
// the marker (big endian for UBJSON, little endian for BJData), or, for a
|
|
// high-precision number, the length and the decimal digits
|
|
std::vector<std::uint8_t> integer_payload(const char marker, const json& value, const bool little_endian)
|
|
{
|
|
std::size_t width = 0;
|
|
switch (marker)
|
|
{
|
|
case 'i':
|
|
case 'U':
|
|
width = 1;
|
|
break;
|
|
case 'I':
|
|
case 'u':
|
|
width = 2;
|
|
break;
|
|
case 'l':
|
|
case 'm':
|
|
width = 4;
|
|
break;
|
|
case 'L':
|
|
case 'M':
|
|
width = 8;
|
|
break;
|
|
default:
|
|
{
|
|
const std::string digits = value.dump();
|
|
std::vector<std::uint8_t> result = {'i', static_cast<std::uint8_t>(digits.size())};
|
|
for (const char c : digits)
|
|
{
|
|
result.push_back(static_cast<std::uint8_t>(c));
|
|
}
|
|
return result;
|
|
}
|
|
}
|
|
|
|
const std::uint64_t bits = value.is_number_unsigned()
|
|
? value.get<std::uint64_t>()
|
|
: static_cast<std::uint64_t>(value.get<std::int64_t>());
|
|
std::vector<std::uint8_t> result(width);
|
|
for (std::size_t i = 0; i < width; ++i)
|
|
{
|
|
result[little_endian ? i : width - 1 - i] = static_cast<std::uint8_t>(bits >> (8 * i));
|
|
}
|
|
return result;
|
|
}
|
|
|
|
json i64(const std::int64_t v)
|
|
{
|
|
return v;
|
|
}
|
|
|
|
json u64(const std::uint64_t v)
|
|
{
|
|
return v;
|
|
}
|
|
} // namespace
|
|
|
|
TEST_CASE("UBJSON and BJData integer markers at every range edge")
|
|
{
|
|
// An optimized container announces the marker of its values after `$` and
|
|
// then writes every value without a marker, so the marker the writer
|
|
// announces and the width it writes must match for every value. This
|
|
// checks both for the values around each edge of the integer types, as
|
|
// scalars and as the values of optimized arrays and objects.
|
|
struct integer_case
|
|
{
|
|
json value;
|
|
char ubjson; // expected UBJSON marker
|
|
char bjdata; // expected BJData marker
|
|
};
|
|
|
|
const std::int64_t int64_min = (std::numeric_limits<std::int64_t>::min)();
|
|
const std::int64_t int64_max = (std::numeric_limits<std::int64_t>::max)();
|
|
const std::uint64_t uint64_max = (std::numeric_limits<std::uint64_t>::max)();
|
|
|
|
const std::vector<integer_case> cases =
|
|
{
|
|
// int8
|
|
{i64(-129), 'I', 'I'},
|
|
{i64(-128), 'i', 'i'},
|
|
{i64(-127), 'i', 'i'},
|
|
{i64(-1), 'i', 'i'},
|
|
{i64(0), 'i', 'i'},
|
|
{u64(0), 'i', 'i'},
|
|
{i64(126), 'i', 'i'},
|
|
{i64(127), 'i', 'i'},
|
|
{u64(127), 'i', 'i'},
|
|
{i64(128), 'U', 'U'},
|
|
{u64(128), 'U', 'U'},
|
|
// uint8
|
|
{i64(254), 'U', 'U'},
|
|
{i64(255), 'U', 'U'},
|
|
{u64(255), 'U', 'U'},
|
|
{i64(256), 'I', 'I'},
|
|
{u64(256), 'I', 'I'},
|
|
// int16
|
|
{i64(-32769), 'l', 'l'},
|
|
{i64(-32768), 'I', 'I'},
|
|
{i64(-32767), 'I', 'I'},
|
|
{i64(32766), 'I', 'I'},
|
|
{i64(32767), 'I', 'I'},
|
|
{u64(32767), 'I', 'I'},
|
|
{i64(32768), 'l', 'u'},
|
|
{u64(32768), 'l', 'u'},
|
|
// uint16 (BJData only)
|
|
{i64(65534), 'l', 'u'},
|
|
{i64(65535), 'l', 'u'},
|
|
{u64(65535), 'l', 'u'},
|
|
{i64(65536), 'l', 'l'},
|
|
{u64(65536), 'l', 'l'},
|
|
// int32
|
|
{i64(-2147483649LL), 'L', 'L'},
|
|
{i64(-2147483648LL), 'l', 'l'},
|
|
{i64(-2147483647LL), 'l', 'l'},
|
|
{i64(2147483646LL), 'l', 'l'},
|
|
{i64(2147483647LL), 'l', 'l'},
|
|
{u64(2147483647ULL), 'l', 'l'},
|
|
{i64(2147483648LL), 'L', 'm'},
|
|
{u64(2147483648ULL), 'L', 'm'},
|
|
// uint32 (BJData only)
|
|
{i64(4294967294LL), 'L', 'm'},
|
|
{i64(4294967295LL), 'L', 'm'},
|
|
{u64(4294967295ULL), 'L', 'm'},
|
|
{i64(4294967296LL), 'L', 'L'},
|
|
{u64(4294967296ULL), 'L', 'L'},
|
|
// int64
|
|
{i64(int64_min), 'L', 'L'},
|
|
{i64(int64_min + 1), 'L', 'L'},
|
|
{i64(int64_max - 1), 'L', 'L'},
|
|
{i64(int64_max), 'L', 'L'},
|
|
{u64(static_cast<std::uint64_t>(int64_max)), 'L', 'L'},
|
|
// uint64 (BJData only; UBJSON writes a high-precision number)
|
|
{u64(static_cast<std::uint64_t>(int64_max) + 1), 'H', 'M'},
|
|
{u64(uint64_max - 1), 'H', 'M'},
|
|
{u64(uint64_max), 'H', 'M'},
|
|
};
|
|
|
|
for (const auto& c : cases)
|
|
{
|
|
for (const bool bjdata :
|
|
{
|
|
false, true
|
|
})
|
|
{
|
|
const char marker = bjdata ? c.bjdata : c.ubjson;
|
|
const std::vector<std::uint8_t> payload = integer_payload(marker, c.value, bjdata);
|
|
const auto to_binary = [bjdata](const json & j, const bool use_size, const bool use_type)
|
|
{
|
|
return bjdata ? json::to_bjdata(j, use_size, use_type) : json::to_ubjson(j, use_size, use_type);
|
|
};
|
|
const auto from_binary = [bjdata](const std::vector<std::uint8_t>& v)
|
|
{
|
|
return bjdata ? json::from_bjdata(v) : json::from_ubjson(v);
|
|
};
|
|
INFO("value = " << c.value.dump() << (c.value.is_number_unsigned() ? " (unsigned)" : "") << ", format = " << (bjdata ? "BJData" : "UBJSON"));
|
|
|
|
// scalar
|
|
std::vector<std::uint8_t> expected = {static_cast<std::uint8_t>(marker)};
|
|
expected.insert(expected.end(), payload.begin(), payload.end());
|
|
for (const bool use_size :
|
|
{
|
|
false, true
|
|
})
|
|
{
|
|
CHECK(to_binary(c.value, use_size, false) == expected);
|
|
}
|
|
CHECK(from_binary(expected) == c.value);
|
|
|
|
const json arr = {c.value, c.value, c.value};
|
|
|
|
// array without count or type: every value has its marker
|
|
expected = {'['};
|
|
for (int i = 0; i < 3; ++i)
|
|
{
|
|
expected.push_back(static_cast<std::uint8_t>(marker));
|
|
expected.insert(expected.end(), payload.begin(), payload.end());
|
|
}
|
|
expected.push_back(']');
|
|
CHECK(to_binary(arr, false, false) == expected);
|
|
CHECK(from_binary(expected) == arr);
|
|
|
|
// array with count: every value has its marker
|
|
expected = {'[', '#', 'i', 3};
|
|
for (int i = 0; i < 3; ++i)
|
|
{
|
|
expected.push_back(static_cast<std::uint8_t>(marker));
|
|
expected.insert(expected.end(), payload.begin(), payload.end());
|
|
}
|
|
CHECK(to_binary(arr, true, false) == expected);
|
|
CHECK(from_binary(expected) == arr);
|
|
|
|
// array with type and count: the marker once, then the payloads
|
|
expected = {'[', '$', static_cast<std::uint8_t>(marker), '#', 'i', 3};
|
|
for (int i = 0; i < 3; ++i)
|
|
{
|
|
expected.insert(expected.end(), payload.begin(), payload.end());
|
|
}
|
|
CHECK(to_binary(arr, true, true) == expected);
|
|
CHECK(from_binary(expected) == arr);
|
|
|
|
// object with type and count: the marker once, then key and payload
|
|
const json obj = {{"a", c.value}, {"b", c.value}};
|
|
expected = {'{', '$', static_cast<std::uint8_t>(marker), '#', 'i', 2};
|
|
for (const char key :
|
|
{'a', 'b'
|
|
})
|
|
{
|
|
expected.push_back('i');
|
|
expected.push_back(1);
|
|
expected.push_back(static_cast<std::uint8_t>(key));
|
|
expected.insert(expected.end(), payload.begin(), payload.end());
|
|
}
|
|
CHECK(to_binary(obj, true, true) == expected);
|
|
CHECK(from_binary(expected) == obj);
|
|
}
|
|
}
|
|
}
|