mirror of
https://github.com/nlohmann/json.git
synced 2026-11-15 20:16:07 +00:00
* Bound the total number of valueless UBJSON elements per input An optimized UBJSON array of type Z, T or F has no payload, so its declared count is not bounded by the input. The limit of 1048576 applied to each array on its own, so 9 bytes (`[$Z#l` and a count) produced 1048576 values and every further 9 bytes another 1048576: 362 bytes allocated 661 MB in a DOM, and a SAX handler received 1M events per 9 bytes. The reader now adds the counts of all such arrays of one input and rejects the input with out_of_range.408 when the sum exceeds 1048576 plus 4 for every byte read so far. The base is the per-array limit, so an array that was accepted on its own before is still accepted at the start of an input. These arrays add about 80 bytes of memory per input byte (measured 77; one `Z` per byte needs 32), plus the base; the memory of a document as a whole is not bounded by this. The check runs once per valueless array, before its start event, with plain size_t arithmetic that cannot overflow for a 32-bit size_t. Containers with a payload type and objects (every member needs a key) are bounded by the input already and are unchanged. BJData rejects these types. to_ubjson counts the elements of the typed arrays it writes in one call and writes an array in the unoptimized form (`[#n Z Z ...`) once it would bring the total above 1048576, which a reader accepts at any input size. The check is made only for a typed array of a valueless type, so BJData and other arrays do not pay for it. Everything to_ubjson writes in version 3.13.0 can therefore be read back. Two consequences: - to_ubjson output changes for values with more than 1048576 elements in arrays of type Z, T or F: such an array is written at one byte per element, e.g. 270000 rows of 100 nulls take 26.8 MB instead of 1.35 MB. - Output of version 3.12.0 and earlier (typed arrays without such a total) is rejected by from_ubjson when the arrays hold more than 1048576 elements plus 4 per byte of input. 3.12.0 writes a row of up to 255 nulls in 5 bytes (6 above 255), so a table is rejected from 1075 rows of 1000 nulls or 209723 rows of 25 (accepted: 1000 rows of 1000, 209000 rows of 25); rows of 20 or fewer are never rejected. Measured on the test data: no file of json_test_data (converted with to_ubjson in all three modes, 846 round trips) holds an array of this kind with more than 0.17 values per byte. All of them are read back. Reproducer (K arrays of 1048576 nulls), Apple clang -O2: K=40 (362 bytes): before 41943040 values, 664 MB; after rejected, 33 MB K=2 (20 bytes): before 2097152 values, 53 MB; after rejected, 33 MB The new tests count events with a SAX handler where they can: building and comparing million-element values made the first version take 176 s under Valgrind, the current one about 22 s. Behaviour change: an input that holds more than 1048576 elements in arrays of type Z, T or F at more than 4 per byte is now rejected. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Report an empty array for UBJSON `[$N#<n>` instead of n elements A no-op (N) is not a value, so an optimized array of type N holds no element whatever count it declares. The reader skipped the elements but still passed the declared count to the SAX parser: a handler was told start_array(n) and no element followed, and the DOM parser reserved min(n, 16384) values (256 KB) for an array that stays empty. Nine bytes (`[$N#l 0x7FFFFFFF`) therefore committed 256 KB: 2000 of them in one input, 18 KB, committed 512 MB of heap (capacity 163,840,000 values for the 10000 arrays of the original probe: 2.6 GB for 70 KB of input). Untouched pages keep the resident size small, but the commit charge and a 32-bit address space are consumed. The reader now announces 0 elements for an optimized array of type N. The loop that skips the elements is unchanged (it never ran). A count that does not fit into std::size_t is still rejected by the count reader; a count that is too large for the array type of the DOM (before: out_of_range.408 "excessive array size" from the DOM handler) is accepted now, because the array is empty. Behaviour change: sax_parse reports start_array(0) for `[$N#n` (before: n); from_ubjson no longer throws out_of_range.408 for an N-typed array whose count exceeds the maximum size of the array type. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Bound what the DOM parsers reserve for arrays that announce a size Cause: #5476 made the DOM parsers reserve storage (up to 16384 elements) for a counted array of CBOR, MessagePack, UBJSON and BJData when it opens, before any input backs the count. The arrays that are open at the same time keep their storage until the parse ends, so N nested headers that announce 16384 elements and are never completed hold N * 16384 * sizeof(json): 100,000 nested CBOR headers `99 40 00` (300,000 bytes) peaked at 1.65 GiB, 3.12.0 at 12 MiB; the same for MessagePack `DC 40 00`, UBJSON `[#I 16384` and BJData. Fix: json_sax_dom_error_state, the base of the three DOM parsers, keeps a credit of elements that may still be reserved: 65,536 at the start plus 2 for every byte the reader has read. An array that announces no more than the credit left is reserved as before (one comparison and one subtraction per counted array); a larger one is reserved what is left and grows as its elements arrive. binary_reader tells a DOM parser (a handler whose pointer converts to json_sax_dom_error_state*; a handler that derives from one privately or protectedly, or any other SAX handler, is not told anything and reserves as in the base of this change) where to find its count of characters read for the duration of sax_parse(). An RAII guard puts back the previous binding on every exit, also by exception, so the parser never keeps a pointer to a reader that is gone (unbound, only the per-array cap of 16384 applies); the public SAX interface is unchanged, and start_array() still reports the announced number. A bound per array by the remaining input would not help, since the same bytes would back every open level; the bound has to be over all arrays. The elements of an array take at least one byte of input each, so the credit is more than honest input needs: none of the 19 inputs measured (canada, citm_catalog, twitter, mesh, update-center, random, numbers, animation, ik_rig, instruments, geo_response, jeopardy and 7 synthetic ones, each as CBOR, MessagePack, UBJSON and BJData, with and without size/type optimization) was given less than it announced. A value that nests five or more arrays of 16384 elements, each with an array as its first element, is the exception: the later ones grow as before. Measurements (clang -O2, macOS arm64, peak memory footprint of `/usr/bin/time -l`, reproducer: n nested headers): n = 20,000 n = 100,000 develop fix 3.12.0 develop fix 3.12.0 CBOR 338.7 5.9 8.2 1687.7 23.3 11.8 MiB MessagePack 338.9 6.0 7.3 1687.9 23.8 11.9 MiB UBJSON 339.0 7.6 8.0 1687.9 30.1 11.6 MiB BJData 339.0 7.3 7.9 1688.0 30.8 11.6 MiB That is at most (65,536 + 2 per input byte) values: 1 MiB plus 32 bytes per input byte with 16-byte values (a value is 24 bytes with JSON_DIAGNOSTICS and up to 40 with JSON_DIAGNOSTIC_POSITIONS), as documented in untrusted_input.md. Lower fixed caps do not get there: a cap of 64 elements still peaks at 113 MiB (n = 100,000) and costs 4.7 % on arrays of 300 elements (mat300.cbor) and 5.0 % on arrays of 10,000 (arrays_of_big.cbor). Reader instructions retired (min of 7, 108 rows: 18 inputs in the 6 forms CBOR, MessagePack, UBJSON and BJData plain and optimized; jeopardy, 46 MB, is within 0.1 %), compared with the base of this change: -0.19 % to +0.37 % in the formats that announce sizes (canada.cbor 81.9 M -> 82.1 M, many_small.cbor 333.5 M -> 334.2 M); the benefit of #5476 is kept (without any reserve canada.cbor takes 116.8 M, +42.6 %, many_small.cbor +71.5 %, mat300.cbor +81.6 %). Rows of plain UBJSON and BJData, whose arrays announce no size, move by -2.7 % to +2.6 % without any code of theirs changing: the base alternates between two instruction counts per process (452 M and 463 M for many_pairs.ubjson, equal in cycles), and a struct with three unused members added to the same class reproduces the 474 M mode. Tests (CBOR, MessagePack, UBJSON, BJData): 200 nested headers that announce 16384 elements peak at 52,433,600 bytes of DOM allocations on the base and below 128 Ki values with the fix (scaled by sizeof(value), so that it holds with the diagnostic macros) (counting allocator, no process memory measured); an array that completes is still reserved in one allocation; CBOR also checks that a DOM parser is not bound to a finished parse (ASan: stack-use-after-scope without the guard), that wrappers derived publicly (also with a parse inside the parse), privately and protectedly compile and are bound or not as described, that an array that got less than it announced decodes and that a SAX handler is told the announced size. The new tests take 4.6 s under Valgrind at -O0 (gcc:12, all four files). Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Clarify the reservation note for announced array counts Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Report an empty array for UBJSON [$N#<n>] instead of n elements (#6079) * Report an empty array for UBJSON `[$N#<n>` instead of n elements A no-op (N) is not a value, so an optimized array of type N holds no element whatever count it declares. The reader skipped the elements but still passed the declared count to the SAX parser: a handler was told start_array(n) and no element followed, and the DOM parser reserved min(n, 16384) values (256 KB) for an array that stays empty. Nine bytes (`[$N#l 0x7FFFFFFF`) therefore committed 256 KB: 2000 of them in one input, 18 KB, committed 512 MB of heap (capacity 163,840,000 values for the 10000 arrays of the original probe: 2.6 GB for 70 KB of input). Untouched pages keep the resident size small, but the commit charge and a 32-bit address space are consumed. The reader now announces 0 elements for an optimized array of type N. The loop that skips the elements is unchanged (it never ran). A count that does not fit into std::size_t is still rejected by the count reader; a count that is too large for the array type of the DOM (before: out_of_range.408 "excessive array size" from the DOM handler) is accepted now, because the array is empty. Behaviour change: sax_parse reports start_array(0) for `[$N#n` (before: n); from_ubjson no longer throws out_of_range.408 for an N-typed array whose count exceeds the maximum size of the array type. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Clarify the reservation note for announced array counts Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Add JSON_UBJSON_VALUELESS_ELEMENTS_PER_BYTE to set the UBJSON whole-input budget (#6083) * Add JSON_UBJSON_VALUELESS_ELEMENTS_PER_BYTE to set the UBJSON whole-input budget The whole-input budget for the elements of optimized UBJSON arrays of type Z, T and F (1048576 plus 4 per byte read) rejects files written by 3.12.0 that have long arrays of these types, such as 1100 rows of 1000 nulls with a `$Z` annotation, and there was no way to read them. The per-byte rate is now the macro JSON_UBJSON_VALUELESS_ELEMENTS_PER_BYTE. Semantics: - default 4, so the default build is unchanged (the generated code of from_ubjson is identical at -O2 up to source line numbers); - the value must be a positive integer: 0, negative, non-integral, bool, character and wider-than-uintmax_t values fail to compile with a static_assert that names the macro; - a value that does not fit into std::size_t is the largest size_t, so one spelling works for 32- and 64-bit targets; - the limit base + rate * bytes_read saturates at the largest size_t instead of wrapping around, for any rate and input size; a rate of 131072 (2^20 / 8) or more cannot be exceeded by any input (the shortest header of an array of more than 32767 elements is 8 bytes), which switches the whole-input budget off; - the limit of one array (1048576) and the base (1048576) stay fixed constants, so the writer fallback still follows the base only and a reader with any rate reads everything the writer produces; - the macro is not part of the ABI tag and must have the same value in every translation unit; it is documented that way. The memory the arrays can take in a DOM is about 20 bytes per rate and byte of input (measured, macOS, containers of 9 * rate elements: 25 bytes per input byte at rate 1, 81 at 4, 312 at 16, 1226 at 64, 5493 at 256) plus about 33 MB for the base. A 3.12.0 row of N values has a header of b bytes (5 up to 255, 6 up to 32767, 8 above), so a table is read completely if N <= rate * b, and otherwise up to (1048576 + h * rate, h = outer header: 6, 7 or 9 bytes) / (N - rate * b) rows; 1100 rows of 1000 nulls need a rate of 8. The new page of the macro documents the formulas, the measurements, and two examples that are run by the documentation checks. The tests of the budget move to tests/src/unit-ubjson_valueless.cpp, which derives every boundary from the macro and is built with the rates 1, 1000000 and 0xFFFFFFFFFFFFFFFF in addition to the default (the last two check that nothing is refused and that the limit does not wrap around); they fail with the unchanged header for rate 1, 1000000 and the maximum. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Refuse char8_t for JSON_UBJSON_VALUELESS_ELEMENTS_PER_BYTE and clarify its documentation Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Use fseek instead of rewind in the UBJSON FILE* test clang-tidy (bugprone-unsafe-functions) rejects rewind() because it has no error detection. Check the result of fseek() instead. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Charge an announced array only what is reserved for it An array that announced more than reserve_cap() elements but no more than the credit left was charged the announced count, while reserve_array() reserved at most reserve_cap() of them. An outer array of 20000 elements therefore left a third nested array of 16384 with 12796 instead of the full reservation. reserve_count() now caps the count first and charges that, and the DOM parsers reserve it through reserve_array_capped(), which does not cap again, so the fast path costs the same as before (instructions retired within +-0.06 % on canada, citm_catalog and twitter in CBOR, MessagePack and UBJSON). Also re-wrap the comment of valueless_elements_base. Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me>
4929 lines
252 KiB
C++
4929 lines
252 KiB
C++
// __ _____ _____ _____
|
||
// __| | __| | | | JSON for Modern C++ (supporting code)
|
||
// | | |__ | | | | | | version 3.12.0
|
||
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
|
||
//
|
||
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
|
||
// SPDX-License-Identifier: MIT
|
||
|
||
#include "doctest_compatibility.h"
|
||
|
||
#define JSON_TESTS_PRIVATE
|
||
// capture whether JSON_DELETE_DEPRECATED_FUNCTIONS was enabled on the command
|
||
// line *before* including json.hpp, since the library #undefs it once the header
|
||
// has been fully processed (see include/nlohmann/detail/macro_unscope.hpp); the
|
||
// tests of deprecated functions are skipped if these functions are deleted
|
||
#if defined(JSON_DELETE_DEPRECATED_FUNCTIONS) && (JSON_DELETE_DEPRECATED_FUNCTIONS == 1)
|
||
#define JSON_TEST_DEPRECATED_FUNCTIONS_DELETED
|
||
#endif
|
||
|
||
#include <nlohmann/json.hpp>
|
||
using nlohmann::json;
|
||
using nlohmann::ordered_json;
|
||
|
||
#include <algorithm>
|
||
#include <climits>
|
||
#include <limits>
|
||
#include <iostream>
|
||
#include <fstream>
|
||
#include <list>
|
||
#include <set>
|
||
#include "make_test_data_available.hpp"
|
||
#include "test_utils.hpp"
|
||
#include "sax_countdown.hpp"
|
||
#include "allocation_counter.hpp"
|
||
#include "sax_event_loggers.hpp"
|
||
using utils::SaxCountdown;
|
||
using utils::SaxEventLogger;
|
||
|
||
|
||
// trait_test_arg and the "value_in_range_of trait" TEST_CASE_TEMPLATE_DEFINE
|
||
// are shared with unit-32bit.cpp
|
||
#include "value_in_range_of_test.hpp"
|
||
|
||
// NOLINTNEXTLINE(bugprone-throwing-static-initialization)
|
||
TEST_CASE_TEMPLATE_INVOKE(value_in_range_of_test, \
|
||
trait_test_arg<std::int32_t, std::int32_t, true, true>, \
|
||
trait_test_arg<std::int32_t, std::uint32_t, true, false>, \
|
||
trait_test_arg<std::uint32_t, std::int32_t, false, true>, \
|
||
trait_test_arg<std::uint32_t, std::uint32_t, true, true>, \
|
||
trait_test_arg<std::int32_t, std::int64_t, false, false>, \
|
||
trait_test_arg<std::int32_t, std::uint64_t, true, false>, \
|
||
trait_test_arg<std::uint32_t, std::int64_t, false, false>, \
|
||
trait_test_arg<std::uint32_t, std::uint64_t, true, false>, \
|
||
trait_test_arg<std::int64_t, std::int32_t, true, true>, \
|
||
trait_test_arg<std::int64_t, std::uint32_t, true, true>, \
|
||
trait_test_arg<std::uint64_t, std::int32_t, false, true>, \
|
||
trait_test_arg<std::uint64_t, std::uint32_t, true, true>, \
|
||
trait_test_arg<std::int64_t, std::int64_t, true, true>, \
|
||
trait_test_arg<std::int64_t, std::uint64_t, true, false>, \
|
||
trait_test_arg<std::uint64_t, std::int64_t, false, true>, \
|
||
trait_test_arg<std::uint64_t, std::uint64_t, true, true>);
|
||
|
||
#if SIZE_MAX == 0xffffffff
|
||
TEST_CASE_TEMPLATE_INVOKE(value_in_range_of_test, \
|
||
trait_test_arg<std::size_t, std::int32_t, false, true>, \
|
||
trait_test_arg<std::size_t, std::uint32_t, true, true>, \
|
||
trait_test_arg<std::size_t, std::int64_t, false, false>, \
|
||
trait_test_arg<std::size_t, std::uint64_t, true, false>);
|
||
#else
|
||
// NOLINTNEXTLINE(bugprone-throwing-static-initialization)
|
||
TEST_CASE_TEMPLATE_INVOKE(value_in_range_of_test, \
|
||
trait_test_arg<std::size_t, std::int32_t, false, true>, \
|
||
trait_test_arg<std::size_t, std::uint32_t, true, true>, \
|
||
trait_test_arg<std::size_t, std::int64_t, false, true>, \
|
||
trait_test_arg<std::size_t, std::uint64_t, true, true>);
|
||
#endif
|
||
|
||
TEST_CASE("BJData")
|
||
{
|
||
SECTION("binary_reader BJData lookup tables")
|
||
{
|
||
// bjd_type_name() is static; is_valid_optimized_type() needs a BJData reader
|
||
std::vector<std::uint8_t> const data;
|
||
using reader_t = nlohmann::detail::binary_reader<json, decltype(nlohmann::detail::input_adapter(data))>;
|
||
|
||
// the markers BJData excludes as optimized types must match
|
||
// binary_writer's is_bjdata_excluded_type_marker(), which encodes the
|
||
// same 8 markers
|
||
const reader_t reader(nlohmann::detail::input_adapter(data), json::input_format_t::bjdata);
|
||
for (const char marker :
|
||
{'[', '{', 'S', 'H', 'T', 'F', 'N', 'Z', 'x'
|
||
})
|
||
{
|
||
CHECK(!reader.is_valid_optimized_type(static_cast<unsigned char>(marker)));
|
||
}
|
||
for (const char marker :
|
||
{'U', 'i', 'u', 'I', 'm', 'l', 'M', 'L', 'd', 'D', 'C', 'B', 'h'
|
||
})
|
||
{
|
||
CHECK(reader.is_valid_optimized_type(static_cast<unsigned char>(marker)));
|
||
}
|
||
|
||
// every dtype marker must round-trip to its ND-array type name
|
||
const std::vector<std::pair<char, std::string>> types
|
||
{
|
||
{'B', "byte"}, {'C', "char"}, {'D', "double"}, {'I', "int16"},
|
||
{'L', "int64"}, {'M', "uint64"}, {'U', "uint8"}, {'d', "single"},
|
||
{'i', "int8"}, {'l', "int32"}, {'m', "uint32"}, {'u', "uint16"}
|
||
};
|
||
for (const auto& type : types)
|
||
{
|
||
const char* name = reader_t::bjd_type_name(static_cast<unsigned char>(type.first));
|
||
REQUIRE(name != nullptr);
|
||
CHECK(std::string(name) == type.second);
|
||
}
|
||
CHECK(reader_t::bjd_type_name(static_cast<unsigned char>('x')) == nullptr);
|
||
}
|
||
|
||
SECTION("individual values")
|
||
{
|
||
SECTION("discarded")
|
||
{
|
||
// a discarded value cannot be serialized to BJData
|
||
json const j = json::value_t::discarded;
|
||
CHECK_THROWS_WITH_AS(json::to_bjdata(j), "[json.exception.type_error.321] cannot serialize discarded value to BJData", json::type_error&);
|
||
}
|
||
|
||
SECTION("discarded values nested in a container")
|
||
{
|
||
json const discarded = json::value_t::discarded;
|
||
|
||
SECTION("in an array")
|
||
{
|
||
json const j = {1, discarded, 2};
|
||
#if JSON_DIAGNOSTICS
|
||
CHECK_THROWS_WITH_AS(json::to_bjdata(j), "[json.exception.type_error.321] (/1) cannot serialize discarded value to BJData", json::type_error&);
|
||
#else
|
||
CHECK_THROWS_WITH_AS(json::to_bjdata(j), "[json.exception.type_error.321] cannot serialize discarded value to BJData", json::type_error&);
|
||
#endif
|
||
}
|
||
|
||
SECTION("as an object value")
|
||
{
|
||
json j;
|
||
j["a"] = 1;
|
||
j["b"] = discarded;
|
||
#if JSON_DIAGNOSTICS
|
||
CHECK_THROWS_WITH_AS(json::to_bjdata(j), "[json.exception.type_error.321] (/b) cannot serialize discarded value to BJData", json::type_error&);
|
||
#else
|
||
CHECK_THROWS_WITH_AS(json::to_bjdata(j), "[json.exception.type_error.321] cannot serialize discarded value to BJData", json::type_error&);
|
||
#endif
|
||
}
|
||
|
||
SECTION("nested deeper (array in object in array)")
|
||
{
|
||
json inner_array = {1, discarded};
|
||
json middle_object;
|
||
middle_object["x"] = inner_array;
|
||
json const j = {middle_object};
|
||
#if JSON_DIAGNOSTICS
|
||
CHECK_THROWS_WITH_AS(json::to_bjdata(j), "[json.exception.type_error.321] (/0/x/1) cannot serialize discarded value to BJData", json::type_error&);
|
||
#else
|
||
CHECK_THROWS_WITH_AS(json::to_bjdata(j), "[json.exception.type_error.321] cannot serialize discarded value to BJData", json::type_error&);
|
||
#endif
|
||
}
|
||
|
||
SECTION("optimized array of all-discarded elements")
|
||
{
|
||
json const j = {discarded, discarded};
|
||
#if JSON_DIAGNOSTICS
|
||
CHECK_THROWS_WITH_AS(json::to_bjdata(j, true, true), "[json.exception.type_error.321] (/0) cannot serialize discarded value to BJData", json::type_error&);
|
||
#else
|
||
CHECK_THROWS_WITH_AS(json::to_bjdata(j, true, true), "[json.exception.type_error.321] cannot serialize discarded value to BJData", json::type_error&);
|
||
#endif
|
||
}
|
||
}
|
||
|
||
SECTION("null")
|
||
{
|
||
json const j = nullptr;
|
||
std::vector<uint8_t> const expected = {'Z'};
|
||
const auto result = json::to_bjdata(j);
|
||
CHECK(result == expected);
|
||
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
|
||
SECTION("boolean")
|
||
{
|
||
SECTION("true")
|
||
{
|
||
json const j = true;
|
||
std::vector<uint8_t> const expected = {'T'};
|
||
const auto result = json::to_bjdata(j);
|
||
CHECK(result == expected);
|
||
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
|
||
SECTION("false")
|
||
{
|
||
json const j = false;
|
||
std::vector<uint8_t> const expected = {'F'};
|
||
const auto result = json::to_bjdata(j);
|
||
CHECK(result == expected);
|
||
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
}
|
||
|
||
SECTION("byte")
|
||
{
|
||
SECTION("0..255 (uint8)")
|
||
{
|
||
for (size_t i = 0; i <= 255; ++i)
|
||
{
|
||
CAPTURE(i)
|
||
|
||
// create JSON value with integer number (no byte type in JSON)
|
||
json j = -1;
|
||
j.get_ref<json::number_integer_t&>() = static_cast<json::number_integer_t>(i);
|
||
|
||
// check type
|
||
CHECK(j.is_number_integer());
|
||
|
||
// create byte vector
|
||
std::vector<uint8_t> const value
|
||
{
|
||
static_cast<uint8_t>('B'),
|
||
static_cast<uint8_t>(i),
|
||
};
|
||
|
||
// compare value
|
||
CHECK(json::from_bjdata(value) == j);
|
||
}
|
||
}
|
||
}
|
||
|
||
SECTION("number")
|
||
{
|
||
SECTION("signed")
|
||
{
|
||
SECTION("-9223372036854775808..-2147483649 (int64)")
|
||
{
|
||
std::vector<int64_t> const numbers
|
||
{
|
||
(std::numeric_limits<int64_t>::min)(),
|
||
-1000000000000000000LL,
|
||
-100000000000000000LL,
|
||
-10000000000000000LL,
|
||
-1000000000000000LL,
|
||
-100000000000000LL,
|
||
-10000000000000LL,
|
||
-1000000000000LL,
|
||
-100000000000LL,
|
||
-10000000000LL,
|
||
-2147483649LL,
|
||
};
|
||
for (const auto i : numbers)
|
||
{
|
||
CAPTURE(i)
|
||
|
||
// create JSON value with integer number
|
||
json const j = i;
|
||
|
||
// check type
|
||
CHECK(j.is_number_integer());
|
||
|
||
// create expected byte vector
|
||
std::vector<uint8_t> const expected
|
||
{
|
||
static_cast<uint8_t>('L'),
|
||
static_cast<uint8_t>(i & 0xff),
|
||
static_cast<uint8_t>((i >> 8) & 0xff),
|
||
static_cast<uint8_t>((i >> 16) & 0xff),
|
||
static_cast<uint8_t>((i >> 24) & 0xff),
|
||
static_cast<uint8_t>((i >> 32) & 0xff),
|
||
static_cast<uint8_t>((i >> 40) & 0xff),
|
||
static_cast<uint8_t>((i >> 48) & 0xff),
|
||
static_cast<uint8_t>((i >> 56) & 0xff),
|
||
};
|
||
|
||
// compare result + size
|
||
const auto result = json::to_bjdata(j);
|
||
CHECK(result == expected);
|
||
CHECK(result.size() == 9);
|
||
|
||
// check individual bytes
|
||
CHECK(result[0] == 'L');
|
||
int64_t const restored = (static_cast<int64_t>(result[8]) << 070) +
|
||
(static_cast<int64_t>(result[7]) << 060) +
|
||
(static_cast<int64_t>(result[6]) << 050) +
|
||
(static_cast<int64_t>(result[5]) << 040) +
|
||
(static_cast<int64_t>(result[4]) << 030) +
|
||
(static_cast<int64_t>(result[3]) << 020) +
|
||
(static_cast<int64_t>(result[2]) << 010) +
|
||
static_cast<int64_t>(result[1]);
|
||
CHECK(restored == i);
|
||
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
}
|
||
|
||
SECTION("-2147483648..-32769 (int32)")
|
||
{
|
||
std::vector<int32_t> const numbers
|
||
{
|
||
-32769,
|
||
-100000,
|
||
-1000000,
|
||
-10000000,
|
||
-100000000,
|
||
-1000000000,
|
||
-2147483647 - 1, // https://stackoverflow.com/a/29356002/266378
|
||
};
|
||
for (const auto i : numbers)
|
||
{
|
||
CAPTURE(i)
|
||
|
||
// create JSON value with integer number
|
||
json const j = i;
|
||
|
||
// check type
|
||
CHECK(j.is_number_integer());
|
||
|
||
// create expected byte vector
|
||
std::vector<uint8_t> const expected
|
||
{
|
||
static_cast<uint8_t>('l'),
|
||
static_cast<uint8_t>(i & 0xff),
|
||
static_cast<uint8_t>((i >> 8) & 0xff),
|
||
static_cast<uint8_t>((i >> 16) & 0xff),
|
||
static_cast<uint8_t>((i >> 24) & 0xff),
|
||
};
|
||
|
||
// compare result + size
|
||
const auto result = json::to_bjdata(j);
|
||
CHECK(result == expected);
|
||
CHECK(result.size() == 5);
|
||
|
||
// check individual bytes
|
||
CHECK(result[0] == 'l');
|
||
int32_t const restored = (static_cast<int32_t>(result[4]) << 030) +
|
||
(static_cast<int32_t>(result[3]) << 020) +
|
||
(static_cast<int32_t>(result[2]) << 010) +
|
||
static_cast<int32_t>(result[1]);
|
||
CHECK(restored == i);
|
||
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
}
|
||
|
||
SECTION("-32768..-129 (int16)")
|
||
{
|
||
for (int32_t i = -32768; i <= -129; i = utils::next_integer_sample(i, -129, 7))
|
||
{
|
||
CAPTURE(i)
|
||
|
||
// create JSON value with integer number
|
||
json const j = i;
|
||
|
||
// check type
|
||
CHECK(j.is_number_integer());
|
||
|
||
// create expected byte vector
|
||
std::vector<uint8_t> const expected
|
||
{
|
||
static_cast<uint8_t>('I'),
|
||
static_cast<uint8_t>(i & 0xff),
|
||
static_cast<uint8_t>((i >> 8) & 0xff),
|
||
};
|
||
|
||
// compare result + size
|
||
const auto result = json::to_bjdata(j);
|
||
CHECK(result == expected);
|
||
CHECK(result.size() == 3);
|
||
|
||
// check individual bytes
|
||
CHECK(result[0] == 'I');
|
||
auto const restored = static_cast<int16_t>(((result[2] << 8) + result[1]));
|
||
CHECK(restored == i);
|
||
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
}
|
||
|
||
SECTION("-9263 (int16)")
|
||
{
|
||
json const j = -9263;
|
||
std::vector<uint8_t> const expected = {'I', 0xd1, 0xdb};
|
||
|
||
// compare result + size
|
||
const auto result = json::to_bjdata(j);
|
||
CHECK(result == expected);
|
||
CHECK(result.size() == 3);
|
||
|
||
// check individual bytes
|
||
CHECK(result[0] == 'I');
|
||
auto const restored = static_cast<int16_t>(((result[2] << 8) + result[1]));
|
||
CHECK(restored == -9263);
|
||
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
|
||
SECTION("-128..-1 (int8)")
|
||
{
|
||
for (auto i = -128; i <= -1; ++i)
|
||
{
|
||
CAPTURE(i)
|
||
|
||
// create JSON value with integer number
|
||
json const j = i;
|
||
|
||
// check type
|
||
CHECK(j.is_number_integer());
|
||
|
||
// create expected byte vector
|
||
std::vector<uint8_t> const expected
|
||
{
|
||
'i',
|
||
static_cast<uint8_t>(i),
|
||
};
|
||
|
||
// compare result + size
|
||
const auto result = json::to_bjdata(j);
|
||
CHECK(result == expected);
|
||
CHECK(result.size() == 2);
|
||
|
||
// check individual bytes
|
||
CHECK(result[0] == 'i');
|
||
CHECK(static_cast<int8_t>(result[1]) == i);
|
||
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
}
|
||
|
||
SECTION("0..127 (int8)")
|
||
{
|
||
for (size_t i = 0; i <= 127; ++i)
|
||
{
|
||
CAPTURE(i)
|
||
|
||
// create JSON value with integer number
|
||
json j = -1;
|
||
j.get_ref<json::number_integer_t&>() = static_cast<json::number_integer_t>(i);
|
||
|
||
// check type
|
||
CHECK(j.is_number_integer());
|
||
|
||
// create expected byte vector
|
||
std::vector<uint8_t> const expected
|
||
{
|
||
static_cast<uint8_t>('i'),
|
||
static_cast<uint8_t>(i),
|
||
};
|
||
|
||
// compare result + size
|
||
const auto result = json::to_bjdata(j);
|
||
CHECK(result == expected);
|
||
CHECK(result.size() == 2);
|
||
|
||
// check individual bytes
|
||
CHECK(result[0] == 'i');
|
||
CHECK(result[1] == i);
|
||
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
}
|
||
|
||
SECTION("128..255 (uint8)")
|
||
{
|
||
for (size_t i = 128; i <= 255; ++i)
|
||
{
|
||
CAPTURE(i)
|
||
|
||
// create JSON value with integer number
|
||
json j = -1;
|
||
j.get_ref<json::number_integer_t&>() = static_cast<json::number_integer_t>(i);
|
||
|
||
// check type
|
||
CHECK(j.is_number_integer());
|
||
|
||
// create expected byte vector
|
||
std::vector<uint8_t> const expected
|
||
{
|
||
static_cast<uint8_t>('U'),
|
||
static_cast<uint8_t>(i),
|
||
};
|
||
|
||
// compare result + size
|
||
const auto result = json::to_bjdata(j);
|
||
CHECK(result == expected);
|
||
CHECK(result.size() == 2);
|
||
|
||
// check individual bytes
|
||
CHECK(result[0] == 'U');
|
||
CHECK(result[1] == i);
|
||
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
}
|
||
|
||
SECTION("256..32767 (int16)")
|
||
{
|
||
for (size_t i = 256; i <= 32767; i = utils::next_integer_sample(i, static_cast<size_t>(32767), static_cast<size_t>(7)))
|
||
{
|
||
CAPTURE(i)
|
||
|
||
// create JSON value with integer number
|
||
json j = -1;
|
||
j.get_ref<json::number_integer_t&>() = static_cast<json::number_integer_t>(i);
|
||
|
||
// check type
|
||
CHECK(j.is_number_integer());
|
||
|
||
// create expected byte vector
|
||
std::vector<uint8_t> const expected
|
||
{
|
||
static_cast<uint8_t>('I'),
|
||
static_cast<uint8_t>(i & 0xff),
|
||
static_cast<uint8_t>((i >> 8) & 0xff),
|
||
};
|
||
|
||
// compare result + size
|
||
const auto result = json::to_bjdata(j);
|
||
CHECK(result == expected);
|
||
CHECK(result.size() == 3);
|
||
|
||
// check individual bytes
|
||
CHECK(result[0] == 'I');
|
||
auto const restored = static_cast<uint16_t>((static_cast<uint8_t>(result[2]) * 256) + static_cast<uint8_t>(result[1]));
|
||
CHECK(restored == i);
|
||
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
}
|
||
|
||
SECTION("32768..65535 (uint16)")
|
||
{
|
||
for (const uint32_t i :
|
||
{
|
||
32768u, 55555u, 65535u
|
||
})
|
||
{
|
||
CAPTURE(i)
|
||
|
||
// create JSON value with integer number
|
||
json j = -1;
|
||
j.get_ref<json::number_integer_t&>() = static_cast<json::number_integer_t>(i);
|
||
|
||
// check type
|
||
CHECK(j.is_number_integer());
|
||
|
||
// create expected byte vector
|
||
std::vector<uint8_t> const expected
|
||
{
|
||
static_cast<uint8_t>('u'),
|
||
static_cast<uint8_t>(i & 0xff),
|
||
static_cast<uint8_t>((i >> 8) & 0xff),
|
||
};
|
||
|
||
// compare result + size
|
||
const auto result = json::to_bjdata(j);
|
||
CHECK(result == expected);
|
||
CHECK(result.size() == 3);
|
||
|
||
// check individual bytes
|
||
CHECK(result[0] == 'u');
|
||
auto const restored = static_cast<uint16_t>((static_cast<uint8_t>(result[2]) * 256) + static_cast<uint8_t>(result[1]));
|
||
CHECK(restored == i);
|
||
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
}
|
||
|
||
SECTION("65536..2147483647 (int32)")
|
||
{
|
||
for (const uint32_t i :
|
||
{
|
||
65536u, 77777u, 2147483647u
|
||
})
|
||
{
|
||
CAPTURE(i)
|
||
|
||
// create JSON value with integer number
|
||
json j = -1;
|
||
j.get_ref<json::number_integer_t&>() = static_cast<json::number_integer_t>(i);
|
||
|
||
// check type
|
||
CHECK(j.is_number_integer());
|
||
|
||
// create expected byte vector
|
||
std::vector<uint8_t> const expected
|
||
{
|
||
'l',
|
||
static_cast<uint8_t>(i & 0xff),
|
||
static_cast<uint8_t>((i >> 8) & 0xff),
|
||
static_cast<uint8_t>((i >> 16) & 0xff),
|
||
static_cast<uint8_t>((i >> 24) & 0xff),
|
||
};
|
||
|
||
// compare result + size
|
||
const auto result = json::to_bjdata(j);
|
||
CHECK(result == expected);
|
||
CHECK(result.size() == 5);
|
||
|
||
// check individual bytes
|
||
CHECK(result[0] == 'l');
|
||
uint32_t const restored = (static_cast<uint32_t>(result[4]) << 030) +
|
||
(static_cast<uint32_t>(result[3]) << 020) +
|
||
(static_cast<uint32_t>(result[2]) << 010) +
|
||
static_cast<uint32_t>(result[1]);
|
||
CHECK(restored == i);
|
||
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
}
|
||
|
||
SECTION("2147483648..4294967295 (uint32)")
|
||
{
|
||
for (const uint32_t i :
|
||
{
|
||
2147483648u, 3333333333u, 4294967295u
|
||
})
|
||
{
|
||
CAPTURE(i)
|
||
|
||
// create JSON value with integer number
|
||
json j = -1;
|
||
j.get_ref<json::number_integer_t&>() = static_cast<json::number_integer_t>(i);
|
||
|
||
// check type
|
||
CHECK(j.is_number_integer());
|
||
|
||
// create expected byte vector
|
||
std::vector<uint8_t> const expected
|
||
{
|
||
'm',
|
||
static_cast<uint8_t>(i & 0xff),
|
||
static_cast<uint8_t>((i >> 8) & 0xff),
|
||
static_cast<uint8_t>((i >> 16) & 0xff),
|
||
static_cast<uint8_t>((i >> 24) & 0xff),
|
||
};
|
||
|
||
// compare result + size
|
||
const auto result = json::to_bjdata(j);
|
||
CHECK(result == expected);
|
||
CHECK(result.size() == 5);
|
||
|
||
// check individual bytes
|
||
CHECK(result[0] == 'm');
|
||
uint32_t const restored = (static_cast<uint32_t>(result[4]) << 030) +
|
||
(static_cast<uint32_t>(result[3]) << 020) +
|
||
(static_cast<uint32_t>(result[2]) << 010) +
|
||
static_cast<uint32_t>(result[1]);
|
||
CHECK(restored == i);
|
||
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
}
|
||
|
||
SECTION("4294967296..9223372036854775807 (int64)")
|
||
{
|
||
std::vector<uint64_t> const v = {4294967296LU, 9223372036854775807LU};
|
||
for (const uint64_t i : v)
|
||
{
|
||
CAPTURE(i)
|
||
|
||
// create JSON value with integer number
|
||
json j = -1;
|
||
j.get_ref<json::number_integer_t&>() = static_cast<json::number_integer_t>(i);
|
||
|
||
// check type
|
||
CHECK(j.is_number_integer());
|
||
|
||
// create expected byte vector
|
||
std::vector<uint8_t> const expected
|
||
{
|
||
'L',
|
||
static_cast<uint8_t>(i & 0xff),
|
||
static_cast<uint8_t>((i >> 010) & 0xff),
|
||
static_cast<uint8_t>((i >> 020) & 0xff),
|
||
static_cast<uint8_t>((i >> 030) & 0xff),
|
||
static_cast<uint8_t>((i >> 040) & 0xff),
|
||
static_cast<uint8_t>((i >> 050) & 0xff),
|
||
static_cast<uint8_t>((i >> 060) & 0xff),
|
||
static_cast<uint8_t>((i >> 070) & 0xff),
|
||
};
|
||
|
||
// compare result + size
|
||
const auto result = json::to_bjdata(j);
|
||
CHECK(result == expected);
|
||
CHECK(result.size() == 9);
|
||
|
||
// check individual bytes
|
||
CHECK(result[0] == 'L');
|
||
uint64_t const restored = (static_cast<uint64_t>(result[8]) << 070) +
|
||
(static_cast<uint64_t>(result[7]) << 060) +
|
||
(static_cast<uint64_t>(result[6]) << 050) +
|
||
(static_cast<uint64_t>(result[5]) << 040) +
|
||
(static_cast<uint64_t>(result[4]) << 030) +
|
||
(static_cast<uint64_t>(result[3]) << 020) +
|
||
(static_cast<uint64_t>(result[2]) << 010) +
|
||
static_cast<uint64_t>(result[1]);
|
||
CHECK(restored == i);
|
||
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
}
|
||
|
||
SECTION("9223372036854775808..18446744073709551615 (uint64)")
|
||
{
|
||
std::vector<uint64_t> const v = {9223372036854775808ull, 18446744073709551615ull};
|
||
for (const uint64_t i : v)
|
||
{
|
||
CAPTURE(i)
|
||
|
||
// create JSON value with integer number
|
||
json const j = i;
|
||
|
||
// check type
|
||
CHECK(j.is_number_unsigned());
|
||
|
||
// create expected byte vector
|
||
std::vector<uint8_t> const expected
|
||
{
|
||
'M',
|
||
static_cast<uint8_t>(i & 0xff),
|
||
static_cast<uint8_t>((i >> 010) & 0xff),
|
||
static_cast<uint8_t>((i >> 020) & 0xff),
|
||
static_cast<uint8_t>((i >> 030) & 0xff),
|
||
static_cast<uint8_t>((i >> 040) & 0xff),
|
||
static_cast<uint8_t>((i >> 050) & 0xff),
|
||
static_cast<uint8_t>((i >> 060) & 0xff),
|
||
static_cast<uint8_t>((i >> 070) & 0xff),
|
||
};
|
||
|
||
// compare result + size
|
||
const auto result = json::to_bjdata(j);
|
||
CHECK(result == expected);
|
||
CHECK(result.size() == 9);
|
||
|
||
// check individual bytes
|
||
CHECK(result[0] == 'M');
|
||
uint64_t const restored = (static_cast<uint64_t>(result[8]) << 070) +
|
||
(static_cast<uint64_t>(result[7]) << 060) +
|
||
(static_cast<uint64_t>(result[6]) << 050) +
|
||
(static_cast<uint64_t>(result[5]) << 040) +
|
||
(static_cast<uint64_t>(result[4]) << 030) +
|
||
(static_cast<uint64_t>(result[3]) << 020) +
|
||
(static_cast<uint64_t>(result[2]) << 010) +
|
||
static_cast<uint64_t>(result[1]);
|
||
CHECK(restored == i);
|
||
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
}
|
||
}
|
||
|
||
SECTION("unsigned")
|
||
{
|
||
SECTION("0..127 (int8)")
|
||
{
|
||
for (size_t i = 0; i <= 127; ++i)
|
||
{
|
||
CAPTURE(i)
|
||
|
||
// create JSON value with unsigned integer number
|
||
json const j = i;
|
||
|
||
// check type
|
||
CHECK(j.is_number_unsigned());
|
||
|
||
// create expected byte vector
|
||
std::vector<uint8_t> const expected{'i', static_cast<uint8_t>(i)};
|
||
|
||
// compare result + size
|
||
const auto result = json::to_bjdata(j);
|
||
CHECK(result == expected);
|
||
CHECK(result.size() == 2);
|
||
|
||
// check individual bytes
|
||
CHECK(result[0] == 'i');
|
||
auto const restored = static_cast<uint8_t>(result[1]);
|
||
CHECK(restored == i);
|
||
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
}
|
||
|
||
SECTION("128..255 (uint8)")
|
||
{
|
||
for (size_t i = 128; i <= 255; ++i)
|
||
{
|
||
CAPTURE(i)
|
||
|
||
// create JSON value with unsigned integer number
|
||
json const j = i;
|
||
|
||
// check type
|
||
CHECK(j.is_number_unsigned());
|
||
|
||
// create expected byte vector
|
||
std::vector<uint8_t> const expected{'U', static_cast<uint8_t>(i)};
|
||
|
||
// compare result + size
|
||
const auto result = json::to_bjdata(j);
|
||
CHECK(result == expected);
|
||
CHECK(result.size() == 2);
|
||
|
||
// check individual bytes
|
||
CHECK(result[0] == 'U');
|
||
auto const restored = static_cast<uint8_t>(result[1]);
|
||
CHECK(restored == i);
|
||
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
}
|
||
|
||
SECTION("256..32767 (int16)")
|
||
{
|
||
for (size_t i = 256; i <= 32767; i = utils::next_integer_sample(i, static_cast<size_t>(32767), static_cast<size_t>(7)))
|
||
{
|
||
CAPTURE(i)
|
||
|
||
// create JSON value with unsigned integer number
|
||
json const j = i;
|
||
|
||
// check type
|
||
CHECK(j.is_number_unsigned());
|
||
|
||
// create expected byte vector
|
||
std::vector<uint8_t> const expected
|
||
{
|
||
'I',
|
||
static_cast<uint8_t>(i & 0xff),
|
||
static_cast<uint8_t>((i >> 8) & 0xff),
|
||
};
|
||
|
||
// compare result + size
|
||
const auto result = json::to_bjdata(j);
|
||
CHECK(result == expected);
|
||
CHECK(result.size() == 3);
|
||
|
||
// check individual bytes
|
||
CHECK(result[0] == 'I');
|
||
auto const restored = static_cast<uint16_t>((static_cast<uint8_t>(result[2]) * 256) + static_cast<uint8_t>(result[1]));
|
||
CHECK(restored == i);
|
||
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
}
|
||
|
||
SECTION("32768..65535 (uint16)")
|
||
{
|
||
for (const uint32_t i :
|
||
{
|
||
32768u, 55555u, 65535u
|
||
})
|
||
{
|
||
CAPTURE(i)
|
||
|
||
// create JSON value with unsigned integer number
|
||
json const j = i;
|
||
|
||
// check type
|
||
CHECK(j.is_number_unsigned());
|
||
|
||
// create expected byte vector
|
||
std::vector<uint8_t> const expected
|
||
{
|
||
'u',
|
||
static_cast<uint8_t>(i & 0xff),
|
||
static_cast<uint8_t>((i >> 8) & 0xff),
|
||
};
|
||
|
||
// compare result + size
|
||
const auto result = json::to_bjdata(j);
|
||
CHECK(result == expected);
|
||
CHECK(result.size() == 3);
|
||
|
||
// check individual bytes
|
||
CHECK(result[0] == 'u');
|
||
auto const restored = static_cast<uint16_t>((static_cast<uint8_t>(result[2]) * 256) + static_cast<uint8_t>(result[1]));
|
||
CHECK(restored == i);
|
||
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
}
|
||
SECTION("65536..2147483647 (int32)")
|
||
{
|
||
for (const uint32_t i :
|
||
{
|
||
65536u, 77777u, 2147483647u
|
||
})
|
||
{
|
||
CAPTURE(i)
|
||
|
||
// create JSON value with unsigned integer number
|
||
json const j = i;
|
||
|
||
// check type
|
||
CHECK(j.is_number_unsigned());
|
||
|
||
// create expected byte vector
|
||
std::vector<uint8_t> const expected
|
||
{
|
||
'l',
|
||
static_cast<uint8_t>(i & 0xff),
|
||
static_cast<uint8_t>((i >> 8) & 0xff),
|
||
static_cast<uint8_t>((i >> 16) & 0xff),
|
||
static_cast<uint8_t>((i >> 24) & 0xff),
|
||
};
|
||
|
||
// compare result + size
|
||
const auto result = json::to_bjdata(j);
|
||
CHECK(result == expected);
|
||
CHECK(result.size() == 5);
|
||
|
||
// check individual bytes
|
||
CHECK(result[0] == 'l');
|
||
uint32_t const restored = (static_cast<uint32_t>(result[4]) << 030) +
|
||
(static_cast<uint32_t>(result[3]) << 020) +
|
||
(static_cast<uint32_t>(result[2]) << 010) +
|
||
static_cast<uint32_t>(result[1]);
|
||
CHECK(restored == i);
|
||
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
}
|
||
|
||
SECTION("2147483648..4294967295 (uint32)")
|
||
{
|
||
for (const uint32_t i :
|
||
{
|
||
2147483648u, 3333333333u, 4294967295u
|
||
})
|
||
{
|
||
CAPTURE(i)
|
||
|
||
// create JSON value with unsigned integer number
|
||
json const j = i;
|
||
|
||
// check type
|
||
CHECK(j.is_number_unsigned());
|
||
|
||
// create expected byte vector
|
||
std::vector<uint8_t> const expected
|
||
{
|
||
'm',
|
||
static_cast<uint8_t>(i & 0xff),
|
||
static_cast<uint8_t>((i >> 8) & 0xff),
|
||
static_cast<uint8_t>((i >> 16) & 0xff),
|
||
static_cast<uint8_t>((i >> 24) & 0xff),
|
||
};
|
||
|
||
// compare result + size
|
||
const auto result = json::to_bjdata(j);
|
||
CHECK(result == expected);
|
||
CHECK(result.size() == 5);
|
||
|
||
// check individual bytes
|
||
CHECK(result[0] == 'm');
|
||
uint32_t const restored = (static_cast<uint32_t>(result[4]) << 030) +
|
||
(static_cast<uint32_t>(result[3]) << 020) +
|
||
(static_cast<uint32_t>(result[2]) << 010) +
|
||
static_cast<uint32_t>(result[1]);
|
||
CHECK(restored == i);
|
||
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
}
|
||
|
||
SECTION("4294967296..9223372036854775807 (int64)")
|
||
{
|
||
std::vector<uint64_t> const v = {4294967296ul, 9223372036854775807ul};
|
||
for (const uint64_t i : v)
|
||
{
|
||
CAPTURE(i)
|
||
|
||
// create JSON value with integer number
|
||
json const j = i;
|
||
|
||
// check type
|
||
CHECK(j.is_number_unsigned());
|
||
|
||
// create expected byte vector
|
||
std::vector<uint8_t> const expected
|
||
{
|
||
'L',
|
||
static_cast<uint8_t>(i & 0xff),
|
||
static_cast<uint8_t>((i >> 010) & 0xff),
|
||
static_cast<uint8_t>((i >> 020) & 0xff),
|
||
static_cast<uint8_t>((i >> 030) & 0xff),
|
||
static_cast<uint8_t>((i >> 040) & 0xff),
|
||
static_cast<uint8_t>((i >> 050) & 0xff),
|
||
static_cast<uint8_t>((i >> 060) & 0xff),
|
||
static_cast<uint8_t>((i >> 070) & 0xff),
|
||
};
|
||
|
||
// compare result + size
|
||
const auto result = json::to_bjdata(j);
|
||
CHECK(result == expected);
|
||
CHECK(result.size() == 9);
|
||
|
||
// check individual bytes
|
||
CHECK(result[0] == 'L');
|
||
uint64_t const restored = (static_cast<uint64_t>(result[8]) << 070) +
|
||
(static_cast<uint64_t>(result[7]) << 060) +
|
||
(static_cast<uint64_t>(result[6]) << 050) +
|
||
(static_cast<uint64_t>(result[5]) << 040) +
|
||
(static_cast<uint64_t>(result[4]) << 030) +
|
||
(static_cast<uint64_t>(result[3]) << 020) +
|
||
(static_cast<uint64_t>(result[2]) << 010) +
|
||
static_cast<uint64_t>(result[1]);
|
||
CHECK(restored == i);
|
||
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
}
|
||
|
||
SECTION("9223372036854775808..18446744073709551615 (uint64)")
|
||
{
|
||
std::vector<uint64_t> const v = {9223372036854775808ull, 18446744073709551615ull};
|
||
for (const uint64_t i : v)
|
||
{
|
||
CAPTURE(i)
|
||
|
||
// create JSON value with integer number
|
||
json const j = i;
|
||
|
||
// check type
|
||
CHECK(j.is_number_unsigned());
|
||
|
||
// create expected byte vector
|
||
std::vector<uint8_t> const expected
|
||
{
|
||
'M',
|
||
static_cast<uint8_t>(i & 0xff),
|
||
static_cast<uint8_t>((i >> 010) & 0xff),
|
||
static_cast<uint8_t>((i >> 020) & 0xff),
|
||
static_cast<uint8_t>((i >> 030) & 0xff),
|
||
static_cast<uint8_t>((i >> 040) & 0xff),
|
||
static_cast<uint8_t>((i >> 050) & 0xff),
|
||
static_cast<uint8_t>((i >> 060) & 0xff),
|
||
static_cast<uint8_t>((i >> 070) & 0xff),
|
||
};
|
||
|
||
// compare result + size
|
||
const auto result = json::to_bjdata(j);
|
||
CHECK(result == expected);
|
||
CHECK(result.size() == 9);
|
||
|
||
// check individual bytes
|
||
CHECK(result[0] == 'M');
|
||
uint64_t const restored = (static_cast<uint64_t>(result[8]) << 070) +
|
||
(static_cast<uint64_t>(result[7]) << 060) +
|
||
(static_cast<uint64_t>(result[6]) << 050) +
|
||
(static_cast<uint64_t>(result[5]) << 040) +
|
||
(static_cast<uint64_t>(result[4]) << 030) +
|
||
(static_cast<uint64_t>(result[3]) << 020) +
|
||
(static_cast<uint64_t>(result[2]) << 010) +
|
||
static_cast<uint64_t>(result[1]);
|
||
CHECK(restored == i);
|
||
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
}
|
||
}
|
||
SECTION("float64")
|
||
{
|
||
SECTION("3.1415925")
|
||
{
|
||
double v = 3.1415925;
|
||
json const j = v;
|
||
std::vector<uint8_t> const expected =
|
||
{
|
||
'D', 0xfc, 0xde, 0xa6, 0x3f, 0xfb, 0x21, 0x09, 0x40
|
||
};
|
||
const auto result = json::to_bjdata(j);
|
||
CHECK(result == expected);
|
||
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result) == v);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
}
|
||
|
||
SECTION("half-precision float")
|
||
{
|
||
SECTION("simple half floats")
|
||
{
|
||
CHECK(json::parse("0.0") == json::from_bjdata(std::vector<uint8_t>({'h', 0x00, 0x00})));
|
||
CHECK(json::parse("-0.0") == json::from_bjdata(std::vector<uint8_t>({'h', 0x00, 0x80})));
|
||
CHECK(json::parse("1.0") == json::from_bjdata(std::vector<uint8_t>({'h', 0x00, 0x3c})));
|
||
CHECK(json::parse("1.5") == json::from_bjdata(std::vector<uint8_t>({'h', 0x00, 0x3e})));
|
||
CHECK(json::parse("65504.0") == json::from_bjdata(std::vector<uint8_t>({'h', 0xff, 0x7b})));
|
||
}
|
||
|
||
SECTION("errors")
|
||
{
|
||
SECTION("no byte follows")
|
||
{
|
||
json _;
|
||
std::vector<uint8_t> const vec0 = {'h'};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vec0), "[json.exception.parse_error.110] parse error at byte 2: syntax error while parsing BJData number: unexpected end of input", json::parse_error&);
|
||
CHECK(json::from_bjdata(vec0, true, false).is_discarded());
|
||
}
|
||
|
||
SECTION("only one byte follows")
|
||
{
|
||
json _;
|
||
std::vector<uint8_t> const vec1 = {'h', 0x00};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vec1), "[json.exception.parse_error.110] parse error at byte 3: syntax error while parsing BJData number: unexpected end of input", json::parse_error&);
|
||
CHECK(json::from_bjdata(vec1, true, false).is_discarded());
|
||
}
|
||
}
|
||
}
|
||
|
||
SECTION("half-precision float (edge cases)")
|
||
{
|
||
SECTION("exp = 0b00000")
|
||
{
|
||
SECTION("0 (0 00000 0000000000)")
|
||
{
|
||
json const j = json::from_bjdata(std::vector<uint8_t>({'h', 0x00, 0x00}));
|
||
const json::number_float_t d{j};
|
||
CHECK(d == 0.0);
|
||
}
|
||
|
||
SECTION("-0 (1 00000 0000000000)")
|
||
{
|
||
json const j = json::from_bjdata(std::vector<uint8_t>({'h', 0x00, 0x80}));
|
||
const json::number_float_t d{j};
|
||
CHECK(d == -0.0);
|
||
}
|
||
|
||
SECTION("2**-24 (0 00000 0000000001)")
|
||
{
|
||
json const j = json::from_bjdata(std::vector<uint8_t>({'h', 0x01, 0x00}));
|
||
const json::number_float_t d{j};
|
||
CHECK(d == std::pow(2.0, -24.0));
|
||
}
|
||
}
|
||
|
||
SECTION("exp = 0b11111")
|
||
{
|
||
SECTION("infinity (0 11111 0000000000)")
|
||
{
|
||
json const j = json::from_bjdata(std::vector<uint8_t>({'h', 0x00, 0x7c}));
|
||
const json::number_float_t d{j};
|
||
CHECK(d == std::numeric_limits<json::number_float_t>::infinity());
|
||
CHECK(j.dump() == "null");
|
||
}
|
||
|
||
SECTION("-infinity (1 11111 0000000000)")
|
||
{
|
||
json const j = json::from_bjdata(std::vector<uint8_t>({'h', 0x00, 0xfc}));
|
||
const json::number_float_t d{j};
|
||
CHECK(d == -std::numeric_limits<json::number_float_t>::infinity());
|
||
CHECK(j.dump() == "null");
|
||
}
|
||
}
|
||
|
||
SECTION("other values from https://en.wikipedia.org/wiki/Half-precision_floating-point_format")
|
||
{
|
||
SECTION("1 (0 01111 0000000000)")
|
||
{
|
||
json const j = json::from_bjdata(std::vector<uint8_t>({'h', 0x00, 0x3c}));
|
||
const json::number_float_t d{j};
|
||
CHECK(d == 1);
|
||
}
|
||
|
||
SECTION("-2 (1 10000 0000000000)")
|
||
{
|
||
json const j = json::from_bjdata(std::vector<uint8_t>({'h', 0x00, 0xc0}));
|
||
const json::number_float_t d{j};
|
||
CHECK(d == -2);
|
||
}
|
||
|
||
SECTION("65504 (0 11110 1111111111)")
|
||
{
|
||
json const j = json::from_bjdata(std::vector<uint8_t>({'h', 0xff, 0x7b}));
|
||
const json::number_float_t d{j};
|
||
CHECK(d == 65504);
|
||
}
|
||
}
|
||
|
||
SECTION("infinity")
|
||
{
|
||
json const j = json::from_bjdata(std::vector<uint8_t>({'h', 0x00, 0x7c}));
|
||
json::number_float_t const d{j};
|
||
CHECK_FALSE(std::isfinite(d));
|
||
CHECK(j.dump() == "null");
|
||
}
|
||
|
||
SECTION("NaN")
|
||
{
|
||
json const j = json::from_bjdata(std::vector<uint8_t>({'h', 0x00, 0x7e }));
|
||
json::number_float_t const d{j};
|
||
CHECK(std::isnan(d));
|
||
CHECK(j.dump() == "null");
|
||
}
|
||
}
|
||
|
||
SECTION("high-precision number")
|
||
{
|
||
SECTION("unsigned integer number")
|
||
{
|
||
std::vector<uint8_t> const vec = {'H', 'i', 0x14, '1', '2', '3', '4', '5', '6', '7', '8', '9', '0', '1', '2', '3', '4', '5', '6', '7', '8', '9', '0'};
|
||
const auto j = json::from_bjdata(vec);
|
||
CHECK(j.is_number_unsigned());
|
||
CHECK(j.dump() == "12345678901234567890");
|
||
}
|
||
|
||
SECTION("signed integer number")
|
||
{
|
||
std::vector<uint8_t> const vec = {'H', 'i', 0x13, '-', '1', '2', '3', '4', '5', '6', '7', '8', '9', '0', '1', '2', '3', '4', '5', '6', '7', '8'};
|
||
const auto j = json::from_bjdata(vec);
|
||
CHECK(j.is_number_integer());
|
||
CHECK(j.dump() == "-123456789012345678");
|
||
}
|
||
|
||
SECTION("floating-point number")
|
||
{
|
||
std::vector<uint8_t> const vec = {'H', 'i', 0x16, '3', '.', '1', '4', '1', '5', '9', '2', '6', '5', '3', '5', '8', '9', '7', '9', '3', '2', '3', '8', '4', '6'};
|
||
const auto j = json::from_bjdata(vec);
|
||
CHECK(j.is_number_float());
|
||
CHECK(j.dump() == "3.141592653589793");
|
||
}
|
||
|
||
SECTION("errors")
|
||
{
|
||
// error while parsing length
|
||
std::vector<uint8_t> const vec0 = {'H', 'i'};
|
||
CHECK(json::from_bjdata(vec0, true, false).is_discarded());
|
||
// error while parsing string
|
||
std::vector<uint8_t> const vec1 = {'H', 'i', '1'};
|
||
CHECK(json::from_bjdata(vec1, true, false).is_discarded());
|
||
|
||
json _;
|
||
std::vector<uint8_t> const vec2 = {'H', 'i', 2, '1', 'A', '3'};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vec2), "[json.exception.parse_error.115] parse error at byte 5: syntax error while parsing BJData high-precision number: invalid number text: 1A", json::parse_error);
|
||
std::vector<uint8_t> const vec3 = {'H', 'i', 2, '1', '.'};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vec3), "[json.exception.parse_error.115] parse error at byte 5: syntax error while parsing BJData high-precision number: invalid number text: 1.", json::parse_error);
|
||
// Reject NULs where they are read, including trailing NULs and payloads cut off after one.
|
||
SECTION("NUL in high-precision number (issue #5753)")
|
||
{
|
||
for (const auto& vec : std::vector<std::vector<uint8_t>>
|
||
{
|
||
{'H', 'i', 3, '1', 0, 'x'},
|
||
{'H', 'i', 2, '1', 0},
|
||
{'H', 'i', 3, '1', 0}
|
||
})
|
||
{
|
||
CAPTURE(vec)
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vec), "[json.exception.parse_error.115] parse error at byte 5: syntax error while parsing BJData high-precision number: invalid number text; last byte: 0x00", json::parse_error);
|
||
CHECK(json::from_bjdata(vec, true, false).is_discarded());
|
||
CHECK(json::from_bjdata(vec, false, false).is_discarded());
|
||
}
|
||
|
||
std::vector<uint8_t> const nested = {'[', 'H', 'i', 3, '1', 0, 'x', ']'};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(nested), "[json.exception.parse_error.115] parse error at byte 6: syntax error while parsing BJData high-precision number: invalid number text; last byte: 0x00", json::parse_error);
|
||
CHECK(json::from_bjdata(nested, true, false).is_discarded());
|
||
|
||
std::vector<uint8_t> const valid = {'H', 'i', 1, '1'};
|
||
const auto j = json::from_bjdata(valid);
|
||
CHECK(j.is_number_unsigned());
|
||
CHECK(j == json(1));
|
||
}
|
||
|
||
std::vector<uint8_t> const vec_overflow = {'H', 'i', 5, '1', 'e', '4', '0', '0'};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vec_overflow), "[json.exception.out_of_range.406] number overflow parsing '1e400'", json::out_of_range);
|
||
std::vector<uint8_t> const vec4 = {'H', 2, '1', '0'};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vec4), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing BJData size: expected length type specification (U, i, u, I, m, l, M, L) after '#'; last byte: 0x02", json::parse_error);
|
||
}
|
||
}
|
||
}
|
||
|
||
SECTION("string")
|
||
{
|
||
SECTION("N = 0..127")
|
||
{
|
||
for (size_t N = 0; N <= 127; ++N)
|
||
{
|
||
CAPTURE(N)
|
||
|
||
// create JSON value with string containing of N * 'x'
|
||
const auto s = std::string(N, 'x');
|
||
json const j = s;
|
||
|
||
// create expected byte vector
|
||
std::vector<uint8_t> expected;
|
||
expected.push_back('S');
|
||
expected.push_back('i');
|
||
expected.push_back(static_cast<uint8_t>(N));
|
||
for (size_t i = 0; i < N; ++i)
|
||
{
|
||
expected.push_back('x');
|
||
}
|
||
|
||
// compare result + size
|
||
const auto result = json::to_bjdata(j);
|
||
CHECK(result == expected);
|
||
CHECK(result.size() == N + 3);
|
||
// check that no null byte is appended
|
||
if (N > 0)
|
||
{
|
||
CHECK(result.back() != '\x00');
|
||
}
|
||
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
}
|
||
|
||
SECTION("N = 128..255")
|
||
{
|
||
for (size_t N = 128; N <= 255; ++N)
|
||
{
|
||
CAPTURE(N)
|
||
|
||
// create JSON value with string containing of N * 'x'
|
||
const auto s = std::string(N, 'x');
|
||
json const j = s;
|
||
|
||
// create expected byte vector
|
||
std::vector<uint8_t> expected;
|
||
expected.push_back('S');
|
||
expected.push_back('U');
|
||
expected.push_back(static_cast<uint8_t>(N));
|
||
for (size_t i = 0; i < N; ++i)
|
||
{
|
||
expected.push_back('x');
|
||
}
|
||
|
||
// compare result + size
|
||
const auto result = json::to_bjdata(j);
|
||
CHECK(result == expected);
|
||
CHECK(result.size() == N + 3);
|
||
// check that no null byte is appended
|
||
CHECK(result.back() != '\x00');
|
||
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
}
|
||
|
||
SECTION("N = 256..32767")
|
||
{
|
||
for (const size_t N :
|
||
{
|
||
256u, 999u, 1025u, 3333u, 2048u, 32767u
|
||
})
|
||
{
|
||
CAPTURE(N)
|
||
|
||
// create JSON value with string containing of N * 'x'
|
||
const auto s = std::string(N, 'x');
|
||
json const j = s;
|
||
|
||
// create expected byte vector (hack: create string first)
|
||
std::vector<uint8_t> expected(N, 'x');
|
||
// reverse order of commands, because we insert at begin()
|
||
expected.insert(expected.begin(), static_cast<uint8_t>((N >> 8) & 0xff));
|
||
expected.insert(expected.begin(), static_cast<uint8_t>(N & 0xff));
|
||
expected.insert(expected.begin(), 'I');
|
||
expected.insert(expected.begin(), 'S');
|
||
|
||
// compare result + size
|
||
const auto result = json::to_bjdata(j);
|
||
CHECK(result == expected);
|
||
CHECK(result.size() == N + 4);
|
||
// check that no null byte is appended
|
||
CHECK(result.back() != '\x00');
|
||
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
}
|
||
|
||
SECTION("N = 32768..65535")
|
||
{
|
||
for (const size_t N :
|
||
{
|
||
32768u, 55555u, 65535u
|
||
})
|
||
{
|
||
CAPTURE(N)
|
||
|
||
// create JSON value with string containing of N * 'x'
|
||
const auto s = std::string(N, 'x');
|
||
json const j = s;
|
||
|
||
// create expected byte vector (hack: create string first)
|
||
std::vector<uint8_t> expected(N, 'x');
|
||
// reverse order of commands, because we insert at begin()
|
||
expected.insert(expected.begin(), static_cast<uint8_t>((N >> 8) & 0xff));
|
||
expected.insert(expected.begin(), static_cast<uint8_t>(N & 0xff));
|
||
expected.insert(expected.begin(), 'u');
|
||
expected.insert(expected.begin(), 'S');
|
||
|
||
// compare result + size
|
||
const auto result = json::to_bjdata(j);
|
||
CHECK(result == expected);
|
||
CHECK(result.size() == N + 4);
|
||
// check that no null byte is appended
|
||
CHECK(result.back() != '\x00');
|
||
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
}
|
||
|
||
SECTION("N = 65536..2147483647")
|
||
{
|
||
for (const size_t N :
|
||
{
|
||
65536u, 77777u, 1048576u
|
||
})
|
||
{
|
||
CAPTURE(N)
|
||
|
||
// create JSON value with string containing of N * 'x'
|
||
const auto s = std::string(N, 'x');
|
||
json const j = s;
|
||
|
||
// create expected byte vector (hack: create string first)
|
||
std::vector<uint8_t> expected(N, 'x');
|
||
// reverse order of commands, because we insert at begin()
|
||
expected.insert(expected.begin(), static_cast<uint8_t>((N >> 24) & 0xff));
|
||
expected.insert(expected.begin(), static_cast<uint8_t>((N >> 16) & 0xff));
|
||
expected.insert(expected.begin(), static_cast<uint8_t>((N >> 8) & 0xff));
|
||
expected.insert(expected.begin(), static_cast<uint8_t>(N & 0xff));
|
||
expected.insert(expected.begin(), 'l');
|
||
expected.insert(expected.begin(), 'S');
|
||
|
||
// compare result + size
|
||
const auto result = json::to_bjdata(j);
|
||
CHECK(result == expected);
|
||
CHECK(result.size() == N + 6);
|
||
// check that no null byte is appended
|
||
CHECK(result.back() != '\x00');
|
||
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
}
|
||
}
|
||
|
||
SECTION("binary")
|
||
{
|
||
for (json::bjdata_version_t bjdata_version :
|
||
{
|
||
json::bjdata_version_t::draft2, json::bjdata_version_t::draft3
|
||
})
|
||
{
|
||
CAPTURE(bjdata_version)
|
||
const bool draft3 = (bjdata_version == json::bjdata_version_t::draft3);
|
||
|
||
SECTION("N = 0..127")
|
||
{
|
||
for (std::size_t N = 0; N <= 127; ++N)
|
||
{
|
||
CAPTURE(N)
|
||
|
||
// create JSON value with byte array containing of N * 'x'
|
||
const auto s = std::vector<std::uint8_t>(N, 'x');
|
||
json const j = json::binary(s);
|
||
|
||
// create expected byte vector
|
||
std::vector<std::uint8_t> expected;
|
||
expected.push_back(static_cast<std::uint8_t>('['));
|
||
if (draft3 || N != 0)
|
||
{
|
||
expected.push_back(static_cast<std::uint8_t>('$'));
|
||
expected.push_back(static_cast<std::uint8_t>(draft3 ? 'B' : 'U'));
|
||
}
|
||
expected.push_back(static_cast<std::uint8_t>('#'));
|
||
expected.push_back(static_cast<std::uint8_t>('i'));
|
||
expected.push_back(static_cast<std::uint8_t>(N));
|
||
for (size_t i = 0; i < N; ++i)
|
||
{
|
||
expected.push_back(0x78);
|
||
}
|
||
|
||
// compare result + size
|
||
const auto result = json::to_bjdata(j, true, true, bjdata_version);
|
||
CHECK(result == expected);
|
||
if (!draft3 && N == 0)
|
||
{
|
||
CHECK(result.size() == N + 4);
|
||
}
|
||
else
|
||
{
|
||
CHECK(result.size() == N + 6);
|
||
}
|
||
|
||
// check that no null byte is appended
|
||
if (N > 0)
|
||
{
|
||
CHECK(result.back() != '\x00');
|
||
}
|
||
|
||
if (draft3)
|
||
{
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
else
|
||
{
|
||
// roundtrip only works to an array of numbers
|
||
json j_out = s;
|
||
CHECK(json::from_bjdata(result) == j_out);
|
||
CHECK(json::from_bjdata(result, true, false) == j_out);
|
||
}
|
||
}
|
||
}
|
||
|
||
SECTION("N = 128..255")
|
||
{
|
||
for (std::size_t N = 128; N <= 255; ++N)
|
||
{
|
||
CAPTURE(N)
|
||
|
||
// create JSON value with byte array containing of N * 'x'
|
||
const auto s = std::vector<std::uint8_t>(N, 'x');
|
||
json const j = json::binary(s);
|
||
|
||
// create expected byte vector
|
||
std::vector<uint8_t> expected;
|
||
expected.push_back(static_cast<std::uint8_t>('['));
|
||
expected.push_back(static_cast<std::uint8_t>('$'));
|
||
expected.push_back(static_cast<std::uint8_t>(draft3 ? 'B' : 'U'));
|
||
expected.push_back(static_cast<std::uint8_t>('#'));
|
||
expected.push_back(static_cast<std::uint8_t>('U'));
|
||
expected.push_back(static_cast<std::uint8_t>(N));
|
||
for (size_t i = 0; i < N; ++i)
|
||
{
|
||
expected.push_back(0x78);
|
||
}
|
||
|
||
// compare result + size
|
||
const auto result = json::to_bjdata(j, true, true, bjdata_version);
|
||
CHECK(result == expected);
|
||
CHECK(result.size() == N + 6);
|
||
// check that no null byte is appended
|
||
CHECK(result.back() != '\x00');
|
||
|
||
if (draft3)
|
||
{
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
else
|
||
{
|
||
// roundtrip only works to an array of numbers
|
||
json j_out = s;
|
||
CHECK(json::from_bjdata(result) == j_out);
|
||
CHECK(json::from_bjdata(result, true, false) == j_out);
|
||
}
|
||
}
|
||
}
|
||
|
||
SECTION("N = 256..32767")
|
||
{
|
||
for (const std::size_t N :
|
||
{
|
||
256u, 999u, 1025u, 3333u, 2048u, 32767u
|
||
})
|
||
{
|
||
CAPTURE(N)
|
||
|
||
// create JSON value with byte array containing of N * 'x'
|
||
const auto s = std::vector<std::uint8_t>(N, 'x');
|
||
json const j = json::binary(s);
|
||
|
||
// create expected byte vector
|
||
std::vector<std::uint8_t> expected(N + 7, 'x');
|
||
expected[0] = '[';
|
||
expected[1] = '$';
|
||
expected[2] = draft3 ? 'B' : 'U';
|
||
expected[3] = '#';
|
||
expected[4] = 'I';
|
||
expected[5] = static_cast<std::uint8_t>(N & 0xFF);
|
||
expected[6] = static_cast<std::uint8_t>((N >> 8) & 0xFF);
|
||
|
||
// compare result + size
|
||
const auto result = json::to_bjdata(j, true, true, bjdata_version);
|
||
CHECK(result == expected);
|
||
CHECK(result.size() == N + 7);
|
||
// check that no null byte is appended
|
||
CHECK(result.back() != '\x00');
|
||
|
||
if (draft3)
|
||
{
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
else
|
||
{
|
||
// roundtrip only works to an array of numbers
|
||
json j_out = s;
|
||
CHECK(json::from_bjdata(result) == j_out);
|
||
CHECK(json::from_bjdata(result, true, false) == j_out);
|
||
}
|
||
}
|
||
}
|
||
|
||
SECTION("N = 32768..65535")
|
||
{
|
||
for (const std::size_t N :
|
||
{
|
||
32768u, 55555u, 65535u
|
||
})
|
||
{
|
||
CAPTURE(N)
|
||
|
||
// create JSON value with byte array containing of N * 'x'
|
||
const auto s = std::vector<std::uint8_t>(N, 'x');
|
||
json const j = json::binary(s);
|
||
|
||
// create expected byte vector
|
||
std::vector<std::uint8_t> expected(N + 7, 'x');
|
||
expected[0] = '[';
|
||
expected[1] = '$';
|
||
expected[2] = draft3 ? 'B' : 'U';
|
||
expected[3] = '#';
|
||
expected[4] = 'u';
|
||
expected[5] = static_cast<std::uint8_t>(N & 0xFF);
|
||
expected[6] = static_cast<std::uint8_t>((N >> 8) & 0xFF);
|
||
|
||
// compare result + size
|
||
const auto result = json::to_bjdata(j, true, true, bjdata_version);
|
||
CHECK(result == expected);
|
||
CHECK(result.size() == N + 7);
|
||
// check that no null byte is appended
|
||
CHECK(result.back() != '\x00');
|
||
|
||
if (draft3)
|
||
{
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
else
|
||
{
|
||
// roundtrip only works to an array of numbers
|
||
json j_out = s;
|
||
CHECK(json::from_bjdata(result) == j_out);
|
||
CHECK(json::from_bjdata(result, true, false) == j_out);
|
||
}
|
||
}
|
||
}
|
||
|
||
SECTION("N = 65536..2147483647")
|
||
{
|
||
for (const std::size_t N :
|
||
{
|
||
65536u, 77777u, 1048576u
|
||
})
|
||
{
|
||
CAPTURE(N)
|
||
|
||
// create JSON value with byte array containing of N * 'x'
|
||
const auto s = std::vector<std::uint8_t>(N, 'x');
|
||
json const j = json::binary(s);
|
||
|
||
// create expected byte vector
|
||
std::vector<std::uint8_t> expected(N + 9, 'x');
|
||
expected[0] = '[';
|
||
expected[1] = '$';
|
||
expected[2] = draft3 ? 'B' : 'U';
|
||
expected[3] = '#';
|
||
expected[4] = 'l';
|
||
expected[5] = static_cast<std::uint8_t>(N & 0xFF);
|
||
expected[6] = static_cast<std::uint8_t>((N >> 8) & 0xFF);
|
||
expected[7] = static_cast<std::uint8_t>((N >> 16) & 0xFF);
|
||
expected[8] = static_cast<std::uint8_t>((N >> 24) & 0xFF);
|
||
|
||
// compare result + size
|
||
const auto result = json::to_bjdata(j, true, true, bjdata_version);
|
||
CHECK(result == expected);
|
||
CHECK(result.size() == N + 9);
|
||
// check that no null byte is appended
|
||
CHECK(result.back() != '\x00');
|
||
|
||
if (draft3)
|
||
{
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
else
|
||
{
|
||
// roundtrip only works to an array of numbers
|
||
json j_out = s;
|
||
CHECK(json::from_bjdata(result) == j_out);
|
||
CHECK(json::from_bjdata(result, true, false) == j_out);
|
||
}
|
||
}
|
||
}
|
||
|
||
SECTION("Other Serializations")
|
||
{
|
||
const std::size_t N = 10;
|
||
const auto s = std::vector<std::uint8_t>(N, 'x');
|
||
json const j = json::binary(s);
|
||
|
||
SECTION("No Count No Type")
|
||
{
|
||
std::vector<uint8_t> expected;
|
||
expected.push_back(static_cast<std::uint8_t>('['));
|
||
for (std::size_t i = 0; i < N; ++i)
|
||
{
|
||
expected.push_back(static_cast<std::uint8_t>(draft3 ? 'B' : 'U'));
|
||
expected.push_back(static_cast<std::uint8_t>(0x78));
|
||
}
|
||
expected.push_back(static_cast<std::uint8_t>(']'));
|
||
|
||
// compare result + size
|
||
const auto result = json::to_bjdata(j, false, false, bjdata_version);
|
||
CHECK(result == expected);
|
||
CHECK(result.size() == N + 12);
|
||
// check that no null byte is appended
|
||
CHECK(result.back() != '\x00');
|
||
|
||
// roundtrip only works to an array of numbers
|
||
json j_out = s;
|
||
CHECK(json::from_bjdata(result) == j_out);
|
||
CHECK(json::from_bjdata(result, true, false) == j_out);
|
||
}
|
||
|
||
SECTION("Yes Count No Type")
|
||
{
|
||
std::vector<std::uint8_t> expected;
|
||
expected.push_back(static_cast<std::uint8_t>('['));
|
||
expected.push_back(static_cast<std::uint8_t>('#'));
|
||
expected.push_back(static_cast<std::uint8_t>('i'));
|
||
expected.push_back(static_cast<std::uint8_t>(N));
|
||
|
||
for (size_t i = 0; i < N; ++i)
|
||
{
|
||
expected.push_back(static_cast<std::uint8_t>(draft3 ? 'B' : 'U'));
|
||
expected.push_back(static_cast<std::uint8_t>(0x78));
|
||
}
|
||
|
||
// compare result + size
|
||
const auto result = json::to_bjdata(j, true, false, bjdata_version);
|
||
CHECK(result == expected);
|
||
CHECK(result.size() == N + 14);
|
||
// check that no null byte is appended
|
||
CHECK(result.back() != '\x00');
|
||
|
||
// roundtrip only works to an array of numbers
|
||
json j_out = s;
|
||
CHECK(json::from_bjdata(result) == j_out);
|
||
CHECK(json::from_bjdata(result, true, false) == j_out);
|
||
}
|
||
}
|
||
}
|
||
}
|
||
SECTION("array")
|
||
{
|
||
SECTION("empty")
|
||
{
|
||
SECTION("size=false type=false")
|
||
{
|
||
json const j = json::array();
|
||
std::vector<uint8_t> const expected = {'[', ']'};
|
||
const auto result = json::to_bjdata(j);
|
||
CHECK(result == expected);
|
||
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
|
||
SECTION("size=true type=false")
|
||
{
|
||
json const j = json::array();
|
||
std::vector<uint8_t> const expected = {'[', '#', 'i', 0};
|
||
const auto result = json::to_bjdata(j, true);
|
||
CHECK(result == expected);
|
||
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
|
||
SECTION("size=true type=true")
|
||
{
|
||
json const j = json::array();
|
||
std::vector<uint8_t> const expected = {'[', '#', 'i', 0};
|
||
const auto result = json::to_bjdata(j, true, true);
|
||
CHECK(result == expected);
|
||
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
}
|
||
|
||
SECTION("[null]")
|
||
{
|
||
SECTION("size=false type=false")
|
||
{
|
||
json const j = {nullptr};
|
||
std::vector<uint8_t> const expected = {'[', 'Z', ']'};
|
||
const auto result = json::to_bjdata(j);
|
||
CHECK(result == expected);
|
||
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
|
||
SECTION("size=true type=false")
|
||
{
|
||
json const j = {nullptr};
|
||
std::vector<uint8_t> const expected = {'[', '#', 'i', 1, 'Z'};
|
||
const auto result = json::to_bjdata(j, true);
|
||
CHECK(result == expected);
|
||
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
|
||
SECTION("size=true type=true")
|
||
{
|
||
json const j = {nullptr};
|
||
std::vector<uint8_t> const expected = {'[', '#', 'i', 1, 'Z'};
|
||
const auto result = json::to_bjdata(j, true, true);
|
||
CHECK(result == expected);
|
||
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
}
|
||
|
||
SECTION("[1,2,3,4,5]")
|
||
{
|
||
SECTION("size=false type=false")
|
||
{
|
||
json const j = json::parse("[1,2,3,4,5]");
|
||
std::vector<uint8_t> const expected = {'[', 'i', 1, 'i', 2, 'i', 3, 'i', 4, 'i', 5, ']'};
|
||
const auto result = json::to_bjdata(j);
|
||
CHECK(result == expected);
|
||
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
|
||
SECTION("size=true type=false")
|
||
{
|
||
json const j = json::parse("[1,2,3,4,5]");
|
||
std::vector<uint8_t> const expected = {'[', '#', 'i', 5, 'i', 1, 'i', 2, 'i', 3, 'i', 4, 'i', 5};
|
||
const auto result = json::to_bjdata(j, true);
|
||
CHECK(result == expected);
|
||
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
|
||
SECTION("size=true type=true")
|
||
{
|
||
json const j = json::parse("[1,2,3,4,5]");
|
||
std::vector<uint8_t> const expected = {'[', '$', 'i', '#', 'i', 5, 1, 2, 3, 4, 5};
|
||
const auto result = json::to_bjdata(j, true, true);
|
||
CHECK(result == expected);
|
||
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
}
|
||
|
||
SECTION("[[[[]]]]")
|
||
{
|
||
SECTION("size=false type=false")
|
||
{
|
||
json const j = json::parse("[[[[]]]]");
|
||
std::vector<uint8_t> const expected = {'[', '[', '[', '[', ']', ']', ']', ']'};
|
||
const auto result = json::to_bjdata(j);
|
||
CHECK(result == expected);
|
||
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
|
||
SECTION("size=true type=false")
|
||
{
|
||
json const j = json::parse("[[[[]]]]");
|
||
std::vector<uint8_t> const expected = {'[', '#', 'i', 1, '[', '#', 'i', 1, '[', '#', 'i', 1, '[', '#', 'i', 0};
|
||
const auto result = json::to_bjdata(j, true);
|
||
CHECK(result == expected);
|
||
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
|
||
SECTION("size=true type=true")
|
||
{
|
||
json const j = json::parse("[[[[]]]]");
|
||
std::vector<uint8_t> const expected = {'[', '#', 'i', 1, '[', '#', 'i', 1, '[', '#', 'i', 1, '[', '#', 'i', 0};
|
||
const auto result = json::to_bjdata(j, true, true);
|
||
CHECK(result == expected);
|
||
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
}
|
||
|
||
SECTION("array with int16_t elements")
|
||
{
|
||
SECTION("size=false type=false")
|
||
{
|
||
json j(257, nullptr);
|
||
std::vector<uint8_t> expected(j.size() + 2, 'Z'); // all null
|
||
expected[0] = '['; // opening array
|
||
expected[258] = ']'; // closing array
|
||
const auto result = json::to_bjdata(j);
|
||
CHECK(result == expected);
|
||
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
|
||
SECTION("size=true type=false")
|
||
{
|
||
json j(257, nullptr);
|
||
std::vector<uint8_t> expected(j.size() + 5, 'Z'); // all null
|
||
expected[0] = '['; // opening array
|
||
expected[1] = '#'; // array size
|
||
expected[2] = 'I'; // int16
|
||
expected[3] = 0x01; // 0x0101, first byte
|
||
expected[4] = 0x01; // 0x0101, second byte
|
||
const auto result = json::to_bjdata(j, true);
|
||
CHECK(result == expected);
|
||
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
}
|
||
|
||
SECTION("array with uint16_t elements")
|
||
{
|
||
SECTION("size=false type=false")
|
||
{
|
||
json j(32768, nullptr);
|
||
std::vector<uint8_t> expected(j.size() + 2, 'Z'); // all null
|
||
expected[0] = '['; // opening array
|
||
expected[32769] = ']'; // closing array
|
||
const auto result = json::to_bjdata(j);
|
||
CHECK(result == expected);
|
||
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
|
||
SECTION("size=true type=false")
|
||
{
|
||
json j(32768, nullptr);
|
||
std::vector<uint8_t> expected(j.size() + 5, 'Z'); // all null
|
||
expected[0] = '['; // opening array
|
||
expected[1] = '#'; // array size
|
||
expected[2] = 'u'; // int16
|
||
expected[3] = 0x00; // 0x0101, first byte
|
||
expected[4] = 0x80; // 0x0101, second byte
|
||
const auto result = json::to_bjdata(j, true);
|
||
CHECK(result == expected);
|
||
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
}
|
||
|
||
SECTION("array with int32_t elements")
|
||
{
|
||
SECTION("size=false type=false")
|
||
{
|
||
json j(65793, nullptr);
|
||
std::vector<uint8_t> expected(j.size() + 2, 'Z'); // all null
|
||
expected[0] = '['; // opening array
|
||
expected[65794] = ']'; // closing array
|
||
const auto result = json::to_bjdata(j);
|
||
CHECK(result == expected);
|
||
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
|
||
SECTION("size=true type=false")
|
||
{
|
||
json j(65793, nullptr);
|
||
std::vector<uint8_t> expected(j.size() + 7, 'Z'); // all null
|
||
expected[0] = '['; // opening array
|
||
expected[1] = '#'; // array size
|
||
expected[2] = 'l'; // int32
|
||
expected[3] = 0x01; // 0x00010101, fourth byte
|
||
expected[4] = 0x01; // 0x00010101, third byte
|
||
expected[5] = 0x01; // 0x00010101, second byte
|
||
expected[6] = 0x00; // 0x00010101, first byte
|
||
const auto result = json::to_bjdata(j, true);
|
||
CHECK(result == expected);
|
||
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
}
|
||
}
|
||
|
||
SECTION("object")
|
||
{
|
||
SECTION("empty")
|
||
{
|
||
SECTION("size=false type=false")
|
||
{
|
||
json const j = json::object();
|
||
std::vector<uint8_t> const expected = {'{', '}'};
|
||
const auto result = json::to_bjdata(j);
|
||
CHECK(result == expected);
|
||
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
|
||
SECTION("size=true type=false")
|
||
{
|
||
json const j = json::object();
|
||
std::vector<uint8_t> const expected = {'{', '#', 'i', 0};
|
||
const auto result = json::to_bjdata(j, true);
|
||
CHECK(result == expected);
|
||
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
|
||
SECTION("size=true type=true")
|
||
{
|
||
json const j = json::object();
|
||
std::vector<uint8_t> const expected = {'{', '#', 'i', 0};
|
||
const auto result = json::to_bjdata(j, true, true);
|
||
CHECK(result == expected);
|
||
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
}
|
||
|
||
SECTION("{\"\":null}")
|
||
{
|
||
SECTION("size=false type=false")
|
||
{
|
||
json const j = {{"", nullptr}};
|
||
std::vector<uint8_t> const expected = {'{', 'i', 0, 'Z', '}'};
|
||
const auto result = json::to_bjdata(j);
|
||
CHECK(result == expected);
|
||
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
|
||
SECTION("size=true type=false")
|
||
{
|
||
json const j = {{"", nullptr}};
|
||
std::vector<uint8_t> const expected = {'{', '#', 'i', 1, 'i', 0, 'Z'};
|
||
const auto result = json::to_bjdata(j, true);
|
||
CHECK(result == expected);
|
||
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
}
|
||
|
||
SECTION("{\"a\": {\"b\": {\"c\": {}}}}")
|
||
{
|
||
SECTION("size=false type=false")
|
||
{
|
||
json const j = json::parse(R"({"a": {"b": {"c": {}}}})");
|
||
std::vector<uint8_t> const expected =
|
||
{
|
||
'{', 'i', 1, 'a', '{', 'i', 1, 'b', '{', 'i', 1, 'c', '{', '}', '}', '}', '}'
|
||
};
|
||
const auto result = json::to_bjdata(j);
|
||
CHECK(result == expected);
|
||
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
|
||
SECTION("size=true type=false")
|
||
{
|
||
json const j = json::parse(R"({"a": {"b": {"c": {}}}})");
|
||
std::vector<uint8_t> const expected =
|
||
{
|
||
'{', '#', 'i', 1, 'i', 1, 'a', '{', '#', 'i', 1, 'i', 1, 'b', '{', '#', 'i', 1, 'i', 1, 'c', '{', '#', 'i', 0
|
||
};
|
||
const auto result = json::to_bjdata(j, true);
|
||
CHECK(result == expected);
|
||
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
|
||
SECTION("size=true type=true ignore object type marker")
|
||
{
|
||
json const j = json::parse(R"({"a": {"b": {"c": {}}}})");
|
||
std::vector<uint8_t> const expected =
|
||
{
|
||
'{', '#', 'i', 1, 'i', 1, 'a', '{', '#', 'i', 1, 'i', 1, 'b', '{', '#', 'i', 1, 'i', 1, 'c', '{', '#', 'i', 0
|
||
};
|
||
const auto result = json::to_bjdata(j, true, true);
|
||
CHECK(result == expected);
|
||
|
||
// roundtrip
|
||
CHECK(json::from_bjdata(result) == j);
|
||
CHECK(json::from_bjdata(result, true, false) == j);
|
||
}
|
||
}
|
||
}
|
||
}
|
||
|
||
SECTION("errors")
|
||
{
|
||
SECTION("strict mode")
|
||
{
|
||
std::vector<uint8_t> const vec = {'Z', 'Z'};
|
||
SECTION("non-strict mode")
|
||
{
|
||
const auto result = json::from_bjdata(vec, false);
|
||
CHECK(result == json());
|
||
}
|
||
|
||
SECTION("strict mode")
|
||
{
|
||
json _;
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vec),
|
||
"[json.exception.parse_error.110] parse error at byte 2: syntax error while parsing BJData value: expected end of input; last byte: 0x5A", json::parse_error&);
|
||
}
|
||
}
|
||
}
|
||
|
||
SECTION("SAX aborts")
|
||
{
|
||
SECTION("start_array()")
|
||
{
|
||
std::vector<uint8_t> const v = {'[', 'T', 'F', ']'};
|
||
SaxCountdown scp(0);
|
||
CHECK_FALSE(json::sax_parse(v, &scp, json::input_format_t::bjdata));
|
||
}
|
||
|
||
SECTION("start_object()")
|
||
{
|
||
std::vector<uint8_t> const v = {'{', 'i', 3, 'f', 'o', 'o', 'F', '}'};
|
||
SaxCountdown scp(0);
|
||
CHECK_FALSE(json::sax_parse(v, &scp, json::input_format_t::bjdata));
|
||
}
|
||
|
||
SECTION("key() in object")
|
||
{
|
||
std::vector<uint8_t> const v = {'{', 'i', 3, 'f', 'o', 'o', 'F', '}'};
|
||
SaxCountdown scp(1);
|
||
CHECK_FALSE(json::sax_parse(v, &scp, json::input_format_t::bjdata));
|
||
}
|
||
|
||
SECTION("start_array(len)")
|
||
{
|
||
std::vector<uint8_t> const v = {'[', '#', 'i', '2', 'T', 'F'};
|
||
SaxCountdown scp(0);
|
||
CHECK_FALSE(json::sax_parse(v, &scp, json::input_format_t::bjdata));
|
||
}
|
||
|
||
SECTION("start_object(len)")
|
||
{
|
||
std::vector<uint8_t> const v = {'{', '#', 'i', '1', 3, 'f', 'o', 'o', 'F'};
|
||
SaxCountdown scp(0);
|
||
CHECK_FALSE(json::sax_parse(v, &scp, json::input_format_t::bjdata));
|
||
}
|
||
|
||
SECTION("key() in object with length")
|
||
{
|
||
std::vector<uint8_t> const v = {'{', 'i', 3, 'f', 'o', 'o', 'F', '}'};
|
||
SaxCountdown scp(1);
|
||
CHECK_FALSE(json::sax_parse(v, &scp, json::input_format_t::bjdata));
|
||
}
|
||
|
||
SECTION("start_array() in ndarray _ArraySize_")
|
||
{
|
||
// _ArrayType_ (2 events: key + string) is now emitted before
|
||
// _ArraySize_ (see GitHub issue #5661), which shifts the events
|
||
// below later by the same 2 events
|
||
std::vector<uint8_t> const v = {'[', '$', 'i', '#', '[', '$', 'i', '#', 'i', 2, 2, 1, 1, 2};
|
||
SaxCountdown scp(4);
|
||
CHECK_FALSE(json::sax_parse(v, &scp, json::input_format_t::bjdata));
|
||
}
|
||
|
||
SECTION("number_integer() in ndarray _ArraySize_")
|
||
{
|
||
std::vector<uint8_t> const v = {'[', '$', 'U', '#', '[', '$', 'i', '#', 'i', 2, 2, 1, 1, 2};
|
||
SaxCountdown scp(5);
|
||
CHECK_FALSE(json::sax_parse(v, &scp, json::input_format_t::bjdata));
|
||
}
|
||
|
||
SECTION("key() in ndarray _ArrayType_")
|
||
{
|
||
// _ArrayType_ is emitted right after start_object(), before _ArraySize_
|
||
std::vector<uint8_t> const v = {'[', '$', 'U', '#', '[', '$', 'U', '#', 'i', 2, 2, 2, 1, 2, 3, 4};
|
||
SaxCountdown scp(1);
|
||
CHECK_FALSE(json::sax_parse(v, &scp, json::input_format_t::bjdata));
|
||
}
|
||
|
||
SECTION("string() in ndarray _ArrayType_")
|
||
{
|
||
std::vector<uint8_t> const v = {'[', '$', 'U', '#', '[', '$', 'U', '#', 'i', 2, 2, 2, 1, 2, 3, 4};
|
||
SaxCountdown scp(2);
|
||
CHECK_FALSE(json::sax_parse(v, &scp, json::input_format_t::bjdata));
|
||
}
|
||
|
||
SECTION("key() in ndarray _ArrayData_")
|
||
{
|
||
std::vector<uint8_t> const v = {'[', '$', 'U', '#', '[', '$', 'U', '#', 'i', 2, 2, 2, 1, 2, 3, 4};
|
||
SaxCountdown scp(8);
|
||
CHECK_FALSE(json::sax_parse(v, &scp, json::input_format_t::bjdata));
|
||
}
|
||
|
||
SECTION("string() in ndarray _ArrayData_")
|
||
{
|
||
std::vector<uint8_t> const v = {'[', '$', 'U', '#', '[', '$', 'U', '#', 'i', 2, 2, 2, 1, 2, 3, 4};
|
||
SaxCountdown scp(9);
|
||
CHECK_FALSE(json::sax_parse(v, &scp, json::input_format_t::bjdata));
|
||
}
|
||
|
||
SECTION("string() in ndarray _ArrayType_")
|
||
{
|
||
std::vector<uint8_t> const v = {'[', '$', 'U', '#', '[', '$', 'i', '#', 'i', 2, 3, 2, 6, 5, 4, 3, 2, 1};
|
||
SaxCountdown scp(11);
|
||
CHECK_FALSE(json::sax_parse(v, &scp, json::input_format_t::bjdata));
|
||
}
|
||
|
||
SECTION("every event of an ndarray of chars")
|
||
{
|
||
// the elements of an ndarray of 'C' are read one by one
|
||
std::vector<uint8_t> const v = {'[', '$', 'C', '#', '[', '$', 'U', '#', 'U', 2, 2, 2, 'a', 'b', 'c', 'd'};
|
||
// start_object, key, string, key, start_array, 2 numbers, end_array,
|
||
// key, start_array, 4 chars, end_array, end_object
|
||
constexpr int events = 16;
|
||
for (int allowed = 0; allowed < events; ++allowed)
|
||
{
|
||
CAPTURE(allowed)
|
||
SaxCountdown scp(allowed);
|
||
CHECK_FALSE(json::sax_parse(v, &scp, json::input_format_t::bjdata));
|
||
}
|
||
SaxCountdown scp(events);
|
||
CHECK(json::sax_parse(v, &scp, json::input_format_t::bjdata));
|
||
}
|
||
|
||
SECTION("start_array() in ndarray _ArrayData_")
|
||
{
|
||
std::vector<uint8_t> const v = {'[', '$', 'U', '#', '[', 'i', 2, 'i', 3, ']', 6, 5, 4, 3, 2, 1};
|
||
SaxCountdown scp(13);
|
||
CHECK_FALSE(json::sax_parse(v, &scp, json::input_format_t::bjdata));
|
||
}
|
||
}
|
||
|
||
SECTION("parsing values")
|
||
{
|
||
SECTION("strings")
|
||
{
|
||
// create a single-character string for all number types
|
||
std::vector<uint8_t> s_i = {'S', 'i', 1, 'a'};
|
||
std::vector<uint8_t> const s_U = {'S', 'U', 1, 'a'};
|
||
std::vector<uint8_t> const s_I = {'S', 'I', 1, 0, 'a'};
|
||
std::vector<uint8_t> const s_u = {'S', 'u', 1, 0, 'a'};
|
||
std::vector<uint8_t> const s_l = {'S', 'l', 1, 0, 0, 0, 'a'};
|
||
std::vector<uint8_t> const s_m = {'S', 'm', 1, 0, 0, 0, 'a'};
|
||
std::vector<uint8_t> const s_L = {'S', 'L', 1, 0, 0, 0, 0, 0, 0, 0, 'a'};
|
||
std::vector<uint8_t> const s_M = {'S', 'M', 1, 0, 0, 0, 0, 0, 0, 0, 'a'};
|
||
|
||
// check if string is parsed correctly to "a"
|
||
CHECK(json::from_bjdata(s_i) == "a");
|
||
CHECK(json::from_bjdata(s_U) == "a");
|
||
CHECK(json::from_bjdata(s_I) == "a");
|
||
CHECK(json::from_bjdata(s_u) == "a");
|
||
CHECK(json::from_bjdata(s_l) == "a");
|
||
CHECK(json::from_bjdata(s_m) == "a");
|
||
CHECK(json::from_bjdata(s_L) == "a");
|
||
CHECK(json::from_bjdata(s_M) == "a");
|
||
|
||
// roundtrip: output should be optimized
|
||
CHECK(json::to_bjdata(json::from_bjdata(s_i)) == s_i);
|
||
CHECK(json::to_bjdata(json::from_bjdata(s_U)) == s_i);
|
||
CHECK(json::to_bjdata(json::from_bjdata(s_I)) == s_i);
|
||
CHECK(json::to_bjdata(json::from_bjdata(s_u)) == s_i);
|
||
CHECK(json::to_bjdata(json::from_bjdata(s_l)) == s_i);
|
||
CHECK(json::to_bjdata(json::from_bjdata(s_m)) == s_i);
|
||
CHECK(json::to_bjdata(json::from_bjdata(s_L)) == s_i);
|
||
CHECK(json::to_bjdata(json::from_bjdata(s_M)) == s_i);
|
||
}
|
||
|
||
SECTION("number")
|
||
{
|
||
SECTION("float")
|
||
{
|
||
// float32
|
||
std::vector<uint8_t> const v_d = {'d', 0xd0, 0x0f, 0x49, 0x40};
|
||
CHECK(json::from_bjdata(v_d) == 3.14159f);
|
||
|
||
// float64
|
||
std::vector<uint8_t> const v_D = {'D', 0x6e, 0x86, 0x1b, 0xf0, 0xf9, 0x21, 0x09, 0x40};
|
||
CHECK(json::from_bjdata(v_D) == 3.14159);
|
||
|
||
// float32 is serialized as float64 as the library does not support float32
|
||
CHECK(json::to_bjdata(json::from_bjdata(v_d)) == json::to_bjdata(3.14159f));
|
||
}
|
||
}
|
||
|
||
SECTION("array")
|
||
{
|
||
SECTION("optimized version (length only)")
|
||
{
|
||
// create vector with two elements of the same type
|
||
std::vector<uint8_t> const v_TU = {'[', '#', 'U', 2, 'T', 'T'};
|
||
std::vector<uint8_t> const v_T = {'[', '#', 'i', 2, 'T', 'T'};
|
||
std::vector<uint8_t> const v_F = {'[', '#', 'i', 2, 'F', 'F'};
|
||
std::vector<uint8_t> const v_Z = {'[', '#', 'i', 2, 'Z', 'Z'};
|
||
std::vector<uint8_t> const v_i = {'[', '#', 'i', 2, 'i', 0x7F, 'i', 0x7F};
|
||
std::vector<uint8_t> const v_U = {'[', '#', 'i', 2, 'U', 0xFF, 'U', 0xFF};
|
||
std::vector<uint8_t> const v_I = {'[', '#', 'i', 2, 'I', 0xFF, 0x7F, 'I', 0xFF, 0x7F};
|
||
std::vector<uint8_t> const v_u = {'[', '#', 'i', 2, 'u', 0x0F, 0xA7, 'u', 0x0F, 0xA7};
|
||
std::vector<uint8_t> const v_l = {'[', '#', 'i', 2, 'l', 0xFF, 0xFF, 0xFF, 0x7F, 'l', 0xFF, 0xFF, 0xFF, 0x7F};
|
||
std::vector<uint8_t> const v_m = {'[', '#', 'i', 2, 'm', 0xFF, 0xC9, 0x9A, 0xBB, 'm', 0xFF, 0xC9, 0x9A, 0xBB};
|
||
std::vector<uint8_t> const v_L = {'[', '#', 'i', 2, 'L', 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0x7F, 'L', 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0x7F};
|
||
std::vector<uint8_t> const v_M = {'[', '#', 'i', 2, 'M', 0xFF, 0xFF, 0x63, 0xA7, 0xB3, 0xB6, 0xE0, 0x8D, 'M', 0xFF, 0xFF, 0x63, 0xA7, 0xB3, 0xB6, 0xE0, 0x8D};
|
||
std::vector<uint8_t> const v_D = {'[', '#', 'i', 2, 'D', 0x4a, 0xd8, 0x12, 0x4d, 0xfb, 0x21, 0x09, 0x40, 'D', 0x4a, 0xd8, 0x12, 0x4d, 0xfb, 0x21, 0x09, 0x40};
|
||
std::vector<uint8_t> const v_S = {'[', '#', 'i', 2, 'S', 'i', 1, 'a', 'S', 'i', 1, 'a'};
|
||
std::vector<uint8_t> const v_C = {'[', '#', 'i', 2, 'C', 'a', 'C', 'a'};
|
||
std::vector<uint8_t> const v_B = {'[', '#', 'i', 2, 'B', 0xFF, 'B', 0xFF};
|
||
|
||
// check if vector is parsed correctly
|
||
CHECK(json::from_bjdata(v_TU) == json({true, true}));
|
||
CHECK(json::from_bjdata(v_T) == json({true, true}));
|
||
CHECK(json::from_bjdata(v_F) == json({false, false}));
|
||
CHECK(json::from_bjdata(v_Z) == json({nullptr, nullptr}));
|
||
CHECK(json::from_bjdata(v_i) == json({127, 127}));
|
||
CHECK(json::from_bjdata(v_U) == json({255, 255}));
|
||
CHECK(json::from_bjdata(v_I) == json({32767, 32767}));
|
||
CHECK(json::from_bjdata(v_u) == json({42767, 42767}));
|
||
CHECK(json::from_bjdata(v_l) == json({2147483647, 2147483647}));
|
||
CHECK(json::from_bjdata(v_m) == json({3147483647, 3147483647}));
|
||
CHECK(json::from_bjdata(v_L) == json({9223372036854775807, 9223372036854775807}));
|
||
CHECK(json::from_bjdata(v_M) == json({10223372036854775807ull, 10223372036854775807ull}));
|
||
CHECK(json::from_bjdata(v_D) == json({3.1415926, 3.1415926}));
|
||
CHECK(json::from_bjdata(v_S) == json({"a", "a"}));
|
||
CHECK(json::from_bjdata(v_C) == json({"a", "a"}));
|
||
CHECK(json::from_bjdata(v_B) == json({255, 255}));
|
||
|
||
// roundtrip: output should be optimized
|
||
CHECK(json::to_bjdata(json::from_bjdata(v_T), true) == v_T);
|
||
CHECK(json::to_bjdata(json::from_bjdata(v_F), true) == v_F);
|
||
CHECK(json::to_bjdata(json::from_bjdata(v_Z), true) == v_Z);
|
||
CHECK(json::to_bjdata(json::from_bjdata(v_i), true) == v_i);
|
||
CHECK(json::to_bjdata(json::from_bjdata(v_U), true) == v_U);
|
||
CHECK(json::to_bjdata(json::from_bjdata(v_I), true) == v_I);
|
||
CHECK(json::to_bjdata(json::from_bjdata(v_u), true) == v_u);
|
||
CHECK(json::to_bjdata(json::from_bjdata(v_l), true) == v_l);
|
||
CHECK(json::to_bjdata(json::from_bjdata(v_m), true) == v_m);
|
||
CHECK(json::to_bjdata(json::from_bjdata(v_L), true) == v_L);
|
||
CHECK(json::to_bjdata(json::from_bjdata(v_M), true) == v_M);
|
||
CHECK(json::to_bjdata(json::from_bjdata(v_D), true) == v_D);
|
||
CHECK(json::to_bjdata(json::from_bjdata(v_S), true) == v_S);
|
||
CHECK(json::to_bjdata(json::from_bjdata(v_C), true) == v_S); // char is serialized to string
|
||
CHECK(json::to_bjdata(json::from_bjdata(v_B), true) == v_U); // byte is serialized to uint8
|
||
}
|
||
|
||
SECTION("optimized version (type and length)")
|
||
{
|
||
// create vector with two elements of the same type
|
||
std::vector<uint8_t> const v_i = {'[', '$', 'i', '#', 'i', 2, 0x7F, 0x7F};
|
||
std::vector<uint8_t> const v_U = {'[', '$', 'U', '#', 'i', 2, 0xFF, 0xFF};
|
||
std::vector<uint8_t> const v_I = {'[', '$', 'I', '#', 'i', 2, 0xFF, 0x7F, 0xFF, 0x7F};
|
||
std::vector<uint8_t> const v_u = {'[', '$', 'u', '#', 'i', 2, 0x0F, 0xA7, 0x0F, 0xA7};
|
||
std::vector<uint8_t> const v_l = {'[', '$', 'l', '#', 'i', 2, 0xFF, 0xFF, 0xFF, 0x7F, 0xFF, 0xFF, 0xFF, 0x7F};
|
||
std::vector<uint8_t> const v_m = {'[', '$', 'm', '#', 'i', 2, 0xFF, 0xC9, 0x9A, 0xBB, 0xFF, 0xC9, 0x9A, 0xBB};
|
||
std::vector<uint8_t> const v_L = {'[', '$', 'L', '#', 'i', 2, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0x7F, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0x7F};
|
||
std::vector<uint8_t> const v_M = {'[', '$', 'M', '#', 'i', 2, 0xFF, 0xFF, 0x63, 0xA7, 0xB3, 0xB6, 0xE0, 0x8D, 0xFF, 0xFF, 0x63, 0xA7, 0xB3, 0xB6, 0xE0, 0x8D};
|
||
std::vector<uint8_t> const v_D = {'[', '$', 'D', '#', 'i', 2, 0x4a, 0xd8, 0x12, 0x4d, 0xfb, 0x21, 0x09, 0x40, 0x4a, 0xd8, 0x12, 0x4d, 0xfb, 0x21, 0x09, 0x40};
|
||
std::vector<uint8_t> const v_S = {'[', '#', 'i', 2, 'S', 'i', 1, 'a', 'S', 'i', 1, 'a'};
|
||
std::vector<uint8_t> const v_C = {'[', '$', 'C', '#', 'i', 2, 'a', 'a'};
|
||
std::vector<uint8_t> const v_B = {'[', '$', 'B', '#', 'i', 2, 0xFF, 0xFF};
|
||
|
||
// check if vector is parsed correctly
|
||
CHECK(json::from_bjdata(v_i) == json({127, 127}));
|
||
CHECK(json::from_bjdata(v_U) == json({255, 255}));
|
||
CHECK(json::from_bjdata(v_I) == json({32767, 32767}));
|
||
CHECK(json::from_bjdata(v_u) == json({42767, 42767}));
|
||
CHECK(json::from_bjdata(v_l) == json({2147483647, 2147483647}));
|
||
CHECK(json::from_bjdata(v_m) == json({3147483647, 3147483647}));
|
||
CHECK(json::from_bjdata(v_L) == json({9223372036854775807, 9223372036854775807}));
|
||
CHECK(json::from_bjdata(v_M) == json({10223372036854775807ull, 10223372036854775807ull}));
|
||
CHECK(json::from_bjdata(v_D) == json({3.1415926, 3.1415926}));
|
||
CHECK(json::from_bjdata(v_S) == json({"a", "a"}));
|
||
CHECK(json::from_bjdata(v_C) == json({"a", "a"}));
|
||
CHECK(json::from_bjdata(v_B) == json::binary(std::vector<uint8_t>({static_cast<uint8_t>(255), static_cast<uint8_t>(255)})));
|
||
|
||
// roundtrip: output should be optimized
|
||
std::vector<uint8_t> const v_empty = {'[', '#', 'i', 0};
|
||
CHECK(json::to_bjdata(json::from_bjdata(v_i), true, true) == v_i);
|
||
CHECK(json::to_bjdata(json::from_bjdata(v_U), true, true) == v_U);
|
||
CHECK(json::to_bjdata(json::from_bjdata(v_I), true, true) == v_I);
|
||
CHECK(json::to_bjdata(json::from_bjdata(v_u), true, true) == v_u);
|
||
CHECK(json::to_bjdata(json::from_bjdata(v_l), true, true) == v_l);
|
||
CHECK(json::to_bjdata(json::from_bjdata(v_m), true, true) == v_m);
|
||
CHECK(json::to_bjdata(json::from_bjdata(v_L), true, true) == v_L);
|
||
CHECK(json::to_bjdata(json::from_bjdata(v_M), true, true) == v_M);
|
||
CHECK(json::to_bjdata(json::from_bjdata(v_D), true, true) == v_D);
|
||
CHECK(json::to_bjdata(json::from_bjdata(v_S), true, true) == v_S);
|
||
CHECK(json::to_bjdata(json::from_bjdata(v_C), true, true) == v_S); // char is serialized to string
|
||
CHECK(json::to_bjdata(json::from_bjdata(v_B), true, true, json::bjdata_version_t::draft2) == v_U);
|
||
CHECK(json::to_bjdata(json::from_bjdata(v_B), true, true, json::bjdata_version_t::draft3) == v_B);
|
||
}
|
||
|
||
SECTION("optimized ndarray (type and vector-size as optimized 1D array)")
|
||
{
|
||
// create vector with two elements of the same type
|
||
std::vector<uint8_t> const v_0 = {'[', '$', 'i', '#', '[', '$', 'i', '#', 'i', 1, 0};
|
||
std::vector<uint8_t> const v_1 = {'[', '$', 'i', '#', '[', '$', 'i', '#', 'i', 1, 2, 0x7F, 0x7F};
|
||
std::vector<uint8_t> const v_i = {'[', '$', 'i', '#', '[', '$', 'i', '#', 'i', 2, 1, 2, 0x7F, 0x7F};
|
||
std::vector<uint8_t> const v_U = {'[', '$', 'U', '#', '[', '$', 'i', '#', 'i', 2, 1, 2, 0xFF, 0xFF};
|
||
std::vector<uint8_t> const v_I = {'[', '$', 'I', '#', '[', '$', 'i', '#', 'i', 2, 1, 2, 0xFF, 0x7F, 0xFF, 0x7F};
|
||
std::vector<uint8_t> const v_u = {'[', '$', 'u', '#', '[', '$', 'i', '#', 'i', 2, 1, 2, 0x0F, 0xA7, 0x0F, 0xA7};
|
||
std::vector<uint8_t> const v_l = {'[', '$', 'l', '#', '[', '$', 'i', '#', 'i', 2, 1, 2, 0xFF, 0xFF, 0xFF, 0x7F, 0xFF, 0xFF, 0xFF, 0x7F};
|
||
std::vector<uint8_t> const v_m = {'[', '$', 'm', '#', '[', '$', 'i', '#', 'i', 2, 1, 2, 0xFF, 0xC9, 0x9A, 0xBB, 0xFF, 0xC9, 0x9A, 0xBB};
|
||
std::vector<uint8_t> const v_L = {'[', '$', 'L', '#', '[', '$', 'i', '#', 'i', 2, 1, 2, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0x7F, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0x7F};
|
||
std::vector<uint8_t> const v_M = {'[', '$', 'M', '#', '[', '$', 'i', '#', 'i', 2, 1, 2, 0xFF, 0xFF, 0x63, 0xA7, 0xB3, 0xB6, 0xE0, 0x8D, 0xFF, 0xFF, 0x63, 0xA7, 0xB3, 0xB6, 0xE0, 0x8D};
|
||
std::vector<uint8_t> const v_D = {'[', '$', 'D', '#', '[', '$', 'i', '#', 'i', 2, 1, 2, 0x4a, 0xd8, 0x12, 0x4d, 0xfb, 0x21, 0x09, 0x40, 0x4a, 0xd8, 0x12, 0x4d, 0xfb, 0x21, 0x09, 0x40};
|
||
std::vector<uint8_t> const v_S = {'[', '#', '[', '$', 'i', '#', 'i', 2, 1, 2, 'S', 'i', 1, 'a', 'S', 'i', 1, 'a'};
|
||
std::vector<uint8_t> const v_C = {'[', '$', 'C', '#', '[', '$', 'i', '#', 'i', 2, 1, 2, 'a', 'a'};
|
||
std::vector<uint8_t> const v_B = {'[', '$', 'B', '#', '[', '$', 'i', '#', 'i', 2, 1, 2, 0xFF, 0xFF};
|
||
|
||
// check if vector is parsed correctly
|
||
CHECK(json::from_bjdata(v_0) == json::array());
|
||
CHECK(json::from_bjdata(v_1) == json({127, 127}));
|
||
CHECK(json::from_bjdata(v_i) == json({127, 127}));
|
||
CHECK(json::from_bjdata(v_U) == json({255, 255}));
|
||
CHECK(json::from_bjdata(v_I) == json({32767, 32767}));
|
||
CHECK(json::from_bjdata(v_u) == json({42767, 42767}));
|
||
CHECK(json::from_bjdata(v_l) == json({2147483647, 2147483647}));
|
||
CHECK(json::from_bjdata(v_m) == json({3147483647, 3147483647}));
|
||
CHECK(json::from_bjdata(v_L) == json({9223372036854775807, 9223372036854775807}));
|
||
CHECK(json::from_bjdata(v_M) == json({10223372036854775807ull, 10223372036854775807ull}));
|
||
CHECK(json::from_bjdata(v_D) == json({3.1415926, 3.1415926}));
|
||
CHECK(json::from_bjdata(v_S) == json({"a", "a"}));
|
||
CHECK(json::from_bjdata(v_C) == json({"a", "a"}));
|
||
CHECK(json::from_bjdata(v_B) == json::binary(std::vector<uint8_t>({static_cast<uint8_t>(255), static_cast<uint8_t>(255)})));
|
||
}
|
||
|
||
SECTION("optimized ndarray (type and vector-size ndarray with JData annotations)")
|
||
{
|
||
// create vector with 0, 1, 2 elements of the same type
|
||
std::vector<uint8_t> const v_e = {'[', '$', 'U', '#', '[', '$', 'i', '#', 'i', 2, 2, 1, 0xFE, 0xFF};
|
||
std::vector<uint8_t> const v_U = {'[', '$', 'U', '#', '[', '$', 'i', '#', 'i', 2, 2, 3, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06};
|
||
std::vector<uint8_t> const v_i = {'[', '$', 'i', '#', '[', '$', 'i', '#', 'i', 2, 2, 3, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06};
|
||
std::vector<uint8_t> const v_u = {'[', '$', 'u', '#', '[', '$', 'i', '#', 'i', 2, 2, 3, 0x01, 0x00, 0x02, 0x00, 0x03, 0x00, 0x04, 0x00, 0x05, 0x00, 0x06, 0x00};
|
||
std::vector<uint8_t> const v_I = {'[', '$', 'I', '#', '[', '$', 'i', '#', 'i', 2, 2, 3, 0x01, 0x00, 0x02, 0x00, 0x03, 0x00, 0x04, 0x00, 0x05, 0x00, 0x06, 0x00};
|
||
std::vector<uint8_t> const v_m = {'[', '$', 'm', '#', '[', '$', 'i', '#', 'i', 2, 2, 3, 0x01, 0x00, 0x00, 0x00, 0x02, 0x00, 0x00, 0x00, 0x03, 0x00, 0x00, 0x00, 0x04, 0x00, 0x00, 0x00, 0x05, 0x00, 0x00, 0x00, 0x06, 0x00, 0x00, 0x00};
|
||
std::vector<uint8_t> const v_l = {'[', '$', 'l', '#', '[', '$', 'i', '#', 'i', 2, 2, 3, 0x01, 0x00, 0x00, 0x00, 0x02, 0x00, 0x00, 0x00, 0x03, 0x00, 0x00, 0x00, 0x04, 0x00, 0x00, 0x00, 0x05, 0x00, 0x00, 0x00, 0x06, 0x00, 0x00, 0x00};
|
||
std::vector<uint8_t> const v_M = {'[', '$', 'M', '#', '[', '$', 'i', '#', 'i', 2, 2, 3, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x03, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x04, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x05, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x06, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00};
|
||
std::vector<uint8_t> const v_L = {'[', '$', 'L', '#', '[', '$', 'i', '#', 'i', 2, 2, 3, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x03, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x04, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x05, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x06, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00};
|
||
std::vector<uint8_t> const v_d = {'[', '$', 'd', '#', '[', '$', 'i', '#', 'i', 2, 2, 3, 0x00, 0x00, 0x80, 0x3F, 0x00, 0x00, 0x00, 0x40, 0x00, 0x00, 0x40, 0x40, 0x00, 0x00, 0x80, 0x40, 0x00, 0x00, 0xA0, 0x40, 0x00, 0x00, 0xC0, 0x40};
|
||
std::vector<uint8_t> const v_D = {'[', '$', 'D', '#', '[', '$', 'i', '#', 'i', 2, 2, 3, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xF0, 0x3F, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x40, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x08, 0x40, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x10, 0x40, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x14, 0x40, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x18, 0x40};
|
||
std::vector<uint8_t> const v_C = {'[', '$', 'C', '#', '[', '$', 'i', '#', 'i', 2, 2, 3, 'a', 'b', 'c', 'd', 'e', 'f'};
|
||
std::vector<uint8_t> const v_B = {'[', '$', 'B', '#', '[', '$', 'i', '#', 'i', 2, 2, 3, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06};
|
||
|
||
// check if vector is parsed correctly
|
||
CHECK(json::from_bjdata(v_e) == json({{"_ArrayData_", {254, 255}}, {"_ArraySize_", {2, 1}}, {"_ArrayType_", "uint8"}}));
|
||
CHECK(json::from_bjdata(v_U) == json({{"_ArrayData_", {1, 2, 3, 4, 5, 6}}, {"_ArraySize_", {2, 3}}, {"_ArrayType_", "uint8"}}));
|
||
CHECK(json::from_bjdata(v_i) == json({{"_ArrayData_", {1, 2, 3, 4, 5, 6}}, {"_ArraySize_", {2, 3}}, {"_ArrayType_", "int8"}}));
|
||
CHECK(json::from_bjdata(v_i) == json({{"_ArrayData_", {1, 2, 3, 4, 5, 6}}, {"_ArraySize_", {2, 3}}, {"_ArrayType_", "int8"}}));
|
||
CHECK(json::from_bjdata(v_u) == json({{"_ArrayData_", {1, 2, 3, 4, 5, 6}}, {"_ArraySize_", {2, 3}}, {"_ArrayType_", "uint16"}}));
|
||
CHECK(json::from_bjdata(v_I) == json({{"_ArrayData_", {1, 2, 3, 4, 5, 6}}, {"_ArraySize_", {2, 3}}, {"_ArrayType_", "int16"}}));
|
||
CHECK(json::from_bjdata(v_m) == json({{"_ArrayData_", {1, 2, 3, 4, 5, 6}}, {"_ArraySize_", {2, 3}}, {"_ArrayType_", "uint32"}}));
|
||
CHECK(json::from_bjdata(v_l) == json({{"_ArrayData_", {1, 2, 3, 4, 5, 6}}, {"_ArraySize_", {2, 3}}, {"_ArrayType_", "int32"}}));
|
||
CHECK(json::from_bjdata(v_M) == json({{"_ArrayData_", {1, 2, 3, 4, 5, 6}}, {"_ArraySize_", {2, 3}}, {"_ArrayType_", "uint64"}}));
|
||
CHECK(json::from_bjdata(v_L) == json({{"_ArrayData_", {1, 2, 3, 4, 5, 6}}, {"_ArraySize_", {2, 3}}, {"_ArrayType_", "int64"}}));
|
||
CHECK(json::from_bjdata(v_d) == json({{"_ArrayData_", {1.f, 2.f, 3.f, 4.f, 5.f, 6.f}}, {"_ArraySize_", {2, 3}}, {"_ArrayType_", "single"}}));
|
||
CHECK(json::from_bjdata(v_D) == json({{"_ArrayData_", {1., 2., 3., 4., 5., 6.}}, {"_ArraySize_", {2, 3}}, {"_ArrayType_", "double"}}));
|
||
CHECK(json::from_bjdata(v_C) == json({{"_ArrayData_", {'a', 'b', 'c', 'd', 'e', 'f'}}, {"_ArraySize_", {2, 3}}, {"_ArrayType_", "char"}}));
|
||
CHECK(json::from_bjdata(v_B) == json({{"_ArrayData_", {1, 2, 3, 4, 5, 6}}, {"_ArraySize_", {2, 3}}, {"_ArrayType_", "byte"}}));
|
||
|
||
// roundtrip: output should be optimized
|
||
CHECK(json::to_bjdata(json::from_bjdata(v_e), true, true) == v_e);
|
||
CHECK(json::to_bjdata(json::from_bjdata(v_U), true, true) == v_U);
|
||
CHECK(json::to_bjdata(json::from_bjdata(v_i), true, true) == v_i);
|
||
CHECK(json::to_bjdata(json::from_bjdata(v_u), true, true) == v_u);
|
||
CHECK(json::to_bjdata(json::from_bjdata(v_I), true, true) == v_I);
|
||
CHECK(json::to_bjdata(json::from_bjdata(v_m), true, true) == v_m);
|
||
CHECK(json::to_bjdata(json::from_bjdata(v_l), true, true) == v_l);
|
||
CHECK(json::to_bjdata(json::from_bjdata(v_M), true, true) == v_M);
|
||
CHECK(json::to_bjdata(json::from_bjdata(v_L), true, true) == v_L);
|
||
CHECK(json::to_bjdata(json::from_bjdata(v_d), true, true) == v_d);
|
||
CHECK(json::to_bjdata(json::from_bjdata(v_D), true, true) == v_D);
|
||
CHECK(json::to_bjdata(json::from_bjdata(v_C), true, true) == v_C);
|
||
// v_B uses the Draft-3-only 'B' marker, so it round-trips only when
|
||
// Draft 3 is explicitly selected (see GitHub issue #5404); the
|
||
// default Draft 2 falls back to a plain object instead, covered by
|
||
// the "ndarray with _ArrayType_ "byte" is gated by the BJData draft
|
||
// version" section below
|
||
CHECK(json::to_bjdata(json::from_bjdata(v_B), true, true, json::bjdata_version_t::draft3) == v_B);
|
||
}
|
||
|
||
SECTION("ndarray with data not matching _ArrayType_ is written as an object")
|
||
{
|
||
// A JData-annotated object is only serialized as an ndarray when
|
||
// its _ArrayData_ elements are actually stored as the number kind
|
||
// named by _ArrayType_. Otherwise the writer would read the wrong
|
||
// union member (e.g. a std::string's heap pointer as a uint64) and
|
||
// emit it, so such an object falls back to a plain object encoding
|
||
// that still round-trips.
|
||
|
||
// string data declared as a uint64 array
|
||
json const j_str = json({{"_ArrayType_", "uint64"}, {"_ArraySize_", {2, 1}}, {"_ArrayData_", {"pointer", "value"}}});
|
||
const auto out_str = json::to_bjdata(j_str);
|
||
CHECK(out_str.at(0) == '{');
|
||
CHECK(json::from_bjdata(out_str) == j_str);
|
||
|
||
// integer data declared as a double array
|
||
json const j_float = json({{"_ArrayType_", "double"}, {"_ArraySize_", {2, 1}}, {"_ArrayData_", {1, 2}}});
|
||
const auto out_float = json::to_bjdata(j_float);
|
||
CHECK(out_float.at(0) == '{');
|
||
CHECK(json::from_bjdata(out_float) == j_float);
|
||
|
||
// a non-integer shape entry is likewise not treated as an ndarray
|
||
json const j_size = json({{"_ArrayType_", "uint8"}, {"_ArraySize_", {"x", 1}}, {"_ArrayData_", {1}}});
|
||
const auto out_size = json::to_bjdata(j_size);
|
||
CHECK(out_size.at(0) == '{');
|
||
CHECK(json::from_bjdata(out_size) == j_size);
|
||
|
||
// a negative shape entry is not a usable dimension either
|
||
json const j_neg = json::parse(R"({"_ArrayType_":"uint8","_ArraySize_":[-1,1],"_ArrayData_":[1]})");
|
||
const auto out_neg = json::to_bjdata(j_neg);
|
||
CHECK(out_neg.at(0) == '{');
|
||
CHECK(json::from_bjdata(out_neg) == j_neg);
|
||
}
|
||
|
||
SECTION("ndarray parsed from text is written as a typed array")
|
||
{
|
||
// json::parse stores a non-negative integer as number_unsigned while
|
||
// the C++ API stores an int literal as number_integer, so _ArrayType_
|
||
// names the wire type rather than the storage. Both storages have to
|
||
// produce the same typed array for every type.
|
||
// "byte" is checked separately below since it additionally requires
|
||
// BJData Draft 3 to be selected explicitly (see GitHub issue #5404).
|
||
for (const char* type :
|
||
{"uint8", "int8", "uint16", "int16", "uint32", "int32", "uint64", "int64", "char"
|
||
})
|
||
{
|
||
CAPTURE(type)
|
||
const std::string text = std::string(R"({"_ArrayType_":")") + type +
|
||
R"(","_ArraySize_":[2,3],"_ArrayData_":[1,2,3,4,5,6]})";
|
||
const auto from_text = json::to_bjdata(json::parse(text));
|
||
CHECK(from_text.at(0) == '[');
|
||
CHECK(from_text == json::to_bjdata(json({{"_ArrayType_", type}, {"_ArraySize_", {2, 3}}, {"_ArrayData_", {1, 2, 3, 4, 5, 6}}})));
|
||
}
|
||
|
||
{
|
||
const std::string text = R"({"_ArrayType_":"byte","_ArraySize_":[2,3],"_ArrayData_":[1,2,3,4,5,6]})";
|
||
const auto from_text = json::to_bjdata(json::parse(text), true, true, json::bjdata_version_t::draft3);
|
||
CHECK(from_text.at(0) == '[');
|
||
CHECK(from_text == json::to_bjdata(json({{"_ArrayType_", "byte"}, {"_ArraySize_", {2, 3}}, {"_ArrayData_", {1, 2, 3, 4, 5, 6}}}),
|
||
true, true, json::bjdata_version_t::draft3));
|
||
}
|
||
|
||
// negative values under a signed type behave the same way
|
||
const auto from_neg = json::to_bjdata(json::parse(R"({"_ArrayType_":"int32","_ArraySize_":[2,1],"_ArrayData_":[-5,7]})"));
|
||
CHECK(from_neg.at(0) == '[');
|
||
CHECK(from_neg == json::to_bjdata(json({{"_ArrayType_", "int32"}, {"_ArraySize_", {2, 1}}, {"_ArrayData_", {-5, 7}}})));
|
||
|
||
// and so do the floating point types
|
||
const auto from_float = json::to_bjdata(json::parse(R"({"_ArrayType_":"double","_ArraySize_":[2,1],"_ArrayData_":[1.5,2.5]})"));
|
||
CHECK(from_float.at(0) == '[');
|
||
CHECK(from_float == json::to_bjdata(json({{"_ArrayType_", "double"}, {"_ArraySize_", {2, 1}}, {"_ArrayData_", {1.5, 2.5}}})));
|
||
}
|
||
|
||
SECTION("optimized ndarray (type and vector-size as 1D array)")
|
||
{
|
||
// create vector with two elements of the same type
|
||
std::vector<uint8_t> const v_0 = {'[', '$', 'i', '#', '[', ']'};
|
||
std::vector<uint8_t> const v_E = {'[', '$', 'i', '#', '[', 'i', 2, 'i', 0, ']'};
|
||
std::vector<uint8_t> const v_i = {'[', '$', 'i', '#', '[', 'i', 1, 'i', 2, ']', 0x7F, 0x7F};
|
||
std::vector<uint8_t> const v_U = {'[', '$', 'U', '#', '[', 'i', 1, 'i', 2, ']', 0xFF, 0xFF};
|
||
std::vector<uint8_t> const v_I = {'[', '$', 'I', '#', '[', 'i', 1, 'i', 2, ']', 0xFF, 0x7F, 0xFF, 0x7F};
|
||
std::vector<uint8_t> const v_u = {'[', '$', 'u', '#', '[', 'i', 1, 'i', 2, ']', 0x0F, 0xA7, 0x0F, 0xA7};
|
||
std::vector<uint8_t> const v_l = {'[', '$', 'l', '#', '[', 'i', 1, 'i', 2, ']', 0xFF, 0xFF, 0xFF, 0x7F, 0xFF, 0xFF, 0xFF, 0x7F};
|
||
std::vector<uint8_t> const v_m = {'[', '$', 'm', '#', '[', 'i', 1, 'i', 2, ']', 0xFF, 0xC9, 0x9A, 0xBB, 0xFF, 0xC9, 0x9A, 0xBB};
|
||
std::vector<uint8_t> const v_L = {'[', '$', 'L', '#', '[', 'i', 1, 'i', 2, ']', 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0x7F, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0x7F};
|
||
std::vector<uint8_t> const v_M = {'[', '$', 'M', '#', '[', 'i', 1, 'i', 2, ']', 0xFF, 0xFF, 0x63, 0xA7, 0xB3, 0xB6, 0xE0, 0x8D, 0xFF, 0xFF, 0x63, 0xA7, 0xB3, 0xB6, 0xE0, 0x8D};
|
||
std::vector<uint8_t> const v_D = {'[', '$', 'D', '#', '[', 'i', 1, 'i', 2, ']', 0x4a, 0xd8, 0x12, 0x4d, 0xfb, 0x21, 0x09, 0x40, 0x4a, 0xd8, 0x12, 0x4d, 0xfb, 0x21, 0x09, 0x40};
|
||
std::vector<uint8_t> const v_S = {'[', '#', '[', 'i', 1, 'i', 2, ']', 'S', 'i', 1, 'a', 'S', 'i', 1, 'a'};
|
||
std::vector<uint8_t> const v_C = {'[', '$', 'C', '#', '[', 'i', 1, 'i', 2, ']', 'a', 'a'};
|
||
std::vector<uint8_t> const v_B = {'[', '$', 'B', '#', '[', 'i', 1, 'i', 2, ']', 0xFF, 0xFF};
|
||
std::vector<uint8_t> const v_R = {'[', '#', '[', 'i', 2, ']', 'i', 6, 'U', 7};
|
||
|
||
// check if vector is parsed correctly
|
||
CHECK(json::from_bjdata(v_0) == json::array());
|
||
CHECK(json::from_bjdata(v_E) == json::array());
|
||
CHECK(json::from_bjdata(v_i) == json({127, 127}));
|
||
CHECK(json::from_bjdata(v_U) == json({255, 255}));
|
||
CHECK(json::from_bjdata(v_I) == json({32767, 32767}));
|
||
CHECK(json::from_bjdata(v_u) == json({42767, 42767}));
|
||
CHECK(json::from_bjdata(v_l) == json({2147483647, 2147483647}));
|
||
CHECK(json::from_bjdata(v_m) == json({3147483647, 3147483647}));
|
||
CHECK(json::from_bjdata(v_L) == json({9223372036854775807, 9223372036854775807}));
|
||
CHECK(json::from_bjdata(v_M) == json({10223372036854775807ull, 10223372036854775807ull}));
|
||
CHECK(json::from_bjdata(v_D) == json({3.1415926, 3.1415926}));
|
||
CHECK(json::from_bjdata(v_S) == json({"a", "a"}));
|
||
CHECK(json::from_bjdata(v_C) == json({"a", "a"}));
|
||
CHECK(json::from_bjdata(v_B) == json::binary(std::vector<uint8_t>({static_cast<uint8_t>(255), static_cast<uint8_t>(255)})));
|
||
CHECK(json::from_bjdata(v_R) == json({6, 7}));
|
||
}
|
||
|
||
SECTION("optimized ndarray (type and vector-size as size-optimized array)")
|
||
{
|
||
// create vector with two elements of the same type
|
||
std::vector<uint8_t> const v_i = {'[', '$', 'i', '#', '[', '#', 'i', 2, 'i', 1, 'i', 2, 0x7F, 0x7F};
|
||
std::vector<uint8_t> const v_U = {'[', '$', 'U', '#', '[', '#', 'i', 2, 'i', 1, 'i', 2, 0xFF, 0xFF};
|
||
std::vector<uint8_t> const v_I = {'[', '$', 'I', '#', '[', '#', 'i', 2, 'i', 1, 'i', 2, 0xFF, 0x7F, 0xFF, 0x7F};
|
||
std::vector<uint8_t> const v_u = {'[', '$', 'u', '#', '[', '#', 'i', 2, 'i', 1, 'i', 2, 0x0F, 0xA7, 0x0F, 0xA7};
|
||
std::vector<uint8_t> const v_l = {'[', '$', 'l', '#', '[', '#', 'i', 2, 'i', 1, 'i', 2, 0xFF, 0xFF, 0xFF, 0x7F, 0xFF, 0xFF, 0xFF, 0x7F};
|
||
std::vector<uint8_t> const v_m = {'[', '$', 'm', '#', '[', '#', 'i', 2, 'i', 1, 'i', 2, 0xFF, 0xC9, 0x9A, 0xBB, 0xFF, 0xC9, 0x9A, 0xBB};
|
||
std::vector<uint8_t> const v_L = {'[', '$', 'L', '#', '[', '#', 'i', 2, 'i', 1, 'i', 2, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0x7F, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0x7F};
|
||
std::vector<uint8_t> const v_M = {'[', '$', 'M', '#', '[', '#', 'i', 2, 'i', 1, 'i', 2, 0xFF, 0xFF, 0x63, 0xA7, 0xB3, 0xB6, 0xE0, 0x8D, 0xFF, 0xFF, 0x63, 0xA7, 0xB3, 0xB6, 0xE0, 0x8D};
|
||
std::vector<uint8_t> const v_D = {'[', '$', 'D', '#', '[', '#', 'i', 2, 'i', 1, 'i', 2, 0x4a, 0xd8, 0x12, 0x4d, 0xfb, 0x21, 0x09, 0x40, 0x4a, 0xd8, 0x12, 0x4d, 0xfb, 0x21, 0x09, 0x40};
|
||
std::vector<uint8_t> const v_S = {'[', '#', '[', '#', 'i', 2, 'i', 1, 'i', 2, 'S', 'i', 1, 'a', 'S', 'i', 1, 'a'};
|
||
std::vector<uint8_t> const v_C = {'[', '$', 'C', '#', '[', '#', 'i', 2, 'i', 1, 'i', 2, 'a', 'a'};
|
||
std::vector<uint8_t> const v_B = {'[', '$', 'B', '#', '[', '#', 'i', 2, 'i', 1, 'i', 2, 0xFF, 0xFF};
|
||
|
||
// check if vector is parsed correctly
|
||
CHECK(json::from_bjdata(v_i) == json({127, 127}));
|
||
CHECK(json::from_bjdata(v_U) == json({255, 255}));
|
||
CHECK(json::from_bjdata(v_I) == json({32767, 32767}));
|
||
CHECK(json::from_bjdata(v_u) == json({42767, 42767}));
|
||
CHECK(json::from_bjdata(v_l) == json({2147483647, 2147483647}));
|
||
CHECK(json::from_bjdata(v_m) == json({3147483647, 3147483647}));
|
||
CHECK(json::from_bjdata(v_L) == json({9223372036854775807, 9223372036854775807}));
|
||
CHECK(json::from_bjdata(v_M) == json({10223372036854775807ull, 10223372036854775807ull}));
|
||
CHECK(json::from_bjdata(v_D) == json({3.1415926, 3.1415926}));
|
||
CHECK(json::from_bjdata(v_S) == json({"a", "a"}));
|
||
CHECK(json::from_bjdata(v_C) == json({"a", "a"}));
|
||
CHECK(json::from_bjdata(v_B) == json::binary(std::vector<uint8_t>({static_cast<uint8_t>(255), static_cast<uint8_t>(255)})));
|
||
}
|
||
|
||
SECTION("invalid ndarray annotations remains as object")
|
||
{
|
||
// check if invalid ND array annotations stay as object
|
||
json j_type = json({{"_ArrayData_", {1, 2, 3, 4, 5, 6}}, {"_ArraySize_", {2, 3}}, {"_ArrayType_", "invalidtype"}});
|
||
json j_size = json({{"_ArrayData_", {1, 2, 3, 4, 5}}, {"_ArraySize_", {2, 3}}, {"_ArrayType_", "uint8"}});
|
||
|
||
// roundtrip: output should stay as object
|
||
CHECK(json::from_bjdata(json::to_bjdata(j_type), true, true) == j_type);
|
||
CHECK(json::from_bjdata(json::to_bjdata(j_size), true, true) == j_size);
|
||
}
|
||
|
||
SECTION("ndarray whose _ArrayType_ is not a string stays as object")
|
||
{
|
||
// the type name is looked up as a string below the annotation
|
||
// check; a non-string _ArrayType_ cannot name a known dtype,
|
||
// so calling get<string_t>() on it would throw type_error.302
|
||
// instead of falling back like an unrecognized type name
|
||
// already does (see GitHub issue #5398)
|
||
json const j_number = json({{"_ArrayType_", 1}, {"_ArraySize_", {2}}, {"_ArrayData_", {1, 2}}});
|
||
const auto out_number = json::to_bjdata(j_number);
|
||
CHECK(out_number.at(0) == '{');
|
||
CHECK(json::from_bjdata(out_number) == j_number);
|
||
|
||
json const j_null = json({{"_ArrayType_", nullptr}, {"_ArraySize_", {2}}, {"_ArrayData_", {1, 2}}});
|
||
const auto out_null = json::to_bjdata(j_null);
|
||
CHECK(out_null.at(0) == '{');
|
||
CHECK(json::from_bjdata(out_null) == j_null);
|
||
|
||
json const j_bool = json({{"_ArrayType_", true}, {"_ArraySize_", {2}}, {"_ArrayData_", {1, 2}}});
|
||
const auto out_bool = json::to_bjdata(j_bool);
|
||
CHECK(out_bool.at(0) == '{');
|
||
CHECK(json::from_bjdata(out_bool) == j_bool);
|
||
|
||
json const j_array = json({{"_ArrayType_", {"uint8"}}, {"_ArraySize_", {2}}, {"_ArrayData_", {1, 2}}});
|
||
const auto out_array = json::to_bjdata(j_array);
|
||
CHECK(out_array.at(0) == '{');
|
||
CHECK(json::from_bjdata(out_array) == j_array);
|
||
|
||
json const j_object = json({{"_ArrayType_", {{"a", 1}}}, {"_ArraySize_", {2}}, {"_ArrayData_", {1, 2}}});
|
||
const auto out_object = json::to_bjdata(j_object);
|
||
CHECK(out_object.at(0) == '{');
|
||
CHECK(json::from_bjdata(out_object) == j_object);
|
||
}
|
||
|
||
SECTION("re-serializing a value containing a plain-array-of-bytes is value-stable but not byte-stable")
|
||
{
|
||
// OSS-Fuzz found this input (an array whose first element is a
|
||
// binary_t byte, followed by an object whose _ArrayType_ is
|
||
// not a string) while exercising the fix for #5398 above: once
|
||
// the fix stops to_bjdata() from throwing type_error.302 for
|
||
// the third element, serialization proceeds far enough to
|
||
// reach a pre-existing, unrelated round-trip quirk in how a
|
||
// single-byte binary_t value is re-encoded.
|
||
std::vector<std::uint8_t> const input
|
||
{
|
||
0x5b, 0x5b, 0x24, 0x42, 0x23, 0x5b, 0x69, 0x01, 0x5d, 0x5b, 0x5b, 0x5d, 0x7b, 0x55, 0x0b,
|
||
0x5f, 0x41, 0x72, 0x72, 0x61, 0x79, 0x44, 0x61, 0x74, 0x61, 0x5f, 0x54, 0x55, 0x0b, 0x5f,
|
||
0x41, 0x72, 0x72, 0x61, 0x79, 0x53, 0x69, 0x7a, 0x65, 0x5f, 0x5a, 0x55, 0x0b, 0x5f, 0x41,
|
||
0x72, 0x72, 0x61, 0x79, 0x54, 0x79, 0x70, 0x65, 0x5f, 0x54, 0x7d, 0x5d
|
||
};
|
||
json const j1 = json::from_bjdata(input);
|
||
|
||
// to_bjdata() must not throw (this is what #5398 fixes)
|
||
std::vector<std::uint8_t> vec2;
|
||
CHECK_NOTHROW(vec2 = json::to_bjdata(j1, false, false));
|
||
|
||
// parsing back a plain (non-optimized) array of bytes cannot
|
||
// recover that it used to be a binary_t: from_bjdata() has no
|
||
// way to distinguish "array of uint8 numbers" from "array of
|
||
// bytes" unless the compact "$U#" array header is used, so
|
||
// the binary_t collapses into a plain JSON array
|
||
json const j2 = json::from_bjdata(vec2);
|
||
CHECK(j1 != j2);
|
||
CHECK(j2 == json({{91}, json::array(), {{"_ArrayData_", true}, {"_ArraySize_", nullptr}, {"_ArrayType_", true}}}));
|
||
|
||
// re-serializing j2 no longer goes through the dedicated
|
||
// binary_t writer (which always uses the 'U' marker for raw
|
||
// bytes); the now-plain number 91 goes through the generic
|
||
// smallest-type writer instead, which - like the rest of the
|
||
// UBJSON/BJData writer, and unchanged by this fix - prefers
|
||
// the 'i' (int8) marker over 'U' (uint8) for values that fit
|
||
// both. Both markers are valid BJData and both decode back to
|
||
// 91, so this is not byte-for-byte identical to vec2, but it
|
||
// is value-stable: parsing it again reproduces j2 exactly.
|
||
std::vector<std::uint8_t> const vec3 = json::to_bjdata(j2, false, false);
|
||
CHECK(json::from_bjdata(vec3) == j2);
|
||
}
|
||
|
||
SECTION("ndarray whose dimensions overflow stays as object")
|
||
{
|
||
// the product of the dimensions wraps around std::size_t to 0
|
||
// and so matches the size of the empty _ArrayData_; writing this
|
||
// as an ndarray would announce an element count no reader can
|
||
// honor, so it has to stay a plain object
|
||
json j_overflow = json({{"_ArrayData_", json::array()}, {"_ArraySize_", {9223372036854775808ull, 2}}, {"_ArrayType_", "uint8"}});
|
||
CHECK(json::from_bjdata(json::to_bjdata(j_overflow), true, true) == j_overflow);
|
||
|
||
// a single dimension that does not fit into std::size_t is
|
||
// rejected for the same reason (only observable where
|
||
// std::size_t is narrower than 64 bit)
|
||
json j_huge = json({{"_ArrayData_", json::array()}, {"_ArraySize_", {18446744073709551615ull, 2}}, {"_ArrayType_", "uint8"}});
|
||
CHECK(json::from_bjdata(json::to_bjdata(j_huge), true, true) == j_huge);
|
||
|
||
// a well-formed ndarray is still encoded as one
|
||
json j_ok = json({{"_ArrayData_", {1, 2, 3, 4, 5, 6}}, {"_ArraySize_", {2, 3}}, {"_ArrayType_", "uint8"}});
|
||
CHECK(json::to_bjdata(j_ok) == std::vector<uint8_t>({'[', '$', 'U', '#', '[', 'i', 2, 'i', 3, ']', 1, 2, 3, 4, 5, 6}));
|
||
CHECK(json::from_bjdata(json::to_bjdata(j_ok), true, true) == j_ok);
|
||
}
|
||
|
||
SECTION("ndarray whose _ArraySize_ is not an array stays as object")
|
||
{
|
||
// the shape is written verbatim as the header length, so a
|
||
// value that is not an array cannot produce a valid one: null
|
||
// would emit 'Z' and an object '{', neither of which a reader
|
||
// accepts after '#'. Both have to stay plain objects.
|
||
json const j_null = json({{"_ArrayType_", "uint8"}, {"_ArraySize_", nullptr}, {"_ArrayData_", json::array()}});
|
||
const auto out_null = json::to_bjdata(j_null);
|
||
CHECK(out_null.at(0) == '{');
|
||
CHECK(json::from_bjdata(out_null) == j_null);
|
||
|
||
// an object shape passes the per-entry check by iterating its
|
||
// values rather than dimensions, so it needs rejecting too
|
||
json const j_obj = json({{"_ArrayType_", "uint8"}, {"_ArraySize_", {{"a", 1}}}, {"_ArrayData_", {1}}});
|
||
const auto out_obj = json::to_bjdata(j_obj);
|
||
CHECK(out_obj.at(0) == '{');
|
||
CHECK(json::from_bjdata(out_obj) == j_obj);
|
||
|
||
// a scalar shape is not a dimension list either
|
||
json const j_num = json({{"_ArrayType_", "uint8"}, {"_ArraySize_", 1}, {"_ArrayData_", {1}}});
|
||
const auto out_num = json::to_bjdata(j_num);
|
||
CHECK(out_num.at(0) == '{');
|
||
CHECK(json::from_bjdata(out_num) == j_num);
|
||
|
||
// OSS-Fuzz issue 474400817: an empty object _ArraySize_ was
|
||
// written as the ND-array header length, which from_bjdata()
|
||
// could not read back
|
||
const std::vector<uint8_t> input =
|
||
{
|
||
'[', '{', 'U', 11, '_', 'A', 'r', 'r', 'a', 'y', 'D', 'a', 't', 'a', '_', 'Z',
|
||
'U', 11, '_', 'A', 'r', 'r', 'a', 'y', 'T', 'y', 'p', 'e', '_', 'S', 'i', 5, 'i', 'n', 't', '1', '6',
|
||
'U', 11, '_', 'A', 'r', 'r', 'a', 'y', 'S', 'i', 'z', 'e', '_', '{', '}', '}', ']'
|
||
};
|
||
const json j1 = json::from_bjdata(input);
|
||
CHECK(j1 == json::parse(R"([{"_ArrayType_":"int16","_ArraySize_":{},"_ArrayData_":null}])"));
|
||
json j2;
|
||
CHECK_NOTHROW(j2 = json::from_bjdata(json::to_bjdata(j1, false, false)));
|
||
CHECK(j2 == j1);
|
||
}
|
||
|
||
SECTION("ndarray with out-of-range _ArrayData_ elements stays as object")
|
||
{
|
||
// each element is cast to the (possibly narrower) C++ type
|
||
// named by _ArrayType_ before being written; a value that
|
||
// does not fit that type would silently wrap instead of
|
||
// being reported, so such an object falls back to a plain
|
||
// object encoding that still round-trips (see GitHub issue #5403)
|
||
|
||
// an unsigned element that does not fit uint8
|
||
json const j_uint8 = json({{"_ArrayType_", "uint8"}, {"_ArraySize_", {2, 1}}, {"_ArrayData_", {1, 256}}});
|
||
const auto out_uint8 = json::to_bjdata(j_uint8);
|
||
CHECK(out_uint8.at(0) == '{');
|
||
CHECK(json::from_bjdata(out_uint8) == j_uint8);
|
||
|
||
// a signed element that does not fit int8
|
||
json const j_int8 = json({{"_ArrayType_", "int8"}, {"_ArraySize_", {2, 1}}, {"_ArrayData_", {1, 200}}});
|
||
const auto out_int8 = json::to_bjdata(j_int8);
|
||
CHECK(out_int8.at(0) == '{');
|
||
CHECK(json::from_bjdata(out_int8) == j_int8);
|
||
|
||
// a negative element is likewise out of range for an
|
||
// unsigned _ArrayType_
|
||
json const j_uint16_neg = json({{"_ArrayType_", "uint16"}, {"_ArraySize_", {2, 1}}, {"_ArrayData_", {1, -1}}});
|
||
const auto out_uint16_neg = json::to_bjdata(j_uint16_neg);
|
||
CHECK(out_uint16_neg.at(0) == '{');
|
||
CHECK(json::from_bjdata(out_uint16_neg) == j_uint16_neg);
|
||
|
||
// a double element that overflows to infinity when narrowed
|
||
// to the "single" (float) precision named by _ArrayType_
|
||
json const j_single = json({{"_ArrayType_", "single"}, {"_ArraySize_", {2, 1}}, {"_ArrayData_", {1.5, 1e40}}});
|
||
const auto out_single = json::to_bjdata(j_single);
|
||
CHECK(out_single.at(0) == '{');
|
||
CHECK(json::from_bjdata(out_single) == j_single);
|
||
|
||
// a double element that is finite and within the range of "single"
|
||
// but is not exactly representable as a float, so narrowing it would
|
||
// silently round it (0.1 is read back as 0.10000000149011612); this,
|
||
// like the overflow case above, falls back to a plain object (see
|
||
// GitHub issue #5661)
|
||
json const j_single_rounded = json({{"_ArrayType_", "single"}, {"_ArraySize_", {2, 1}}, {"_ArrayData_", {1.5, 0.1}}});
|
||
const auto out_single_rounded = json::to_bjdata(j_single_rounded);
|
||
CHECK(out_single_rounded.at(0) == '{');
|
||
CHECK(json::from_bjdata(out_single_rounded) == j_single_rounded);
|
||
|
||
// a double element that underflows to 0 when narrowed to "single"
|
||
json const j_single_underflow = json({{"_ArrayType_", "single"}, {"_ArraySize_", {2, 1}}, {"_ArrayData_", {1.5, 1e-300}}});
|
||
const auto out_single_underflow = json::to_bjdata(j_single_underflow);
|
||
CHECK(out_single_underflow.at(0) == '{');
|
||
CHECK(json::from_bjdata(out_single_underflow) == j_single_underflow);
|
||
|
||
// in-range boundary values still use the compact ndarray encoding
|
||
json const j_uint8_ok = json({{"_ArrayType_", "uint8"}, {"_ArraySize_", {2, 1}}, {"_ArrayData_", {0, 255}}});
|
||
CHECK(json::to_bjdata(j_uint8_ok) == std::vector<uint8_t>({'[', '$', 'U', '#', '[', 'i', 2, 'i', 1, ']', 0, 255}));
|
||
|
||
json const j_int8_ok = json({{"_ArrayType_", "int8"}, {"_ArraySize_", {2, 1}}, {"_ArrayData_", {-128, 127}}});
|
||
CHECK(json::to_bjdata(j_int8_ok) == std::vector<uint8_t>({'[', '$', 'i', '#', '[', 'i', 2, 'i', 1, ']', 0x80, 0x7F}));
|
||
|
||
json const j_single_ok = json({{"_ArrayType_", "single"}, {"_ArraySize_", {2, 1}}, {"_ArrayData_", {1.5, -1.5}}});
|
||
const auto out_single_ok = json::to_bjdata(j_single_ok);
|
||
CHECK(out_single_ok.at(0) == '[');
|
||
CHECK(json::from_bjdata(out_single_ok) == json({{"_ArrayType_", "single"}, {"_ArraySize_", {2, 1}}, {"_ArrayData_", {1.5f, -1.5f}}}));
|
||
}
|
||
|
||
SECTION("ndarray with _ArrayType_ \"char\" is limited to 0..127")
|
||
{
|
||
// both BJData drafts limit a char to 0..127; larger values must not
|
||
// be written as 'C' elements (the reader rejects them), so such an
|
||
// object stays a plain object that round-trips
|
||
for (const auto version :
|
||
{
|
||
json::bjdata_version_t::draft2, json::bjdata_version_t::draft3
|
||
})
|
||
{
|
||
json const j_ok = json({{"_ArrayType_", "char"}, {"_ArraySize_", {2, 2}}, {"_ArrayData_", {0, 97, 98, 127}}});
|
||
const auto out_ok = json::to_bjdata(j_ok, true, true, version);
|
||
CHECK(out_ok == std::vector<uint8_t>({'[', '$', 'C', '#', '[', '$', 'i', '#', 'i', 2, 2, 2, 0, 97, 98, 127}));
|
||
CHECK(json::from_bjdata(out_ok) == j_ok);
|
||
|
||
for (const int bad :
|
||
{
|
||
128, 200, 255, 256, -1
|
||
})
|
||
{
|
||
CAPTURE(bad)
|
||
json const j_bad = json({{"_ArrayType_", "char"}, {"_ArraySize_", {2, 2}}, {"_ArrayData_", {97, 98, 99, bad}}});
|
||
const auto out_bad = json::to_bjdata(j_bad, true, true, version);
|
||
CHECK(out_bad.at(0) == '{');
|
||
CHECK(json::from_bjdata(out_bad) == j_bad);
|
||
}
|
||
}
|
||
}
|
||
|
||
SECTION("ndarray annotation keys are read back in the documented order")
|
||
{
|
||
// from_bjdata() must emit the annotation object's keys in the order
|
||
// used throughout the documentation, _ArrayType_, _ArraySize_,
|
||
// _ArrayData_: the type marker precedes the dimension vector on the
|
||
// wire (see get_ubjson_size_type()), so it is known, and emitted,
|
||
// before _ArraySize_. For a plain json this key order is invisible
|
||
// (its comparison ignores it), but for an ordered_json it is not (see
|
||
// GitHub issue #5661).
|
||
const ordered_json o = ordered_json::parse(R"({"_ArrayType_":"uint8","_ArraySize_":[2,2],"_ArrayData_":[1,2,3,4]})");
|
||
const auto packed = ordered_json::to_bjdata(o);
|
||
CHECK(packed.at(0) == '[');
|
||
const ordered_json o_back = ordered_json::from_bjdata(packed);
|
||
CHECK(o_back == o);
|
||
CHECK(o_back.dump() == o.dump());
|
||
}
|
||
|
||
SECTION("ndarray that would not be read back as an annotated object stays as object")
|
||
{
|
||
// the reader only restores an annotated object from an ND-array
|
||
// with at least two non-zero dimensions that is not a 1xN row
|
||
// vector; any other shape is read back as a plain array. Writing
|
||
// such an object as an ND-array would drop its annotation, so it
|
||
// falls back to a plain object encoding that round-trips.
|
||
for (const char* text :
|
||
{
|
||
R"({"_ArrayType_":"int16","_ArraySize_":[],"_ArrayData_":[]})",
|
||
R"({"_ArrayType_":"int16","_ArraySize_":[2],"_ArrayData_":[1,2]})",
|
||
R"({"_ArrayType_":"int16","_ArraySize_":[1,2],"_ArrayData_":[1,2]})",
|
||
R"({"_ArrayType_":"int16","_ArraySize_":[0],"_ArrayData_":[]})",
|
||
R"({"_ArrayType_":"int16","_ArraySize_":[2,0],"_ArrayData_":[]})",
|
||
R"({"_ArrayType_":"int16","_ArraySize_":[0,2],"_ArrayData_":[]})"
|
||
})
|
||
{
|
||
CAPTURE(text)
|
||
const json j = json::parse(text);
|
||
for (const bool use_size :
|
||
{
|
||
false, true
|
||
})
|
||
{
|
||
const auto out = json::to_bjdata(j, use_size, use_size);
|
||
CHECK(out.at(0) == '{');
|
||
CHECK(json::from_bjdata(out) == j);
|
||
}
|
||
}
|
||
|
||
// a genuine ND-array still uses the compact encoding and round-trips
|
||
const json j_2d = json::parse(R"({"_ArrayType_":"int16","_ArraySize_":[2,1],"_ArrayData_":[1,2]})");
|
||
const auto out_2d = json::to_bjdata(j_2d);
|
||
CHECK(out_2d.at(0) == '[');
|
||
CHECK(json::from_bjdata(out_2d) == j_2d);
|
||
}
|
||
|
||
SECTION("ndarray with non-array _ArrayData_ stays as object")
|
||
{
|
||
// the elements are written from _ArrayData_ as a flat list, so it
|
||
// has to be an array: null has size 0, any other scalar has size 1,
|
||
// and iterating an object visits its values, so each of these could
|
||
// match the dimensions and be encoded as an unrelated ND-array
|
||
for (const char* text :
|
||
{
|
||
R"({"_ArrayType_":"int16","_ArraySize_":[2,1],"_ArrayData_":null})",
|
||
R"({"_ArrayType_":"int16","_ArraySize_":[2,1],"_ArrayData_":{"a":1,"b":2}})",
|
||
R"({"_ArrayType_":"int16","_ArraySize_":[1],"_ArrayData_":5})",
|
||
R"({"_ArrayType_":"int16","_ArraySize_":[],"_ArrayData_":null})"
|
||
})
|
||
{
|
||
CAPTURE(text)
|
||
const json j = json::parse(text);
|
||
const auto out = json::to_bjdata(j);
|
||
CHECK(out.at(0) == '{');
|
||
CHECK(json::from_bjdata(out) == j);
|
||
}
|
||
|
||
// OSS-Fuzz issue 563659413: an empty binary _ArraySize_ is written
|
||
// as a plain object and read back as an empty array, after which
|
||
// the object with a null _ArrayData_ was encoded as an empty
|
||
// ND-array and re-read as [], so a second round trip lost the value
|
||
const std::vector<uint8_t> input =
|
||
{
|
||
'{', 'U', 11, '_', 'A', 'r', 'r', 'a', 'y', 'D', 'a', 't', 'a', '_', 'Z',
|
||
'U', 11, '_', 'A', 'r', 'r', 'a', 'y', 'T', 'y', 'p', 'e', '_', 'S', 'i', 5, 'i', 'n', 't', '1', '6',
|
||
'U', 11, '_', 'A', 'r', 'r', 'a', 'y', 'S', 'i', 'z', 'e', '_', '[', '$', 'B', '#', '[', ']', '}'
|
||
};
|
||
const json j1 = json::from_bjdata(input);
|
||
const json j2 = json::from_bjdata(json::to_bjdata(j1, false, false));
|
||
CHECK(j2 == json::parse(R"({"_ArrayType_":"int16","_ArraySize_":[],"_ArrayData_":null})"));
|
||
CHECK(json::from_bjdata(json::to_bjdata(j2, false, false)) == j2);
|
||
}
|
||
|
||
SECTION("ndarray with _ArrayType_ \"byte\" is gated by the BJData draft version")
|
||
{
|
||
// the 'B' (byte) marker used by _ArrayType_ "byte" is only defined
|
||
// by BJData Draft 3; Draft 2 (the default) has no such marker, so
|
||
// emitting it unconditionally produced a stream that a Draft 2
|
||
// reader could not parse as intended (see GitHub issue #5404).
|
||
// Two dimensions are used so that a successfully written ndarray
|
||
// round-trips back into the annotated object (a single dimension
|
||
// is, by the BJData ndarray convention, read back as a plain
|
||
// binary value rather than the annotated object, same as every
|
||
// other single-dimension ndarray of a non-"byte" type is read
|
||
// back as a plain array instead of the annotated object).
|
||
json const j_byte = json({{"_ArrayType_", "byte"}, {"_ArraySize_", {2, 3}}, {"_ArrayData_", {1, 2, 3, 4, 5, 6}}});
|
||
|
||
// default (Draft 2): falls back to a plain object and round-trips
|
||
const auto out_draft2 = json::to_bjdata(j_byte);
|
||
CHECK(out_draft2.at(0) == '{');
|
||
CHECK(json::from_bjdata(out_draft2) == j_byte);
|
||
|
||
// explicit Draft 2: same as the default
|
||
const auto out_draft2_explicit = json::to_bjdata(j_byte, true, true, json::bjdata_version_t::draft2);
|
||
CHECK(out_draft2_explicit.at(0) == '{');
|
||
CHECK(json::from_bjdata(out_draft2_explicit) == j_byte);
|
||
|
||
// Draft 3 explicitly selected: still uses the compact 'B' ndarray encoding
|
||
const auto out_draft3 = json::to_bjdata(j_byte, true, true, json::bjdata_version_t::draft3);
|
||
CHECK(out_draft3 == std::vector<uint8_t>({'[', '$', 'B', '#', '[', '$', 'i', '#', 'i', 2, 2, 3, 1, 2, 3, 4, 5, 6}));
|
||
CHECK(json::from_bjdata(out_draft3) == j_byte);
|
||
}
|
||
}
|
||
}
|
||
|
||
SECTION("parse errors")
|
||
{
|
||
SECTION("empty byte vector")
|
||
{
|
||
json _;
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(std::vector<uint8_t>()),
|
||
"[json.exception.parse_error.110] parse error at byte 1: syntax error while parsing BJData value: unexpected end of input", json::parse_error&);
|
||
}
|
||
|
||
SECTION("char")
|
||
{
|
||
SECTION("eof after C byte")
|
||
{
|
||
std::vector<uint8_t> const v = {'C'};
|
||
json _;
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v), "[json.exception.parse_error.110] parse error at byte 2: syntax error while parsing BJData char: unexpected end of input", json::parse_error&);
|
||
}
|
||
|
||
SECTION("byte out of range")
|
||
{
|
||
std::vector<uint8_t> const v = {'C', 130};
|
||
json _;
|
||
CHECK_THROWS_WITH(_ = json::from_bjdata(v), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing BJData char: byte after 'C' must be in range 0x00..0x7F; last byte: 0x82");
|
||
}
|
||
|
||
SECTION("ndarray of chars cut off by the end of the input")
|
||
{
|
||
// 2x2 chars announced, three present
|
||
std::vector<uint8_t> const v = {'[', '$', 'C', '#', '[', '$', 'U', '#', 'U', 2, 2, 2, 'a', 'b', 'c'};
|
||
json _;
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v), "[json.exception.parse_error.110] parse error at byte 16: syntax error while parsing BJData char: unexpected end of input", json::parse_error&);
|
||
CHECK(json::from_bjdata(v, true, false).is_discarded());
|
||
}
|
||
}
|
||
|
||
SECTION("byte")
|
||
{
|
||
SECTION("parse bjdata markers in ubjson")
|
||
{
|
||
std::vector<uint8_t> const v = {'B', 1};
|
||
|
||
json _;
|
||
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(v), "[json.exception.parse_error.112] parse error at byte 1: syntax error while parsing UBJSON value: invalid byte: 0x42", json::parse_error&);
|
||
}
|
||
}
|
||
|
||
SECTION("strings")
|
||
{
|
||
SECTION("eof after S byte")
|
||
{
|
||
std::vector<uint8_t> const v = {'S'};
|
||
json _;
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v), "[json.exception.parse_error.110] parse error at byte 2: syntax error while parsing BJData value: unexpected end of input", json::parse_error&);
|
||
}
|
||
|
||
SECTION("invalid byte")
|
||
{
|
||
std::vector<uint8_t> const v = {'S', '1', 'a'};
|
||
json _;
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing BJData string: expected length type specification (U, i, u, I, m, l, M, L); last byte: 0x31", json::parse_error&);
|
||
}
|
||
|
||
SECTION("negative length")
|
||
{
|
||
json _;
|
||
|
||
std::vector<uint8_t> const vi = {'S', 'i', 0xFF};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vi), "[json.exception.parse_error.113] parse error at byte 3: syntax error while parsing BJData string: string length must not be negative", json::parse_error&);
|
||
CHECK(json::from_bjdata(vi, true, false).is_discarded());
|
||
|
||
std::vector<uint8_t> const vl = {'S', 'l', 0xFF, 0xFF, 0xFF, 0xFF};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vl), "[json.exception.parse_error.113] parse error at byte 6: syntax error while parsing BJData string: string length must not be negative", json::parse_error&);
|
||
CHECK(json::from_bjdata(vl, true, false).is_discarded());
|
||
}
|
||
|
||
SECTION("parse bjdata markers in ubjson")
|
||
{
|
||
// create a single-character string for all number types
|
||
std::vector<uint8_t> const s_u = {'S', 'u', 1, 0, 'a'};
|
||
std::vector<uint8_t> const s_m = {'S', 'm', 1, 0, 0, 0, 'a'};
|
||
std::vector<uint8_t> const s_M = {'S', 'M', 1, 0, 0, 0, 0, 0, 0, 0, 'a'};
|
||
|
||
json _;
|
||
// check if string is parsed correctly to "a"
|
||
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(s_u), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing UBJSON string: expected length type specification (U, i, I, l, L); last byte: 0x75", json::parse_error&);
|
||
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(s_m), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing UBJSON string: expected length type specification (U, i, I, l, L); last byte: 0x6D", json::parse_error&);
|
||
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(s_M), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing UBJSON string: expected length type specification (U, i, I, l, L); last byte: 0x4D", json::parse_error&);
|
||
}
|
||
}
|
||
|
||
SECTION("array")
|
||
{
|
||
SECTION("optimized array: no size following type")
|
||
{
|
||
std::vector<uint8_t> const v = {'[', '$', 'i', 2};
|
||
json _;
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v), "[json.exception.parse_error.112] parse error at byte 4: syntax error while parsing BJData size: expected '#' after type information; last byte: 0x02", json::parse_error&);
|
||
}
|
||
|
||
SECTION("optimized array: negative size")
|
||
{
|
||
std::vector<uint8_t> const v1 = {'[', '#', 'i', 0xF1};
|
||
std::vector<uint8_t> const v2 = {'[', '$', 'I', '#', 'i', 0xF2};
|
||
std::vector<uint8_t> const v3 = {'[', '$', 'I', '#', '[', 'i', 0xF4, 'i', 0x02, ']'};
|
||
std::vector<uint8_t> const v4 = {'[', '$', 'l', '#', 'i', 0xF7};
|
||
std::vector<uint8_t> const v5 = {'[', '$', 'I', '#', '[', 'i', 0xF5, 'i', 0xF1, ']'};
|
||
std::vector<uint8_t> const v6 = {'[', '#', '[', 'i', 0xF3, 'i', 0x02, ']'};
|
||
|
||
std::vector<uint8_t> const vI = {'[', '#', 'I', 0x00, 0xF1};
|
||
std::vector<uint8_t> const vl = {'[', '#', 'l', 0x00, 0x00, 0x00, 0xF2};
|
||
std::vector<uint8_t> const vL = {'[', '#', 'L', 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xF3};
|
||
std::vector<uint8_t> const vM = {'[', '$', 'M', '#', '[', 'I', 0x00, 0x20, 'M', 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x20, 0xFF, ']'};
|
||
std::vector<uint8_t> const vMX = {'[', '$', 'U', '#', '[', 'M', 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 'U', 0x01, ']'};
|
||
|
||
json _;
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v1), "[json.exception.parse_error.113] parse error at byte 4: syntax error while parsing BJData size: count in an optimized container must be positive", json::parse_error&);
|
||
CHECK(json::from_bjdata(v1, true, false).is_discarded());
|
||
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v2), "[json.exception.parse_error.113] parse error at byte 6: syntax error while parsing BJData size: count in an optimized container must be positive", json::parse_error&);
|
||
CHECK(json::from_bjdata(v2, true, false).is_discarded());
|
||
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v3), "[json.exception.parse_error.113] parse error at byte 7: syntax error while parsing BJData size: count in an optimized container must be positive", json::parse_error&);
|
||
CHECK(json::from_bjdata(v3, true, false).is_discarded());
|
||
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v4), "[json.exception.parse_error.113] parse error at byte 6: syntax error while parsing BJData size: count in an optimized container must be positive", json::parse_error&);
|
||
CHECK(json::from_bjdata(v4, true, false).is_discarded());
|
||
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v5), "[json.exception.parse_error.113] parse error at byte 7: syntax error while parsing BJData size: count in an optimized container must be positive", json::parse_error&);
|
||
CHECK(json::from_bjdata(v5, true, false).is_discarded());
|
||
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v6), "[json.exception.parse_error.113] parse error at byte 5: syntax error while parsing BJData size: count in an optimized container must be positive", json::parse_error&);
|
||
CHECK(json::from_bjdata(v6, true, false).is_discarded());
|
||
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vI), "[json.exception.parse_error.113] parse error at byte 5: syntax error while parsing BJData size: count in an optimized container must be positive", json::parse_error&);
|
||
CHECK(json::from_bjdata(vI, true, false).is_discarded());
|
||
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vl), "[json.exception.parse_error.113] parse error at byte 7: syntax error while parsing BJData size: count in an optimized container must be positive", json::parse_error&);
|
||
CHECK(json::from_bjdata(vl, true, false).is_discarded());
|
||
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vL), "[json.exception.parse_error.113] parse error at byte 11: syntax error while parsing BJData size: count in an optimized container must be positive", json::parse_error&);
|
||
CHECK(json::from_bjdata(vL, true, false).is_discarded());
|
||
|
||
#if SIZE_MAX != 0xffffffff
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vM), "[json.exception.out_of_range.408] syntax error while parsing BJData size: excessive ndarray size caused overflow", json::out_of_range&);
|
||
#else
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vM), "[json.exception.out_of_range.408] syntax error while parsing BJData size: integer value overflow", json::out_of_range&);
|
||
#endif
|
||
CHECK(json::from_bjdata(vM, true, false).is_discarded());
|
||
|
||
// the first dimension is 2^64-1, which is rejected as a count (see below)
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vMX), "[json.exception.out_of_range.408] syntax error while parsing BJData size: integer value overflow", json::out_of_range&);
|
||
CHECK(json::from_bjdata(vMX, true, false).is_discarded());
|
||
}
|
||
|
||
SECTION("optimized array: count equal to SIZE_MAX")
|
||
{
|
||
// the reader marks a container without a count by SIZE_MAX, so
|
||
// a count of that value must not be read as "no count"
|
||
// (it used to parse as [1])
|
||
json _;
|
||
#if SIZE_MAX != 0xffffffff
|
||
std::vector<uint8_t> const vM = {'[', '#', 'M', 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 'i', 0x01, ']'};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vM), "[json.exception.out_of_range.408] syntax error while parsing BJData size: integer value overflow", json::out_of_range&);
|
||
CHECK(json::from_bjdata(vM, true, false).is_discarded());
|
||
// one less is an ordinary excessive count
|
||
std::vector<uint8_t> const vM1 = {'[', '#', 'M', 0xFE, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 'i', 0x01, ']'};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vM1), "[json.exception.out_of_range.408] excessive array size: 18446744073709551614", json::out_of_range&);
|
||
#else
|
||
std::vector<uint8_t> const vm = {'[', '#', 'm', 0xFF, 0xFF, 0xFF, 0xFF, 'i', 0x01, ']'};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vm), "[json.exception.out_of_range.408] syntax error while parsing BJData size: integer value overflow", json::out_of_range&);
|
||
CHECK(json::from_bjdata(vm, true, false).is_discarded());
|
||
std::vector<uint8_t> const vL = {'[', '#', 'L', 0xFF, 0xFF, 0xFF, 0xFF, 0x00, 0x00, 0x00, 0x00, 'i', 0x01, ']'};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vL), "[json.exception.out_of_range.408] syntax error while parsing BJData size: integer value overflow", json::out_of_range&);
|
||
CHECK(json::from_bjdata(vL, true, false).is_discarded());
|
||
#endif
|
||
}
|
||
|
||
SECTION("optimized array: integer value overflow")
|
||
{
|
||
#if SIZE_MAX == 0xffffffff
|
||
std::vector<uint8_t> const vL = {'[', '#', 'L', 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x7F};
|
||
std::vector<uint8_t> const vM = {'[', '$', 'M', '#', '[', 'I', 0x00, 0x20, 'M', 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x20, 0xFF, ']'};
|
||
|
||
json _;
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vL), "[json.exception.out_of_range.408] syntax error while parsing BJData size: integer value overflow", json::out_of_range&);
|
||
CHECK(json::from_bjdata(vL, true, false).is_discarded());
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vM), "[json.exception.out_of_range.408] syntax error while parsing BJData size: integer value overflow", json::out_of_range&);
|
||
CHECK(json::from_bjdata(vM, true, false).is_discarded());
|
||
#endif
|
||
}
|
||
|
||
SECTION("overflow detection in dimension multiplication")
|
||
{
|
||
// Simple SAX handler just to monitor if overflow is detected
|
||
struct SimpleOverflowSaxHandler : public nlohmann::json_sax<json>
|
||
{
|
||
bool overflow_detected = false;
|
||
|
||
// Implement all required virtual methods with minimal implementation
|
||
bool null() override
|
||
{
|
||
return true;
|
||
}
|
||
bool boolean(bool /*val*/) override
|
||
{
|
||
return true;
|
||
}
|
||
bool number_integer(json::number_integer_t /*val*/) override
|
||
{
|
||
return true;
|
||
}
|
||
bool number_unsigned(json::number_unsigned_t /*val*/) override
|
||
{
|
||
return true;
|
||
}
|
||
bool number_float(json::number_float_t /*val*/, const std::string& /*s*/) override
|
||
{
|
||
return true;
|
||
}
|
||
bool string(std::string& /*val*/) override
|
||
{
|
||
return true;
|
||
}
|
||
bool binary(json::binary_t& /*val*/) override
|
||
{
|
||
return true;
|
||
}
|
||
bool start_object(std::size_t /*elements*/) override
|
||
{
|
||
return true;
|
||
}
|
||
bool key(std::string& /*val*/) override
|
||
{
|
||
return true;
|
||
}
|
||
bool end_object() override
|
||
{
|
||
return true;
|
||
}
|
||
bool start_array(std::size_t /*elements*/) override
|
||
{
|
||
return true;
|
||
}
|
||
bool end_array() override
|
||
{
|
||
return true;
|
||
}
|
||
|
||
// This is the only method we care about - detecting error 408
|
||
bool parse_error(std::size_t /*position*/, const std::string& /*last_token*/, const json::exception& ex) override
|
||
{
|
||
if (ex.id == 408)
|
||
{
|
||
overflow_detected = true;
|
||
}
|
||
return false;
|
||
}
|
||
};
|
||
|
||
// Create BJData payload with overflow-causing dimensions (2^32+1) × (2^32)
|
||
const std::vector<uint8_t> bjdata_payload =
|
||
{
|
||
0x5B, // '[' start array
|
||
0x24, 0x55, // '$', 'U' (type uint8)
|
||
0x23, 0x5B, // '#', '[' (dimensions array)
|
||
0x4D, // 'M' (uint64)
|
||
0x01, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, // 2^32 + 1 (4294967297) as little-endian
|
||
0x4D, // 'M' (uint64)
|
||
0x00, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, // 2^32 (4294967296) as little-endian
|
||
0x5D // ']' end dimensions
|
||
// No data - we don't need it for this test, we just want to hit the overflow check
|
||
};
|
||
|
||
// Test with overflow dimensions using SAX parser
|
||
{
|
||
SimpleOverflowSaxHandler handler;
|
||
const auto result = json::sax_parse(bjdata_payload, &handler,
|
||
nlohmann::detail::input_format_t::bjdata, false);
|
||
|
||
// Should detect overflow
|
||
CHECK(handler.overflow_detected == true);
|
||
CHECK(result == false);
|
||
}
|
||
|
||
// Test with DOM parser (should throw)
|
||
{
|
||
json _;
|
||
CHECK_THROWS_AS(_ = json::from_bjdata(bjdata_payload), json::out_of_range);
|
||
}
|
||
|
||
// Test with normal dimensions
|
||
const std::vector<uint8_t> normal_payload =
|
||
{
|
||
0x5B, // '[' start array
|
||
0x24, 0x55, // '$', 'U' (type uint8)
|
||
0x23, 0x5B, // '#', '[' (dimensions array)
|
||
0x55, 0x02, // 'U', 2 (uint8)
|
||
0x55, 0x03, // 'U', 3 (uint8)
|
||
0x5D, // ']' end dimensions
|
||
// 6 data bytes for a 2×3 array (enough to avoid EOF but not entire array)
|
||
0x01, 0x02, 0x03, 0x04, 0x05, 0x06
|
||
};
|
||
|
||
// For normal dimensions, overflow should not be detected
|
||
{
|
||
SimpleOverflowSaxHandler handler;
|
||
const auto result = json::sax_parse(normal_payload, &handler,
|
||
nlohmann::detail::input_format_t::bjdata, false);
|
||
|
||
CHECK(handler.overflow_detected == false);
|
||
CHECK(result == true);
|
||
}
|
||
}
|
||
|
||
SECTION("ndarray of half-precision floats can be read")
|
||
{
|
||
// the 'h' element type is read as _ArrayType_ "half"
|
||
json _;
|
||
const std::vector<uint8_t> half = {'[', '$', 'h', '#', '[', '$', 'U', '#', 'U', 2, 2, 2, 0x00, 0x3C, 0x00, 0x40, 0x00, 0x42, 0x00, 0x44};
|
||
const json j_half = json::parse(R"({"_ArrayType_":"half","_ArraySize_":[2,2],"_ArrayData_":[1.0,2.0,3.0,4.0]})");
|
||
CHECK(json::from_bjdata(half) == j_half);
|
||
|
||
// the writer does not pack "half": it writes a plain object that reads back equal
|
||
CHECK(json::from_bjdata(json::to_bjdata(j_half)) == j_half);
|
||
CHECK(json::from_bjdata(json::to_bjdata(j_half, true, true, json::bjdata_version_t::draft3)) == j_half);
|
||
|
||
// truncated half data
|
||
CHECK_THROWS_AS(_ = json::from_bjdata(std::vector<uint8_t>(half.begin(), half.end() - 1)), json::parse_error&);
|
||
}
|
||
|
||
SECTION("ndarray of char rejects values above 127")
|
||
{
|
||
json _;
|
||
const std::vector<uint8_t> ok = {'[', '$', 'C', '#', '[', '$', 'U', '#', 'U', 2, 2, 2, 'a', 'b', 'c', 0x7F};
|
||
CHECK(json::from_bjdata(ok) == json::parse(R"({"_ArrayType_":"char","_ArraySize_":[2,2],"_ArrayData_":[97,98,99,127]})"));
|
||
|
||
std::vector<uint8_t> bad = ok;
|
||
bad.back() = 0xC8;
|
||
CHECK_THROWS_AS(_ = json::from_bjdata(bad), json::parse_error&);
|
||
CHECK(json::from_bjdata(bad, true, false).is_discarded());
|
||
|
||
// the same restriction holds for 1-D arrays
|
||
CHECK_THROWS_AS(_ = json::from_bjdata(std::vector<uint8_t>({'[', '$', 'C', '#', 'i', 1, 0xC8})), json::parse_error&);
|
||
|
||
// truncated input
|
||
CHECK_THROWS_AS(_ = json::from_bjdata(std::vector<uint8_t>(ok.begin(), ok.end() - 1)), json::parse_error&);
|
||
|
||
// 'B' (byte) elements may use the full range
|
||
const std::vector<uint8_t> byte_nd = {'[', '$', 'B', '#', '[', '$', 'U', '#', 'U', 2, 2, 2, 1, 2, 3, 0xC8};
|
||
CHECK(json::from_bjdata(byte_nd) == json::parse(R"({"_ArrayType_":"byte","_ArraySize_":[2,2],"_ArrayData_":[1,2,3,200]})"));
|
||
}
|
||
|
||
SECTION("do not accept NTFZ markers in ndarray optimized type (with count)")
|
||
{
|
||
json _;
|
||
std::vector<uint8_t> const v_N = {'[', '$', 'N', '#', '[', '#', 'i', 2, 'i', 1, 'i', 2};
|
||
std::vector<uint8_t> const v_T = {'[', '$', 'T', '#', '[', '#', 'i', 2, 'i', 1, 'i', 2};
|
||
std::vector<uint8_t> const v_F = {'[', '$', 'F', '#', '[', '#', 'i', 2, 'i', 1, 'i', 2};
|
||
std::vector<uint8_t> const v_Z = {'[', '$', 'Z', '#', '[', '#', 'i', 2, 'i', 1, 'i', 2};
|
||
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v_N), "[json.exception.parse_error.112] parse error at byte 3: syntax error while parsing BJData type: marker 0x4E is not a permitted optimized array type", json::parse_error&);
|
||
CHECK(json::from_bjdata(v_N, true, false).is_discarded());
|
||
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v_T), "[json.exception.parse_error.112] parse error at byte 3: syntax error while parsing BJData type: marker 0x54 is not a permitted optimized array type", json::parse_error&);
|
||
CHECK(json::from_bjdata(v_T, true, false).is_discarded());
|
||
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v_F), "[json.exception.parse_error.112] parse error at byte 3: syntax error while parsing BJData type: marker 0x46 is not a permitted optimized array type", json::parse_error&);
|
||
CHECK(json::from_bjdata(v_F, true, false).is_discarded());
|
||
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v_Z), "[json.exception.parse_error.112] parse error at byte 3: syntax error while parsing BJData type: marker 0x5A is not a permitted optimized array type", json::parse_error&);
|
||
CHECK(json::from_bjdata(v_Z, true, false).is_discarded());
|
||
}
|
||
|
||
SECTION("do not accept NTFZ markers in ndarray optimized type (without count)")
|
||
{
|
||
json _;
|
||
std::vector<uint8_t> const v_N = {'[', '$', 'N', '#', '[', 'i', 1, 'i', 2, ']'};
|
||
std::vector<uint8_t> const v_T = {'[', '$', 'T', '#', '[', 'i', 1, 'i', 2, ']'};
|
||
std::vector<uint8_t> const v_F = {'[', '$', 'F', '#', '[', 'i', 1, 'i', 2, ']'};
|
||
std::vector<uint8_t> const v_Z = {'[', '$', 'Z', '#', '[', 'i', 1, 'i', 2, ']'};
|
||
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v_N), "[json.exception.parse_error.112] parse error at byte 3: syntax error while parsing BJData type: marker 0x4E is not a permitted optimized array type", json::parse_error&);
|
||
CHECK(json::from_bjdata(v_N, true, false).is_discarded());
|
||
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v_T), "[json.exception.parse_error.112] parse error at byte 3: syntax error while parsing BJData type: marker 0x54 is not a permitted optimized array type", json::parse_error&);
|
||
CHECK(json::from_bjdata(v_T, true, false).is_discarded());
|
||
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v_F), "[json.exception.parse_error.112] parse error at byte 3: syntax error while parsing BJData type: marker 0x46 is not a permitted optimized array type", json::parse_error&);
|
||
CHECK(json::from_bjdata(v_F, true, false).is_discarded());
|
||
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v_Z), "[json.exception.parse_error.112] parse error at byte 3: syntax error while parsing BJData type: marker 0x5A is not a permitted optimized array type", json::parse_error&);
|
||
CHECK(json::from_bjdata(v_Z, true, false).is_discarded());
|
||
}
|
||
}
|
||
|
||
SECTION("strings")
|
||
{
|
||
std::vector<uint8_t> const vS = {'S'};
|
||
json _;
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vS), "[json.exception.parse_error.110] parse error at byte 2: syntax error while parsing BJData value: unexpected end of input", json::parse_error&);
|
||
CHECK(json::from_bjdata(vS, true, false).is_discarded());
|
||
|
||
std::vector<uint8_t> const v = {'S', 'i', '2', 'a'};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v), "[json.exception.parse_error.110] parse error at byte 5: syntax error while parsing BJData string: unexpected end of input", json::parse_error&);
|
||
CHECK(json::from_bjdata(v, true, false).is_discarded());
|
||
|
||
std::vector<uint8_t> const vC = {'C'};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vC), "[json.exception.parse_error.110] parse error at byte 2: syntax error while parsing BJData char: unexpected end of input", json::parse_error&);
|
||
CHECK(json::from_bjdata(vC, true, false).is_discarded());
|
||
}
|
||
|
||
SECTION("sizes")
|
||
{
|
||
std::vector<uint8_t> const vU = {'[', '#', 'U'};
|
||
json _;
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vU), "[json.exception.parse_error.110] parse error at byte 4: syntax error while parsing BJData number: unexpected end of input", json::parse_error&);
|
||
CHECK(json::from_bjdata(vU, true, false).is_discarded());
|
||
|
||
std::vector<uint8_t> const vi = {'[', '#', 'i'};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vi), "[json.exception.parse_error.110] parse error at byte 4: syntax error while parsing BJData number: unexpected end of input", json::parse_error&);
|
||
CHECK(json::from_bjdata(vi, true, false).is_discarded());
|
||
|
||
std::vector<uint8_t> const vI = {'[', '#', 'I'};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vI), "[json.exception.parse_error.110] parse error at byte 4: syntax error while parsing BJData number: unexpected end of input", json::parse_error&);
|
||
CHECK(json::from_bjdata(vI, true, false).is_discarded());
|
||
|
||
std::vector<uint8_t> const vu = {'[', '#', 'u'};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vu), "[json.exception.parse_error.110] parse error at byte 4: syntax error while parsing BJData number: unexpected end of input", json::parse_error&);
|
||
CHECK(json::from_bjdata(vu, true, false).is_discarded());
|
||
|
||
std::vector<uint8_t> const vl = {'[', '#', 'l'};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vl), "[json.exception.parse_error.110] parse error at byte 4: syntax error while parsing BJData number: unexpected end of input", json::parse_error&);
|
||
CHECK(json::from_bjdata(vl, true, false).is_discarded());
|
||
|
||
std::vector<uint8_t> const vm = {'[', '#', 'm'};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vm), "[json.exception.parse_error.110] parse error at byte 4: syntax error while parsing BJData number: unexpected end of input", json::parse_error&);
|
||
CHECK(json::from_bjdata(vm, true, false).is_discarded());
|
||
|
||
std::vector<uint8_t> const vL = {'[', '#', 'L'};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vL), "[json.exception.parse_error.110] parse error at byte 4: syntax error while parsing BJData number: unexpected end of input", json::parse_error&);
|
||
CHECK(json::from_bjdata(vL, true, false).is_discarded());
|
||
|
||
std::vector<uint8_t> const vM = {'[', '#', 'M'};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vM), "[json.exception.parse_error.110] parse error at byte 4: syntax error while parsing BJData number: unexpected end of input", json::parse_error&);
|
||
CHECK(json::from_bjdata(vM, true, false).is_discarded());
|
||
|
||
std::vector<uint8_t> const v0 = {'[', '#', 'T', ']'};
|
||
CHECK_THROWS_WITH(_ = json::from_bjdata(v0), "[json.exception.parse_error.113] parse error at byte 3: syntax error while parsing BJData size: expected length type specification (U, i, u, I, m, l, M, L) after '#'; last byte: 0x54");
|
||
CHECK(json::from_bjdata(v0, true, false).is_discarded());
|
||
|
||
std::vector<uint8_t> const vB = {'[', '#', 'B', ']'};
|
||
CHECK_THROWS_WITH(_ = json::from_bjdata(vB), "[json.exception.parse_error.113] parse error at byte 3: syntax error while parsing BJData size: expected length type specification (U, i, u, I, m, l, M, L) after '#'; last byte: 0x42");
|
||
CHECK(json::from_bjdata(v0, true, false).is_discarded());
|
||
}
|
||
|
||
SECTION("parse bjdata markers as array size in ubjson")
|
||
{
|
||
json _;
|
||
std::vector<uint8_t> const vu = {'[', '#', 'u'};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(vu), "[json.exception.parse_error.113] parse error at byte 3: syntax error while parsing UBJSON size: expected length type specification (U, i, I, l, L) after '#'; last byte: 0x75", json::parse_error&);
|
||
CHECK(json::from_ubjson(vu, true, false).is_discarded());
|
||
|
||
std::vector<uint8_t> const vm = {'[', '#', 'm'};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(vm), "[json.exception.parse_error.113] parse error at byte 3: syntax error while parsing UBJSON size: expected length type specification (U, i, I, l, L) after '#'; last byte: 0x6D", json::parse_error&);
|
||
CHECK(json::from_ubjson(vm, true, false).is_discarded());
|
||
|
||
std::vector<uint8_t> const vM = {'[', '#', 'M'};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(vM), "[json.exception.parse_error.113] parse error at byte 3: syntax error while parsing UBJSON size: expected length type specification (U, i, I, l, L) after '#'; last byte: 0x4D", json::parse_error&);
|
||
CHECK(json::from_ubjson(vM, true, false).is_discarded());
|
||
|
||
std::vector<uint8_t> const v0 = {'[', '#', '['};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(v0), "[json.exception.parse_error.113] parse error at byte 3: syntax error while parsing UBJSON size: expected length type specification (U, i, I, l, L) after '#'; last byte: 0x5B", json::parse_error&);
|
||
CHECK(json::from_ubjson(v0, true, false).is_discarded());
|
||
}
|
||
|
||
SECTION("types")
|
||
{
|
||
std::vector<uint8_t> const v0 = {'[', '$'};
|
||
json _;
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v0), "[json.exception.parse_error.110] parse error at byte 3: syntax error while parsing BJData type: unexpected end of input", json::parse_error&);
|
||
CHECK(json::from_bjdata(v0, true, false).is_discarded());
|
||
|
||
std::vector<uint8_t> const vi = {'[', '$', '#'};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vi), "[json.exception.parse_error.112] parse error at byte 3: syntax error while parsing BJData type: marker 0x23 is not a permitted optimized array type", json::parse_error&);
|
||
CHECK(json::from_bjdata(vi, true, false).is_discarded());
|
||
|
||
std::vector<uint8_t> const vU = {'[', '$', 'U'};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vU), "[json.exception.parse_error.110] parse error at byte 4: syntax error while parsing BJData value: unexpected end of input", json::parse_error&);
|
||
CHECK(json::from_bjdata(vU, true, false).is_discarded());
|
||
|
||
std::vector<uint8_t> const v1 = {'[', '$', '['};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v1), "[json.exception.parse_error.112] parse error at byte 3: syntax error while parsing BJData type: marker 0x5B is not a permitted optimized array type", json::parse_error&);
|
||
CHECK(json::from_bjdata(v1, true, false).is_discarded());
|
||
}
|
||
|
||
SECTION("arrays")
|
||
{
|
||
std::vector<uint8_t> const vST = {'[', '$', 'i', '#', 'i', 2, 1};
|
||
json _;
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vST), "[json.exception.parse_error.110] parse error at byte 8: syntax error while parsing BJData number: unexpected end of input", json::parse_error&);
|
||
CHECK(json::from_bjdata(vST, true, false).is_discarded());
|
||
|
||
std::vector<uint8_t> const vS = {'[', '#', 'i', 2, 'i', 1};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vS), "[json.exception.parse_error.110] parse error at byte 7: syntax error while parsing BJData value: unexpected end of input", json::parse_error&);
|
||
CHECK(json::from_bjdata(vS, true, false).is_discarded());
|
||
|
||
std::vector<uint8_t> const v = {'[', 'i', 2, 'i', 1};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v), "[json.exception.parse_error.110] parse error at byte 6: syntax error while parsing BJData value: unexpected end of input", json::parse_error&);
|
||
CHECK(json::from_bjdata(v, true, false).is_discarded());
|
||
}
|
||
|
||
SECTION("ndarrays")
|
||
{
|
||
std::vector<uint8_t> const vST = {'[', '$', 'i', '#', '[', '$', 'i', '#'};
|
||
json _;
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vST), "[json.exception.parse_error.113] parse error at byte 9: syntax error while parsing BJData size: expected length type specification (U, i, u, I, m, l, M, L) after '#'; last byte: 0xFF", json::parse_error&);
|
||
CHECK(json::from_bjdata(vST, true, false).is_discarded());
|
||
|
||
std::vector<uint8_t> const v = {'[', '$', 'i', '#', '[', '$', 'i', '#', 'i', 2, 1, 2};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v), "[json.exception.parse_error.110] parse error at byte 13: syntax error while parsing BJData number: unexpected end of input", json::parse_error&);
|
||
CHECK(json::from_bjdata(v, true, false).is_discarded());
|
||
|
||
std::vector<uint8_t> const vS0 = {'[', '$', 'i', '#', '[', '$', 'i', '#', 'i', 2, 1};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vS0), "[json.exception.parse_error.110] parse error at byte 12: syntax error while parsing BJData number: unexpected end of input", json::parse_error&);
|
||
CHECK(json::from_bjdata(vS0, true, false).is_discarded());
|
||
|
||
std::vector<uint8_t> const vS = {'[', '$', 'i', '#', '[', '#', 'i', 2, 1, 2, 1};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vS), "[json.exception.parse_error.113] parse error at byte 9: syntax error while parsing BJData size: expected length type specification (U, i, u, I, m, l, M, L) after '#'; last byte: 0x01", json::parse_error&);
|
||
CHECK(json::from_bjdata(vS, true, false).is_discarded());
|
||
|
||
std::vector<uint8_t> const vT = {'[', '$', 'i', '#', '[', 'i', 2, 'i'};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vT), "[json.exception.parse_error.110] parse error at byte 9: syntax error while parsing BJData number: unexpected end of input", json::parse_error&);
|
||
CHECK(json::from_bjdata(vT, true, false).is_discarded());
|
||
|
||
std::vector<uint8_t> const vT0 = {'[', '$', 'i', '#', '[', 'i'};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vT0), "[json.exception.parse_error.110] parse error at byte 7: syntax error while parsing BJData number: unexpected end of input", json::parse_error&);
|
||
CHECK(json::from_bjdata(vT0, true, false).is_discarded());
|
||
|
||
std::vector<uint8_t> const vu = {'[', '$', 'i', '#', '[', '$', 'i', '#', 'u', 1, 0};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vu), "[json.exception.parse_error.110] parse error at byte 12: syntax error while parsing BJData number: unexpected end of input", json::parse_error&);
|
||
CHECK(json::from_bjdata(vu, true, false).is_discarded());
|
||
|
||
std::vector<uint8_t> const vm = {'[', '$', 'i', '#', '[', '$', 'i', '#', 'm', 1, 0, 0, 0};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vm), "[json.exception.parse_error.110] parse error at byte 14: syntax error while parsing BJData number: unexpected end of input", json::parse_error&);
|
||
CHECK(json::from_bjdata(vm, true, false).is_discarded());
|
||
|
||
std::vector<uint8_t> const vM = {'[', '$', 'i', '#', '[', '$', 'i', '#', 'M', 1, 0, 0, 0, 0, 0, 0, 0};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vM), "[json.exception.parse_error.110] parse error at byte 18: syntax error while parsing BJData number: unexpected end of input", json::parse_error&);
|
||
CHECK(json::from_bjdata(vM, true, false).is_discarded());
|
||
|
||
std::vector<uint8_t> const vU = {'[', '$', 'U', '#', '[', '$', 'i', '#', 'i', 2, 2, 3, 1, 2, 3, 4, 5};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vU), "[json.exception.parse_error.110] parse error at byte 18: syntax error while parsing BJData number: unexpected end of input", json::parse_error&);
|
||
CHECK(json::from_bjdata(vU, true, false).is_discarded());
|
||
|
||
std::vector<uint8_t> const vB = {'[', '$', 'B', '#', '[', '$', 'i', '#', 'i', 2, 2, 3, 1, 2, 3, 4, 5};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vU), "[json.exception.parse_error.110] parse error at byte 18: syntax error while parsing BJData number: unexpected end of input", json::parse_error&);
|
||
CHECK(json::from_bjdata(vU, true, false).is_discarded());
|
||
|
||
std::vector<uint8_t> const vT1 = {'[', '$', 'T', '#', '[', '$', 'i', '#', 'i', 2, 2, 3};
|
||
CHECK(json::from_bjdata(vT1, true, false).is_discarded());
|
||
|
||
std::vector<uint8_t> const vh = {'[', '$', 'h', '#', '[', '$', 'i', '#', 'i', 2, 2, 3};
|
||
CHECK(json::from_bjdata(vh, true, false).is_discarded());
|
||
|
||
std::vector<uint8_t> const vR = {'[', '$', 'i', '#', '[', 'i', 1, '[', ']', ']', 1};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vR), "[json.exception.parse_error.113] parse error at byte 8: syntax error while parsing BJData size: ndarray dimensional vector is not allowed", json::parse_error&);
|
||
CHECK(json::from_bjdata(vR, true, false).is_discarded());
|
||
|
||
std::vector<uint8_t> const vRo = {'[', '$', 'i', '#', '[', 'i', 0, '{', '}', ']', 1};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vRo), "[json.exception.parse_error.113] parse error at byte 8: syntax error while parsing BJData size: expected length type specification (U, i, u, I, m, l, M, L) after '#'; last byte: 0x7B", json::parse_error&);
|
||
CHECK(json::from_bjdata(vRo, true, false).is_discarded());
|
||
|
||
std::vector<uint8_t> const vR1 = {'[', '$', 'i', '#', '[', '[', 'i', 1, ']', ']', 1};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vR1), "[json.exception.parse_error.113] parse error at byte 6: syntax error while parsing BJData size: ndarray dimensional vector is not allowed", json::parse_error&);
|
||
CHECK(json::from_bjdata(vR1, true, false).is_discarded());
|
||
|
||
// a dimension vector that opens another one is rejected where the
|
||
// nested '[' is read, rather than after it has been descended into
|
||
std::vector<uint8_t> const vR2 = {'[', '$', 'i', '#', '[', '#', '[', 'i', 1, ']', ']', 1};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vR2), "[json.exception.parse_error.113] parse error at byte 7: syntax error while parsing BJData size: ndarray dimensional vector is not allowed", json::parse_error&);
|
||
CHECK(json::from_bjdata(vR2, true, false).is_discarded());
|
||
|
||
std::vector<uint8_t> const vR3 = {'[', '#', '[', 'i', '2', 'i', 2, ']'};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vR3), "[json.exception.parse_error.112] parse error at byte 8: syntax error while parsing BJData size: ndarray requires both type and size", json::parse_error&);
|
||
CHECK(json::from_bjdata(vR3, true, false).is_discarded());
|
||
|
||
std::vector<uint8_t> const vR4 = {'[', '$', 'i', '#', '[', '$', 'i', '#', '[', 'i', 1, ']', 1};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vR4), "[json.exception.parse_error.113] parse error at byte 9: syntax error while parsing BJData size: ndarray dimensional vector is not allowed", json::parse_error&);
|
||
CHECK(json::from_bjdata(vR4, true, false).is_discarded());
|
||
|
||
std::vector<uint8_t> const vR5 = {'[', '$', 'i', '#', '[', '[', '[', ']', ']', ']'};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vR5), "[json.exception.parse_error.113] parse error at byte 6: syntax error while parsing BJData size: ndarray dimensional vector is not allowed", json::parse_error&);
|
||
CHECK(json::from_bjdata(vR5, true, false).is_discarded());
|
||
|
||
std::vector<uint8_t> const vR6 = {'[', '$', 'i', '#', '[', '$', 'i', '#', '[', 'i', '2', 'i', 2, ']'};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vR6), "[json.exception.parse_error.113] parse error at byte 9: syntax error while parsing BJData size: ndarray dimensional vector is not allowed", json::parse_error&);
|
||
CHECK(json::from_bjdata(vR6, true, false).is_discarded());
|
||
|
||
std::vector<uint8_t> const vH = {'[', 'H', '[', '#', '[', '$', 'i', '#', '[', 'i', '2', 'i', 2, ']'};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vH), "[json.exception.parse_error.113] parse error at byte 3: syntax error while parsing BJData size: ndarray dimensional vector is not allowed", json::parse_error&);
|
||
CHECK(json::from_bjdata(vH, true, false).is_discarded());
|
||
|
||
// Every "#[" of this chain used to open another dimension vector
|
||
// and cost several stack frames before anything was rejected, so a
|
||
// long enough chain crashed the process (see #5104). The nested
|
||
// vector is refused where it is read, so the length is irrelevant.
|
||
std::vector<uint8_t> vRdeep = {'['};
|
||
for (std::size_t i = 0; i < 100000; ++i)
|
||
{
|
||
vRdeep.push_back('#');
|
||
vRdeep.push_back('[');
|
||
}
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vRdeep), "[json.exception.parse_error.113] parse error at byte 5: syntax error while parsing BJData size: ndarray dimensional vector is not allowed", json::parse_error&);
|
||
CHECK(json::from_bjdata(vRdeep, true, false).is_discarded());
|
||
}
|
||
|
||
SECTION("objects")
|
||
{
|
||
std::vector<uint8_t> const vST = {'{', '$', 'i', '#', 'i', 2, 'i', 1, 'a', 1};
|
||
json _;
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vST), "[json.exception.parse_error.110] parse error at byte 11: syntax error while parsing BJData value: unexpected end of input", json::parse_error&);
|
||
CHECK(json::from_bjdata(vST, true, false).is_discarded());
|
||
|
||
std::vector<uint8_t> const vT = {'{', '$', 'i', 'i', 1, 'a', 1};
|
||
CHECK_THROWS_WITH(_ = json::from_bjdata(vT), "[json.exception.parse_error.112] parse error at byte 4: syntax error while parsing BJData size: expected '#' after type information; last byte: 0x69");
|
||
CHECK(json::from_bjdata(vT, true, false).is_discarded());
|
||
|
||
std::vector<uint8_t> const vS = {'{', '#', 'i', 2, 'i', 1, 'a', 'i', 1};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vS), "[json.exception.parse_error.110] parse error at byte 10: syntax error while parsing BJData value: unexpected end of input", json::parse_error&);
|
||
CHECK(json::from_bjdata(vS, true, false).is_discarded());
|
||
|
||
std::vector<uint8_t> const v = {'{', 'i', 1, 'a', 'i', 1};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v), "[json.exception.parse_error.110] parse error at byte 7: syntax error while parsing BJData value: unexpected end of input", json::parse_error&);
|
||
CHECK(json::from_bjdata(v, true, false).is_discarded());
|
||
|
||
std::vector<uint8_t> const v2 = {'{', 'i', 1, 'a', 'i', 1, 'i'};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v2), "[json.exception.parse_error.110] parse error at byte 8: syntax error while parsing BJData number: unexpected end of input", json::parse_error&);
|
||
CHECK(json::from_bjdata(v2, true, false).is_discarded());
|
||
|
||
std::vector<uint8_t> const v3 = {'{', 'i', 1, 'a'};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v3), "[json.exception.parse_error.110] parse error at byte 5: syntax error while parsing BJData value: unexpected end of input", json::parse_error&);
|
||
CHECK(json::from_bjdata(v3, true, false).is_discarded());
|
||
|
||
std::vector<uint8_t> const vST1 = {'{', '$', 'd', '#', 'i', 2, 'i', 1, 'a'};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vST1), "[json.exception.parse_error.110] parse error at byte 10: syntax error while parsing BJData number: unexpected end of input", json::parse_error&);
|
||
CHECK(json::from_bjdata(vST1, true, false).is_discarded());
|
||
|
||
std::vector<uint8_t> const vST2 = {'{', '#', 'i', 2, 'i', 1, 'a'};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vST2), "[json.exception.parse_error.110] parse error at byte 8: syntax error while parsing BJData value: unexpected end of input", json::parse_error&);
|
||
CHECK(json::from_bjdata(vST2, true, false).is_discarded());
|
||
|
||
std::vector<uint8_t> const vO = {'{', '#', '[', 'i', 2, 'i', 1, ']', 'i', 1, 'a', 'i', 1, 'i', 1, 'b', 'i', 2};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vO), "[json.exception.parse_error.112] parse error at byte 8: syntax error while parsing BJData size: ndarray requires both type and size", json::parse_error&);
|
||
CHECK(json::from_bjdata(vO, true, false).is_discarded());
|
||
|
||
std::vector<uint8_t> const vO2 = {'{', '$', 'i', '#', '[', 'i', 2, 'i', 1, ']', 'i', 1, 'a', 1, 'i', 1, 'b', 2};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vO2), "[json.exception.parse_error.112] parse error at byte 10: syntax error while parsing BJData object: BJData object does not support ND-array size in optimized format", json::parse_error&);
|
||
CHECK(json::from_bjdata(vO2, true, false).is_discarded());
|
||
}
|
||
}
|
||
|
||
SECTION("writing optimized values")
|
||
{
|
||
SECTION("integer")
|
||
{
|
||
SECTION("array of i")
|
||
{
|
||
json const j = {1, -1};
|
||
std::vector<uint8_t> const expected = {'[', '$', 'i', '#', 'i', 2, 1, 0xff};
|
||
CHECK(json::to_bjdata(j, true, true) == expected);
|
||
}
|
||
|
||
SECTION("array of U")
|
||
{
|
||
json const j = {200, 201};
|
||
std::vector<uint8_t> const expected = {'[', '$', 'U', '#', 'i', 2, 0xC8, 0xC9};
|
||
CHECK(json::to_bjdata(j, true, true) == expected);
|
||
}
|
||
|
||
SECTION("array of I")
|
||
{
|
||
json const j = {30000, -30000};
|
||
std::vector<uint8_t> const expected = {'[', '$', 'I', '#', 'i', 2, 0x30, 0x75, 0xd0, 0x8a};
|
||
CHECK(json::to_bjdata(j, true, true) == expected);
|
||
}
|
||
|
||
SECTION("array of u")
|
||
{
|
||
json const j = {50000, 50001};
|
||
std::vector<uint8_t> const expected = {'[', '$', 'u', '#', 'i', 2, 0x50, 0xC3, 0x51, 0xC3};
|
||
CHECK(json::to_bjdata(j, true, true) == expected);
|
||
}
|
||
|
||
SECTION("array of l")
|
||
{
|
||
json const j = {70000, -70000};
|
||
std::vector<uint8_t> const expected = {'[', '$', 'l', '#', 'i', 2, 0x70, 0x11, 0x01, 0x00, 0x90, 0xEE, 0xFE, 0xFF};
|
||
CHECK(json::to_bjdata(j, true, true) == expected);
|
||
}
|
||
|
||
SECTION("array of m")
|
||
{
|
||
json const j = {3147483647, 3147483648};
|
||
std::vector<uint8_t> const expected = {'[', '$', 'm', '#', 'i', 2, 0xFF, 0xC9, 0x9A, 0xBB, 0x00, 0xCA, 0x9A, 0xBB};
|
||
CHECK(json::to_bjdata(j, true, true) == expected);
|
||
}
|
||
|
||
SECTION("array of L")
|
||
{
|
||
json const j = {5000000000, -5000000000};
|
||
std::vector<uint8_t> const expected = {'[', '$', 'L', '#', 'i', 2, 0x00, 0xF2, 0x05, 0x2A, 0x01, 0x00, 0x00, 0x00, 0x00, 0x0E, 0xFA, 0xD5, 0xFE, 0xFF, 0xFF, 0xFF};
|
||
CHECK(json::to_bjdata(j, true, true) == expected);
|
||
}
|
||
}
|
||
|
||
SECTION("unsigned integer")
|
||
{
|
||
SECTION("array of i")
|
||
{
|
||
json const j = {1u, 2u};
|
||
std::vector<uint8_t> const expected = {'[', '$', 'i', '#', 'i', 2, 1, 2};
|
||
std::vector<uint8_t> const expected_size = {'[', '#', 'i', 2, 'i', 1, 'i', 2};
|
||
CHECK(json::to_bjdata(j, true, true) == expected);
|
||
CHECK(json::to_bjdata(j, true) == expected_size);
|
||
}
|
||
|
||
SECTION("array of U")
|
||
{
|
||
json const j = {200u, 201u};
|
||
std::vector<uint8_t> const expected = {'[', '$', 'U', '#', 'i', 2, 0xC8, 0xC9};
|
||
std::vector<uint8_t> const expected_size = {'[', '#', 'i', 2, 'U', 0xC8, 'U', 0xC9};
|
||
CHECK(json::to_bjdata(j, true, true) == expected);
|
||
CHECK(json::to_bjdata(j, true) == expected_size);
|
||
}
|
||
|
||
SECTION("array of I")
|
||
{
|
||
json const j = {30000u, 30001u};
|
||
std::vector<uint8_t> const expected = {'[', '$', 'I', '#', 'i', 2, 0x30, 0x75, 0x31, 0x75};
|
||
std::vector<uint8_t> const expected_size = {'[', '#', 'i', 2, 'I', 0x30, 0x75, 'I', 0x31, 0x75};
|
||
CHECK(json::to_bjdata(j, true, true) == expected);
|
||
CHECK(json::to_bjdata(j, true) == expected_size);
|
||
}
|
||
|
||
SECTION("array of u")
|
||
{
|
||
json const j = {50000u, 50001u};
|
||
std::vector<uint8_t> const expected = {'[', '$', 'u', '#', 'i', 2, 0x50, 0xC3, 0x51, 0xC3};
|
||
std::vector<uint8_t> const expected_size = {'[', '#', 'i', 2, 'u', 0x50, 0xC3, 'u', 0x51, 0xC3};
|
||
CHECK(json::to_bjdata(j, true, true) == expected);
|
||
CHECK(json::to_bjdata(j, true) == expected_size);
|
||
}
|
||
|
||
SECTION("array of l")
|
||
{
|
||
json const j = {70000u, 70001u};
|
||
std::vector<uint8_t> const expected = {'[', '$', 'l', '#', 'i', 2, 0x70, 0x11, 0x01, 0x00, 0x71, 0x11, 0x01, 0x00};
|
||
std::vector<uint8_t> const expected_size = {'[', '#', 'i', 2, 'l', 0x70, 0x11, 0x01, 0x00, 'l', 0x71, 0x11, 0x01, 0x00};
|
||
CHECK(json::to_bjdata(j, true, true) == expected);
|
||
CHECK(json::to_bjdata(j, true) == expected_size);
|
||
}
|
||
|
||
SECTION("array of m")
|
||
{
|
||
json const j = {3147483647u, 3147483648u};
|
||
std::vector<uint8_t> const expected = {'[', '$', 'm', '#', 'i', 2, 0xFF, 0xC9, 0x9A, 0xBB, 0x00, 0xCA, 0x9A, 0xBB};
|
||
std::vector<uint8_t> const expected_size = {'[', '#', 'i', 2, 'm', 0xFF, 0xC9, 0x9A, 0xBB, 'm', 0x00, 0xCA, 0x9A, 0xBB};
|
||
CHECK(json::to_bjdata(j, true, true) == expected);
|
||
CHECK(json::to_bjdata(j, true) == expected_size);
|
||
}
|
||
|
||
SECTION("array of L")
|
||
{
|
||
json const j = {5000000000u, 5000000001u};
|
||
std::vector<uint8_t> const expected = {'[', '$', 'L', '#', 'i', 2, 0x00, 0xF2, 0x05, 0x2A, 0x01, 0x00, 0x00, 0x00, 0x01, 0xF2, 0x05, 0x2A, 0x01, 0x00, 0x00, 0x00};
|
||
std::vector<uint8_t> const expected_size = {'[', '#', 'i', 2, 'L', 0x00, 0xF2, 0x05, 0x2A, 0x01, 0x00, 0x00, 0x00, 'L', 0x01, 0xF2, 0x05, 0x2A, 0x01, 0x00, 0x00, 0x00};
|
||
CHECK(json::to_bjdata(j, true, true) == expected);
|
||
CHECK(json::to_bjdata(j, true) == expected_size);
|
||
}
|
||
|
||
SECTION("array of M")
|
||
{
|
||
json const j = {10223372036854775807ull, 10223372036854775808ull};
|
||
std::vector<uint8_t> const expected = {'[', '$', 'M', '#', 'i', 2, 0xFF, 0xFF, 0x63, 0xA7, 0xB3, 0xB6, 0xE0, 0x8D, 0x00, 0x00, 0x64, 0xA7, 0xB3, 0xB6, 0xE0, 0x8D};
|
||
std::vector<uint8_t> const expected_size = {'[', '#', 'i', 2, 'M', 0xFF, 0xFF, 0x63, 0xA7, 0xB3, 0xB6, 0xE0, 0x8D, 'M', 0x00, 0x00, 0x64, 0xA7, 0xB3, 0xB6, 0xE0, 0x8D};
|
||
CHECK(json::to_bjdata(j, true, true) == expected);
|
||
CHECK(json::to_bjdata(j, true) == expected_size);
|
||
}
|
||
}
|
||
}
|
||
}
|
||
|
||
TEST_CASE("BJData input that cannot be read is discarded by every overload")
|
||
{
|
||
std::vector<std::uint8_t> input = json::to_bjdata(json({{"a", {1, 2}}}));
|
||
input.pop_back();
|
||
|
||
json _;
|
||
CHECK_THROWS_AS(_ = json::from_bjdata(input.begin(), input.end()), json::parse_error&);
|
||
CHECK(json::from_bjdata(input, true, false).is_discarded());
|
||
CHECK(json::from_bjdata(input.begin(), input.end(), true, false).is_discarded());
|
||
}
|
||
|
||
TEST_CASE("BJData SAX parsing stops at every event")
|
||
{
|
||
// Containers are opened and closed by the loop that reads them; a SAX
|
||
// handler that rejects any event - including the end of a nested
|
||
// container - must stop the parse right there.
|
||
const auto count_events = [](const std::vector<std::uint8_t>& input)
|
||
{
|
||
int events = 0;
|
||
while (true)
|
||
{
|
||
SaxCountdown scp(events);
|
||
if (json::sax_parse(input, &scp, json::input_format_t::bjdata))
|
||
{
|
||
return events;
|
||
}
|
||
++events;
|
||
REQUIRE(events < 1000);
|
||
}
|
||
};
|
||
|
||
// 20 events: every container kind closes inside another one
|
||
const json j = json::parse(R"({"a": [1, {"b": []}], "c": {"d": [[2]]}})");
|
||
CHECK(count_events(json::to_bjdata(j)) == 20);
|
||
CHECK(count_events(json::to_bjdata(j, true)) == 20);
|
||
CHECK(count_events(json::to_bjdata(j, true, true)) == 20);
|
||
|
||
// an ND-array is announced as an annotated object: start_object, then
|
||
// _ArrayType_, _ArraySize_ and _ArrayData_ with its elements
|
||
const json ndarray = json::parse(R"({"_ArrayType_": "uint8", "_ArraySize_": [2, 2], "_ArrayData_": [1, 2, 3, 4]})");
|
||
CHECK(count_events(json::to_bjdata(ndarray, true, true)) == 16);
|
||
}
|
||
|
||
TEST_CASE("issue #5405 - array reserve for definite-length BJData arrays")
|
||
{
|
||
#if !defined(JSON_NOEXCEPTION)
|
||
// this SECTION relies on catching a thrown exception to distinguish
|
||
// which of two acceptable, bounded rejections a hostile header took;
|
||
// under JSON_NOEXCEPTION, JSON_THROW never produces a catchable C++
|
||
// exception (it aborts instead), so this cannot be tested that way here
|
||
SECTION("a huge claimed length with no element data must not over-allocate")
|
||
{
|
||
// optimized form [$type#count: type 'i' (int8), count as a four-byte
|
||
// little-endian 'l' (int32) of 0x7FFFFFFF (2147483647), but no
|
||
// element data at all. max_size() for a std::vector is far larger
|
||
// than this count, so it does not reject the header outright; the
|
||
// (capped) reservation must not attempt to allocate space for
|
||
// billions of elements before the missing data is detected.
|
||
json _;
|
||
const std::vector<uint8_t> input = {'[', '$', 'i', '#', 'l', 0xFF, 0xFF, 0xFF, 0x7F};
|
||
// On a platform where std::vector<json>::max_size() is smaller than
|
||
// the claimed count (e.g. 32-bit, where max_size() is bounded by a
|
||
// 32-bit SIZE_MAX divided by sizeof(json)), the SAX consumer's own
|
||
// check rejects the header outright (out_of_range.408, with the
|
||
// claimed count in the message) instead of accepting it and only
|
||
// finding it short of data once the (capped) reservation looks for
|
||
// element bytes that were never provided (parse_error.110). Either
|
||
// is an acceptable, bounded rejection of the hostile header -- the
|
||
// property under test is that no path attempts to allocate space
|
||
// for billions of elements.
|
||
bool threw = false;
|
||
try
|
||
{
|
||
_ = json::from_bjdata(input);
|
||
}
|
||
catch (const json::parse_error& e)
|
||
{
|
||
threw = true;
|
||
CHECK(e.id == 110);
|
||
CHECK(std::string(e.what()) == "[json.exception.parse_error.110] parse error at byte 10: syntax error while parsing BJData number: unexpected end of input");
|
||
}
|
||
catch (const json::out_of_range& e)
|
||
{
|
||
threw = true;
|
||
CHECK(e.id == 408);
|
||
CHECK(std::string(e.what()).find("excessive array size") != std::string::npos);
|
||
}
|
||
CHECK(threw);
|
||
|
||
// json_sax_dom_parser::start_array()'s max_size() check (unlike the
|
||
// scanner's own parse_error path) throws unconditionally via
|
||
// JSON_THROW rather than going through sax->parse_error(), so it is
|
||
// not gated by allow_exceptions=false on a platform where this
|
||
// header hits that check (e.g. 32-bit, see above) -- allow either
|
||
// a discarded result or the same out_of_range it throws with
|
||
// exceptions enabled.
|
||
try
|
||
{
|
||
CHECK(json::from_bjdata(input, true, false).is_discarded());
|
||
}
|
||
catch (const json::out_of_range& e)
|
||
{
|
||
CHECK(e.id == 408);
|
||
}
|
||
}
|
||
#endif
|
||
|
||
SECTION("arrays of various sizes decode to the same value as before the reserve optimization")
|
||
{
|
||
for (const auto size :
|
||
{
|
||
std::size_t{0}, std::size_t{1}, std::size_t{5}, // small
|
||
std::size_t{16384}, // exactly at the reserve cap
|
||
std::size_t{20000} // above the reserve cap
|
||
})
|
||
{
|
||
CAPTURE(size)
|
||
json j = json::array();
|
||
for (std::size_t i = 0; i < size; ++i)
|
||
{
|
||
j.push_back(static_cast<int>(i % 1000));
|
||
}
|
||
|
||
// exercise both the plain and the optimized [$type#count encoding
|
||
const auto packed_plain = json::to_bjdata(j);
|
||
CHECK(json::from_bjdata(packed_plain) == j);
|
||
|
||
const auto packed_optimized = json::to_bjdata(j, true, true);
|
||
CHECK(json::from_bjdata(packed_optimized) == j);
|
||
}
|
||
}
|
||
|
||
SECTION("a user-defined SAX consumer is unaffected by the internal DOM reserve optimization")
|
||
{
|
||
// the reserve() call is local to json_sax_dom_parser / json_sax_dom_callback_parser;
|
||
// a custom SAX consumer that does not touch a DOM array sees identical events
|
||
json j = json::array();
|
||
for (int i = 0; i < 100; ++i)
|
||
{
|
||
j.push_back(i);
|
||
}
|
||
const auto packed = json::to_bjdata(j, true, true);
|
||
|
||
SaxCountdown scp(1000000); // large enough to never trigger an abort
|
||
CHECK(json::sax_parse(packed, &scp, json::input_format_t::bjdata));
|
||
}
|
||
}
|
||
|
||
TEST_CASE("BJData - open arrays reserve only what the input has backed")
|
||
{
|
||
// an array that announces its size gets storage for (up to 16384 of) its
|
||
// elements when it opens; the arrays that are open at the same time must
|
||
// not hold more than the input read so far can back
|
||
SECTION("nested headers that announce 16384 elements and are never completed")
|
||
{
|
||
// 200 levels would hold 200 * 16384 * sizeof(json),
|
||
// if each of them reserved what it announces
|
||
const auto input = utils::repeated({'[', '#', 'I', 0x00, 0x40}, 200);
|
||
|
||
utils::counts() = utils::allocation_counts{};
|
||
const auto j = utils::counted_json::from_bjdata(input, true, false);
|
||
CHECK(j.is_discarded());
|
||
CAPTURE(utils::counts().peak)
|
||
// the credit is 65,536 values; the base peaks at more than 3 million
|
||
CHECK(utils::counts().peak < sizeof(utils::counted_json) * 128 * 1024);
|
||
}
|
||
|
||
SECTION("a completed array is still reserved in one allocation")
|
||
{
|
||
json array = json::array();
|
||
for (int i = 0; i < 10000; ++i)
|
||
{
|
||
array.push_back(i % 100);
|
||
}
|
||
|
||
// an array without a size is not reserved, so only these two
|
||
{
|
||
const auto packed = json::to_bjdata(array, true);
|
||
|
||
utils::counts() = utils::allocation_counts{};
|
||
const auto j = utils::counted_json::from_bjdata(packed);
|
||
CHECK(j.size() == 10000);
|
||
// the array and its storage; growing the storage as the elements
|
||
// arrive would take more than a dozen allocations
|
||
CAPTURE(utils::counts().allocations)
|
||
CHECK(utils::counts().allocations <= 4);
|
||
}
|
||
|
||
{
|
||
const auto packed = json::to_bjdata(array, true, true);
|
||
|
||
utils::counts() = utils::allocation_counts{};
|
||
const auto j = utils::counted_json::from_bjdata(packed);
|
||
CHECK(j.size() == 10000);
|
||
// the array and its storage; growing the storage as the elements
|
||
// arrive would take more than a dozen allocations
|
||
CAPTURE(utils::counts().allocations)
|
||
CHECK(utils::counts().allocations <= 4);
|
||
}
|
||
}
|
||
}
|
||
|
||
TEST_CASE("Universal Binary JSON Specification Examples 1")
|
||
{
|
||
SECTION("Null Value")
|
||
{
|
||
json const j = {{"passcode", nullptr}};
|
||
std::vector<uint8_t> v = {'{', 'i', 8, 'p', 'a', 's', 's', 'c', 'o', 'd', 'e', 'Z', '}'};
|
||
CHECK(json::to_bjdata(j) == v);
|
||
CHECK(json::from_bjdata(v) == j);
|
||
}
|
||
|
||
SECTION("No-Op Value")
|
||
{
|
||
json const j = {"foo", "bar", "baz"};
|
||
std::vector<uint8_t> v = {'[', 'S', 'i', 3, 'f', 'o', 'o',
|
||
'S', 'i', 3, 'b', 'a', 'r',
|
||
'S', 'i', 3, 'b', 'a', 'z', ']'
|
||
};
|
||
std::vector<uint8_t> const v2 = {'[', 'S', 'i', 3, 'f', 'o', 'o', 'N',
|
||
'S', 'i', 3, 'b', 'a', 'r', 'N', 'N', 'N',
|
||
'S', 'i', 3, 'b', 'a', 'z', 'N', 'N', ']'
|
||
};
|
||
CHECK(json::to_bjdata(j) == v);
|
||
CHECK(json::from_bjdata(v) == j);
|
||
CHECK(json::from_bjdata(v2) == j);
|
||
}
|
||
|
||
SECTION("Boolean Types")
|
||
{
|
||
json const j = {{"authorized", true}, {"verified", false}};
|
||
std::vector<uint8_t> v = {'{', 'i', 10, 'a', 'u', 't', 'h', 'o', 'r', 'i', 'z', 'e', 'd', 'T',
|
||
'i', 8, 'v', 'e', 'r', 'i', 'f', 'i', 'e', 'd', 'F', '}'
|
||
};
|
||
CHECK(json::to_bjdata(j) == v);
|
||
CHECK(json::from_bjdata(v) == j);
|
||
}
|
||
|
||
SECTION("Numeric Types")
|
||
{
|
||
json j =
|
||
{
|
||
{"int8", 16},
|
||
{"uint8", 255},
|
||
{"int16", 32767},
|
||
{"uint16", 42767},
|
||
{"int32", 2147483647},
|
||
{"uint32", 3147483647},
|
||
{"int64", 9223372036854775807},
|
||
{"uint64", 10223372036854775807ull},
|
||
{"float64", 113243.7863123}
|
||
};
|
||
std::vector<uint8_t> v = {'{',
|
||
'i', 7, 'f', 'l', 'o', 'a', 't', '6', '4', 'D', 0xcf, 0x34, 0xbc, 0x94, 0xbc, 0xa5, 0xfb, 0x40,
|
||
'i', 5, 'i', 'n', 't', '1', '6', 'I', 0xff, 0x7f,
|
||
'i', 5, 'i', 'n', 't', '3', '2', 'l', 0xff, 0xff, 0xff, 0x7f,
|
||
'i', 5, 'i', 'n', 't', '6', '4', 'L', 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0x7f,
|
||
'i', 4, 'i', 'n', 't', '8', 'i', 16,
|
||
'i', 6, 'u', 'i', 'n', 't', '1', '6', 'u', 0x0F, 0xA7,
|
||
'i', 6, 'u', 'i', 'n', 't', '3', '2', 'm', 0xFF, 0xC9, 0x9A, 0xBB,
|
||
'i', 6, 'u', 'i', 'n', 't', '6', '4', 'M', 0xFF, 0xFF, 0x63, 0xA7, 0xB3, 0xB6, 0xE0, 0x8D,
|
||
'i', 5, 'u', 'i', 'n', 't', '8', 'U', 0xff,
|
||
'}'
|
||
};
|
||
CHECK(json::to_bjdata(j) == v);
|
||
CHECK(json::from_bjdata(v) == j);
|
||
}
|
||
|
||
SECTION("Char Type")
|
||
{
|
||
json const j = {{"rolecode", "a"}, {"delim", ";"}};
|
||
std::vector<uint8_t> const v = {'{', 'i', 5, 'd', 'e', 'l', 'i', 'm', 'C', ';', 'i', 8, 'r', 'o', 'l', 'e', 'c', 'o', 'd', 'e', 'C', 'a', '}'};
|
||
//CHECK(json::to_bjdata(j) == v);
|
||
CHECK(json::from_bjdata(v) == j);
|
||
}
|
||
|
||
SECTION("Byte Type")
|
||
{
|
||
const auto s = std::vector<std::uint8_t>(
|
||
{
|
||
static_cast<std::uint8_t>(222),
|
||
static_cast<std::uint8_t>(173),
|
||
static_cast<std::uint8_t>(190),
|
||
static_cast<std::uint8_t>(239)
|
||
});
|
||
json const j = {{"binary", json::binary(s)}, {"val", 123}};
|
||
std::vector<uint8_t> const v = {'{', 'i', 6, 'b', 'i', 'n', 'a', 'r', 'y', '[', '$', 'B', '#', 'i', 4, 222, 173, 190, 239, 'i', 3, 'v', 'a', 'l', 'i', 123, '}'};
|
||
//CHECK(json::to_bjdata(j) == v); // 123 value gets encoded as uint8
|
||
CHECK(json::from_bjdata(v) == j);
|
||
}
|
||
|
||
SECTION("String Type")
|
||
{
|
||
SECTION("English")
|
||
{
|
||
json const j = "hello";
|
||
std::vector<uint8_t> v = {'S', 'i', 5, 'h', 'e', 'l', 'l', 'o'};
|
||
CHECK(json::to_bjdata(j) == v);
|
||
CHECK(json::from_bjdata(v) == j);
|
||
}
|
||
|
||
SECTION("Russian")
|
||
{
|
||
json const j = "привет";
|
||
std::vector<uint8_t> v = {'S', 'i', 12, 0xD0, 0xBF, 0xD1, 0x80, 0xD0, 0xB8, 0xD0, 0xB2, 0xD0, 0xB5, 0xD1, 0x82};
|
||
CHECK(json::to_bjdata(j) == v);
|
||
CHECK(json::from_bjdata(v) == j);
|
||
}
|
||
|
||
SECTION("Russian")
|
||
{
|
||
json const j = "مرحبا";
|
||
std::vector<uint8_t> v = {'S', 'i', 10, 0xD9, 0x85, 0xD8, 0xB1, 0xD8, 0xAD, 0xD8, 0xA8, 0xD8, 0xA7};
|
||
CHECK(json::to_bjdata(j) == v);
|
||
CHECK(json::from_bjdata(v) == j);
|
||
}
|
||
|
||
SECTION("ill-formed UTF-8 (see #5529, #5651)")
|
||
{
|
||
// none of the binary format specs requires a decoder to reject
|
||
// ill-formed UTF-8 in a text string, so a value whose bytes are
|
||
// not valid UTF-8 (0xC0 0xAE is an overlong encoding of '.')
|
||
// round-trips byte for byte as a string value; to_bjdata() writes
|
||
// the bytes unchanged, as before 3.13.0, unless
|
||
// JSON_STRICT_BINARY_UTF8 is enabled (see
|
||
// unit-binary_utf8_strict.cpp)
|
||
const std::vector<uint8_t> v = {'S', 'i', 2, 0xc0, 0xae};
|
||
json j;
|
||
CHECK_NOTHROW(j = json::from_bjdata(v));
|
||
REQUIRE(j.is_string());
|
||
CHECK(j.get_ref<const json::string_t&>() == std::string("\xc0\xae"));
|
||
CHECK_THROWS_AS(utils::ignore_return_value(j.dump()), json::type_error&);
|
||
CHECK(json::from_bjdata(json::to_bjdata(j)) == j);
|
||
|
||
// the same bytes as an object key round-trip as well
|
||
const std::vector<uint8_t> v_key = {'{', 'i', 2, 0xc0, 0xae, 'i', 1, '}'};
|
||
json j_key;
|
||
CHECK_NOTHROW(j_key = json::from_bjdata(v_key));
|
||
REQUIRE(j_key.is_object());
|
||
CHECK(j_key.contains(std::string("\xc0\xae")));
|
||
CHECK(json::from_bjdata(json::to_bjdata(j_key)) == j_key);
|
||
|
||
CHECK(json::from_bjdata(json::to_bjdata(json("\xFF"))) == json("\xFF"));
|
||
// a truncated multi-byte sequence
|
||
CHECK(json::from_bjdata(json::to_bjdata(json("\xC3"))) == json("\xC3"));
|
||
// an encoded surrogate half (U+D800)
|
||
CHECK(json::from_bjdata(json::to_bjdata(json("\xED\xA0\x80"))) == json("\xED\xA0\x80"));
|
||
// an overlong encoding of '.'
|
||
CHECK(json::from_bjdata(json::to_bjdata(json("\xC0\xAF"))) == json("\xC0\xAF"));
|
||
|
||
// an object key with ill-formed UTF-8 is kept the same way
|
||
CHECK(json::from_bjdata(json::to_bjdata(json{{"\xFF", 1}})) == json{{"\xFF", 1}});
|
||
}
|
||
}
|
||
|
||
SECTION("Array Type")
|
||
{
|
||
SECTION("size=false type=false")
|
||
{
|
||
// note the float has been replaced by a double
|
||
json const j = {nullptr, true, false, 4782345193, 153.132, "ham"};
|
||
std::vector<uint8_t> v = {'[', 'Z', 'T', 'F', 'L', 0xE9, 0xCB, 0x0C, 0x1D, 0x01, 0x00, 0x00, 0x00, 'D', 0x4e, 0x62, 0x10, 0x58, 0x39, 0x24, 0x63, 0x40, 'S', 'i', 3, 'h', 'a', 'm', ']'};
|
||
CHECK(json::to_bjdata(j) == v);
|
||
CHECK(json::from_bjdata(v) == j);
|
||
}
|
||
|
||
SECTION("size=true type=false")
|
||
{
|
||
// note the float has been replaced by a double
|
||
json const j = {nullptr, true, false, 4782345193, 153.132, "ham"};
|
||
std::vector<uint8_t> v = {'[', '#', 'i', 6, 'Z', 'T', 'F', 'L', 0xE9, 0xCB, 0x0C, 0x1D, 0x01, 0x00, 0x00, 0x00, 'D', 0x4e, 0x62, 0x10, 0x58, 0x39, 0x24, 0x63, 0x40, 'S', 'i', 3, 'h', 'a', 'm'};
|
||
CHECK(json::to_bjdata(j, true) == v);
|
||
CHECK(json::from_bjdata(v) == j);
|
||
}
|
||
|
||
SECTION("size=true type=true")
|
||
{
|
||
// note the float has been replaced by a double
|
||
json const j = {nullptr, true, false, 4782345193, 153.132, "ham"};
|
||
std::vector<uint8_t> v = {'[', '#', 'i', 6, 'Z', 'T', 'F', 'L', 0xE9, 0xCB, 0x0C, 0x1D, 0x01, 0x00, 0x00, 0x00, 'D', 0x4e, 0x62, 0x10, 0x58, 0x39, 0x24, 0x63, 0x40, 'S', 'i', 3, 'h', 'a', 'm'};
|
||
CHECK(json::to_bjdata(j, true, true) == v);
|
||
CHECK(json::from_bjdata(v) == j);
|
||
}
|
||
}
|
||
|
||
SECTION("Object Type")
|
||
{
|
||
SECTION("size=false type=false")
|
||
{
|
||
json j =
|
||
{
|
||
{
|
||
"post", {
|
||
{"id", 1137},
|
||
{"author", "rkalla"},
|
||
{"timestamp", 1364482090592},
|
||
{"body", "I totally agree!"}
|
||
}
|
||
}
|
||
};
|
||
std::vector<uint8_t> v = {'{', 'i', 4, 'p', 'o', 's', 't', '{',
|
||
'i', 6, 'a', 'u', 't', 'h', 'o', 'r', 'S', 'i', 6, 'r', 'k', 'a', 'l', 'l', 'a',
|
||
'i', 4, 'b', 'o', 'd', 'y', 'S', 'i', 16, 'I', ' ', 't', 'o', 't', 'a', 'l', 'l', 'y', ' ', 'a', 'g', 'r', 'e', 'e', '!',
|
||
'i', 2, 'i', 'd', 'I', 0x71, 0x04,
|
||
'i', 9, 't', 'i', 'm', 'e', 's', 't', 'a', 'm', 'p', 'L', 0x60, 0x66, 0x78, 0xB1, 0x3D, 0x01, 0x00, 0x00,
|
||
'}', '}'
|
||
};
|
||
CHECK(json::to_bjdata(j) == v);
|
||
CHECK(json::from_bjdata(v) == j);
|
||
}
|
||
|
||
SECTION("size=true type=false")
|
||
{
|
||
json j =
|
||
{
|
||
{
|
||
"post", {
|
||
{"id", 1137},
|
||
{"author", "rkalla"},
|
||
{"timestamp", 1364482090592},
|
||
{"body", "I totally agree!"}
|
||
}
|
||
}
|
||
};
|
||
std::vector<uint8_t> v = {'{', '#', 'i', 1, 'i', 4, 'p', 'o', 's', 't', '{', '#', 'i', 4,
|
||
'i', 6, 'a', 'u', 't', 'h', 'o', 'r', 'S', 'i', 6, 'r', 'k', 'a', 'l', 'l', 'a',
|
||
'i', 4, 'b', 'o', 'd', 'y', 'S', 'i', 16, 'I', ' ', 't', 'o', 't', 'a', 'l', 'l', 'y', ' ', 'a', 'g', 'r', 'e', 'e', '!',
|
||
'i', 2, 'i', 'd', 'I', 0x71, 0x04,
|
||
'i', 9, 't', 'i', 'm', 'e', 's', 't', 'a', 'm', 'p', 'L', 0x60, 0x66, 0x78, 0xB1, 0x3D, 0x01, 0x00, 0x00,
|
||
};
|
||
CHECK(json::to_bjdata(j, true) == v);
|
||
CHECK(json::from_bjdata(v) == j);
|
||
}
|
||
|
||
SECTION("size=true type=true")
|
||
{
|
||
json j =
|
||
{
|
||
{
|
||
"post", {
|
||
{"id", 1137},
|
||
{"author", "rkalla"},
|
||
{"timestamp", 1364482090592},
|
||
{"body", "I totally agree!"}
|
||
}
|
||
}
|
||
};
|
||
std::vector<uint8_t> v = {'{', '#', 'i', 1, 'i', 4, 'p', 'o', 's', 't', '{', '#', 'i', 4,
|
||
'i', 6, 'a', 'u', 't', 'h', 'o', 'r', 'S', 'i', 6, 'r', 'k', 'a', 'l', 'l', 'a',
|
||
'i', 4, 'b', 'o', 'd', 'y', 'S', 'i', 16, 'I', ' ', 't', 'o', 't', 'a', 'l', 'l', 'y', ' ', 'a', 'g', 'r', 'e', 'e', '!',
|
||
'i', 2, 'i', 'd', 'I', 0x71, 0x04,
|
||
'i', 9, 't', 'i', 'm', 'e', 's', 't', 'a', 'm', 'p', 'L', 0x60, 0x66, 0x78, 0xB1, 0x3D, 0x01, 0x00, 0x00,
|
||
};
|
||
CHECK(json::to_bjdata(j, true, true) == v);
|
||
CHECK(json::from_bjdata(v) == j);
|
||
}
|
||
}
|
||
|
||
SECTION("Optimized Format")
|
||
{
|
||
SECTION("Array Example")
|
||
{
|
||
SECTION("No Optimization")
|
||
{
|
||
// note the floats have been replaced by doubles
|
||
json const j = {29.97, 31.13, 67.0, 2.113, 23.888};
|
||
std::vector<uint8_t> v = {'[',
|
||
'D', 0xb8, 0x1e, 0x85, 0xeb, 0x51, 0xf8, 0x3d, 0x40,
|
||
'D', 0xe1, 0x7a, 0x14, 0xae, 0x47, 0x21, 0x3f, 0x40,
|
||
'D', 0x00, 0x00, 0x00, 0x00, 0x00, 0xc0, 0x50, 0x40,
|
||
'D', 0x81, 0x95, 0x43, 0x8b, 0x6c, 0xe7, 0x00, 0x40,
|
||
'D', 0x17, 0xd9, 0xce, 0xf7, 0x53, 0xe3, 0x37, 0x40,
|
||
']'
|
||
};
|
||
CHECK(json::to_bjdata(j) == v);
|
||
CHECK(json::from_bjdata(v) == j);
|
||
}
|
||
|
||
SECTION("Optimized with count")
|
||
{
|
||
// note the floats have been replaced by doubles
|
||
json const j = {29.97, 31.13, 67.0, 2.113, 23.888};
|
||
std::vector<uint8_t> v = {'[', '#', 'i', 5,
|
||
'D', 0xb8, 0x1e, 0x85, 0xeb, 0x51, 0xf8, 0x3d, 0x40,
|
||
'D', 0xe1, 0x7a, 0x14, 0xae, 0x47, 0x21, 0x3f, 0x40,
|
||
'D', 0x00, 0x00, 0x00, 0x00, 0x00, 0xc0, 0x50, 0x40,
|
||
'D', 0x81, 0x95, 0x43, 0x8b, 0x6c, 0xe7, 0x00, 0x40,
|
||
'D', 0x17, 0xd9, 0xce, 0xf7, 0x53, 0xe3, 0x37, 0x40,
|
||
};
|
||
CHECK(json::to_bjdata(j, true) == v);
|
||
CHECK(json::from_bjdata(v) == j);
|
||
}
|
||
|
||
SECTION("Optimized with type & count")
|
||
{
|
||
// note the floats have been replaced by doubles
|
||
json const j = {29.97, 31.13, 67.0, 2.113, 23.888};
|
||
std::vector<uint8_t> v = {'[', '$', 'D', '#', 'i', 5,
|
||
0xb8, 0x1e, 0x85, 0xeb, 0x51, 0xf8, 0x3d, 0x40,
|
||
0xe1, 0x7a, 0x14, 0xae, 0x47, 0x21, 0x3f, 0x40,
|
||
0x00, 0x00, 0x00, 0x00, 0x00, 0xc0, 0x50, 0x40,
|
||
0x81, 0x95, 0x43, 0x8b, 0x6c, 0xe7, 0x00, 0x40,
|
||
0x17, 0xd9, 0xce, 0xf7, 0x53, 0xe3, 0x37, 0x40,
|
||
};
|
||
CHECK(json::to_bjdata(j, true, true) == v);
|
||
CHECK(json::from_bjdata(v) == j);
|
||
}
|
||
}
|
||
|
||
SECTION("Object Example")
|
||
{
|
||
SECTION("No Optimization")
|
||
{
|
||
// note the floats have been replaced by doubles
|
||
json const j = { {"lat", 29.976}, {"long", 31.131}, {"alt", 67.0} };
|
||
std::vector<uint8_t> v = {'{',
|
||
'i', 3, 'a', 'l', 't', 'D', 0x00, 0x00, 0x00, 0x00, 0x00, 0xc0, 0x50, 0x40,
|
||
'i', 3, 'l', 'a', 't', 'D', 0x60, 0xe5, 0xd0, 0x22, 0xdb, 0xf9, 0x3d, 0x40,
|
||
'i', 4, 'l', 'o', 'n', 'g', 'D', 0xa8, 0xc6, 0x4b, 0x37, 0x89, 0x21, 0x3f, 0x40,
|
||
'}'
|
||
};
|
||
CHECK(json::to_bjdata(j) == v);
|
||
CHECK(json::from_bjdata(v) == j);
|
||
}
|
||
|
||
SECTION("Optimized with count")
|
||
{
|
||
// note the floats have been replaced by doubles
|
||
json const j = { {"lat", 29.976}, {"long", 31.131}, {"alt", 67.0} };
|
||
std::vector<uint8_t> v = {'{', '#', 'i', 3,
|
||
'i', 3, 'a', 'l', 't', 'D', 0x00, 0x00, 0x00, 0x00, 0x00, 0xc0, 0x50, 0x40,
|
||
'i', 3, 'l', 'a', 't', 'D', 0x60, 0xe5, 0xd0, 0x22, 0xdb, 0xf9, 0x3d, 0x40,
|
||
'i', 4, 'l', 'o', 'n', 'g', 'D', 0xa8, 0xc6, 0x4b, 0x37, 0x89, 0x21, 0x3f, 0x40,
|
||
};
|
||
CHECK(json::to_bjdata(j, true) == v);
|
||
CHECK(json::from_bjdata(v) == j);
|
||
}
|
||
|
||
SECTION("Optimized with type & count")
|
||
{
|
||
// note the floats have been replaced by doubles
|
||
json const j = { {"lat", 29.976}, {"long", 31.131}, {"alt", 67.0} };
|
||
std::vector<uint8_t> v = {'{', '$', 'D', '#', 'i', 3,
|
||
'i', 3, 'a', 'l', 't', 0x00, 0x00, 0x00, 0x00, 0x00, 0xc0, 0x50, 0x40,
|
||
'i', 3, 'l', 'a', 't', 0x60, 0xe5, 0xd0, 0x22, 0xdb, 0xf9, 0x3d, 0x40,
|
||
'i', 4, 'l', 'o', 'n', 'g', 0xa8, 0xc6, 0x4b, 0x37, 0x89, 0x21, 0x3f, 0x40,
|
||
};
|
||
CHECK(json::to_bjdata(j, true, true) == v);
|
||
CHECK(json::from_bjdata(v) == j);
|
||
}
|
||
}
|
||
|
||
SECTION("Special Cases (Null, No-Op and Boolean)")
|
||
{
|
||
SECTION("Array")
|
||
{
|
||
json _;
|
||
std::vector<uint8_t> const v = {'[', '$', 'N', '#', 'I', 0x00, 0x02};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v), "[json.exception.parse_error.112] parse error at byte 3: syntax error while parsing BJData type: marker 0x4E is not a permitted optimized array type", json::parse_error&);
|
||
CHECK(json::from_bjdata(v, true, false).is_discarded());
|
||
}
|
||
|
||
SECTION("Object")
|
||
{
|
||
json _;
|
||
std::vector<uint8_t> const v = {'{', '$', 'Z', '#', 'i', 3, 'i', 4, 'n', 'a', 'm', 'e', 'i', 8, 'p', 'a', 's', 's', 'w', 'o', 'r', 'd', 'i', 5, 'e', 'm', 'a', 'i', 'l'};
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v), "[json.exception.parse_error.112] parse error at byte 3: syntax error while parsing BJData type: marker 0x5A is not a permitted optimized array type", json::parse_error&);
|
||
CHECK(json::from_bjdata(v, true, false).is_discarded());
|
||
}
|
||
}
|
||
}
|
||
}
|
||
|
||
TEST_CASE("Parse BJData directly from a file using iterator and sentinel")
|
||
{
|
||
std::string const filename = TEST_DATA_DIRECTORY "/json_testsuite/sample.json.bjdata";
|
||
std::ifstream file(filename, std::ios::binary);
|
||
const std::istreambuf_iterator<char> first(file);
|
||
const json parsed = json::from_bjdata(first, utils::istreambuf_sentinel{});
|
||
CHECK((parsed.is_object() || parsed.is_array()));
|
||
}
|
||
|
||
#if !defined(JSON_NOEXCEPTION)
|
||
TEST_CASE("all BJData first bytes")
|
||
{
|
||
// these bytes will fail immediately with exception parse_error.112
|
||
std::set<uint8_t> supported =
|
||
{
|
||
'T', 'F', 'Z', 'B', 'U', 'i', 'I', 'l', 'L', 'd', 'D', 'C', 'S', '[', '{', 'N', 'H', 'u', 'm', 'M', 'h'
|
||
};
|
||
|
||
for (auto i = 0; i < 256; ++i)
|
||
{
|
||
const auto byte = static_cast<uint8_t>(i);
|
||
CAPTURE(byte)
|
||
|
||
try
|
||
{
|
||
auto res = json::from_bjdata(std::vector<uint8_t>(1, byte));
|
||
}
|
||
catch (const json::parse_error& e)
|
||
{
|
||
// check that parse_error.112 is only thrown if the
|
||
// first byte is not in the supported set
|
||
INFO_WITH_TEMP(e.what());
|
||
if (supported.find(byte) == supported.end())
|
||
{
|
||
CHECK(e.id == 112);
|
||
}
|
||
else
|
||
{
|
||
CHECK(e.id != 112);
|
||
}
|
||
}
|
||
}
|
||
}
|
||
#endif
|
||
|
||
TEST_CASE("BJData and UBJSON can be written to a string")
|
||
{
|
||
const std::vector<json> values =
|
||
{
|
||
{{"a", {1, 2.5, "x", nullptr}}, {"b", json::binary({1, 2})}},
|
||
// an annotated ND-array, and objects that only look like one
|
||
json::parse(R"({"_ArrayType_": "uint8", "_ArraySize_": [2, 2], "_ArrayData_": [1, 2, 3, 4]})"),
|
||
json::parse(R"({"_ArrayType_": 1, "_ArraySize_": [2, 2], "_ArrayData_": [1, 2, 3, 4]})"),
|
||
json::parse(R"({"_ArrayType_": "uint8", "_ArraySize_": 4, "_ArrayData_": [1, 2, 3, 4]})"),
|
||
json::parse(R"({"_ArrayType_": "uint8", "_ArraySize_": [2, -2], "_ArrayData_": [1, 2, 3, 4]})"),
|
||
json::parse(R"({"_ArrayType_": "uint8", "_ArraySize_": [2, 2], "_ArrayData_": [1, 2, 3]})"),
|
||
json::parse(R"({"_ArrayType_": "uint8", "_ArraySize_": [2, 2], "_ArrayData_": 1})"),
|
||
};
|
||
|
||
// compared byte by byte: building a std::string from the bytes would
|
||
// convert them implicitly, which -fsanitize=integer reports for bytes of
|
||
// 0x80 and above
|
||
const auto same_bytes = [](const std::vector<std::uint8_t>& bytes, const std::string & text)
|
||
{
|
||
return bytes.size() == text.size() && std::equal(bytes.begin(), bytes.end(), text.begin(), [](std::uint8_t byte, char c)
|
||
{
|
||
return byte == static_cast<std::uint8_t>(c);
|
||
});
|
||
};
|
||
|
||
for (const auto& j : values)
|
||
{
|
||
CAPTURE(j.dump())
|
||
for (const bool use_size :
|
||
{
|
||
false, true
|
||
})
|
||
{
|
||
for (const bool use_type :
|
||
{
|
||
false, true
|
||
})
|
||
{
|
||
if (use_type && !use_size)
|
||
{
|
||
continue;
|
||
}
|
||
CAPTURE(use_size)
|
||
CAPTURE(use_type)
|
||
|
||
const auto bjdata = json::to_bjdata(j, use_size, use_type);
|
||
std::string bjdata_string;
|
||
json::to_bjdata(j, bjdata_string, use_size, use_type);
|
||
CHECK(same_bytes(bjdata, bjdata_string));
|
||
|
||
const auto ubjson = json::to_ubjson(j, use_size, use_type);
|
||
std::string ubjson_string;
|
||
json::to_ubjson(j, ubjson_string, use_size, use_type);
|
||
CHECK(same_bytes(ubjson, ubjson_string));
|
||
}
|
||
}
|
||
}
|
||
}
|
||
|
||
TEST_CASE("BJData use_type requires use_size")
|
||
{
|
||
SECTION("non-empty object throws other_error.502")
|
||
{
|
||
const json j = {{"a", 1}, {"b", 2}};
|
||
CHECK_THROWS_WITH_AS(json::to_bjdata(j, false, true),
|
||
"[json.exception.other_error.502] use_type requires use_size = true",
|
||
json::other_error&);
|
||
}
|
||
|
||
SECTION("non-empty array throws other_error.502")
|
||
{
|
||
const json j = {1, 2, 3};
|
||
CHECK_THROWS_WITH_AS(json::to_bjdata(j, false, true),
|
||
"[json.exception.other_error.502] use_type requires use_size = true",
|
||
json::other_error&);
|
||
}
|
||
|
||
SECTION("non-empty binary value throws other_error.502")
|
||
{
|
||
const json j = json::binary({1, 2, 3});
|
||
CHECK_THROWS_WITH_AS(json::to_bjdata(j, false, true),
|
||
"[json.exception.other_error.502] use_type requires use_size = true",
|
||
json::other_error&);
|
||
CHECK_THROWS_WITH_AS(json::to_ubjson(j, false, true),
|
||
"[json.exception.other_error.502] use_type requires use_size = true",
|
||
json::other_error&);
|
||
}
|
||
|
||
SECTION("scalars do not throw with use_type=true, use_count=false")
|
||
{
|
||
CHECK_NOTHROW(json::to_bjdata(42, false, true));
|
||
CHECK_NOTHROW(json::to_bjdata(3.14, false, true));
|
||
CHECK_NOTHROW(json::to_bjdata("hello", false, true));
|
||
CHECK_NOTHROW(json::to_bjdata(true, false, true));
|
||
CHECK_NOTHROW(json::to_bjdata(nullptr, false, true));
|
||
}
|
||
|
||
SECTION("empty containers do not throw with use_type=true, use_count=false")
|
||
{
|
||
CHECK_NOTHROW(json::to_bjdata(json::array(), false, true));
|
||
CHECK_NOTHROW(json::to_bjdata(json::object(), false, true));
|
||
}
|
||
|
||
SECTION("valid combinations on non-empty containers")
|
||
{
|
||
const json j = {{"a", 1}, {"b", 2}};
|
||
CHECK_NOTHROW(json::to_bjdata(j, false, false));
|
||
CHECK_NOTHROW(json::to_bjdata(j, true, false));
|
||
CHECK_NOTHROW(json::to_bjdata(j, true, true));
|
||
}
|
||
}
|
||
|
||
TEST_CASE("BJData round trip of a binary value is value-stable, not byte-stable")
|
||
{
|
||
// OSS-Fuzz issue 474480402: a Draft 3 optimized binary array is read as a
|
||
// binary value, which to_bjdata() writes in the default Draft 2 mode as a
|
||
// plain array of uint8 numbers. That is read back as an array of numbers,
|
||
// for which the writer then picks the smallest type marker, int8 ('i'),
|
||
// so re-serializing changes the bytes, but not the value. This is the
|
||
// exception described in the "Round trips" note of the BJData
|
||
// documentation, and why the fuzzer checks value stability (see #5494).
|
||
const std::vector<uint8_t> input = {'[', '$', 'B', '#', 'U', 1, 0x20};
|
||
const json j1 = json::from_bjdata(input);
|
||
CHECK(j1 == json::binary({0x20}));
|
||
|
||
const std::vector<uint8_t> vec = json::to_bjdata(j1, false, false);
|
||
CHECK(vec == std::vector<uint8_t>({'[', 'U', 0x20, ']'}));
|
||
const json j2 = json::from_bjdata(vec);
|
||
CHECK(j2 == json::array({0x20}));
|
||
|
||
const std::vector<uint8_t> vec2 = json::to_bjdata(j2, false, false);
|
||
CHECK(vec2 == std::vector<uint8_t>({'[', 'i', 0x20, ']'}));
|
||
CHECK(json::from_bjdata(vec2) == j2);
|
||
}
|
||
|
||
TEST_CASE("BJData roundtrips" * doctest::skip())
|
||
{
|
||
SECTION("input from self-generated BJData files")
|
||
{
|
||
for (const std::string filename :
|
||
{
|
||
TEST_DATA_DIRECTORY "/json_nlohmann_tests/all_unicode.json",
|
||
TEST_DATA_DIRECTORY "/json.org/1.json",
|
||
TEST_DATA_DIRECTORY "/json.org/2.json",
|
||
TEST_DATA_DIRECTORY "/json.org/3.json",
|
||
TEST_DATA_DIRECTORY "/json.org/4.json",
|
||
TEST_DATA_DIRECTORY "/json.org/5.json",
|
||
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip01.json",
|
||
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip02.json",
|
||
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip03.json",
|
||
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip04.json",
|
||
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip05.json",
|
||
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip06.json",
|
||
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip07.json",
|
||
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip08.json",
|
||
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip09.json",
|
||
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip10.json",
|
||
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip11.json",
|
||
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip12.json",
|
||
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip13.json",
|
||
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip14.json",
|
||
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip15.json",
|
||
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip16.json",
|
||
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip17.json",
|
||
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip18.json",
|
||
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip19.json",
|
||
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip20.json",
|
||
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip21.json",
|
||
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip22.json",
|
||
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip23.json",
|
||
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip24.json",
|
||
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip25.json",
|
||
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip26.json",
|
||
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip27.json",
|
||
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip28.json",
|
||
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip29.json",
|
||
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip30.json",
|
||
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip31.json",
|
||
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip32.json",
|
||
TEST_DATA_DIRECTORY "/json_testsuite/sample.json",
|
||
TEST_DATA_DIRECTORY "/json_tests/pass1.json",
|
||
TEST_DATA_DIRECTORY "/json_tests/pass2.json",
|
||
TEST_DATA_DIRECTORY "/json_tests/pass3.json"
|
||
})
|
||
{
|
||
CAPTURE(filename)
|
||
|
||
std::ifstream f_json(filename);
|
||
const json j1 = json::parse(f_json);
|
||
auto packed = utils::read_binary_file(filename + ".bjdata");
|
||
|
||
{
|
||
INFO_WITH_TEMP(filename + ": std::vector<uint8_t>");
|
||
json j2;
|
||
CHECK_NOTHROW(j2 = json::from_bjdata(packed));
|
||
CHECK(j1 == j2);
|
||
}
|
||
|
||
{
|
||
INFO_WITH_TEMP(filename + ": std::ifstream");
|
||
std::ifstream f_bjdata(filename + ".bjdata", std::ios::binary);
|
||
json j2;
|
||
CHECK_NOTHROW(j2 = json::from_bjdata(f_bjdata));
|
||
CHECK(j1 == j2);
|
||
}
|
||
|
||
{
|
||
INFO_WITH_TEMP(filename + ": output to output adapters");
|
||
{
|
||
INFO_WITH_TEMP(filename + ": output adapters: std::vector<uint8_t>");
|
||
std::vector<uint8_t> vec;
|
||
json::to_bjdata(j1, vec);
|
||
CHECK(vec == packed);
|
||
}
|
||
}
|
||
}
|
||
}
|
||
}
|
||
|
||
TEST_CASE("issue #5648 - from_bjdata(ptr, len) must read len bytes, not treat ptr as a C string")
|
||
{
|
||
// to_bjdata() encodes the integer 0 as the two bytes 'i' 0x00 (a BJData
|
||
// type marker followed by the value byte 0x00), so the packed data
|
||
// below contains a 0x00 byte before its end.
|
||
const json j = {{"a", 0}};
|
||
const std::vector<std::uint8_t> packed = json::to_bjdata(j);
|
||
bool contains_nul = false;
|
||
for (const auto byte : packed)
|
||
{
|
||
contains_nul |= (byte == 0x00);
|
||
}
|
||
REQUIRE(contains_nul);
|
||
|
||
#ifndef JSON_TEST_DEPRECATED_FUNCTIONS_DELETED
|
||
// before the fix, from_bjdata had no (ptr, len) overload, so this call
|
||
// bound to from_bjdata(InputType&&, bool strict) instead: ptr was read
|
||
// as a NUL-terminated C string (stopping at the embedded 0x00 byte), and
|
||
// len was silently converted to the strict flag. The deprecated
|
||
// overload added for this issue forwards to from_bjdata(ptr, ptr + len,
|
||
// ...) instead, like from_ubjson's deprecated (ptr, len) overload does.
|
||
json result;
|
||
CHECK_NOTHROW(result = json::from_bjdata(packed.data(), packed.size()));
|
||
CHECK(result == j);
|
||
|
||
// len must not collapse into the strict flag either
|
||
CHECK(json::from_bjdata(packed.data(), packed.size(), false) == j);
|
||
#endif
|
||
}
|
||
|
||
TEST_CASE("BJData large strings and binaries (chunked reader)")
|
||
{
|
||
// Strings share get_ubjson_string() -> get_string() -> get_bytes() with
|
||
// plain UBJSON. Binary values are different: only a Draft 3 optimized
|
||
// array (type marker 'B') is read back as a binary value, through
|
||
// get_binary() -> get_bytes() (see parse_ubjson_internal()'s "If BJData
|
||
// type marker is 'B'" branch); Draft 2 (the default) writes a binary
|
||
// value as a plain array of uint8_t numbers instead (see the "round trip
|
||
// of a binary value is value-stable, not byte-stable" test above), which
|
||
// never reaches get_bytes(). Both reads happen in bounded chunks
|
||
// (binary_reader.hpp, chunk_size == 4096); check lengths around and
|
||
// beyond that size, for both vector (iterator) and pointer inputs.
|
||
for (const std::size_t len :
|
||
{
|
||
std::size_t{0}, std::size_t{1}, std::size_t{4095}, std::size_t{4096},
|
||
std::size_t{4097}, std::size_t{8192}, std::size_t{100000}
|
||
})
|
||
{
|
||
CAPTURE(len)
|
||
|
||
// string
|
||
const json j_string = std::string(len, 'x');
|
||
const std::vector<std::uint8_t> v_string = json::to_bjdata(j_string);
|
||
CHECK(json::from_bjdata(v_string) == j_string);
|
||
// pointer input exercises the std::memcpy fast path
|
||
CHECK(json::from_bjdata(reinterpret_cast<const char*>(v_string.data()),
|
||
reinterpret_cast<const char*>(v_string.data()) + v_string.size()) == j_string);
|
||
|
||
// binary, forced into the Draft 3 optimized ('B' marker) encoding
|
||
const json j_binary = json::binary(std::vector<std::uint8_t>(len, 0xCD));
|
||
const std::vector<std::uint8_t> v_binary = json::to_bjdata(j_binary, true, true, json::bjdata_version_t::draft3);
|
||
CHECK(json::from_bjdata(v_binary) == j_binary);
|
||
CHECK(json::from_bjdata(reinterpret_cast<const char*>(v_binary.data()),
|
||
reinterpret_cast<const char*>(v_binary.data()) + v_binary.size()) == j_binary);
|
||
|
||
// a truncated payload must still be reported as an error
|
||
if (len > 16)
|
||
{
|
||
std::vector<std::uint8_t> truncated = v_string;
|
||
truncated.resize(truncated.size() - 8);
|
||
json _;
|
||
CHECK_THROWS_AS(_ = json::from_bjdata(truncated), json::parse_error);
|
||
}
|
||
}
|
||
}
|
||
|
||
TEST_CASE("BJData optimized container type marker is validated for count 0")
|
||
{
|
||
// the '$' type marker must be a legal marker even if no element follows
|
||
SECTION("invalid marker")
|
||
{
|
||
// 'Z', 'T', 'F', 'N', 'S', 'H', '[' and '{' are excluded in BJData
|
||
for (const char marker :
|
||
{'x', 'E', 'Z', 'T', 'F', 'N', 'S', 'H', '[', '{'
|
||
})
|
||
{
|
||
for (const char open :
|
||
{'[', '{'
|
||
})
|
||
{
|
||
for (const int count :
|
||
{
|
||
0, 1
|
||
})
|
||
{
|
||
CAPTURE(marker)
|
||
CAPTURE(open)
|
||
CAPTURE(count)
|
||
const std::vector<std::uint8_t> v = {static_cast<std::uint8_t>(open), '$', static_cast<std::uint8_t>(marker), '#', 'i', static_cast<std::uint8_t>(count)};
|
||
json _;
|
||
CHECK_THROWS_AS(_ = json::from_bjdata(v), json::parse_error&);
|
||
CHECK(json::from_bjdata(v, true, false).is_discarded());
|
||
}
|
||
}
|
||
}
|
||
}
|
||
|
||
SECTION("error message")
|
||
{
|
||
const std::vector<std::uint8_t> v = {'[', '$', 'x', '#', 'i', 0};
|
||
json _;
|
||
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v), "[json.exception.parse_error.112] parse error at byte 3: syntax error while parsing BJData type: marker 0x78 is not a permitted optimized array type", json::parse_error&);
|
||
}
|
||
|
||
SECTION("valid marker")
|
||
{
|
||
for (const char marker :
|
||
{'i', 'U', 'I', 'u', 'l', 'm', 'L', 'M', 'h', 'd', 'D', 'C'
|
||
})
|
||
{
|
||
CAPTURE(marker)
|
||
const std::vector<std::uint8_t> va = {'[', '$', static_cast<std::uint8_t>(marker), '#', 'i', 0};
|
||
const std::vector<std::uint8_t> vo = {'{', '$', static_cast<std::uint8_t>(marker), '#', 'i', 0};
|
||
CHECK(json::from_bjdata(va) == json::array());
|
||
CHECK(json::from_bjdata(vo) == json::object());
|
||
}
|
||
|
||
// an optimized array of bytes is read as a binary value
|
||
const std::vector<std::uint8_t> vb = {'[', '$', 'B', '#', 'i', 0};
|
||
CHECK(json::from_bjdata(vb) == json::binary({}));
|
||
}
|
||
}
|
||
|
||
TEST_CASE("BJData optimized arrays from contiguous and non-contiguous input")
|
||
{
|
||
// contiguous input reads the numbers of an array with a type marker in
|
||
// one loop, other input number by number; both must give the same events,
|
||
// value, or error for the complete document and for every truncation of
|
||
// it, and stop at every event a SAX handler rejects
|
||
json document =
|
||
{
|
||
{"u8", {0, 200, 255}},
|
||
{"i8", {-5, 100, -128}},
|
||
{"i16", {-30000, 1000, 30000}},
|
||
{"i32", {-2000000000, 5, 70000}},
|
||
{"i64", {-9000000000000000000LL, 1, 5000000000LL}},
|
||
{"f64", {1.5, -0.1, 2.5}},
|
||
{"pairs", {{1.25, -3.75}, {1.25, -3.75}, json::array()}},
|
||
{"mixed", {1, "a", 2.5, nullptr}},
|
||
{"u16", {40000, 1, 65535}},
|
||
{"u32", {4000000000U, 1, 70000}},
|
||
{"u64", {18446744073709551615ULL, 1, 5000000000ULL}},
|
||
{"nd", {{"_ArrayType_", "double"}, {"_ArraySize_", {2, 2}}, {"_ArrayData_", {1.5, -2.5, 3.5, -4.5}}}},
|
||
{"nd8", {{"_ArrayType_", "uint8"}, {"_ArraySize_", {3}}, {"_ArrayData_", {1, 2, 3}}}},
|
||
};
|
||
const std::vector<std::uint8_t> encoded = json::to_bjdata(document, true, true);
|
||
CHECK(json::from_bjdata(encoded) == document);
|
||
|
||
for (std::size_t length = 0; length <= encoded.size(); ++length)
|
||
{
|
||
CAPTURE(length)
|
||
const std::vector<std::uint8_t> contiguous(encoded.begin(), encoded.begin() + static_cast<std::ptrdiff_t>(length));
|
||
const std::list<std::uint8_t> bytewise(contiguous.begin(), contiguous.end());
|
||
|
||
CHECK(json::from_bjdata(contiguous, true, false).dump() == json::from_bjdata(bytewise.begin(), bytewise.end(), true, false).dump());
|
||
#if !defined(JSON_NOEXCEPTION)
|
||
std::string contiguous_error;
|
||
std::string bytewise_error;
|
||
try
|
||
{
|
||
json _;
|
||
_ = json::from_bjdata(contiguous);
|
||
}
|
||
catch (const json::exception& e)
|
||
{
|
||
contiguous_error = e.what();
|
||
}
|
||
try
|
||
{
|
||
json _;
|
||
_ = json::from_bjdata(bytewise.begin(), bytewise.end());
|
||
}
|
||
catch (const json::exception& e)
|
||
{
|
||
bytewise_error = e.what();
|
||
}
|
||
CHECK(contiguous_error == bytewise_error);
|
||
#endif
|
||
|
||
SaxEventLogger from_contiguous;
|
||
SaxEventLogger from_bytewise;
|
||
CHECK(json::sax_parse(contiguous, &from_contiguous, json::input_format_t::bjdata)
|
||
== json::sax_parse(bytewise.begin(), bytewise.end(), &from_bytewise, json::input_format_t::bjdata));
|
||
CHECK(from_contiguous.events == from_bytewise.events);
|
||
}
|
||
|
||
for (int events = 0; events <= 80; ++events)
|
||
{
|
||
CAPTURE(events)
|
||
const std::list<std::uint8_t> bytewise(encoded.begin(), encoded.end());
|
||
SaxCountdown from_contiguous(events);
|
||
SaxCountdown from_bytewise(events);
|
||
CHECK(json::sax_parse(encoded, &from_contiguous, json::input_format_t::bjdata)
|
||
== json::sax_parse(bytewise.begin(), bytewise.end(), &from_bytewise, json::input_format_t::bjdata));
|
||
}
|
||
}
|