* Run the README test case in JSON_FastTests jobs
The "README" test case was marked doctest::skip() when the tests
moved from Catch to doctest in 2019, where it replaced Catch's hidden
tag. It is not slow (17 assertions, about 0.00 s), but cmake/test.cmake
only passes --no-skip when JSON_FastTests is off, so the per-compiler
ci_test_*_cxxNN matrix, macOS, Windows Release/ARM, icpc, icpx and
nvhpc compiled the README examples without running them.
Drop the skip decorator so every job runs the case. Test-only change.
Part of #5713
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Remove stale clang ranges guards in unit-iterators2.cpp
The "algorithms" and "views" sections were guarded by clang/libstdc++
checks written for a clang 15 (04/2022) bug. The first guard's
condition contradicts its own comment: it skips clang+libc++ and
keeps clang+libstdc++. Both sections already sit inside
`#if JSON_HAS_RANGES`, which macro_scope.hpp excludes for the
toolchains these guards targeted, so the inner guards never let the
sections run on the platforms they meant to protect and are
redundant on the rest. Verified locally with Apple clang 21/libc++
and clang 16.0.6/libstdc++ 12 (Docker): both pass all 1355
assertions with the guards removed.
Part of #5713
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix copy-pasted CBOR half-float checks; enable stale encode checks
In the RFC 8949 Appendix A test case, the decode checks for
5.960464477539063e-8 (0xf9 0x00 0x01) and 0.00006103515625
(0xf9 0x04 0x00) were copy-pasted from the neighboring -4.0 example,
so those two half-float byte sequences were never actually decoded
and checked, and -4.0 was checked three times instead. The two
float32 encode checks for 100000.0 and 3.4028234663852886e+38 were
commented out before the writer supported emitting float32 and are
now verified to match byte for byte, so they are enabled. The
remaining commented-out half-precision to_cbor checks are collapsed
into a single explanatory comment, since the writer never emits
half-precision floats.
Part of #5713
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Assert on the result of STL container conversions in tests
The "object-like STL containers" and "array-like STL containers"
sections converted json values into std::map, unordered_map,
multimap, unordered_multimap, list, forward_list, array, valarray,
vector, deque, set and unordered_set and discarded the result, so
these ~60 conversions only proved that the code compiles and does
not throw; a conversion that dropped or reordered elements would
still pass. Bind each result and compare it against the expected
container. Also fix a copy-paste slip in the deque section
(`j2.get<std::deque<double>>()` instead of j3, so j3's doubles were
never converted to a deque), and remove the dead
`// CHECK(m5["one"] == "eins")` comments that referred to a variable
that did not exist by asserting the equivalent through the bound
result.
Part of #5713
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Deduplicate SaxCountdown and other test helpers across formats
SaxCountdown was copied byte-for-byte into six binary-format test
files (unit-cbor.cpp, unit-msgpack.cpp, unit-ubjson.cpp,
unit-bjdata.cpp, unit-bon8.cpp, unit-bson.cpp), about 370 redundant
lines. Move it into tests/src/sax_countdown.hpp (namespace utils,
alongside test_utils.hpp and round_trip_corpus.hpp) and include it
from all six.
trait_test_arg and the "value_in_range_of trait"
TEST_CASE_TEMPLATE_DEFINE were duplicated between unit-32bit.cpp and
unit-bjdata.cpp; the trait is a detail/meta trait, not specific to
either file. Move it into tests/src/value_in_range_of_test.hpp;
unit-32bit.cpp keeps its own include, since JSON_32bitTest=ONLY
builds only that file. Each file keeps its own
TEST_CASE_TEMPLATE_INVOKE list.
sax_no_exception and the "issue #2824" section were duplicated in
unit-regression2.cpp and unit-disabled_exceptions.cpp. Drop the copy
from unit-regression2.cpp; unit-disabled_exceptions.cpp already
covers the no-exceptions case that #2824 was about, and
ci_test_noexceptions reruns it.
No behavior change. Verified by building and running unit-cbor,
unit-msgpack, unit-ubjson, unit-bjdata, unit-bon8, unit-bson,
unit-32bit, unit-regression2 and unit-disabled_exceptions against
include/ (clang++ -std=c++11, ASan/UBSan where applicable); assertion
counts are unchanged from before the refactor.
Part of #5714
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Remove the unreferenced vendored libFuzzer
tests/thirdparty/Fuzzer (155 files, ~776 KB of vendored Apache-2.0
LLVM code from the 2016 OSS-Fuzz import) is not referenced by any
CMakeLists, Makefile or workflow: the fuzz drivers link against
-fsanitize=fuzzer or the repo's own
tests/src/fuzzer-driver_afl.cpp. Its vendored README only points at
llvm.org's own libFuzzer docs. Being dead code, it also adds noise
to the flawfinder code-scanning workflow, which scans the whole
tree. Remove the directory and its .reuse/dep5 entry.
Part of #5714
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Remove unreferenced 2016 benchmark and fuzz reports
tests/reports (1.6 MB) holds AFL status pages and plots from
2016-08-29 and 2016-10-02, and a nativejson-benchmark snapshot from
2016 with links to rawgit.com, which shut down in 2019. Nothing
references this directory: no doc, README section, script or
workflow points at it, and it describes a ten-years-old, pre-2.0
snapshot of the library.
Part of #5714
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Run the CBOR, MessagePack, BSON and BON8 round-trip invariants in CI
tests/src/round_trip_corpus.hpp exists so that the byte-stability
invariant the fuzzer drivers check also runs on a fixed corpus in CI,
instead of only at OSS-Fuzz. So far only the UBJSON and BJData drivers
had a matching unit test; the CBOR, MessagePack, BSON and BON8 drivers
assert the same invariant (assert(to_X(j2) == vec)) but nothing ran it
outside OSS-Fuzz.
Add "<FORMAT> round-trip invariants" test cases to unit-cbor.cpp,
unit-msgpack.cpp, unit-bson.cpp and unit-bon8.cpp, modeled on the
UBJSON case: seed j1 from the corpus (skipping values that do not
survive the format's own round trip, as the fuzzer drivers only ever
see values from_X() actually produced), then require from_X(to_X(j1))
not to throw and check to_X(j2) == to_X(j1). BSON only serializes
objects, so non-object corpus values are skipped. Update the comments
in round_trip_corpus.hpp and tests/fuzzing.md to name all six formats.
The stream-versus-contiguous check in the BON8 driver is left out, as
#5601 reworks it.
A local probe confirms no violations on the current corpus (CBOR 3849
checked, MessagePack 3909, BSON 2958, BON8 3841 - matching the counts
already recorded for this probe in the issue).
Closes#5714 item 1.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix fuzzer driver step lists to match the checks the code performs
The header comment of six of the seven binary-format fuzzer drivers
listed an invariant the code does not check: CBOR, MessagePack, BSON
and BON8 said "assert(j1 == j2)", but the code checks byte stability,
assert(to_X(j2) == vec). UBJSON and BJData still described the old
"assert(j1 == j2/j3/j4)" byte-exact check from before PR #5494 replaced
it with a use_size/use_type-aware round trip (UBJSON) and a
value-stability check (BJData); BJData's added paragraph already
explained the new check, but the step list above it did not.
Also remove a dead branch in fuzzer-parse_bson.cpp: from_bson() is
called with allow_exceptions = true, so it throws instead of returning
a discarded value, and the "if (j1.is_discarded()) return 0;" guard
could never trigger. Drop the unused <iostream> include from all seven
drivers and <sstream> from all but fuzzer-parse_bon8.cpp, which is the
only one that uses std::istringstream.
Overlaps #5601, which edits all seven drivers in the same hunks.
Closes#5714 item 4.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Silence the CMP0169 deprecation in cmake_fetch_content, fix stale guards
tests/cmake_fetch_content/project calls the single-argument
FetchContent_Populate(json) after FetchContent_Declare(), which CMake
3.30 deprecated as CMP0169. Since the project declares
cmake_minimum_required(VERSION 3.11...3.14), the policy stays unset,
so every configure with a current CMake prints the deprecation
warning. The test is kept on purpose: it is the only coverage of the
FetchContent_Populate + add_subdirectory pattern for CMake 3.11-3.13
users, which the docs still describe as supported. Explicitly set
CMP0169 to OLD, with a comment explaining why.
Also fix two stale version guards:
- tests/cmake_fetch_content/CMakeLists.txt guarded the test with
VERSION_GREATER "3.11.0", which is dead now that tests/CMakeLists.txt
requires CMake 3.13.
- tests/cmake_fetch_content2/CMakeLists.txt guarded with
VERSION_GREATER "3.14.0", which skips exactly 3.14.0, the first
version with FetchContent_MakeAvailable. Change it to
VERSION_GREATER_EQUAL "3.14".
Verified locally: `ctest -R cmake_fetch_content` passes with CMake
4.1, and the CMP0169 deprecation warning that appeared before this
change is gone.
Closes#5714 item 5.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Make the CMake integration-test wrappers consistent
The six tests/cmake_* integration-test wrappers had drifted:
- Only cmake_import and cmake_import_minver forwarded
-A "${CMAKE_GENERATOR_PLATFORM}" to the inner configure, and none
forwarded -T "${CMAKE_GENERATOR_TOOLSET}". The Windows workflow
configures the outer build with -A Win32 -T ClangCL, so without
forwarding, the inner projects of cmake_add_subdirectory,
cmake_fetch_content, cmake_fetch_content2 and
cmake_target_include_directories built with the generator defaults
instead of matching the outer build's platform and toolset. Forward
both consistently from all six wrappers.
- cmake_fetch_content and cmake_fetch_content2 passed
-Dnlohmann_json_source to their inner projects, which never read it
(CMake warns "manually-specified variables were not used"); the
inner projects fetch their own copy of the library instead. Drop it.
- tests/CMakeLists.txt set JSON_FORCED_GLOBAL_COMPILE_OPTIONS from the
matching environment variable but never read the cache variable
again; the lines right below it read $ENV{JSON_FORCED_GLOBAL_COMPILE_OPTIONS}
directly, like the LINK_OPTIONS counterpart already does. Remove the
dead set().
This changes which platform and toolset the Win32 and ClangCL CI jobs
build the four newly-forwarding wrappers' inner projects with, which
may surface new failures there; CI has to confirm those jobs.
Verified locally with Ninja (empty -A ""/-T "" is accepted): all 12
cmake_* tests still pass, and the inner fetch_content configures no
longer warn about the unused variable.
Closes#5714 item 6.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Turn the #972 fifo_map regression test into a real test
The #972 regression test in unit-regression1.cpp only built a
my_json array from a string literal (the original crash) and had no
CHECK, so the fifo_map object type it exists to demonstrate was never
exercised. Meanwhile the docs recommend fifo_map for keeping object
keys in insertion order (object_order.md, template_parameters.md),
and nothing tested that recommendation.
Extend the section: after the original array assignment, parse an
object with my_json::parse() (not via the "..."_json UDL, which
returns a plain nlohmann::json and would exercise the cross-basic_json
conversion constructor instead of the parser's own key insertion -
and, as tried locally, does not keep fifo order for this stateful
comparator) and check that dump() keeps insertion order, and that it
survives erase() and inserting a new key.
Also narrow thirdparty/fifo_map off the include path of every other
test-* target: it was a PUBLIC include directory of test_main, even
though unit-regression1.cpp is its only user. Add a small
fifo_map_include INTERFACE library with that include directory and
attach it to test-regression1 only via json_test_set_test_options().
Verified locally (test-regression1_cpp11, default build and
-fsanitize=address,undefined): the new checks pass; `git grep fifo_map
tests` still only finds unit-regression1.cpp and the vendored header.
Closes#5714 item 7.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Document the vendored doctest.h patch; fix stale doctest_compatibility.h comments
tests/thirdparty/doctest/doctest.h is doctest 2.4.12, imported in
#4771. Two weeks later, #4801 hand-edited translateActiveException()
to declare "String res;" inside the translator loop instead of before
it, so a translator that does not match does not leave a previous
translator's result in "res" for the next iteration to see. Nothing
recorded this, so re-vendoring doctest.h from upstream would silently
drop the fix. Add a comment at the patched site naming the version,
the PR and the reason, so a future re-vendor knows to re-apply it.
Also fix two stale comments in doctest_compatibility.h:
- The DOCTEST_THREAD_LOCAL comment referenced Xcode 6/7, which is no
longer supported; reword it to explain why the define must stay
regardless (it keeps doctest's own thread_local usage out of the way
of the same Clang/MinGW crash that JSON_NO_THREAD_LOCAL works around
in the library, see ci_test_no_thread_local).
- The <iosfwd> include's comment justified it with tests that define
"private" as "public"; no test under tests/src does that any more
(removed by #2352). Reword the comment instead of dropping the
include, since confirming it is safe to drop needs the full CI
matrix including MSVC 2015+.
Verified locally that tests/src/unit-readme.cpp still builds and
passes 17/17 with these headers.
Closes#5714 item 9.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Stop compiling unit-wstring.cpp out entirely on classic ICC
tests/src/unit-wstring.cpp wrapped the whole file in
#ifndef __INTEL_COMPILER, with the comment "ICPC errors out on
multibyte character sequences in source files". The ci_icpc job
(intel/oneapi-hpckit:2023.2.1) still exists, so that job ran none of
the wstring/u16string/u32string input adapter tests, including the
malformed-input checks #5704 (open) extends.
Only 9 lines contained non-ASCII bytes: the three *_is_utf16()/
*_is_utf32() probe functions, and three std::wstring/u16string/
u32string literals plus their narrow-string dump() expectations.
Rewrite all of them with \u/\U escapes in the wide/u16/u32 literals
and \x escapes (split into separate string-literal tokens so a
following byte is never read as part of the same hex escape, e.g.
"\xE1\x83\x85" "a") in the narrow ones. Remove the
#ifndef __INTEL_COMPILER/#endif guard along with it.
The *_is_utf16()/*_is_utf32() probes compared a raw multibyte literal
against an escape-based one to detect a compiler that misreads the
source file's encoding; with no raw literals left to misread, the
comparison is now tautological, so drop the probes and the "if"
guards around each SECTION's body instead of leaving them in as dead
checks.
The same non-ASCII-in-source-and-in-a-narrow-comparison pattern
existed once more in unit-deserialization.cpp's "Using _json with
char8_t literals #4945" test: a raw emoji character in a u8R"(...)"
literal, guarded by a check_utf8() that returned false for ICC (same
reason) and for Windows without the active UTF-8 code page. Rewrite
the literal with a \U escape and compare it against a \x-escaped
expectation instead of a second raw literal, and drop check_utf8()
and the now-unused <windows.h> include along with the guard.
Verified locally (clang, -std=c++11 and -std=c++20,
-fsanitize=address,undefined, and a plain build): test-wstring keeps
18/18 assertions and unit-deserialization keeps 466/466 (c++11) and
477/477 (c++20) assertions, matching this branch before the change
exactly - no coverage was gained or lost, only the source-encoding
dependency was removed. ci_icpc has to confirm classic ICC actually
builds and passes test-wstring now; if it does not, that is a real
finding, not a reason to restore the guard.
Overlaps #5704 (open), which edits unit-wstring.cpp inside the
previously-guarded region (an include near the top, checks in the
invalid-string sections, and a new section at the end).
Closes#5713 item 5.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Parsing from a std::istream crashed in two unusual but valid stream
states, both in input_stream_adapter:
- With eofbit in the stream's exceptions() mask, get_character() sets
eofbit via is->clear(), which throws std::ios_base::failure. While
that exception unwinds, ~input_stream_adapter() called clear() again
to reset eofbit, which is still set and still in the exception mask,
so it throws a second time out of the (implicitly noexcept)
destructor and std::terminate() is called. The destructor now only
calls clear() if a bit other than eofbit remains set, so the first
exception can propagate normally.
- For an std::istream without a stream buffer (rdbuf() == nullptr,
e.g. std::istream(nullptr)), the constructor stored the null
pointer without checking it, and get_character() dereferenced it.
input_adapter(std::istream&) now throws parse_error.101 for such a
stream, the same as it already does for a null FILE* or char*.
Added regression tests to unit-deserialization.cpp and, for the
JSON_PRECISE_STREAM_POSITION variant of get_character(), to
unit-precise-stream-position.cpp; both crashed before this fix.
Documented the two exceptions in parse.md and operator_gtgt.md.
Fixes#5646.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
#5344 added two lines to unit-deserialization.cpp that clang-tidy
reports: modernize-return-braced-init-list for the remaining() helper
and readability-isolate-declaration for "json j1, j2, j3;".
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* docs: qualify the operator>> stream positioning guarantee
operator>>'s notes state that it leaves the stream positioned right
after the parsed value, so that concatenated JSON values can be read
back to back. That does not hold when the value is a number: a number
is only terminated by the character that follows it, and the lexer's
unget() is simulated (it rewinds only the lexer's own bookkeeping),
so that character stays consumed from the stream.
Document the actual behaviour: the guarantee holds for all value types
except numbers, which must be followed by whitespace. Also qualify the
cross-reference on the JSON Lines page, which repeated the unqualified
claim.
Documentation only; the behaviour itself is tracked in #5340.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* fix: restore the character that terminates a number (#5340)
operator>> is documented to leave the stream positioned right after the
parsed value, so that concatenated JSON values can be read back to back.
That did not hold for numbers: a number is only terminated by the
character following it, and lexer::scan_number() reads that character
and calls unget() -- which is simulated and rewinds only the lexer's own
bookkeeping. input_stream_adapter consumes via sbumpc() with no matching
sungetc(), so the terminating character stayed consumed and the next
extraction started one byte too late ('1true' left the stream at 'rue').
Propagating unget() to the adapter directly does not work: next_unget
makes the following get() replay the cached character, so the terminator
would be delivered twice. Instead, restore the still-pending character
once at the end of a non-strict parse, where the input is handed back to
the caller:
- input_stream_adapter gains unget_character() (sungetc()) and advertises
it via supports_unget, detected the same way as supports_seek.
- lexer::restore_pending_unget() turns a pending simulated unget of a
real (non-EOF) character into a real one and clears next_unget so the
character is not also replayed. It is a no-op for adapters that cannot
unget, and reports failure when sungetc() fails, in which case the
input is left as it was before.
- parser calls it on the three non-strict paths, i.e. for operator>> and
sax_parse(strict = false).
Strict parse()/accept() are unaffected: they require the input to end
after the value, so the character is consumed by the end-of-input check
anyway. Parse error messages and reported positions are unchanged.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* tests: fix CI failures in the #5340 test helpers
Four CI failures, all in the new test code:
- GCC (-Werror=useless-cast): drop the `json(...)` wrapper around
`json::parse(...)`, which already returns a `json`.
- GCC (-Werror=unused-result): assign the discarded `json::parse()`
result to a dummy, the idiom used elsewhere in the test suite, and
catch `json::parse_error&` for consistency.
- clang-tidy (google-default-arguments): remove the default argument
from the `pbackfail()` override; `sungetc()` supplies the base
declaration's default.
- MSVC (bad allocation): `no_putback_streambuf::underflow()` set a
one-character get area without advancing `m_pos`, so an implementation
whose `istream::get` peeks before it bumps re-read the same character
forever. Keep no get area at all: `underflow()` peeks, `uflow()`
consumes, and `sungetc()` still always lands in `pbackfail()`, which
is what the test needs.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* fix: leave the character that terminates a number in the input
Read the character following a number without consuming it, instead of
consuming it and putting it back. input_stream_adapter now peeks with
sgetc() and only steps over the character when the next one is requested
or when the adapter is destroyed, so releasing it cannot fail - no
putback position is required from the streambuf.
Suggested by gregmarr in #5344.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* docs: match the version history wording to the peek-based fix
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* docs: drop the whitespace-separator caveat from the parsing pages
The caveat added in #5343 describes the behavior this branch fixes: a
number no longer consumes the character that terminates it, so
concatenated values need no separator.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* refactor: split the strict and non-strict paths in parser
Folding the release_lookahead() call into the existing strict check left
the "in strict mode" comment on an else-if branch, and made the strict
condition in sax_parse() redundant with the branch it followed.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Put the stream position fix behind JSON_PRECISE_STREAM_POSITION
Leaving the character that terminates a number in the stream is observable:
reading "1,2,3" with repeated operator>> works today only because the comma
after each number is swallowed, and std::getline after a number skips the
line break. Both break with the fix, so make it opt-in for 3.x, as suggested
by @gregmarr in the review.
- JSON_PRECISE_STREAM_POSITION (default 0) selects the peek-based
input_stream_adapter. Without it, the adapter is the consuming one from
develop and has no supports_lookahead, so lexer::release_lookahead() and
the parser's calls to it compile to nothing.
- The macro changes input_stream_adapter's layout and member functions, so
it gets the ABI tag _psp, after _bics. The ABI config tests, the natvis
generator, and nlohmann_json.natvis (regenerated) know the tag.
- The tests for the fix move to unit-precise-stream-position.cpp, which
defines the macro itself and runs in every build, and gain the two cases
above. unit-deserialization.cpp pins the default behavior instead.
- The docs describe the default behavior again and point to the new macro
page; version history says "added in 3.13.0, planned default in 4.0.0".
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* docs: document that a NUL byte in the input is treated as end of input
A NUL byte anywhere in the input - trailing, or embedded ahead of more
otherwise well-formed JSON - is currently treated the same as genuine
end of input, so parsing silently stops there instead of raising the
parse_error.101 any other unexpected byte triggers. This mirrors the
NUL-terminated-C-string convention already used when no explicit input
length is given (json::parse(const char*) already stops at strlen()),
just applied uniformly rather than only when a length is genuinely
unavailable.
This behavior predates this change and is not being altered here -
changing it would be an observable, backwards-incompatible behavior
change for any caller that (knowingly or not) depends on it, which is
not something to do silently in a patch. Documenting the current,
verified behavior as a new FAQ entry instead, so it's an intentional
and discoverable part of the contract rather than a surprise.
Fixes#5530.
Signed-off-by: Niels Lohmann <niels.lohmann@gmail.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N4RQ1Ahan5YAGbnAQGjZTY
* Add JSON_STRICT_NUL_HANDLING opt-in macro for issue #5530
A NUL byte anywhere in the input is currently treated the same as real
end of input, rather than raising parse_error.101 like any other
unexpected byte (documented in the previous commit's FAQ entry). A full
unconditional fix was tried in PR #5532 but rejected as too risky to
ship by default: any caller could depend on the current behavior, even
unknowingly (e.g. a zero-padded buffer). On PR #5534, gregmarr proposed
a compile-time opt-in flag instead, and the maintainer agreed, wanting
it available now and defaulting to the corrected behavior in 4.0.0.
This mirrors the existing JSON_BRACE_INIT_COPY_SEMANTICS precedent as
closely as sensible:
- JSON_STRICT_NUL_HANDLING defaults to 0 (off); the three lexer sites
that treat '\0' as EOF/comment-terminator are gated with
`#if !JSON_STRICT_NUL_HANDLING` so the default-off behavior is
byte-for-byte identical to today's.
- input_adapters.hpp's `T (&array)[N]` overload additionally trims a
single trailing '\0' from a `char` array (e.g. a string literal like
`json::parse("123")`) when the macro is on, so that case keeps
working; every other element type (unsigned char, std::uint8_t, ...)
always keeps its full extent. This intentionally does *not* reuse the
existing strlen()-based pointer overload via SFINAE-excluding `char`
from the array overload, as originally sketched for this change: that
approach is ambiguous against the newer generic container overload
added since PR #5532, and even where it compiles, strlen()-scanning a
`char` array that is not NUL-terminated within its bounds reads past
the end of the array (confirmed with AddressSanitizer). Trimming only
a single trailing byte, without scanning, avoids both problems.
- Documented via docs/mkdocs/docs/api/macros/json_strict_nul_handling.md,
linked from the macros index/nav/features page, the FAQ entry, and
the parse/accept/operator>> reference pages.
- Tested in unit-class_parser.cpp and unit-deserialization.cpp, default
state unguarded and opt-in state guarded. Since the library itself
#undefs the macro at the end of json.hpp (as JSON_BRACE_INIT_COPY_SEMANTICS
already does), a plain `#if defined(JSON_STRICT_NUL_HANDLING)` guard
after the include never actually triggers; the tests instead capture
the command-line value into a test-local macro before including the
header. A few pre-existing fixtures elsewhere (std::array<uint8_t, N>
sized one larger than their literal, relying on value-initialization
to silently add a trailing zero byte) needed the same one-byte
adjustment to keep passing under the opt-in behavior.
Unlike the precedent, this adds a proper `JSON_StrictNulHandling` CMake
option (rather than a raw -DCMAKE_CXX_FLAGS injection) and wires its
ci_test_strict_nul_handling target into the ci_cmake_options job matrix
in .github/workflows/ubuntu.yml, so the opt-in build is actually
exercised in CI -- closing the one gap in the precedent's own CI setup
(ci_test_brace_init_copy_semantics is defined but never referenced by
any workflow, so it has never actually run).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Clarify where JSON_STRICT_NUL_HANDLING does not reject NUL bytes
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <niels.lohmann@gmail.com>
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* Migrate ci_icpc/ci_test_compilers_gcc_old/ci_infer off custom json-ci image
Replaces the last three consumers of ghcr.io/nlohmann/json-ci with official
images: ci_icpc now uses Intel's own intel/oneapi-hpckit:2023.2.1-devel-ubuntu22.04
(the last release with classic icc/icpc before Intel dropped it in oneAPI
2024.0), ci_test_compilers_gcc_old installs old GCCs on official ubuntu:20.04
via the same PPA/archive setup the custom image used (working around
actions/checkout's incompatibility with official gcc:4/5/6 images), and
ci_infer runs directly on ubuntu-latest, fetching Facebook's official Infer
release tarball inline instead of a maintained image. No job in
ubuntu.yml references the custom image anymore.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Update quality-assurance compiler table for CI image migration
Reflects the ci_icpc/ci_test_compilers_gcc_old container migration: the
old-GCC jobs (4.8/4.9/5/6) now compile inside official ubuntu:20.04 rather
than the custom Focal-based json-ci image (same OS, just now attributed to
the official image), and ci_icpc now uses Intel's official
intel/oneapi-hpckit:2023.2.1-devel-ubuntu22.04, bumping the reported ICC
version from 2021.5.0 to 2021.10.0 and the OS from Ubuntu 20.04.3 to 22.04.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix CI failures from the json-ci image migration
- ci_icpc: the official intel/oneapi-hpckit image has no CMake preinstalled
(the custom image bundled one); add the missing lukka/get-cmake step.
- ci_test_compilers_gcc_old: official ubuntu:20.04 has no build tool, so
CMake's default Unix Makefiles generator failed with "CMAKE_MAKE_PROGRAM
is not set"; install make alongside the PPA-provided g++.
- ci_infer: Infer v1.1.0's bundled Clang frontend can't parse GCC 14's
headers (ubuntu-latest's default toolchain), failing with parse errors in
<bits/unicode.h>; bump to the latest release, v1.3.0, whose newer bundled
frontend understands them (release asset also renamed upstream from
infer-linux64-v1.1.0.tar.xz to infer-linux-x86_64-v1.3.0.tar.xz).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix ci_test_compilers_gcc_old: g++-6 missing from xenial-only archives
My inline PPA/archive replication only added the xenial main/universe
suites, but g++-6 isn't available there ("has no installation candidate").
The original custom Dockerfile also pulled from bionic main/universe and
xenial-updates main/universe; add those back to match.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix ci_test_compilers_gcc_old: install git for CMake's FetchContent tests
Official ubuntu:20.04 ships no git at all (actions/checkout only succeeded
via its API-download fallback). The cmake_fetch_content(2) tests invoke
CMake's own ExternalProject_Add, which needs a real git binary and failed
with "could not find git for clone of json-populate". Install git alongside
the other build prerequisites.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix ci_icpc: drop redundant setvars.sh sourcing
Unlike the old custom image, the official intel/oneapi-hpckit image already
has the oneAPI environment (icc/icpc on PATH) baked in at the container
level. Explicitly re-sourcing setvars.sh in the Build step failed with
"setvars.sh has already been run. Skipping re-execution." (exit code 3,
aborting the step under `sh -e`). Drop the now-unnecessary sourcing.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix ci_icpc: exclude classic ICC from the std::span regression test
Bumping to Intel's official intel/oneapi-hpckit:2023.2.1 image (see previous
commit) also bumped classic icc/icpc from 2021.5.0 to 2021.10.0. The newer
version's __has_include(<span>) now returns true, but it still can't
actually compile std::span/std::as_bytes usage:
error: namespace "std" has no member "as_bytes"
error: namespace "std" has no member "span"
Exclude __ICC/__INTEL_COMPILER the same way _LIBCPP_VERSION is already
excluded for issue #4490.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix amalgamation/style check: indent comment per astyle
Verified with the pinned astyle 3.4.13 (make install_astyle) locally;
no further diff.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix ci_icpc: skip UTF-8 u8-literal comparison test on classic ICC
test-deserialization_cpp20 failed:
ERROR: CHECK( j2["emoji"] == "😀" ) is NOT correct!
check_utf8() only guards against MSVC's ANSI-codepage quirk (its docstring
example), but classic ICC has an analogous problem: it doesn't encode a
narrow string literal containing non-ASCII source characters as UTF-8,
so comparing a decoded u8R"(...)" literal against a narrow literal with
the same characters fails. Extend the existing guard.
Verified with the pinned astyle 3.4.13; no diff.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Add implementation to retrieve start and end positions of json during parse
* Add more unit tests and add start/stop parsing for arrays
* Add raw value for all types
* Add more tests and fix compiler warning
* Amalgamate
* Fix CLang GCC warnings
* Fix error in build
* Style using astyle 3.1
* Fix whitespace changes
* revert
* more whitespace reverts
* Address PR comments
* Fix failing issues
* More whitespace reverts
* Address remaining PR comments
* Address comments
* Switch to using custom base class instead of default basic_json
* Adding a basic using for a json using the new base class. Also address PR comments and fix CI failures
* Address decltype comments
* Diagnostic positions macro (#4)
Co-authored-by: Sush Shringarputale <sushring@linux.microsoft.com>
* Fix missed include deletion
* Add docs and address other PR comments (#5)
* Add docs and address other PR comments
---------
Co-authored-by: Sush Shringarputale <sushring@linux.microsoft.com>
* Address new PR comments and fix CI tests for documentation
* Update documentation based on feedback (#6)
---------
Co-authored-by: Sush Shringarputale <sushring@linux.microsoft.com>
* Address std::size_t and other comments
* Fix new CI issues
* Fix lcov
* Improve lcov case with update to handle_diagnostic_positions call for discarded values
* Fix indentation of LCOV_EXCL_STOP comments
* fix amalgamation astyle issue
---------
Co-authored-by: Sush Shringarputale <sushring@linux.microsoft.com>
* Possible fix for #4485
Throw's an exception when i is nullptr,
also added a testcase for this scenario though most likely in the wrong test file.cpp
* quick cleanup
* Fix compile issues
* moved tests around, changed exceptions, removed a possibly unneeded include
* add back include <memory> for testing something
* Ninja doesn't like not having a \n, at end of file, adding it back
* update input_adapter file to deal with empty/null file ptr.
* ran make pretty
* added test for inputadapter
* ran make amalgamate
* Update tests/src/unit-deserialization.cpp
Co-authored-by: Niels Lohmann <niels.lohmann@gmail.com>
* Update tests/src/unit-deserialization.cpp
Co-authored-by: Niels Lohmann <niels.lohmann@gmail.com>
* Update input adapters.hpp with new includes
* fix unabigious use of _, (there was a double declare)
* did the amalagamate
* rm duplicate includes
* make amalgamate again
* reorder
* amalgamate
* moved it above
* amalgamate
---------
Co-authored-by: Jordan <jordan-hoang@users.noreply.github.com>
Co-authored-by: Niels Lohmann <niels.lohmann@gmail.com>
* Move UDLs into nlohmann::literals::json_literals namespace
* Add 'using namespace' to unit tests
* Add 'using namespace' to examples
* Add 'using namespace' to README
* Move UDL mkdocs pages out of basic_json/
* Update documentation
* Update docset index
* Add JSON_GlobalUDLs CMake option
* Add unit test
* Build examples without global UDLs
* Add CI target
When parsing a string containing two JSON values using iterators, after
parsing, iterator_input_adapter should have advanced to the first
character of the second value.
Add a unit test to check that this is true.
