Conflicted only in tests/src/unit-class_lexer.cpp, where develop's #5737
lint fix (CAPTURE(x); -> CAPTURE(x)) collided with this PR's rewrite of
the Eisel-Lemire float tests; kept the PR's new tests and applied the
lint-fixed CAPTURE style. single_include regenerated via make amalgamate.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix the ci_cmake_flags wiring so every option is checked
The CMake 3.31.6 flag list referred to itself before it was defined,
so only JSON_BuildTests was checked with that version. The targets for
the CMake running the build ("_2") were created but never added to
ci_cmake_flags, and the three versions shared one build directory.
JSON_StrictNulHandling was not in the list at all.
Use the 3.5.0 list for 3.31.6, add JSON_StrictNulHandling, and create
one ci_cmake_flag_<flag> target per option for the running CMake with
its own build directory. Also use the function parameter in the
COMMENT, refresh the stale version comment, and let ci_clean remove
the downloaded cmake-<version> directories instead of the long-gone
cmake-3.5.0-Darwin64.
Part of #5715
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix ci_test_clang_libcxx_cxx* jobs silently building without warnings
CMake only seeds CMAKE_CXX_FLAGS from the CXXFLAGS environment variable
when the cache entry is unset, so the explicit -DCMAKE_CXX_FLAGS="-stdlib=libc++"
argument made it ignore CXXFLAGS="${CLANG_CXXFLAGS}" entirely. The six
ci_test_standards_clang (..., libcxx) jobs therefore compiled without
-Weverything/-Werror while their libstdc++ siblings did use them.
Pass -stdlib=libc++ through the same CXXFLAGS value instead of a separate
-D argument, and give the target its own build directory
(build_clang_libcxx_cxx${CXX_STANDARD}) so it no longer shares a CMake
cache with the libstdc++ variant. Suppress the resulting
-Wthread-safety-negative finding from libc++'s std::mutex annotations,
which fires on doctest's reporters in this translation unit only.
Part of #5715
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Remove the no-op AppVeyor with_win_header job
The with_win_header matrix entry patched Windows.h into
single_include/nlohmann/json.hpp before building, but JSON_MultipleHeaders
has defaulted to ON since #3532 (2022-06), so CMakeLists.txt points the
tests at include/ and the patched single header is never compiled. The
job has been a no-op VS2015 build since then.
Windows.h coverage already exists through tests/src/unit-windows_h.cpp
(#3631), which runs in every MSVC job. Delete the dead matrix entry and
its before_build steps, and cite unit-windows_h.cpp from the QA page.
Part of #5715
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Remove unused ci_oclint and ci_pvs_studio targets
No workflow invokes ci_oclint, ci_pvs_studio, or their tool discovery.
ci_oclint also had a side effect on every JSON_CI configure: it copied
the single header into src_single/all.cpp and added an add_executable()
for it without EXCLUDE_FROM_ALL, so a plain build compiled a 1.2 MB
translation unit that only that unused target consumed. ci_pvs_studio
duplicates the Makefile's pvs_studio target, which is kept.
Also drop the duplicate --check-level=exhaustive flag passed twice to
the same ci_cppcheck invocation.
Part of #5715
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Stop Dependabot from proposing astyle bumps
astyle is deliberately pinned at 3.4.13 because newer versions reformat
unrelated lines and this version defines the formatting that
check_amalgamation.yml enforces. Without an ignore rule, Dependabot
keeps opening PRs for every new astyle release (most recently #4580,
#4942, #5445, #5448), each of which fails the amalgamation check and
gets closed unmerged.
Part of #5715
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Move Linux arm64 CI from dead Cirrus CI to ubuntu-24.04-arm
Cirrus CI stopped reporting check runs on develop sometime after
d10879bca (2026-05-26); every commit since has only github-actions
check runs, so .cirrus.yml silently lost its only consumer while
README.md, FILES.md and the QA page kept advertising the coverage.
Add a ci_test_arm64 job to ubuntu.yml using the same pinned
actions/checkout and lukka/get-cmake actions as the other jobs, on the
native ubuntu-24.04-arm runner, with a step that confirms uname -m
reports aarch64. Delete .cirrus.yml and its README badge and FILES.md
section, and update the QA page's arm64 row.
Part of #5715
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Make scan-build fail on findings and drop irrelevant checkers (#5715 item 4a)
ci_clang_analyze ran scan-build without --status-bugs, so the job
passed whenever the ninja build succeeded, no matter what the
analyzer found ("No bugs found" in a green run gave no signal either
way). It is also missing --use-analyzer=${CLANG_TOOL}, so scan-build
picks whichever clang happens to be first on PATH inside the
silkeh/clang:dev container instead of the one this file already
selected and versioned.
Add --status-bugs and --use-analyzer=${CLANG_TOOL} to the scan-build
invocation. While here, drop the osx.*, webkit.*, fuchsia.*, and
optin.mpi.* checkers from CLANG_ANALYZER_CHECKS: none of them apply
to this portable C++ library, and leaving them enabled only adds
noise once the job can actually fail on a finding.
The job is currently clean (0 bugs), so this alone does not surface
any new finding; it only makes the existing "no bugs found" result
authoritative. This is 4a of 3 independent steps in #5715 item 4;
4b (Infer) and 4c (IWYU) still need their existing findings triaged
before --fail-on-issue/-Xiwyu --error can be added, and are handled
in separate commits.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Deduplicate the amalgamation/format check's file set and add BUILD.bazel (#5715 item 5)
The amalgamation/format check existed three times with three different
file sets: the Makefile's pretty/check-amalgamation, the pull_request-only
check_amalgamation.yml workflow, and the ci_test_amalgamation CMake target
that also runs on direct pushes to develop/master/release/*. The CMake
target's glob was a strict subset of the workflow's (missing the
docs/mkdocs/docs/examples/*.hpp headers, tests/abi/, tests/cmake_*/project/,
tests/cuda_example/, tests/fmt_formatter/, and tests/module_cpp20/), and it
never checked BUILD.bazel at all, so a misformatted file in any of those
paths, or a stale BUILD.bazel, could reach develop through a direct push
even though the PR-only workflow would have caught it.
Make ci_test_amalgamation glob the same roots (docs/mkdocs/docs/examples,
include, tests) and extensions (*.hpp, *.cpp, *.cu) as check_amalgamation.yml,
excluding tests/thirdparty/ and tests/abi/include/nlohmann/ the same way, and
regenerate and diff BUILD.bazel next to json.hpp/json_fwd.hpp. Also add
docs/mkdocs/docs/examples/*.hpp to the Makefile's pretty/pretty_format
targets, which were missing the four custom_*_type.hpp example headers, and
drop the stale "called by Travis" comment on check-amalgamation (Travis is
gone; nothing currently calls that Makefile target from CI).
Leaves the workflow itself untouched: it deliberately runs amalgamate.py
from a fresh develop checkout so a PR cannot change the tool that checks it.
Overlaps #5610 and #5621, which each add a new amalgamated header and touch
the same INDENT_FILES/ci_test_amalgamation/check_amalgamation.yml hunks.
Verified with `make check-amalgamation` on this branch: clean, no diff.
#5715 item 5.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Download prebuilt CMake binaries on Linux x86_64 instead of building from source (#5715 item 6)
ci_get_cmake() downloaded the source tarball of CMake 3.5.0, 3.31.6, and
4.0.0 and compiled each one completely (including CMake's own test
helpers) with -DCMAKE_POLICY_VERSION_MINIMUM=3.5 as a workaround for
building old CMake with a newer one. On CI this made the
ci_cmake_options (ci_cmake_flags) job take about 11 minutes, most of it
spent building CMake itself, even though Kitware has published
ready-to-run Linux x86_64 archives for all three of these releases
since 3.20 (lowercase platform name).
On Linux x86_64, download and unpack the prebuilt
cmake-<version>-linux-x86_64.tar.gz archive instead and point the
existing ${var} output at its bin/cmake, skipping the configure/build
steps and CMAKE_POLICY_VERSION_MINIMUM entirely. Keep the previous
source build as a fallback for any other platform (macOS, Linux
aarch64), since Kitware does not publish binaries for every
CMake/platform combination this project might build on.
Verify the downloaded archive against Kitware's own published checksum
before unpacking it: download cmake-<version>-SHA-256.txt alongside the
archive and run `sha256sum -c` on the matching line. A CI job that wgets
and untars a binary from a release page with no integrity check is a
supply-chain gap; Kitware has published this file for every release
since 3.20, so checking it costs one extra download and one grep.
As a separate, mechanical change: the ci_cmake_options job's container
only needed to stay on ubuntu:focal for the source build's
libssl-dev dependency and its own aging toolchain; now that the
Linux/x86_64 path never compiles CMake, drop libssl-dev from its apt
install line and move the job to ubuntu:24.04 (Ubuntu 20.04 left
standard support in May 2025). ci_clean already removes the
cmake-3.5.0/cmake-3.31.6/cmake-4.0.0 directories from the #5715 item 3
fix, and the prebuilt path reuses those same directory names, so no
further cleanup changes are needed.
Overlaps #5598, which edits the same ci_cmake_options matrix line in
ubuntu.yml; a rebase may be needed once that lands.
Verified locally: `cmake -S . -B build -DJSON_CI=On` configures cleanly
on macOS/arm64 (source-build fallback branch) and on Linux/x86_64 in an
ubuntu:24.04 Docker container (47 `ci_cmake_flag_*` targets generated,
one built and run successfully); `.github/workflows/ubuntu.yml` still
parses as valid YAML; downloaded the real v3.31.6 Linux x86_64 archive
and SHA-256 file from Kitware and confirmed the `grep | sha256sum -c`
pipeline both accepts the genuine file and is anchored to the exact
filename (not a prefix match).
CI must confirm: the prebuilt-binary path actually runs on the
ubuntu-latest/ubuntu:24.04 x86_64 runner, all `ci_cmake_options`
entries still pass with the new container's GCC, and the job's
runtime drops from roughly 11 minutes.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Make Infer fail on findings, with a type-level baseline for the ~174 pre-existing ones (#5715 item 4b)
ci_infer ran `infer run` without --fail-on-issue, so the job passed
regardless of what Pulse found; the last recorded run (35829411620,
commit 1054b2097) logged "Found 174 issues" and still went green.
report.txt was also never uploaded, so the full finding list was only
ever visible in the truncated 5-issue console excerpt.
Add a repository-root .inferconfig (auto-discovered by Infer; passing
--project-root on the `infer run` invocation makes sure it is found
even though the analysis runs from build/build_infer) that sets
fail-on-issue and disables the six PULSE issue types that made up all
174 findings in that run: PULSE_UNNECESSARY_COPY_ASSIGNMENT (129),
PULSE_UNNECESSARY_COPY (22), PULSE_UNNECESSARY_COPY_INTERMEDIATE (15),
PULSE_RESOURCE_LEAK (5), PULSE_CONST_REFABLE (2), and
PULSE_UNNECESSARY_COPY_OPTIONAL (1).
This is a deliberate, narrower fix than "triage and fix everything in
this PR": the visible sample is entirely doctest-macro copies in test
code (for example tests/src/unit-algorithms.cpp:141 and
tests/src/unit-bjdata.cpp:3706), but 169 of the 174 findings were never
uploaded anywhere and this PR cannot respectably claim to have fixed
issues it never saw, including the resource-leak and const-refable
ones that are the most likely to be genuine bugs. Disabling by issue
type is a coarser baseline than a per-finding one (Infer has no
built-in per-finding baseline short of the two-run `infer reportdiff`
workflow, which this repository does not have the CI infrastructure
for), but it has the same effect today: the job goes from always green
to green-only-when-clean-of-everything-else, so CI now fails the
moment a *new* issue type appears, and report.txt is uploaded as a
workflow artifact on every run (including failures) so the six
disabled types can be triaged and re-enabled incrementally in follow-up
PRs.
#5715 item 4b. 4a (scan-build) and 4c (IWYU) are handled in separate
commits.
Verified: .inferconfig parses as JSON, ubuntu.yml still parses as
YAML. Infer itself is not available in this environment (v1.3.0 tar.xz
requires a Linux x86_64 runner), so CI must confirm that `infer run
--project-root ... -- make` picks up .inferconfig, that fail-on-issue
takes effect, and that the six disabled types actually suppress the
existing findings without also hiding an unrelated new one.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix IWYU findings for json.hpp/json_fwd.hpp/ordered_map.hpp and make CI fail on new ones (#5715 item 4c)
ci_single_binaries ran IWYU via CMake's CXX_INCLUDE_WHAT_YOU_USE launcher
property, which only printed "Warning: include-what-you-use reported
diagnostics" without failing the build: CMake's own __run_co_compile
wrapper does not propagate the launched tool's exit code, so even
`-Xiwyu --error` could never fail `cmake --build` this way. Verified
this empirically by injecting a deliberately-unused #include and
confirming the build still exited 0.
Fix the findings from the last recorded run (issue #5715 item 4, log
35829411620):
- ordered_map.hpp: add <new> (placement new) and
nlohmann/detail/abi_macros.hpp; drop <memory> (std::allocator is
still visible transitively via <vector>, confirmed by full local and
containerized test suite runs).
- json_fwd.hpp: drop <memory> (same reasoning). Keep every forward
declaration IWYU wanted removed (adl_serializer, basic_json,
json_pointer, ordered_map): this file's only job is to forward-declare
them for downstream users, so "nothing in this TU uses them" is
expected, not a real finding. Mark each with `// IWYU pragma: keep`.
- json.hpp: add <cmath>, <cstdint>, <set>, <type_traits>,
<unordered_map>, and the detail/abi_macros.hpp, detail/input/json_sax.hpp,
detail/meta/detected.hpp, thirdparty/hedley/hedley.hpp includes IWYU
says it needs. Do NOT remove adl_serializer.hpp,
detail/conversions/from_json.hpp, detail/conversions/to_json.hpp,
detail/macro_unscope.hpp, or ordered_map.hpp as IWYU suggests: nothing
else in include/nlohmann includes adl_serializer.hpp or
ordered_map.hpp, so basic_json<>'s own default template arguments
(JSONSerializer = adl_serializer, and ordered_json = basic_json<ordered_map>)
would lose their complete type; detail/macro_unscope.hpp is what
undoes the JSON_* macros detail/macro_scope.hpp defines earlier in
this same file, and removing it leaks those macros into every
translation unit that includes <nlohmann/json.hpp>. Verified by
actually removing them in a scratch test: the header still "compiles"
stand-alone but ordered_json and every macro-using translation unit
break. Marked each `// IWYU pragma: keep`.
Enforce it with `iwyu_tool` (ships with IWYU, e.g. as /usr/bin/iwyu_tool
on Debian/Ubuntu) instead of relying on the launcher property: it reads
compile_commands.json (now exported project-wide under JSON_CI) and
does return a real exit code for its own analysis, independent of
CMake's wrapper. ci_single_binaries now runs it over every
src_single/*.cpp with `-Xiwyu --error`, so a *new* finding fails CI.
json.hpp itself is excluded from that hard gate: even after every fix
above, IWYU's suggestion for one remaining symbol (a container
`swap, operator!=` used somewhere via a templated comparator) is not
deterministic — repeated, otherwise-identical containerized runs
reported <set>, then <unordered_map>, then <map> as "the" header to
add/remove for the exact same source. Gating a whole CI job on a
nondeterministic suggestion would make ci_single_binaries flaky rather
than informative, so json.hpp keeps the existing informational warning
(still shown during its normal compile) without failing the build on
it. Every other one of the ~50 single-header checks is included in the
hard gate.
#5715 item 4c. 4a (scan-build) and 4b (Infer) are separate commits.
Verified: full local ctest suite (129/129) and the ci_single_binaries
target itself both green in a containerized silkeh/clang:dev run
(matching the actual CI job) after this fix; a deliberately-reintroduced
unused #include in ordered_map.hpp was confirmed to fail
`cmake --build ... --target ci_single_binaries` (exit 2) with this
change, and to pass without it, on the same container/IWYU version CI
uses. `make check-amalgamation` is clean. Compiled with Clang and GCC
at -std=c++11/14/17/20 locally with no new warnings.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix math formulas rendering as raw TeX in the published docs
The privacy plugin self-hosts MathJax 2.7.0 but drops its
?config=TeX-MML-AM_CHTML query string, so the rehosted script loads no
input jax and the 10 formulas across 6 pages render as raw TeX to
readers. Remove pymdownx.arithmatex and the MathJax extra_javascript
entry, and rewrite the formulas in plain HTML (<sup>, <i>) instead.
This also drops a nine-year-old third-party script from every page.
Part of #5718
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix publish_documentation triggers and persist unneeded git credentials
publish_documentation.yml only triggered on docs/mkdocs/** pushes, but
the site also embeds .github/CODE_OF_CONDUCT.md, CONTRIBUTING.md,
SECURITY.md, cmake/{clang,gcc}_flags.cmake, .clang-tidy,
tools/astyle/.astylerc and tests/fmt_formatter/project/main.cpp via
pymdownx.snippets, so changes to those files never republished the
site. Extend the path filter to cover them, and switch runs-on from
the long-pinned ubuntu-22.04 to ubuntu-latest to match
ci_test_documentation.
Also add persist-credentials: false to the checkouts in ci_icpx and
ci_nvhpc (ubuntu.yml) and msvc-vs2026/msvc-arm64 (windows.yml), none
of which pushes with git, matching every other checkout in these
workflows.
Part of #5718
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix example build: broken debug echo, deprecations hidden for everything
docs/Makefile's debug echo used a space instead of a comma in
$(call cxx_standard ...), so it always printed an empty standard.
Every example was also compiled with -Wno-deprecated-declarations,
which would silently hide an accidental deprecated-API call in any of
them. Factor the duplicated compile flags into EXAMPLE_CPPFLAGS/
EXAMPLE_WARNFLAGS, build with -Werror=deprecated-declarations by
default, and only allow the three examples that intentionally
document deprecated API (the DEPRECATED_EXAMPLES list) to suppress it.
Part of #5718
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix check_structure.py NOLINT parsing and an off-by-one report line
`line.strip("<!-- NOLINT")` followed by `.strip(" -->")` strips any of
the characters in those sets from both ends, not a literal prefix; it
only happened to work for "Examples". A NOLINT'd section name
starting with N, O, L, I or T (e.g. "Notes", "Template parameters",
"Iterator invalidation", "Literals") was silently mangled, so the
suppression did not apply and the checker could report a spurious
missing/misordered section. Parse the comment with a regex instead.
The same fragile strip() pattern was used for heading text; replace it
with a plain prefix slice. Also fix the admonition_title report, which
used the 0-based line index while every other report in the file uses
lineno+1.
Part of #5718
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix docset README's non-existent make target and stale fallback URL
The README told readers to run `make nlohmann_json.docset`, but the
Makefile's targets are `all`, `JSON_for_Modern_C++.docset` and
`install_docset_zeal`; the documented command has failed with "No
rule to make target" since #2967 (2021). Point the README at the real
target and folder name. Info.plist's DashDocSetFallbackURL also still
pointed at the old nlohmann.github.io/json/ URL instead of the
canonical https://json.nlohmann.me/ from mkdocs.yml's site_url.
Leave list_missing_pages/list_removed_paths alone: they may become
redundant once #5638's check_docset() lands, which is a follow-up.
Part of #5718
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Remove two leftover Doxygen-era .link files from the examples directory
parse__iterator_pair.link and parse__pointers.link each held only a
Wandbox "online" permalink from the old Doxygen docs. #3071 deleted
every other .link file in 2021; these two came in through a parallel
PR (#3100) and were never referenced by any page, script or config.
Part of #5718
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix MacPorts CMake example to include its own snippet files
The MacPorts "Example: CMake" block included
integration/homebrew/example.cpp and
integration/homebrew/CMakeLists.txt instead of the MacPorts files
right next to it, a copy-paste slip from the Homebrew section. Nothing
referenced integration/macports/CMakeLists.txt as a result. The page
rendered correctly only because the homebrew, macports and
vcpkg/CMakeLists.txt snippets are byte-identical, so a future edit to
the MacPorts files would not have shown up on the page.
Part of #5718 item 6
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Do not persist git credentials in publish_documentation's checkout
The checkout step in publish_documentation.yml left the default
persist-credentials: true, so GITHUB_TOKEN stayed writable in
.git/config for the rest of the job (zizmor's artipacked finding). The
Deploy documentation step authenticates through its own github_token
input to peaceiris/actions-gh-pages and does not push with the
checked-out credentials, so persist-credentials: false is safe here,
matching every other checkout in the workflow set.
Overlaps #5638, which edits this same checkout step (adds
fetch-depth: 0); expect a rebase conflict there.
Part of #5718 item 2
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Replace list_missing_pages/list_removed_paths with a comm(1)-based diff
The docset Makefile's list_missing_pages ran one sqlite3 query per
mkdocs page, and list_removed_paths nested a loop over all mkdocs pages
inside a loop over all docset index paths (O(n*m) shell iteration).
Issue #5718 item 5 suggested removing or reducing these targets once
#5638's check_docset() lands, but that PR is still open and covers only
API pages and macros, not the full page set these targets check.
Replace the loops with two sorted path lists (DOCSET_PAGE_PATHS from
mkdocs' markdown sources, DOCSET_INDEX_PATHS from the built docset
index) compared with a single comm(1) call each, verified to produce
output identical to the old loops against the current docSet.dsidx.
The sed expression used '#' as its delimiter, which GNU Make reads as
a comment character even inside a variable assignment, truncating the
line and orphaning the closing paren of $(shell ...) ("unterminated
call to function 'shell': missing ')'"). Use '@' as the delimiter
instead.
Part of #5718 item 5
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The node index section on the architecture page says how save() writes
the nodes and that a change of their layout must raise the image
version, and save's format note links to it.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Add API pages for basic_json_document::save() and load(), including the
image_check enumeration and its three levels. Add examples that show
caching a parsed document as an image, loading it without parsing, the
difference between a borrowed and an owned image, and a full check
rejecting a damaged image that a bounds check still reads safely.
Add an "Images" section to the json_view feature page, register the new
pages in mkdocs.yml and docSet.sql, group basic_json_document's member
list by parsing/access/images/edits, and document parse_error.116 and
type_error.320 on the exceptions page, extending out_of_range.416 for
images.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Add API reference pages for basic_json_document::insert and
basic_json_document::erase, matching the style of set.md and
push_back.md: signatures, parameters, return values, exception safety,
exceptions with their exact ids and messages, complexity, notes on
duplicate keys and view/iterator validity, and an example.
Add example programs basic_json_document__insert.cpp and
basic_json_document__erase.cpp with their expected output, each
comparing an edit on an editable document with the same edit on a
plain json value to show what is preserved: member order, the
spelling of untouched numbers, and, for insert, that a view taken
before the insert keeps referring to the same element after its index
shifts.
Register both new pages in mkdocs.yml, docSet.sql, and the member list
of basic_json_document/index.md, add cross-references to them from
set.md and push_back.md, and mention insert/erase in the "Editing a
document" section of features/json_view.md.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The node index section on the architecture page says how edits are kept
(the edit buffer, moved elements, links, and new nodes), and the feature
page links to it.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
- API pages for set and push_back of basic_json_document and for the
editable aliases; the class overviews name the Editable parameter
- type_error.319 on the exceptions page
- the feature page explains editing, and the examples show when it
helps: a configuration file changed without reformatting it
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Lookups in objects are linear, as for ordered_json. Objects with 128
members or more now get a hash table after parsing (open addressing; the
first of duplicate keys is kept, as for the linear search), so that
operator[], at(), find(), contains(), count(), value(), and JSON pointers
take constant time on average in them; the idea of switching to a hash
table for large objects is Boost.JSON's. The parser notes such objects when
it closes them (out of line, so that the parse loop only has a call for
it), and the object node keeps the number of its table.
Looking up each key of an object with 10,000 members: 59.8 ms -> 0.16 ms.
Parsing (json_document::parse, best of 7, separate processes): most files
within 1%; canada +5%, mesh.pretty +3%, citm +3%.
Tests: objects with 127, 128, 129, and 10,000 members (escaped, empty,
and duplicate keys, missing keys, comparisons), nested large objects, and
documents reused with read().
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Long runs of string bytes are scanned 16 at a time with NEON (AArch64, with
GCC and Clang) and SSE2 (x86-64): both belong to the baseline instruction
sets. A signed compare with 0x20 finds control characters and non-ASCII
bytes at once. Keys keep 16 table checks before the vector loop (their
lengths repeat from record to record, so the branches predict well);
string values have 8, as their lengths vary more.
Non-ASCII text is validated 16 bytes at a time with the "lookup4" check of
simdjson (J. Keiser and D. Lemire, "Validating UTF-8 In Less Than One
Instruction Per Byte", 2021): with NEON, and on x86-64 with SSSE3 if
JSON_VIEW_USE_SSSE3 is defined (SSSE3 is not part of x86-64, and the code
must not depend on the flags of a translation unit). JSON_VIEW_NO_SIMD
selects the portable code. The vector code sits in
detail/view/simd.hpp; the same input is accepted either way.
json_document::parse, best of 7 runs in separate processes (M1 Max):
poet.json (CJK text) -72%, random.json -25%, twitter.json -22%,
gsoc-2018.json -20%, semanticscholar -19%, github_events -11%,
apache_builds -9.5%, canada/citm -5/-6%; lottie +4%, tree-pretty +2.5%.
Tests: every two-byte sequence and three- and four-byte sequences with
continuation bytes at the edges of their ranges, at every offset around
the vector blocks of keys and values, cut short, and long runs of text
with a damaged byte, against json::accept and json::parse. CMake builds
the parser tests again with JSON_VIEW_NO_SIMD, and on x86-64 with
JSON_VIEW_USE_SSSE3 and -mssse3; the macros are documented.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
- API pages for operator== and operator!= of basic_json_view, linked
both ways with the basic_json pages
- the feature page and the class overview list the comparisons
- the examples show when the view helps: detecting a changed document
without building json values
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
- API pages for dump, number_format, and operator<< of basic_json_view,
linked both ways with the basic_json pages
- the feature page describes document order and number_format::source
- the examples show when the view helps: forwarding part of a message
and writing numbers exactly as they were read
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
- API pages for get, get_to, get_string, number_token, and value of
basic_json_view; JSON pointer overloads of operator[], at, and
contains; links both ways with the basic_json pages
- the feature page describes which conversions copy nothing
- the examples show when the view helps: strings without copies, numbers
exactly as written, and paths into a large text
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
- ci_test_noexceptions: the element access tests compare the exceptions
of json_view and basic_json through exception_of(), which catches them
outside a CHECK_THROWS; with JSON_NOEXCEPTION the first one aborted the
test. Compile those comparisons only with exceptions.
- clang 3.6: value-initialize a const json_view, as in the tests of
json-view/10-view-document.
- Format three new documentation examples with the pinned astyle, which
the "check" job runs once it gets past the amalgamation step.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
- API pages for operator[], at, front, back, find, contains, count,
begin, end, cbegin, cend, items, and type_name of basic_json_view,
linked both ways with the basic_json pages
- the feature page and size() describe document order and duplicate
keys
- the examples show when the view helps: reading a few fields of a large
text, probing optional members, and members in source order
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
A new section describes the 16-byte node: its fields, how integers,
floats, and object members are stored, how views navigate without
pointers, and a worked example. The feature page and the pages of
basic_json_document and node_count link to it where they mention the
16 bytes.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
- API pages for basic_json_document and basic_json_view, one per member,
and for the four aliases, each with an example
- features/json_view.md: the problem the view solves, ownership and
lifetime, what matches basic_json::parse() and what differs, and when
to choose json, ordered_json, SAX, or the view
- the examples show why one would use the view, not only how: borrowed
vs. owned input, reading a few fields and materializing one subtree,
reusing a document across many messages
- registered in the mkdocs navigation, llms.txt, the docset, the
exceptions page (out_of_range.416), architecture.md, the integration
page, and the README; the yyjson credit is added to the README and
license.md
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The nlohmann.json module includes json_view.hpp in its global module
fragment and exports basic_json_document, basic_json_view, and the four
aliases next to basic_json, json, and ordered_json. features/modules.md
lists them, and tests/module_cpp20 parses a document, so that a missing
export fails the ci_module_cpp20 job.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Add JSON_DISABLE_TUPLE_REFERENCE_CONVERSION to fix std::tuple conversions
basic_json can be constructed from std::tuple<json&>, which it turns into
a one-element array. Because of this, std::tuple picks its converting
constructor that converts the whole source tuple instead of the
element-wise one. As a result, std::tuple<const json&> built from
std::forward_as_tuple(j) binds to a temporary (a compile error with libc++,
a dangling reference with other standard libraries), and std::tuple<json>
built the same way holds [j] instead of a copy of j.
The new opt-in macro JSON_DISABLE_TUPLE_REFERENCE_CONVERSION (CMake option
JSON_DisableTupleReferenceConversion) removes the conversion from a
one-element tuple holding a reference to the same basic_json type, so
std::tuple converts element-wise. It is off by default, so existing
behavior is unchanged. It does not change any function body and therefore
is not part of the ABI tag.
Fixes#2226
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Convert one-element tuples to arrays on every compiler
to_json for std::tuple assigns a braced list, j = { std::get<Idx>(t)... }.
With a single element that is itself a basic_json, Apple clang 15 and 16
treat j = {x} as a copy of x, so std::tuple<json>{true} became true
instead of [true]. The macOS jobs (Xcode 15.1, 16.1) failed the new
checks in unit-disable-tuple-reference-conversion and unit-regression2.
The one-element overload that already handles
JSON_BRACE_INIT_COPY_SEMANTICS builds the array (or object, for a
[string, value] element) explicitly, the same way the initializer-list
constructor does. Use it unconditionally. The output is unchanged on
compilers that already wrapped the element.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Skip json reference tuple tests on clang < 4 and GCC < 5
ci_test_compilers_gcc_old (4.8) and ci_test_compilers_clang (3.4) could
not compile the new tuple tests. Creating a std::tuple of basic_json
references, e.g. std::forward_as_tuple(j), makes these compilers
instantiate basic_json's conversion operator for libstdc++'s internal
tuple bases, which fails hard. This happens with and without
JSON_DISABLE_TUPLE_REFERENCE_CONVERSION, so it is a limitation of these
compilers, not of the new option.
Tested with the CI images: clang 3.4 to 3.9 and GCC 4.8 and 4.9 fail,
clang 4, 5, and 6 and GCC 5 and 6 compile all cases. Skip only the
checks that create such tuples; the is_constructible checks still run.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The strtold fallback, which is left only for long double formats that
are not binary64 (x87, binary128), substituted the first byte of the
locale's decimal point for '.'. Under a locale whose decimal point is
longer than one byte, such as fa_IR.UTF-8 or ar_EG.UTF-8 (U+066B),
strtold stopped there and the value was truncated at the decimal point.
A longer decimal point is now put into a copy of the token.
The test "locale with a multi-byte decimal point" now compares the long
double values with those of the "C" locale; with x87 long doubles it
failed before.
Fixes#5660.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
float, double, and long double where it is IEEE-754 binary64 (MSVC, Apple
arm64) are now converted by the library itself, correctly rounded and
independent of the locale and of the C and C++ libraries:
- The token is split into sign, significand w (at most 19 digits), and
decimal exponent q, using the positions of the decimal point and the
exponent that the scanners already recorded, so no character is
classified again.
- Clinger's fast path where w and 10^|q| are exact.
- Eisel-Lemire otherwise, now templated for binary32 and binary64.
- For tokens with more than 19 digits whose w and w + 1 round differently,
an exact big-integer comparison with the midpoint between the two
candidates (the digit comparison of fast_float, simplified).
This replaces the separate token walks of Clinger's fast path and of
Eisel-Lemire, the significant-digit gate that avoided the former, and, for
float and double, std::from_chars and the locale-aware strtod. std::from_chars
and strtold remain only for other long double formats (x87, binary128,
double-double) and for types that are not IEEE-754. Values are bit-identical
to before wherever the previous conversion was correctly rounded; tokens
converted in a locale with a multi-byte decimal point are now also exact.
Overflow still gives out_of_range.406, underflow a signed zero.
convert_float() is the entry point for other parsers of JSON text: it
converts like the lexer, without allocation for binary32/binary64.
Tests: exact-bit tests for double and float (ties, subnormal and overflow
boundaries, huge exponents, more digits than any midpoint), Eisel-Lemire for
binary32, the round trips of 200,000 doubles and 100,000 floats without
declines, 508 generated hard cases with the expected bits of both formats
(float_hard_cases.hpp) through the converter and both scanners, and
JSON-level overflow/underflow checks for double and float. The locale tests
now check the values in a locale with a multi-byte decimal point.
Docs: the statements that parsing uses strtod/strtof/strtold; the fast_float
credit now names the digit comparison.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
operator>> parsed directly into its basic_json& target, so a parse
error left the target holding whatever was parsed before the error
instead of its previous value. With JSON_DIAGNOSTICS=1, that partial
value also violated the class invariant, because the parent pointers
of an array or object's elements are only set when the container is
closed, which a failed parse never reaches; copying such a value then
aborted in assert_invariant().
Fix it the way basic_json::parse() already handles this: parse into a
temporary and move it into the target only once parsing succeeds, so
the target is left unchanged if an exception is thrown.
Fixes#5652.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
contains(const json_pointer&) and operator/=(std::size_t) (and hence
operator/(std::size_t)) used string_t operations that the StringType
template parameter documentation explicitly does not require:
comparing string_t with a const char* literal, c_str(), and
constructibility from std::string. This made both functions fail to
compile for a conforming custom StringType, even though the
documentation's own reference StringType satisfies the requirements.
Fix contains() to compare individual chars ('0'..'9') instead of
comparing string_t with const char* literals, and to call data()
(documented to be null-terminated) instead of c_str(). Fix
operator/=(std::size_t) to build the array-index token via the
existing detail::to_string<StringType> helper (ADL int_to_string() or
assignment from std::to_string()) instead of via std::to_string()
directly, matching how diff(), items(), and std::hash already convert
a std::size_t to a StringType.
Add regression tests to tests/src/unit-alt-string.cpp: contains() for
present/missing keys and indices, "-", a leading zero, and a
non-numeric token on an array, plus json_pointer::operator/(std::size_t).
Fixes#5666.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Add JSON_NO_UDLS to leave out the user-defined string literals
The bodies of operator""_json and operator""_json_pointer call the
parser, so every translation unit including the library instantiates it,
even if it never parses anything. Defining JSON_NO_UDLS leaves the
literals out entirely, which saves 15-35% compile time for such
translation units (#5294). Nothing changes if the macro is not defined.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Mention JSON_NO_UDLS in the list of exported module symbols
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Move the user-defined string literals to <nlohmann/json_literals.hpp>
Following the review in #5294, the literals now live in their own header
instead of being removed entirely: <nlohmann/json.hpp> includes it at the
end unless JSON_NO_AUTOMATIC_UDLS (renamed from JSON_NO_UDLS) is defined,
so a project can opt out globally and include the header only where the
literals are used.
The header only uses public and standard macros, because the library's
internal macros are undefined at the end of json.hpp and the amalgamation
inlines macro_scope.hpp only once. For the same reason, the library no
longer defines and undefines JSON_USE_GLOBAL_UDLS, so a user's definition
is still visible to the header. The single-header copy is identical to
the multi-header one, as it only includes <nlohmann/json.hpp>. The module
always exports the literals.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix CI: include cycle, GCC 4.8 literal operator spacing, and global UDLs off in the JSON_NO_AUTOMATIC_UDLS test
- Suppress clang-tidy misc-header-include-cycle on the intentional mutual
include of json.hpp and json_literals.hpp.
- Use operator"" _json with a space for GCC 4.8 in the test's detection
aliases, as the header does.
- Only test the global literal operators when JSON_USE_GLOBAL_UDLS is on.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Declare the literal operators through a local macro
The GCC 4.8 spacing condition was repeated for both operator definitions
and the global using-declarations. NLOHMANN_JSON_LITERAL_OPERATOR(suffix)
now selects operator""##suffix or operator"" suffix in one place and is
undefined at the end of json_literals.hpp.
Suggested by gregmarr in review.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
basic_json(first, last) treated value_t::binary like the structured
types (array, object) in the range check, so it always copied the
whole binary value regardless of the iterators, even for an empty
range such as (b.end(), b.end()). The other primitive types (number,
boolean, string) already reject such a range with
invalid_iterator.204, and erase(first, last) already does the same
for binary values, so this made the constructor inconsistent with
both. Move case value_t::binary into the group of checked primitive
types.
Also update the two matching passages in basic_json.md that describe
overload 7, and add a version-history note.
Fixes#5670.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
When a parser callback rejects an object's or array's start event,
json_sax_dom_callback_parser kept calling it for everything inside
that container anyway: nested keys, values, and the start/end events
of containers below it. This contradicts parser_callback_t's own
documentation, which promises that discarding a container at its
start event also hides its content from the callback.
The same code path also kept a full copy of every key inside such a
discarded container in key_stack until the whole parse finished,
because the early return for values that are not stored skipped the
matching pop. Filtering out a large subtree is the main reason to use
a callback, so this made peak memory during the parse scale with the
size of the very subtree the callback was trying to skip.
Fix start_object(), start_array(), and key() so that a container
whose own start event was discarded, or that is nested inside one, is
never handed to the callback, and no longer pushes onto the key
stacks. A container whose start event was accepted but whose key was
rejected still gets its content reported, as documented ("the
callback is still called for the associated value, but its return
value has no further effect"); only its own bookkeeping is skipped
since it will not be stored.
Fixes#5643.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
j.contains(0), j.find(0), and j.count(0) used to compile: the literal 0 is
a null pointer constant, so it converts to a null const char*, and the
overloads taking const typename object_t::key_type& accepted it by
constructing a std::string from that null pointer, which is undefined
behavior (a crash with both libc++ and libstdc++). value(0, default_value)
had the same problem in C++11, where the object comparator is not
transparent.
Add deleted overloads for integral arguments to contains(), find()
(const and non-const), count(), and value() so that these calls are
compile errors in every supported language mode instead of crashing.
Calls with string, string_view, json_pointer, and size-typed element
access (at(), operator[](), erase()) are unaffected.
Fixes#5657.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
to_bson() rejected a binary value's subtype above 255 (out_of_range.415)
in write_bson_binary(), which only has the binary_t, not the basic_json
value that holds it, so the exception was created with no JSON_DIAGNOSTICS
context even though the equivalent to_msgpack() check names the value's
path. The check also ran after the document size, all preceding elements,
and this element's header and length had already reached the output
adapter, so a caller-provided std::vector or std::string ended up holding
a truncated document.
calc_bson_sizes() already walks every value before anything is written,
to size embedded documents and arrays and to reject invalid keys
(out_of_range.409) up front. The subtype check now runs there instead,
in calc_bson_binary_size(), which is given the basic_json value so the
exception can use it as context. The now-redundant check in
write_bson_binary() is removed, since calc_bson_sizes() always throws
first if any binary value in the document has an oversized subtype.
Fixes#5675.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
With JSON_STRICT_NUL_HANDLING defined to 1, parsing a wide, UTF-16,
UTF-32, or (C++20) UTF-8 string literal (e.g. json::parse(L"[1]"))
failed with parse_error.101 at the terminating NUL of the literal,
and accept() returned false. The array overload of input_adapter()
only dropped the compiler-added trailing '\0' for arrays of char,
so for wchar_t, char16_t, char32_t, and char8_t arrays that
terminator was passed to the parser as data, which the macro then
rejected.
Broaden the trimming to every character type that a string literal
can use (char, wchar_t, char16_t, char32_t, and, since C++20,
char8_t). Arrays of any other element type (unsigned char,
std::uint8_t, ...), as used for CBOR/MessagePack, are unaffected: a
trailing zero byte there is still read as genuine data.
Fixes#5658.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Parsing from a std::istream crashed in two unusual but valid stream
states, both in input_stream_adapter:
- With eofbit in the stream's exceptions() mask, get_character() sets
eofbit via is->clear(), which throws std::ios_base::failure. While
that exception unwinds, ~input_stream_adapter() called clear() again
to reset eofbit, which is still set and still in the exception mask,
so it throws a second time out of the (implicitly noexcept)
destructor and std::terminate() is called. The destructor now only
calls clear() if a bit other than eofbit remains set, so the first
exception can propagate normally.
- For an std::istream without a stream buffer (rdbuf() == nullptr,
e.g. std::istream(nullptr)), the constructor stored the null
pointer without checking it, and get_character() dereferenced it.
input_adapter(std::istream&) now throws parse_error.101 for such a
stream, the same as it already does for a null FILE* or char*.
Added regression tests to unit-deserialization.cpp and, for the
JSON_PRECISE_STREAM_POSITION variant of get_character(), to
unit-precise-stream-position.cpp; both crashed before this fix.
Documented the two exceptions in parse.md and operator_gtgt.md.
Fixes#5646.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
basic_json::swap() (and the friend swap() and the pre-C++20 std::swap
overload that forward to it) only exchanged m_data.m_type/m_data.m_value,
leaving each value's json_base_class_t subobject in place. This is
inconsistent with the copy and move constructors and copy assignment,
which all carry the base class along with the value, so after
a.swap(b) any metadata stored in a CustomBaseClass ended up attached to
the wrong value. Algorithms that mix swap() with moves, such as
std::sort, scrambled the metadata across the whole container.
Fix the member swap() to also exchange the json_base_class_t subobject
and extend the noexcept specifications of swap() and the friend swap()
accordingly.
Fixes#5653.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix to_msgpack() reading the inactive number union member
basic_json stores number_integer and number_unsigned in a union, and
number_unsigned_t only has to be at least as wide as number_integer_t
(with the default types, both are 64-bit and have the same
representation). When number_integer_t is narrower, write_msgpack()
read the wrong union member in two places:
- The number_unsigned case wrote number_integer's bits instead of
number_unsigned's, silently writing the wrong value whenever it
did not fit in number_integer_t.
- The number_integer case (non-negative branch) picked the encoded
width by comparing number_unsigned's bits, which is undefined
behavior, though the value written was still number_integer's, so
at worst a too-wide encoding was chosen.
Read the active member in both cases, like the other binary writers
(CBOR, UBJSON, BJData, BSON, BON8) already do.
Fixes#5644.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Cast number_integer to number_unsigned_t only once in to_msgpack()
Addresses review comment by @gregmarr.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Copy values before inserting an initializer list into an array
insert(pos, {...}) inserted wrong values when the initializer list
contained const references to elements of the array being inserted
into. json_ref stores only a pointer for a const lvalue, so the
initializer_list_t range passed straight to the array's range insert
aliased the array's own storage; std::vector::insert(pos, first, last)
may move or shift elements before copying from that range, so the
source elements were already stale by the time they were read
(different wrong results on libc++ and libstdc++).
Copy the referenced values into a temporary array_t first, then move
that temporary into place, so the source range never aliases the
array being modified.
Fixes#5656.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Use the reserve_array helper in the initializer_list insert fix
The previous commit called array_t::reserve() directly on the
temporary buffer used to copy an ilist's values before inserting.
std::deque, a documented ArrayType (tests/src/unit-custom-array-type.cpp),
has no reserve(), so insert(pos, initializer_list) no longer compiled
for it. Use the existing detail::reserve_array() SFINAE helper (already
used by the SAX DOM parser) instead, which leaves array types without
reserve() untouched.
Added a regression check that deque_json::insert(pos, {...}) compiles
and handles the aliasing case from #5656.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
For idx == SIZE_MAX, the non-const array operator[] computed the new size as
idx + 1, which wraps to 0. resize(0) then emptied the array, and the
subsequent operator[](idx) on the now-empty vector wrote one element before
its buffer. Every other too-large index (e.g. SIZE_MAX - 1) already went
through resize(), which throws std::length_error and leaves the array
unchanged; SIZE_MAX was the one value for which the overflow bypassed that
safety net.
Add a guard that throws std::length_error before computing idx + 1 when idx
is the largest representable size_type value, so the array is left
unchanged, matching the exception vector::resize() already throws for
smaller (but still too large) indices.
Fixes#5647.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix clear() to also reset the subtype of a binary value
clear() on a binary value cleared the bytes but left the subtype
untouched, so the result was not equal to a default-constructed
binary value even though the documentation says clear() has the same
effect as *this = basic_json(type()). The fix calls
byte_container_with_subtype::clear_subtype() alongside the existing
clear() call.
Extended the "filled binary" clear() test in unit-modifiers.cpp with
a case that uses a subtype, since the existing cases only covered
binary values without one.
Fixes#5669.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix the table alignment in clear.md
Addresses review comment by @gregmarr.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
