* Review and extend the documentation, and check it in CI
A review of all documentation pages found factual errors, dead links,
missing cross-references, and gaps in examples. This fixes them and adds
checks so the same problems are caught automatically.
Fixes:
- wrong signatures and version histories (operator!= C++20 member,
binary() subtype type, get<PointerType>(), JSON_NO_THREAD_LOCAL, ...)
- stale descriptions (number parsing since #5283, UBJSON table, SAX
example that no longer compiled, tsl::ordered_map advice)
- dead internal and external links; repology.org badges (the domain is
suspended) replaced by badges that query the registries directly
- deprecation notes link the migration guide; the guide itself fixed
Additions:
- "See also" sections, cross-references, 25 runnable examples, 12
Mermaid diagrams, new API pages for json_pointer::operator<=> and
byte_container_with_subtype::operator==/!=
- landing page, guides for untrusted input and performance
- "unreleased" badge after versions newer than the latest release
Checks:
- strict documentation build (broken links/anchors fail it); CI and
the publish workflow fetch the full history the build needs
- weekly external link check, Mermaid syntax check in CI
- check_structure.py: example titles, heading levels, alt texts,
header links, docset index coverage; its unused-example check works
again
- all examples produce the same output on every platform
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Keep the customer links that could not be fixed
A dead link on the customers page is still the evidence of where the
use of the library was documented. Keep the original URLs of the entries
without a working replacement (Marne, Cisco Webex Desk Camera, Philips
Hue, CyberArk) and exclude exactly these URLs from the link check.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Correct the duplicate-key recipe's claim about SAX positions
The SAX interface's key() receives no position either; only parse_error()
does. Also note that the recipe does not report the path to the repeated
key (see discussion #5085).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Say the library is available as a single header and mention json_fwd.hpp
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Correct documentation errors found while hunting for bugs
- patch/patch_inplace: list the JSON pointer errors parse_error.106-109
and out_of_range.402/404, and quote the actual parse_error.105 message.
- unflatten: list parse_error.106/107/108 and out_of_range.404.
- to_bson: list out_of_range.415 (binary subtype above 255) and note
that 412 and 415 are new in 3.13.0.
- to_string: state that string_t must be convertible to std::string, also
in the StringType requirements table.
- JSON Lines: a `while (input >> j)` loop also throws after the last value
for concatenated JSON values; show a loop that works for both.
- BON8: a string gets 0xFF only if nothing follows it in the message; a
string at the end of an array or object is ended by 0xFE.
- custom_string_type.hpp: add operator+=(char), which the "Always
required" list asks for (json_pointer::to_string, flatten, unflatten,
and diff did not compile), and an ADL int_to_string for diff and items.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Cache the release headers with functools.lru_cache
Codacy (Pylint) flagged the mutable default argument that header() used
as its cache. functools.lru_cache keeps the same memoization without it.
The script's output is unchanged.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix math formulas rendering as raw TeX in the published docs
The privacy plugin self-hosts MathJax 2.7.0 but drops its
?config=TeX-MML-AM_CHTML query string, so the rehosted script loads no
input jax and the 10 formulas across 6 pages render as raw TeX to
readers. Remove pymdownx.arithmatex and the MathJax extra_javascript
entry, and rewrite the formulas in plain HTML (<sup>, <i>) instead.
This also drops a nine-year-old third-party script from every page.
Part of #5718
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix publish_documentation triggers and persist unneeded git credentials
publish_documentation.yml only triggered on docs/mkdocs/** pushes, but
the site also embeds .github/CODE_OF_CONDUCT.md, CONTRIBUTING.md,
SECURITY.md, cmake/{clang,gcc}_flags.cmake, .clang-tidy,
tools/astyle/.astylerc and tests/fmt_formatter/project/main.cpp via
pymdownx.snippets, so changes to those files never republished the
site. Extend the path filter to cover them, and switch runs-on from
the long-pinned ubuntu-22.04 to ubuntu-latest to match
ci_test_documentation.
Also add persist-credentials: false to the checkouts in ci_icpx and
ci_nvhpc (ubuntu.yml) and msvc-vs2026/msvc-arm64 (windows.yml), none
of which pushes with git, matching every other checkout in these
workflows.
Part of #5718
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix example build: broken debug echo, deprecations hidden for everything
docs/Makefile's debug echo used a space instead of a comma in
$(call cxx_standard ...), so it always printed an empty standard.
Every example was also compiled with -Wno-deprecated-declarations,
which would silently hide an accidental deprecated-API call in any of
them. Factor the duplicated compile flags into EXAMPLE_CPPFLAGS/
EXAMPLE_WARNFLAGS, build with -Werror=deprecated-declarations by
default, and only allow the three examples that intentionally
document deprecated API (the DEPRECATED_EXAMPLES list) to suppress it.
Part of #5718
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix check_structure.py NOLINT parsing and an off-by-one report line
`line.strip("<!-- NOLINT")` followed by `.strip(" -->")` strips any of
the characters in those sets from both ends, not a literal prefix; it
only happened to work for "Examples". A NOLINT'd section name
starting with N, O, L, I or T (e.g. "Notes", "Template parameters",
"Iterator invalidation", "Literals") was silently mangled, so the
suppression did not apply and the checker could report a spurious
missing/misordered section. Parse the comment with a regex instead.
The same fragile strip() pattern was used for heading text; replace it
with a plain prefix slice. Also fix the admonition_title report, which
used the 0-based line index while every other report in the file uses
lineno+1.
Part of #5718
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix docset README's non-existent make target and stale fallback URL
The README told readers to run `make nlohmann_json.docset`, but the
Makefile's targets are `all`, `JSON_for_Modern_C++.docset` and
`install_docset_zeal`; the documented command has failed with "No
rule to make target" since #2967 (2021). Point the README at the real
target and folder name. Info.plist's DashDocSetFallbackURL also still
pointed at the old nlohmann.github.io/json/ URL instead of the
canonical https://json.nlohmann.me/ from mkdocs.yml's site_url.
Leave list_missing_pages/list_removed_paths alone: they may become
redundant once #5638's check_docset() lands, which is a follow-up.
Part of #5718
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Remove two leftover Doxygen-era .link files from the examples directory
parse__iterator_pair.link and parse__pointers.link each held only a
Wandbox "online" permalink from the old Doxygen docs. #3071 deleted
every other .link file in 2021; these two came in through a parallel
PR (#3100) and were never referenced by any page, script or config.
Part of #5718
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix MacPorts CMake example to include its own snippet files
The MacPorts "Example: CMake" block included
integration/homebrew/example.cpp and
integration/homebrew/CMakeLists.txt instead of the MacPorts files
right next to it, a copy-paste slip from the Homebrew section. Nothing
referenced integration/macports/CMakeLists.txt as a result. The page
rendered correctly only because the homebrew, macports and
vcpkg/CMakeLists.txt snippets are byte-identical, so a future edit to
the MacPorts files would not have shown up on the page.
Part of #5718 item 6
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Do not persist git credentials in publish_documentation's checkout
The checkout step in publish_documentation.yml left the default
persist-credentials: true, so GITHUB_TOKEN stayed writable in
.git/config for the rest of the job (zizmor's artipacked finding). The
Deploy documentation step authenticates through its own github_token
input to peaceiris/actions-gh-pages and does not push with the
checked-out credentials, so persist-credentials: false is safe here,
matching every other checkout in the workflow set.
Overlaps #5638, which edits this same checkout step (adds
fetch-depth: 0); expect a rebase conflict there.
Part of #5718 item 2
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Replace list_missing_pages/list_removed_paths with a comm(1)-based diff
The docset Makefile's list_missing_pages ran one sqlite3 query per
mkdocs page, and list_removed_paths nested a loop over all mkdocs pages
inside a loop over all docset index paths (O(n*m) shell iteration).
Issue #5718 item 5 suggested removing or reducing these targets once
#5638's check_docset() lands, but that PR is still open and covers only
API pages and macros, not the full page set these targets check.
Replace the loops with two sorted path lists (DOCSET_PAGE_PATHS from
mkdocs' markdown sources, DOCSET_INDEX_PATHS from the built docset
index) compared with a single comm(1) call each, verified to produce
output identical to the old loops against the current docSet.dsidx.
The sed expression used '#' as its delimiter, which GNU Make reads as
a comment character even inside a variable assignment, truncating the
line and orphaning the closing paren of $(shell ...) ("unterminated
call to function 'shell': missing ')'"). Use '@' as the delimiter
instead.
Part of #5718 item 5
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Check the fuzzers' UBJSON/BJData round-trip invariants in the unit tests
The strongest correctness checks for the UBJSON and BJData writers lived
only in the OSS-Fuzz drivers: anything from_ubjson()/from_bjdata()
returns must serialize with every option combination, parse back, and
re-serialize stably. Those checks only run at OSS-Fuzz, so regressions
surfaced days later as external reports - the same BJData assert pair
was reported five times over three years, and #5494's harness change
was followed by OSS-Fuzz 563659413 within a day.
Add "UBJSON round-trip invariants" and "BJData round-trip invariants"
test cases that run the drivers' checks on a fixed, deterministic corpus
(tests/src/round_trip_corpus.hpp): integer and float boundaries,
non-finite numbers, strings, binary values, optimized containers, deep
nesting, the JData annotated-array matrix, and seeded random containers.
They also check two properties the drivers do not: the first round trip
preserves the value, and re-serializing reproduces the exact bytes. For
BJData both exclude values containing a binary value, which is read back
as an array of integers unless it was written as a Draft 3 optimized
binary array; this carve-out is now documented in bjdata.md. Run against
the headers before #5542, the BJData test fails, including on the shape
from OSS-Fuzz 563659413.
Also document how OSS-Fuzz reports are handled (reference them as
"OSS-Fuzz: <id>", turn the reproducer into a unit test, keep drivers and
unit tests in sync) in tests/fuzzing.md, and link it from the PR
template and the quality assurance page.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Add the OSS-Fuzz reproducers for 474400817 and 474480402 as unit tests
Following the convention added to tests/fuzzing.md, the reproducers of
the two BJData fuzzer asserts tracked since January are now unit tests:
- 474400817 (assert(false)): an empty object _ArraySize_ was written as
the ND-array header length, which from_bjdata() could not read back.
Fixed by #5455.
- 474480402 (to_bjdata(j2, false, false) == vec2): a one-byte Draft 3
binary array is written in Draft 2 mode as a uint8 array and then
re-serialized with the int8 marker. This is the documented exception to
byte stability, not a library bug; OSS-Fuzz closed it after #5494
relaxed the harness to value stability. The test pins the exact bytes
so the exception stays deliberate.
The 563659413 reproducer is already a unit test (#5542). A comment also
ties the existing UBJSON excessive-count test to the timeout OSS-Fuzz
reported for that shape (testcase 6347769435193344).
OSS-Fuzz: 474400817
OSS-Fuzz: 474480402
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix GCC -Weffc++ and -Wuseless-cast warnings in the round-trip corpus
Initialize the atoms in the member initialization list, and drop the cast of
the generator's result, which already is std::size_t on 64-bit Linux.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
write_bjdata_ndarray() encoded a JData-annotated object as a BJData
ND-array whenever its dimensions' product matched _ArrayData_.size(),
which lost information in two ways:
- _ArrayData_ was never required to be an array. null has size 0, any
other scalar has size 1, and iterating an object visits its values, so
e.g. {"_ArraySize_":[1],"_ArrayData_":5} was written as the array [5],
and an object _ArrayData_ came back as an array.
- The reader only restores an annotated object from an ND-array with at
least two non-zero dimensions that is not a 1xN row vector; an empty,
1-D, row-vector, or zero-sized shape is read back as a plain array. The
writer nonetheless emitted ND-array headers for these shapes, so the
annotation was silently dropped.
OSS-Fuzz issue 563659413 hit this in parse_bjdata_fuzzer: an empty binary
_ArraySize_ is written as a plain object and read back as an empty array,
after which {"_ArrayType_":"int16","_ArraySize_":[],"_ArrayData_":null}
was encoded as the ND-array header "[$I#[]" and re-read as [], failing the
harness's value-stability check.
Such objects now fall back to a plain object encoding, which round-trips.
Genuine ND-arrays (two or more positive dimensions, not a 1xN row vector)
are encoded exactly as before. Existing fallback tests that used 1-D
shapes are moved to 2-D shapes so they keep exercising the check they
were written for, and the BJData documentation is updated.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix to_bjdata() emitting unparsable output when _ArraySize_ is not an array
write_bjdata_ndarray() never checked that _ArraySize_ is an array. The shape
is written verbatim as the header length, so a null shape emitted 'Z' and an
object shape emitted '{' after the '#', neither of which from_bjdata()
accepts, and the round-trip guarantee in the BJData docs was broken.
Both slipped through the existing validation: for null, empty() is true so
the element count starts at 0 and the per-dimension loop never runs, and for
an object the loop walks its values, which can satisfy the non-negative
integer check. When _ArrayData_ then matched that count, the writer took the
ndarray path.
Require the shape to be an array, so anything else falls back to a plain
object encoding that round-trips, as the fallback rule in the docs already
specifies.
Signed-off-by: qatcod <79017227+qatcod@users.noreply.github.com>
* Document that _ArraySize_ must be an array in the ndarray requirements
The list at bjdata.md is the exhaustive set of conditions for the ndarray
encoding, but it only implied this one through 'every entry of'.
Signed-off-by: qatcod <79017227+qatcod@users.noreply.github.com>
---------
Signed-off-by: qatcod <79017227+qatcod@users.noreply.github.com>
* Do not write BJData ndarrays whose size overflows std::size_t
write_bjdata_ndarray() multiplied the _ArraySize_ dimensions into a
std::size_t without checking for overflow. A product that wraps around
to a value that happens to match the size of _ArrayData_ passed the
length check, and the writer emitted an ndarray header announcing an
element count that cannot be represented:
{"_ArrayType_":"uint8","_ArraySize_":[9223372036854775808,2],"_ArrayData_":[]}
was encoded as 5b 24 55 23 5b 4d 00 00 00 00 00 00 00 80 69 02 5d, an
ndarray of 2^64 elements followed by no data. Reading that back throws
out_of_range.408 ("excessive ndarray size caused overflow"), so to_bjdata
produced output that from_bjdata rejects. This is reachable by parsing
untrusted JSON and re-encoding it as BJData.
Mirror the overflow check the binary reader already performs, and also
reject a single dimension that does not fit into std::size_t, which the
previous cast silently truncated where std::size_t is narrower than 64
bits. Such objects now fall back to a plain object encoding, which is
what the surrounding type and length validation already does for
annotations it cannot represent, and they round-trip unchanged.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Document when to_bjdata converts a JData annotation to an ND-array
The BJData page described the 1-D vector case as the only situation in
which an object carrying _ArrayType_/_ArraySize_/_ArrayData_ is not
written as a compact ND-array. The writer has always had several other
fallbacks -- an unknown _ArrayType_, a dimension that is not a
non-negative integer, an _ArrayData_ whose length does not match the
product of the dimensions, and elements that are not numbers of the
annotated kind -- all of which cause the value to be serialized as a
regular JSON object instead.
Spell out the conditions, including the size-overflow check added in the
preceding commit, so the documented behavior matches the implementation.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
