From c05347dd545faeff7f14614d35f8831002df2518 Mon Sep 17 00:00:00 2001 From: Niels Lohmann Date: Wed, 9 Sep 2026 09:48:17 +0200 Subject: [PATCH] Undefine the four JSON_HEDLEY_* macros that leak after including json.hpp (#5475) * Undefine the four JSON_HEDLEY_* macros that leak after including json.hpp include/nlohmann/detail/macro_unscope.hpp includes hedley_undef.hpp to #undef every JSON_HEDLEY_* macro so none of them leak into the including translation unit. Four macros were missing from that list and therefore stayed defined after #include : - JSON_HEDLEY_PRAGMA - JSON_HEDLEY_PREDICT_TRUE - JSON_HEDLEY_PREDICT_FALSE - JSON_HEDLEY_CLANG_HAS_DECLSPEC_ATTRIBUTE hedley_undef.hpp is generated (via `make update_hedley`) by grepping hedley.hpp for its own internal `#undef JSON_HEDLEY_X` redefinition guards. JSON_HEDLEY_PRAGMA/PREDICT_TRUE/PREDICT_FALSE have no such guard in upstream Hedley, so they were never picked up. The guard for JSON_HEDLEY_CLANG_HAS_DECLSPEC_ATTRIBUTE also has an upstream typo (`JSON_HEDLEY_CLANG_HAS_DECLSPEC_DECLSPEC_ATTRIBUTE`), so hedley_undef.hpp was undefining the wrong (never-defined) name. Fixes: - include/nlohmann/thirdparty/hedley/hedley_undef.hpp: corrected the DECLSPEC_ATTRIBUTE typo and added the three missing #undef lines, keeping the file's alphabetical ordering. - Makefile (update_hedley target): changed hedley_undef.hpp generation to extract macro names directly from every `#define JSON_HEDLEY_...` in hedley.hpp instead of from existing `#undef` guards, so a future `make update_hedley` run undefines every macro Hedley actually defines, even ones without a pre-existing redefinition guard. This was not run in this PR (it would also pull in an unrelated upstream Hedley sync); hedley_undef.hpp was hand-patched instead and single_include was regenerated with `make amalgamate`. - tests/src/unit-no-macro-leak.cpp: new regression test (picked up automatically by tests/CMakeLists.txt's existing unit-*.cpp glob) that includes json.hpp and then #ifdef/#error-checks every JSON_HEDLEY_* macro name, so any future leak of any of the 151 vendored macros fails the build, not just the four fixed here. Fixes #5408. Signed-off-by: Niels Lohmann * Derive the JSON_HEDLEY_* leak-check test from hedley.hpp at build time tests/src/unit-no-macro-leak.cpp previously hardcoded a static list of ~151 #ifdef/#error checks, one per JSON_HEDLEY_* macro name known at the time it was written. That list would silently go stale the next time `make update_hedley` pulls in a vendor update that adds, removes, or renames a macro, since nothing would force it to be regenerated. Add cmake/scripts/gen_hedley_undef_check.cmake, which derives the full list of JSON_HEDLEY_* macro names directly from include/nlohmann/thirdparty/hedley/hedley.hpp: - tests/CMakeLists.txt uses it (MODE=checks) to (re)generate hedley_undef_checks.inc at configure and build time, and wires the generating custom target as a dependency of the test-no-macro-leak_cpp* targets so it can never build against a stale copy. unit-no-macro-leak.cpp now just #include-s the generated file inside its TEST_CASE instead of carrying the checks itself. - The Makefile's `update_hedley` target now delegates hedley_undef.hpp generation to the same script (MODE=undef, new `update_hedley_undef` target), so the vendored header, the generated #undef list, and the generated test checks are all derived from the same extraction logic and cannot drift apart. This mirrors the approach taken independently in #5415 for the same issue (#5408), credited there to a self-regenerating mechanism that "can never drift again" -- ported into this branch instead of the static list originally proposed here. Verified with a local CMake configure + build + ctest, both against include/ (JSON_MultipleHeaders=ON) and against the amalgamated single_include/nlohmann/json.hpp (JSON_MultipleHeaders=OFF), and by temporarily deleting a #undef line from hedley_undef.hpp to confirm the generated test actually fails on a real leak. Signed-off-by: Niels Lohmann * Fix REUSE compliance failure in gen_hedley_undef_check.cmake The generated file's embedded banner contains the literal text 'SPDX-License-Identifier: MIT' as part of the *content* being written to hedley_undef.hpp, not as this .cmake script's own REUSE header (it is already covered by the blanket 'Files: *' rule in .reuse/dep5). The reuse tool matched that embedded line as an SPDX tag for the script itself and failed to parse the trailing 'MIT\n")' as a valid SPDX License Expression, breaking ci_reuse_compliance. Wrap the embedded banner in REUSE-IgnoreStart/REUSE-IgnoreEnd comments, as recommended by the tool's own diagnostic output. Signed-off-by: Niels Lohmann --------- Signed-off-by: Niels Lohmann --- Makefile | 21 +++- cmake/scripts/gen_hedley_undef_check.cmake | 112 ++++++++++++++++++ .../thirdparty/hedley/hedley_undef.hpp | 5 +- single_include/nlohmann/json.hpp | 5 +- tests/CMakeLists.txt | 53 +++++++++ tests/src/unit-no-macro-leak.cpp | 34 ++++++ 6 files changed, 225 insertions(+), 5 deletions(-) create mode 100644 cmake/scripts/gen_hedley_undef_check.cmake create mode 100644 tests/src/unit-no-macro-leak.cpp diff --git a/Makefile b/Makefile index d99d6f5f3..e1a1d2b75 100644 --- a/Makefile +++ b/Makefile @@ -1,4 +1,4 @@ -.PHONY: pretty clean ChangeLog.md release +.PHONY: pretty clean ChangeLog.md release update_hedley update_hedley_undef ########################################################################## # configuration @@ -41,6 +41,8 @@ all: @echo "fuzz_testing_ubjson - prepare fuzz testing of the UBJSON parser" @echo "pretty - beautify code with Artistic Style" @echo "run_benchmarks - build and run benchmarks" + @echo "update_hedley - download Hedley and regenerate hedley.hpp / hedley_undef.hpp" + @echo "update_hedley_undef - rebuild hedley_undef.hpp from the JSON_HEDLEY_* #define names in hedley.hpp" ########################################################################## @@ -241,11 +243,24 @@ update_hedley: rm -f include/nlohmann/thirdparty/hedley/hedley.hpp include/nlohmann/thirdparty/hedley/hedley_undef.hpp curl https://raw.githubusercontent.com/nemequ/hedley/master/hedley.h -o include/nlohmann/thirdparty/hedley/hedley.hpp $(SED) -i 's/HEDLEY_/JSON_HEDLEY_/g' include/nlohmann/thirdparty/hedley/hedley.hpp - grep "[[:blank:]]*#[[:blank:]]*undef" include/nlohmann/thirdparty/hedley/hedley.hpp | grep -v "__" | sort | uniq | $(SED) 's/ //g' | $(SED) 's/undef/undef /g' > include/nlohmann/thirdparty/hedley/hedley_undef.hpp $(SED) -i '1s/^/#pragma once\n\n/' include/nlohmann/thirdparty/hedley/hedley.hpp - $(SED) -i '1s/^/#pragma once\n\n/' include/nlohmann/thirdparty/hedley/hedley_undef.hpp + $(MAKE) update_hedley_undef $(MAKE) amalgamate +# Rebuild hedley_undef.hpp from every JSON_HEDLEY_* name that hedley.hpp +# #defines. Hedley does not #undef all of its public macros internally (see +# #5408), so grepping those #undef lines misses names such as +# JSON_HEDLEY_PRAGMA. cmake/scripts/gen_hedley_undef_check.cmake is the +# single source of truth for this extraction (tests/CMakeLists.txt uses the +# same script, in MODE=checks, to generate the matching leak-check test), so +# the vendored header, the generated #undef list, and the regression test +# cannot drift apart. +update_hedley_undef: + cmake -DHEDLEY_HPP=include/nlohmann/thirdparty/hedley/hedley.hpp \ + -DOUTPUT=include/nlohmann/thirdparty/hedley/hedley_undef.hpp \ + -DMODE=undef \ + -P cmake/scripts/gen_hedley_undef_check.cmake + ########################################################################## # serve_header.py ########################################################################## diff --git a/cmake/scripts/gen_hedley_undef_check.cmake b/cmake/scripts/gen_hedley_undef_check.cmake new file mode 100644 index 000000000..fc8cfec2c --- /dev/null +++ b/cmake/scripts/gen_hedley_undef_check.cmake @@ -0,0 +1,112 @@ +# Shared extractor for the JSON_HEDLEY_* macro names defined in hedley.hpp. +# +# Every macro that hedley.hpp #defines must be #undef-ed again once json.hpp +# has been fully processed (see include/nlohmann/detail/macro_unscope.hpp +# and https://github.com/nlohmann/json/issues/5408). Deriving the macro list +# straight from hedley.hpp here -- instead of hand-maintaining it in two +# places -- means hedley_undef.hpp and the regression test that checks for +# leaked macros can never drift apart, even after a future `make +# update_hedley` pulls in new macros from upstream Hedley. +# +# MODE=undef (default): write hedley_undef.hpp (SPDX header, #pragma once, +# one #undef per macro name) -- used by `make update_hedley_undef` +# MODE=checks: write one #ifdef/FAIL_CHECK/#endif per macro name, +# meant to be #include-d inside a TEST_CASE -- used by +# tests/CMakeLists.txt to (re)generate the include for +# tests/src/unit-no-macro-leak.cpp +# +# Required variables: +# HEDLEY_HPP path to include/nlohmann/thirdparty/hedley/hedley.hpp +# OUTPUT path of the file to (over)write +# Optional: +# MODE "undef" (default) or "checks" + +if(NOT DEFINED HEDLEY_HPP OR NOT DEFINED OUTPUT) + message(FATAL_ERROR "HEDLEY_HPP and OUTPUT must be set") +endif() + +if(NOT EXISTS "${HEDLEY_HPP}") + message(FATAL_ERROR "Hedley header not found: ${HEDLEY_HPP}") +endif() + +if(NOT DEFINED MODE) + set(MODE undef) +endif() + +if(NOT MODE STREQUAL "undef" AND NOT MODE STREQUAL "checks") + message(FATAL_ERROR "MODE must be undef or checks, got: ${MODE}") +endif() + +# Line-anchored, like `grep -oE "^[[:blank:]]*#[[:blank:]]*define[[:blank:]]+JSON_HEDLEY_[A-Za-z0-9_]+"`. +# Unanchored matching would also pick up JSON_HEDLEY_* mentions inside +# comments or string literals elsewhere in the file, which must not turn +# into #undef lines. +file(STRINGS "${HEDLEY_HPP}" hedley_lines) +set(macro_names) +foreach(line IN LISTS hedley_lines) + if("${line}" MATCHES "^[ \t]*#[ \t]*define[ \t]+(JSON_HEDLEY_[A-Za-z0-9_]+)") + list(APPEND macro_names "${CMAKE_MATCH_1}") + endif() +endforeach() + +if(NOT macro_names) + message(FATAL_ERROR "No JSON_HEDLEY_* macros found in ${HEDLEY_HPP}") +endif() + +list(REMOVE_DUPLICATES macro_names) +# Lexicographic, locale-independent (ASCII-only names) -- matches `LC_ALL=C sort`. +list(SORT macro_names COMPARE STRING) +list(LENGTH macro_names macro_count) + +set(generated "") +if(MODE STREQUAL "undef") + # Same banner `make update_hedley_undef` would stamp by hand, so the + # recipe is self-contained and its output is byte-stable across reruns. + # The embedded SPDX tags below are part of the *generated* file's + # content, not a REUSE header for this .cmake script itself (which is + # already covered by the blanket "Files: *" rule in .reuse/dep5) -- keep + # them wrapped in REUSE-IgnoreStart/End so `reuse lint` does not try to + # parse "MIT\n")" as this file's own SPDX-License-Identifier value. + # REUSE-IgnoreStart + string(APPEND generated "// __ _____ _____ _____\n") + string(APPEND generated "// __| | __| | | | JSON for Modern C++\n") + string(APPEND generated "// | | |__ | | | | | | version 3.12.0\n") + string(APPEND generated "// |_____|_____|_____|_|___| https://github.com/nlohmann/json\n") + string(APPEND generated "//\n") + string(APPEND generated "// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann \n") + string(APPEND generated "// SPDX-License-Identifier: MIT\n") + # REUSE-IgnoreEnd + string(APPEND generated "\n") + string(APPEND generated "#pragma once\n") + string(APPEND generated "\n") + foreach(name IN LISTS macro_names) + string(APPEND generated "#undef ${name}\n") + endforeach() +else() + string(APPEND generated "// This file is generated by cmake/scripts/gen_hedley_undef_check.cmake\n") + string(APPEND generated "// from include/nlohmann/thirdparty/hedley/hedley.hpp. Do not edit it by\n") + string(APPEND generated "// hand -- it is regenerated on every build. ${macro_count} macros checked.\n\n") + foreach(name IN LISTS macro_names) + string(APPEND generated "#ifdef ${name}\n") + string(APPEND generated " FAIL_CHECK(\"${name} leaked after including nlohmann/json.hpp\");\n") + string(APPEND generated "#endif\n") + endforeach() +endif() + +get_filename_component(output_dir "${OUTPUT}" DIRECTORY) +if(output_dir) + file(MAKE_DIRECTORY "${output_dir}") +endif() + +# Avoid rewriting the file (and busting downstream incremental rebuilds) +# when the content has not actually changed. +set(write_output TRUE) +if(EXISTS "${OUTPUT}") + file(READ "${OUTPUT}" existing_content) + if(existing_content STREQUAL generated) + set(write_output FALSE) + endif() +endif() +if(write_output) + file(WRITE "${OUTPUT}" "${generated}") +endif() diff --git a/include/nlohmann/thirdparty/hedley/hedley_undef.hpp b/include/nlohmann/thirdparty/hedley/hedley_undef.hpp index 1b8bd4338..e4d9838cb 100644 --- a/include/nlohmann/thirdparty/hedley/hedley_undef.hpp +++ b/include/nlohmann/thirdparty/hedley/hedley_undef.hpp @@ -17,7 +17,7 @@ #undef JSON_HEDLEY_CLANG_HAS_ATTRIBUTE #undef JSON_HEDLEY_CLANG_HAS_BUILTIN #undef JSON_HEDLEY_CLANG_HAS_CPP_ATTRIBUTE -#undef JSON_HEDLEY_CLANG_HAS_DECLSPEC_DECLSPEC_ATTRIBUTE +#undef JSON_HEDLEY_CLANG_HAS_DECLSPEC_ATTRIBUTE #undef JSON_HEDLEY_CLANG_HAS_EXTENSION #undef JSON_HEDLEY_CLANG_HAS_FEATURE #undef JSON_HEDLEY_CLANG_HAS_WARNING @@ -108,7 +108,10 @@ #undef JSON_HEDLEY_PELLES_VERSION_CHECK #undef JSON_HEDLEY_PGI_VERSION #undef JSON_HEDLEY_PGI_VERSION_CHECK +#undef JSON_HEDLEY_PRAGMA #undef JSON_HEDLEY_PREDICT +#undef JSON_HEDLEY_PREDICT_FALSE +#undef JSON_HEDLEY_PREDICT_TRUE #undef JSON_HEDLEY_PRINTF_FORMAT #undef JSON_HEDLEY_PRIVATE #undef JSON_HEDLEY_PUBLIC diff --git a/single_include/nlohmann/json.hpp b/single_include/nlohmann/json.hpp index f247453e8..68691769c 100644 --- a/single_include/nlohmann/json.hpp +++ b/single_include/nlohmann/json.hpp @@ -27090,7 +27090,7 @@ struct formatter // NOLINT(cert-dcl58-c #undef JSON_HEDLEY_CLANG_HAS_ATTRIBUTE #undef JSON_HEDLEY_CLANG_HAS_BUILTIN #undef JSON_HEDLEY_CLANG_HAS_CPP_ATTRIBUTE -#undef JSON_HEDLEY_CLANG_HAS_DECLSPEC_DECLSPEC_ATTRIBUTE +#undef JSON_HEDLEY_CLANG_HAS_DECLSPEC_ATTRIBUTE #undef JSON_HEDLEY_CLANG_HAS_EXTENSION #undef JSON_HEDLEY_CLANG_HAS_FEATURE #undef JSON_HEDLEY_CLANG_HAS_WARNING @@ -27181,7 +27181,10 @@ struct formatter // NOLINT(cert-dcl58-c #undef JSON_HEDLEY_PELLES_VERSION_CHECK #undef JSON_HEDLEY_PGI_VERSION #undef JSON_HEDLEY_PGI_VERSION_CHECK +#undef JSON_HEDLEY_PRAGMA #undef JSON_HEDLEY_PREDICT +#undef JSON_HEDLEY_PREDICT_FALSE +#undef JSON_HEDLEY_PREDICT_TRUE #undef JSON_HEDLEY_PRINTF_FORMAT #undef JSON_HEDLEY_PRIVATE #undef JSON_HEDLEY_PUBLIC diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt index 4383b582c..2d0aaaf70 100644 --- a/tests/CMakeLists.txt +++ b/tests/CMakeLists.txt @@ -125,6 +125,51 @@ json_test_set_test_options(test-unicode4 TEST_PROPERTIES TIMEOUT 3000) # add unit tests ############################################################################# +# Generate the leak checks for every JSON_HEDLEY_* macro defined in +# hedley.hpp; tests/src/unit-no-macro-leak.cpp #include-s the result after +# nlohmann/json.hpp (see issue #5408). Using the shared +# cmake/scripts/gen_hedley_undef_check.cmake script (also used by `make +# update_hedley_undef`) instead of a hand-maintained list of macro names +# means this test can never go stale after a future `make update_hedley`. +set(hedley_hpp "${PROJECT_SOURCE_DIR}/include/nlohmann/thirdparty/hedley/hedley.hpp") +set(hedley_undef_check_script "${PROJECT_SOURCE_DIR}/cmake/scripts/gen_hedley_undef_check.cmake") +set(hedley_undef_checks "${PROJECT_BINARY_DIR}/include/hedley_undef_checks.inc") + +# Reconfigure whenever the vendored header or the generator script changes, +# so a `cmake --build` after `make update_hedley` does not silently keep a +# stale generated file around. +set_property(DIRECTORY APPEND PROPERTY CMAKE_CONFIGURE_DEPENDS + "${hedley_hpp}" + "${hedley_undef_check_script}") + +# Generate once at configure time, so the very first build (before any +# custom-command build step has run) already has an up-to-date file. +execute_process( + COMMAND ${CMAKE_COMMAND} + "-DHEDLEY_HPP=${hedley_hpp}" + "-DOUTPUT=${hedley_undef_checks}" + -DMODE=checks + -P "${hedley_undef_check_script}" + RESULT_VARIABLE hedley_undef_check_result +) +if(NOT hedley_undef_check_result EQUAL 0) + message(FATAL_ERROR "Failed to generate ${hedley_undef_checks}") +endif() + +# Also (re)generate as a build step, so an incremental build after editing +# hedley.hpp without a full reconfigure still picks up the change. +add_custom_command( + OUTPUT "${hedley_undef_checks}" + COMMAND ${CMAKE_COMMAND} + "-DHEDLEY_HPP=${hedley_hpp}" + "-DOUTPUT=${hedley_undef_checks}" + -DMODE=checks + -P "${hedley_undef_check_script}" + DEPENDS "${hedley_hpp}" "${hedley_undef_check_script}" + COMMENT "Generating Hedley undef leak checks" + VERBATIM) +add_custom_target(generate_hedley_undef_checks DEPENDS "${hedley_undef_checks}") + if("${JSON_TestStandards}" STREQUAL "") set(test_cxx_standards 11 14 17 20 23) unset(test_force) @@ -163,6 +208,14 @@ foreach(file ${files}) json_test_add_test_for(${file} MAIN test_main CXX_STANDARDS ${test_cxx_standards} ${test_force}) endforeach() +# tests/src/unit-no-macro-leak.cpp #include-s the generated leak-check file, +# so its test targets must be built after generate_hedley_undef_checks. +foreach(cxx_standard ${test_cxx_standards}) + if(TARGET test-no-macro-leak_cpp${cxx_standard}) + add_dependencies(test-no-macro-leak_cpp${cxx_standard} generate_hedley_undef_checks) + endif() +endforeach() + if(json_32bit_test_only) # Skip all other tests in this file return() diff --git a/tests/src/unit-no-macro-leak.cpp b/tests/src/unit-no-macro-leak.cpp new file mode 100644 index 000000000..c5184c52f --- /dev/null +++ b/tests/src/unit-no-macro-leak.cpp @@ -0,0 +1,34 @@ +// __ _____ _____ _____ +// __| | __| | | | JSON for Modern C++ +// | | |__ | | | | | | version 3.12.0 +// |_____|_____|_____|_|___| https://github.com/nlohmann/json +// +// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann +// SPDX-License-Identifier: MIT + +// This file makes sure that none of the internal JSON_HEDLEY_* macros (vendored +// from https://nemequ.github.io/hedley/, see +// include/nlohmann/thirdparty/hedley/hedley.hpp) leak into the including +// translation unit. include/nlohmann/detail/macro_unscope.hpp is supposed to +// #undef every JSON_HEDLEY_* macro (via hedley_undef.hpp) once json.hpp has +// been fully processed. See https://github.com/nlohmann/json/issues/5408, +// where JSON_HEDLEY_PRAGMA, JSON_HEDLEY_PREDICT_TRUE, JSON_HEDLEY_PREDICT_FALSE, +// and JSON_HEDLEY_CLANG_HAS_DECLSPEC_ATTRIBUTE escaped this cleanup because +// hedley_undef.hpp had no matching #undef for them. +// +// hedley_undef_checks.inc (included below) is generated at CMake configure/ +// build time by cmake/scripts/gen_hedley_undef_check.cmake, which derives the +// full list of JSON_HEDLEY_* macro names directly from hedley.hpp. That way +// this test covers every macro Hedley actually defines -- not a hardcoded +// snapshot that would silently go stale the next time `make update_hedley` +// runs -- and can never drift from the vendored header. + +#include "doctest_compatibility.h" + +#include + +TEST_CASE("JSON_HEDLEY macros do not leak after including json.hpp") +{ +#include "hedley_undef_checks.inc" + CHECK(true); // keep an assertion when nothing leaked +}