mirror of
https://github.com/nlohmann/json.git
synced 2026-09-16 07:14:20 +00:00
fix/insert-array-iterator-check
393 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Niels Lohmann
|
d0a53f5435 |
Reject array insert(pos, first, last) iterators not pointing into an array
The array-range insert() overload checked that pos fits the current value and that first/last share the same owning value, but never verified that value is itself an array. Passing iterators from an object, a primitive, or null handed value-initialized (singular) std::vector iterators straight to array_t::insert(), which is undefined behavior. Add the missing is_array() check, mirroring the equivalent check already present in the object-range insert() overload. Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
bf1c726db4 |
Preserve diff()'s original op ordering and fix a slow-path deletion gap
Splitting removed-key detection and common-key recursion into separate
passes (for the earlier lookup-count fix) changed the emitted patch's
op order: all "remove" ops now came before all recursive per-key diffs,
instead of interleaved in source's iteration order as the original
implementation did. This broke docs/mkdocs/docs/examples/diff.output's
exact-match CI check (ci_test_examples) even though the patch was still
semantically correct.
Defer "remove" emission into the same walk that does the recursive
diffs, so common keys and deleted keys are interleaved in source order
again, matching historical output.
While restructuring that walk, the reordering ("slow path") branch was
only emitting "remove" for keys common to both objects, never for keys
present in source but genuinely absent from target -- a key deleted
alongside an actual reorder would silently survive the patch. Fixed by
removing every source key in the slow path (both deleted and common
keys need removing there; common keys are then re-added in target's
order). Verified with a targeted reorder+deletion case and a fresh
20,000-case round-trip fuzz run (0 failures).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
|
||
|
Niels Lohmann
|
80b50ee987 |
Avoid redundant lookups in diff()'s object-order tracking
The previous fix for ordered_json member order re-derived common-key order and suffix information with extra target.find()/source.find() calls layered on top of the pre-existing removed/added-key passes, instead of reusing those same passes. This roughly tripled the number of map lookups per diff() call for every object, including plain `json`, where the reordering path is never taken. Piggyback the order tracking (and the "add" op construction for new keys) onto the two passes the algorithm already needs to detect removed/added keys, and walk the fast path's recursion in lockstep with the precomputed common-key list instead of re-querying `target`. This restores diff() to its pre-existing lookup count; benchmarked at n=1000 keys, ordered_json::diff() was roughly 2x slower than baseline before this change and is back within noise of baseline after it. Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
29ad7b02db |
Make diff() account for member order in ordered_json objects
diff() compared source/target objects purely by key set, ignoring relative member order. For ordered_json (insertion-ordered, vector- backed object_t), two objects that differ only in member order are unequal via operator==, but diff() never emitted any patch operation to fix the order, so source.patch(diff(source, target)) == target could fail to hold. Fix by detecting when common keys appear in a different relative order in source vs. target (or when a new key would need to land somewhere other than the end), and in that case removing and re-adding the affected keys in target's order, which relies on patch()'s "add" op appending new keys at the end of an ordered_map. For plain json (std::map-backed, always key-sorted iteration) this is a no-op and the original minimal per-key diff path is unchanged. Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
c72f37a40d |
Support custom object/array types and improve template parameter handling (#5443)
* docs: document the implicit requirements on basic_json's template parameters The requirements that basic_json places on its eleven template parameters were only implied by how the library uses the resulting object_t, array_t, string_t, etc. Consumers had to discover them by trial and error. Add "Template Parameter Requirements" collecting them, split into what is always required and what is only required when a particular part of the API is instantiated. Notable findings that were previously undocumented: - ObjectType must provide a key_compare member type (actual_object_comparator names object_t::key_compare in both arms of a std::conditional), and its third template parameter is used as a comparator, so std::unordered_map cannot be used without a wrapper. - ArrayType must provide capacity() -- push_back(), emplace_back(), operator+=(), and operator[](size_type) call it unconditionally -- and needs random-access iterators, so std::deque and std::list do not work. - StringType needs contiguous, null-terminated data(), a one-byte value_type, and either assignability from std::to_string or an ADL int_to_string(). - NumberFloatType must be float, double, or long double for parsing and serialization; the integer types must satisfy std::is_integral. - AllocatorType must be stateless, support incomplete types, and use plain pointers. - BooleanType and the number types are union members and must be trivial. Link the new page from the basic_json overview, the types feature page, and the individual type alias pages, and correct the container examples given for ObjectType (std::unordered_map) and ArrayType (std::list), which do not work. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018hxZxz8svM54c6ATEvXp5E Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Fix object_comparator_t for object types without key_compare detail::actual_object_comparator selected between object_t::key_compare and default_object_comparator_t with std::conditional. Both type arguments of std::conditional are named eagerly, so object_t::key_compare had to exist regardless of the condition, and the has_key_compare guard added in 3.11.0 never took effect: any ObjectType without a key_compare member type failed to compile while instantiating basic_json itself. Use detected_or_t instead, which resolves through a SFINAE partial specialization and only names object_t::key_compare when it exists. The selected type is unchanged for every object type that compiled before, so object_comparator_t -- a public member type -- keeps its meaning and ABI. has_key_compare had no other users and is removed. Add a regression test using an adapter around std::unordered_map, which has no key_compare; it fails to compile without this change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018hxZxz8svM54c6ATEvXp5E Signed-off-by: Niels Lohmann <mail@nlohmann.me> * docs: list the types that are known to work for each template parameter Follow up on the template parameter requirements page: state, for every template parameter, which concrete types work and where they stop working. Each entry was verified by compiling and running a common workload (DOM access, dump, parse, CBOR/MessagePack round-trip, flatten, hash) against that instantiation. Findings worth calling out: - ObjectType no longer needs a key_compare member type, so the std::unordered_map adapter only has to restore the template argument order. A hash-ordered ObjectType works everywhere except unflatten(), which reconstructs an array only when it meets the reference token 0 before the other indices. - ArrayType: std::deque works when wrapped to add capacity(); std::list does not. - StringType: std::pmr::string and std::basic_string with a custom allocator compile for the DOM, dump, and parse, but not for the binary readers, flatten, or diff, because the library assigns std::string values to string_t and int_to_string cannot be overloaded for a type in namespace std. - NumberFloatType: long double works for dump and parse but not for the binary formats, which have no encoding for it. - BinaryType: std::vector<std::byte> supports assignment, get, and the binary formats, but neither dump nor std::hash<basic_json>. Also record the object_comparator_t fix in its version history. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018hxZxz8svM54c6ATEvXp5E Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Fix unflatten and binary dumping for non-default configurations unflatten() decided between array and object by looking at the first reference token it happened to see for a node: it started an array only when that token was 0. With a sorted object type the token 0 always arrives first, so the result was correct by accident; with an object type whose iteration order is unspecified, {"/c/2":3,"/c/1":2,"/c/0":1} unflattened to an object with the keys "0", "1", and "2" instead of an array. Collect the pointer prefixes that have a reference token 0 among their children before building the result, and let get_and_create() consult that set. The outcome is now independent of the iteration order and matches, for every input, what a sorted object type produced before: a value is restored as an array if and only if one of its keys is 0. Iterating the flattened object in a different order would have been simpler, but it would have changed the key order of the result for insertion-ordered object types. The serializer, std::hash, and the UBJSON writer converted the elements of a binary value to an integer implicitly, which does not compile for a BinaryType whose value type is std::byte, and which made dump() write the bytes of a signed value type as negative numbers. Convert to std::uint8_t explicitly in all three places, so every byte type dumps as 0..255. The default std::vector<std::uint8_t> configuration is unaffected. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018hxZxz8svM54c6ATEvXp5E Signed-off-by: Niels Lohmann <mail@nlohmann.me> * docs: note which Abseil containers can be used as template arguments Checked against Abseil release 20250127.0 with the same workload as the other entries on the page (DOM access, dump, parse, CBOR/MessagePack/UBJSON round-trip, flatten, hash), with and without JSON_DIAGNOSTICS. absl::flat_hash_map and absl::node_hash_map work as ObjectType through an adapter that restores the template argument order and makes erase(iterator) return the following iterator, which Abseil's returns as void. The page now carries that adapter, and notes that absl::flat_hash_map does not keep references to the mapped values valid across insertions while absl::node_hash_map does. Both have a capacity() member, so JSON_DIAGNOSTICS already refreshes the parent pointers conservatively for them. absl::btree_map and absl::InlinedVector cannot be used at all: object_t and array_t are formed while basic_json is still incomplete, and both inspect their value type at class scope. std::map and std::vector are required by the standard to tolerate this, third-party containers generally are not, so the page states the constraint on its own rather than only per container. absl::InlinedVector does work as BinaryType, where it is instantiated with a complete type. absl::FixedArray and absl::Cord are not usable. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018hxZxz8svM54c6ATEvXp5E Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Relax the ArrayType and ObjectType requirements Two requirements forced users of otherwise suitable containers to write a wrapper, and neither was load-bearing. array_t::capacity() was read in push_back(), emplace_back(), operator+=(), and operator[](size_type), but set_parent() only looks at the value under JSON_DIAGNOSTICS; without diagnostics it was computed and discarded. Read it through array_capacity(), which reports unknown_size() when diagnostics are off or when the array type has no capacity() at all, and treat an unknown capacity as "the elements may have moved" so the parent pointers are refreshed conservatively. std::deque now works as ArrayType, in both builds, and capacity() is no longer named at all in a default build. Since the capacity is now only meaningful for array insertions, it moves out of set_parent() into set_parent_after_array_insert(). basic_json::erase(iterator) assigned the object's erase() return value, which requires the container to return the following iterator. Abseil's hash maps return void to avoid computing a successor the caller may not need. Detect that and compute the successor before erasing; containers that return an iterator, including the vector-backed ordered_map where a precomputed successor would be wrong, keep the existing path. Together these leave an Abseil hash map needing only an alias that restores the template argument order, and no adapter at all for std::deque. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018hxZxz8svM54c6ATEvXp5E Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Do not require string_t to be convertible from std::string Three places built a std::string and handed it to something expecting a string_t: the UBJSON high-precision number reader, which every binary reader instantiates, and the BSON writer's array element size calculation and write. That silently required string_t to be implicitly convertible from std::string, which std::string itself and types with a string_view conversion satisfy, but many string types do not. Construct the string_t explicitly from the data and size, which the requirements already cover. This makes boost::container::string, eastl::string, std::pmr::string, and std::basic_string with a custom allocator work as StringType, none of which could previously be used with any binary format. Add binary format coverage to the alt_string test, which had none, including a UBJSON high-precision number -- the case that goes through the reader path. BSON stays uncovered there: it additionally needs string_t::find(value_type), which alt_string does not provide. Also record which containers from Boost, Abseil, and EASTL work for each template parameter, and correct two claims: std::pmr::string is usable after this change, and tsl::ordered_map is not usable at all, because its iterators expose the mapped value as const while basic_json modifies it in place. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * docs: record compatibility for the common header-only hash maps ankerl::unordered_dense (map and segmented_map), phmap (flat_hash_map and node_hash_map), and robin_hood::unordered_flat_map all work as ObjectType through the same adapter as Abseil's and Boost's hash maps, which only has to restore the template argument order. phmap::btree_map and robin_hood::unordered_node_map do not: like the other btree containers they require a complete value type. Note that none of these hash maps defines key_compare, so every one of them depends on object_comparator_t falling back to default_object_comparator_t. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * docs: record Folly and the remaining vector replacements Folly works, with the caveat that its headers need C++20: folly::fbstring as StringType, folly::fbvector and folly::small_vector as ArrayType, folly::fbvector<std::uint8_t> as BinaryType, and folly::F14NodeMap as ObjectType through the usual argument-order adapter. folly::F14FastMap is the exception and requires a complete value type. For ArrayType, boost::container::devector, boost::container::static_vector (within its fixed capacity), and std::pmr::vector work as well. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * docs: cover fifo_map, gtl, folly::sorted_vector_map, and Qt nlohmann::fifo_map works through the adapter that has always been documented for it, and preserves the insertion order. Restore its mention in the object order page, which was dropped together with the tsl::ordered_map one: unlike ordered_map it keeps a lookup index, so it is the insertion-ordered option without the quadratic cost. gtl::flat_hash_map and folly::sorted_vector_map work as well, the latter through an alias that drops the allocator, whose value type it disagrees on. gtl::btree_map does not, for the same reason as the other btree containers. None of the Qt containers can be used, each for its own reason: QMap has no value_type, QHash iterators yield the mapped value rather than a pair, QList has no max_size(), QByteArray spells empty() as isEmpty(), and QString is UTF-16. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * docs: qualify the std::pmr::string support claim Listing std::pmr::string as fully supported was an overclaim: it was only ever checked with the default memory resource, which is not what PMR is for. basic_json cannot be given an allocator or a memory resource, so a pmr string inside a value always allocates from std::pmr::get_default_resource(), and assigning an arena-backed string into a value silently drops its resource, because polymorphic_allocator does not propagate on copy construction. Passing polymorphic_allocator as AllocatorType does not compile either. Only the process-global set_default_resource() redirects these allocations. Say so, and separate the row from std::basic_string with a custom stateless allocator, which is unaffected. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * docs: remove a duplicated StringType compatibility section The StringType section carried two 'Compatible types' tables and two copies of the reference-implementation tip. The second table was a stale copy from before the binary format string fixes and still listed std::pmr::string and std::basic_string with a custom allocator as unusable, contradicting the corrected table a few lines above it, and it dragged along the old explanation that blamed int_to_string. Drop the stale copy and put the surviving table before the notes, so the 'see below' in the std::pmr::string row points forwards. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * docs: correct the template parameter requirements after independent verification Every claim on the page was re-checked by compiling and running it, including the rows that say a type cannot be used, which were checked to fail for the documented reason and not merely to fail. Twenty-four claims were wrong. The most consequential: the incomplete-type constraint applies to ObjectType only. object_t is instantiated inside the class definition, because it is probed for key_compare; array_t is only named there and is not instantiated until basic_json is complete. So eastl::vector, QList and QVector are not excluded by incomplete types at all -- they simply have no max_size() -- and absl::InlinedVector is excluded for a subtler reason of its own. Further corrections: ObjectType does not need erase(key), which has a fallback, but does need at(key) for UBJSON output; only == and < are used, or == and <=> under C++20, not all six; the documented adapter does not fit ankerl or robin_hood. ArrayType needs no initializer-list insert, and value_type, the (count, value) constructor and swappability are per-function, not always. BinaryType needs a range insert for CBOR indefinite-length byte strings and does not need push_back. StringType needs append(const StringType&) unconditionally, and does not need operator!= or operator== against const char*; empty(), resize(n) and reserve(n) are per-subsystem; int_to_string is needed by diff, items and std::hash rather than by JSON Pointer or flatten. BooleanType must be implicitly convertible from bool, and JSONSerializer's second parameter need not carry a default. std::pmr::string was wrong in the other direction this time: a moved-in string does keep its memory resource, and later growth allocates from it. Only copies land on the default resource. Five requirement violations are not caught at compile time rather than the two the page claimed; they are now listed together up front. Split every compatibility table into what works and what does not, as the reasons in the second half are the useful part. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Reduce the string_t and array_t members the library requires Several members were required only because of how the library happened to be written, not because the functionality needs them. Dropping them widens the set of usable string and array types, and one of them was also a performance problem. string_t: - c_str() is gone. Every call site already knew the length and passed it along, so data() is enough. The one place that did not, the diagnostics path in exceptions.hpp, now builds the token from data() and size(), which also stops it from truncating keys that contain a null byte. - back() is gone; the serializer indexes the last character instead. - find(str, pos), replace(), and substr() are gone. escape() and unescape() rebuilt the string with one replace() per escaped character, which moves the tail every time: escaping a string of n characters that all need escaping cost O(n^2). Both now scan with find_first_of() -- a member the pointer parser already required -- and append whole runs, so the common case is one search and one copy. Escaping 64000 tildes drops from 717 ms to 20 ms; a string with nothing to escape gets faster too (8.4 ms to 5.8 ms), because the scan is still a single memchr per pass. json_pointer::split() takes its reference tokens with the (const char*, size_type) constructor rather than substr(). - json_pointer::to_string() accumulates with concat<string_t> instead of letting concat default to std::string and converting afterwards, so streaming a json_pointer no longer requires string_t to be assignable from a std::string. array_t: - at(size_type) is gone. basic_json::at(size_type) checked the index by calling array_t::at() and translating std::out_of_range, which also required the array type to throw that exact exception. It now compares against size() and uses operator[]. The thrown exception, its message, and the behaviour under JSON_NOEXCEPTION are unchanged. The BSON writer wrote the terminating null byte out of the string's own buffer (size() + 1). It now writes the byte itself, so string_t::data() need not be null-terminated for to_bson(). The tests pin the reduced API: alt_string loses the five dropped members and gains coverage of the escaping paths, and a std::vector whose at() is hidden is used as an ArrayType. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * docs: record the reduced string_t and array_t requirements Drop c_str(), back(), find(str, pos), replace(), and substr() from the StringType requirements and at(size_type) from the ArrayType ones, and note the string assignment the JSON pointer code performs. Streaming a json_pointer no longer needs assignability from a std::string. Add the non-null-terminated data() to the list of violations that are not diagnosed at compile time -- it was described in the StringType section but missing from the summary at the top -- and correct the QString row, which no longer fails for the c_str() it lacks. JSON_CATCH_USER no longer wraps a catch of std::out_of_range: the last one went away with array_t::at(). Describe what the library actually catches. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Use character literals for the signed BinaryType test MSVC rejects char(0xFF) with C4310 (cast truncates constant value), which the Windows workflow treats as an error. The character literals carry the same byte values without a narrowing cast. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Do not instantiate a hash map with an incomplete basic_json in the tests object_t is probed for key_compare inside the definition of basic_json, so it is instantiated while basic_json is still incomplete. Whether a hash map survives that depends on the standard library: libstdc++ 9 needs the size of the mapped type to instantiate std::unordered_map's node type and rejects the adapter, which broke the GCC 9 builds. The test now derives its no-key_compare object type from std::map -- which does cope -- and shadows the inherited key_compare member type with an entity that is not a type, so the library's probe finds none, exactly as for a hash map. The unflatten() order-independence checks in unit-json_pointer already cover the behaviour that the unordered object type was there for. The limitation is documented for std::unordered_map. Also address two Clang-Tidy findings the earlier commits introduced: erase_from_object() declares its iterator with auto, and at(size_type) checks the type first and then falls through to the return instead of throwing from an else branch. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Keep diagnostic key paths null-terminated Building the token from data() and size() kept an embedded null byte in the key, and since what() hands out a C string, that truncated the whole message rather than just the key: to_bson() on a key containing U+0000 reported "[json.exception.out_of_range.409] (/en" instead of the full explanation. This broke test-bson under JSON_DIAGNOSTICS. Constructing from data() alone stops at the first null byte, which is what c_str() did before, so the message is unchanged -- without requiring string_t to provide c_str(). Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Do not parse the value in the array-at() test JSON_DIAGNOSTIC_POSITIONS adds the byte range of the value to the exception message, which a parsed value has and an in-memory one does not, so the two message checks failed in that configuration. Build the array in memory instead of parsing it; the test is about at(size_type) not needing array_t::at(), and the byte range is beside the point. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Move the custom BinaryType tests into their own translation unit The two sections added to unit-regression2.cpp brought a third full basic_json instantiation into a translation unit that was already large. With Clang on MinGW that pushed the object over the reach of a 32-bit relocation and test-regression2_cpp20.exe failed to link: relocation truncated to fit: IMAGE_REL_AMD64_REL32 against `.rdata' unit-regression2.cpp is restored to exactly what it was before, and the coverage moves to unit-custom-binary-type.cpp, next to the object and array type tests it belongs with. The signed value type is now also covered in C++11, where std::byte is not available. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Do not require the container iterators to be nothrow move constructible iter_impl declared its defaulted move operations noexcept. The exception specification a defaulted function gets implicitly follows from its members, here internal_iterator, which holds the object and array iterators. libstdc++ gives std::deque's iterator a user-provided copy constructor without noexcept before version 11, so the implicit specification is noexcept(false) and does not match the declared one. That deletes the function -- and with g++ 4.8, which predates CWG 1778, it is an error outright: error: function 'iter_impl<basic_json<std::map, std::deque> >::iter_impl( iter_impl&&)' defaulted on its first declaration with an exception-specification that differs from the implicit declaration So std::deque, which this branch documents as a usable array type, could not be used with an older standard library. Leaving the specification to be computed cannot mismatch; iteration_proxy_value already spells out the same condition next door. The default configuration is unaffected: json::iterator, json::const_iterator and ordered_json::iterator stay nothrow move constructible and move assignable, which the test now checks so it cannot regress unnoticed. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Address two Clang-Tidy findings the custom container tests exposed Both come from instantiating basic_json with containers other than the default ones, and neither shows up with the Clang-Tidy version available outside CI: - insert(const_iterator, basic_json&&) forwards its by-value iterator to the const-reference overload. performance-unnecessary-value-param asks for the copy to be a move; it only fires for an iterator that is not trivially copyable, as std::deque's is not. The NOLINT on the function does not cover it, because the finding is reported where the parameter is used rather than where it is declared. Move it, which is what the check asks for and is a (very small) improvement in its own right. - cppcoreguidelines-use-enum-class rejects the unnamed enum that shadowed the inherited key_compare member type. An enum class would not do, since it declares a type of that name and the probe would find it again; a member function declaration hides the name just as well. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Assert the iterators' exception specification relative to the container The test pinned that nlohmann::json's iterators stay nothrow movable after iter_impl's defaulted move operations lost their declared noexcept. That is not a property of the library, though: the exception specification is now computed from the container iterators, so it holds only for standard library implementations whose iterators are themselves nothrow movable. MSVC's checked iterators before VS2017 are not -- _Iterator_base12 registers the iterator with the container's debug proxy in a copy constructor that carries no noexcept -- so the assertions fail on a Visual Studio 2015 debug build, which is the one debug configuration in the AppVeyor matrix and has no counterpart in the GitHub Actions matrix. Assert what the change actually guarantees instead: the iterators are nothrow movable exactly when the object and array iterators they are built from are. That still pins the default configuration against a silent regression, and it is true whatever the standard library provides. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Detect a void-returning erase() through a named trait erase_from_object() distinguished its two overloads with a decltype of a member call written inline in a default template argument. Every other detection in the library goes through the detector machinery in detected.hpp instead -- has_erase_with_key_type is the same question about the same member function -- and the inline form is the one shape older compilers are least reliable about. Express it the same way: detect_erase_with_iterator plus is_detected_exact, both of which the library already relies on elsewhere. No behaviour changes. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Give the custom container types only the constructors the library uses The three container types in the new tests inherited every constructor of their base with using Base::Base. That asks for more than the test needs: the library builds an object or an array by default construction, by copy or move, and -- when converting between two basic_json types or from an initializer list -- from an iterator range. Declaring those directly makes the requirement visible in the test, and keeps object types out of a corner where a compiler has to declare std::map's whole constructor set for a derived class while basic_json is still incomplete. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Temporarily disable the new custom container tests AppVeyor is the only CI that builds MSVC 2015 and 2017, and it has now rejected three heads of this branch. Its build log is not reachable from where this is being worked on, so the verdict is a single bit and the cause has to be narrowed down by bisection. Everything else stays: the library changes, the reduced alt_string, and the unflatten() tests. If AppVeyor passes with these three translation units disabled, the cause is one of the six basic_json instantiations they add; if it fails, it is in the library. Either way this commit is reverted. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Guard the disabled tests with a macro rather than #if 0 Clang-Tidy's readability-avoid-unconditional-preprocessor-if rejects a literal #if 0. Use a macro that is never defined instead, which the check does not look at. Still temporary, and reverted together with the previous commit. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Re-enable the array and binary container tests AppVeyor passed with all three new translation units disabled, so the library changes, the reduced alt_string, and the unflatten() tests are fine on MSVC 2015 and 2017; the cause is one of the six basic_json instantiations the new tests add. Bring back two of the three. If AppVeyor passes again, the cause is in unit-custom-object-type.cpp, which is the one still disabled; if it fails, it is in one of these two and needs one more split. Still temporary. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Diagnose two silently violated template parameter requirements Both were on the list of requirements that are not caught at compile time and corrupt values rather than failing, and both are a plain size comparison: - A BinaryType whose value_type is wider than one byte, which the readers and writers reinterpret as raw bytes anyway. - A NumberUnsignedType too narrow to hold the absolute value of every NumberIntegerType value, which makes basic_json(INT64_MIN).dump() yield -0 for std::int64_t with std::uint32_t. Neither static_assert rejects a configuration that worked before: both only fire where the result was already wrong. Also add the two comments the review asked for, in write_bson_string() and calc_bson_array_size(), matching the ones their counterparts already carry. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Align the template parameter tables and record what is now diagnosed Every table in the page is reformatted so each column is exactly as wide as its widest cell, which is what the review asked for in a dozen places: the separator rows that ran two dashes long, the stray spaces, and the columns padded well past their content. The row listing six containers that require a complete mapped type is split in two so that one cell no longer sets the width of the whole table. Content changes: NumberUnsignedType is described as any unsigned integer type at least as wide as NumberIntegerType rather than any unsigned integer type; the two requirements that are now static_asserts move out of the list of violations that are not caught at compile time; and the two places that require a non-const operator[] say why data() will not do (std::string has no non-const data() before C++17). Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Bisect the other way: only the object container tests The previous head touched only docs/, which AppVeyor's only_commits filter skips, so it produced no build and no status at all -- the pull request looked green without ever having been built on MSVC 2015 or 2017. Swap the guards instead of repeating that step: unit-custom-object-type.cpp is enabled and the array and binary translation units are disabled. AppVeyor already passed with all three disabled, so a failure here pins the cause on no_key_compare_json or void_erase_json, and a pass pins it on the array or binary file. Still temporary. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Split the two object types apart AppVeyor failed with only unit-custom-object-type.cpp enabled and passed with all three new translation units disabled, so the cause is one of the two object types in this file and not the array or binary ones. Guard out void_erase_map and leave no_key_compare_map, which separates the two constructs under suspicion: shadowing the inherited key_compare member type with an entity that is not a type, and hiding the inherited erase with a void-returning overload. A failure here points at the first, a pass at the second. Still temporary. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Build the no-key_compare object type by composition, not inheritance The "object type without key_compare" test failed on AppVeyor's MSVC 2017 jobs (/std:c++17): its no_key_compare_map derived publicly from std::map and shadowed the inherited key_compare type with a same-named member function, relying on ordinary member hiding to make key_compare unreachable as a type for the library's detection trait. MSVC 2017 does not honor that hiding for a typename-qualified lookup performed from outside the class and still resolves key_compare to the base's comparator type, so object_comparator_t incorrectly picked it up instead of falling back to default_object_comparator_t. Wrapping a std::map by composition instead removes the base class entirely, so there is no key_compare to find under any lookup rule, on any compiler. Also drops the now-unneeded JSON_BISECT_CUSTOM_CONTAINER_TESTS guard left over from narrowing this down: the void_erase_map test in the same file was never the cause and is re-enabled unconditionally. Verified locally with clang++ and g++ under C++17 and C++20. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Re-enable the array and binary custom-container tests unit-custom-array-type.cpp and unit-custom-binary-type.cpp were still guarded behind JSON_BISECT_CUSTOM_CONTAINER_TESTS from bisecting the AppVeyor failure fixed in |
||
|
Niels Lohmann
|
6487678bc5 |
Fix swap(array_t&)/swap(object_t&) to update parent pointers under JSON_DIAGNOSTICS (#5464)
Both overloads swapped the underlying container storage but never called set_parents(), leaving elements moved into *this with stale m_parent pointers (typically nullptr from the free-standing array_t/object_t). This produced wrong JSON Pointer paths in diagnostic messages and could trip assert_invariant() on subsequent copies. Mirrors the fix already applied in swap(reference other). Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
c0b2878a44 |
Swap diagnostic positions in basic_json::swap() (#5493)
basic_json::swap() (and the friend swap() that forwards to it) only exchanged m_data.m_type/m_data.m_value, leaving start_position/end_position untouched under JSON_DIAGNOSTIC_POSITIONS. This is inconsistent with copy-assignment's operator=(basic_json), which swaps positions as part of its copy-and-swap implementation, so after swap(a, b) each value ended up with the other value's content but its own original position. Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
da42a627dc |
Stop dump() from heap-allocating its output adapter per call (#5449)
* Stop dump() from heap-allocating its output adapter per call The serializer held its output sink as output_adapter_t<char> (a std::shared_ptr<output_adapter_protocol<char>>), which dump() and operator<< built via make_shared -- one heap allocation per call for a sink that only wraps a reference to the caller's string or stream. Hold the sink as a non-owning output_adapter_protocol<char>* instead and construct the concrete adapter on the stack at the call site. The write path (o->write_characters) is unchanged, so output is byte-for-byte identical; a compact dump() of a small object drops from 2 heap allocations to 1 (only the returned string remains), ~3% faster. Completes the per-call allocation cleanup on this branch, which already removed the indent_string buffer (both were reported in #5413). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01L1oJ2ggRHS37zeVe94QTA1 Signed-off-by: Claude <noreply@anthropic.com> * Take the output adapter by reference at the serializer ctor Per review: the serializer still holds the adapter as a non-owning pointer, but the constructor now takes output_adapter_protocol<char>& and takes its address internally, so every call site passes a reference. A reference cannot be null and reads as a borrow, which makes the lifetime contract harder to get wrong than handing over a raw pointer. The stored member and the write path are unchanged. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XAYM1qhSA2FDaDcGfPW3fG Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Claude <noreply@anthropic.com> Signed-off-by: Niels Lohmann <mail@nlohmann.me> Co-authored-by: Claude <noreply@anthropic.com> |
||
|
Niels Lohmann
|
ff80ed3295 |
Speed up dump(), and keep it from overflowing the stack (#5285)
* Add SWAR bulk fast path to string serialization (dump_escaped) When ensure_ascii is false, dump_escaped previously ran every byte of every string and object key through the UTF-8 DFA decoder, even for the common case of ordinary text with nothing to escape. This mirrors the per-byte cost the parser had before the contiguous fast paths. At a character boundary, bulk-copy the longest run of bytes that need no escaping using string_bulk_run() - the same SWAR scanner and UTF-8 bulk validator the lexer's contiguous path uses - and only fall back to the byte-at-a-time DFA loop for the first byte that needs individual handling (a quote, backslash, control character, or ill-formed/truncated UTF-8). Because every "hard" or invalid byte is still processed by the unchanged byte path, escaping output and error handling (including strict-mode error 316 position and message) are byte-identical to before. The ensure_ascii=true path is unchanged: it must escape non-ASCII and 0x7F, which string_bulk_run does not stop on, so a separate predicate would be needed for it. Verified byte-for-byte identical dump output against the pre-change implementation across ~20k randomized byte strings plus curated edge cases (all escapes, control chars, valid multibyte, surrogates, overlong, truncated sequences) for both ensure_ascii settings and all three error handlers, in C++11/17/20 at -O2/-O3. Throughput (g++ -O3, ensure_ascii=false, vs pre-change): long ASCII strings 4.2x twitter-like objects 2.3x dense CJK 1.4x (further headroom with JSON_USE_SIMDUTF) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XAYM1qhSA2FDaDcGfPW3fG Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Buffer serializer output and add ensure_ascii string fast path Two further serialization speedups on top of the ensure_ascii=false bulk copy, both reusing the SWAR primitives in detail/input/string_scan.hpp. 1. Internal write buffer (devirtualization). Every structural character ('{', '"', ',', ...) previously went straight to the output adapter through a virtual call. Route all writes through put_char/put_chars into a 1 KiB buffer that flushes in bulk; the public dump() flushes once the top-level value is done (the recursive worker is split out as dump_internal). Runs larger than the buffer are written straight through, so large payloads are not copied twice. This is the dominant cost for object/array-heavy values. 2. ensure_ascii fast path. dump_escaped previously ran the UTF-8 DFA over every byte when escaping non-ASCII. Add find_ascii_copyable_run() (a SWAR scan stopping at '"', '\\', < 0x20, 0x7F, and >= 0x80) so runs of printable ASCII are bulk-copied, with the byte path handling each escape/non-ASCII byte exactly as before. Behavior is unchanged: dump output is byte-for-byte identical to the previous implementation across ~20k randomized byte strings plus curated edge cases (all escapes, control chars, 0x7F, valid multibyte, surrogates, overlong, truncated), for object/array/pretty output, both ensure_ascii settings, and all three error handlers, in C++11/17/20 at -O2/-O3. New unit tests cover the buffer flush boundaries, the escape and 0x7F handling, multibyte under both settings, and invalid-UTF-8 handling. Throughput (g++ -O3, vs the ensure_ascii=false-only baseline): long ASCII, ensure_ascii=0 4.2x long ASCII, ensure_ascii=1 4.1x twitter-like objects 2.7x dense CJK 1.8x Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XAYM1qhSA2FDaDcGfPW3fG Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Flush serializer buffer in dump_escaped unit test test-convenience failed (macOS finished first; the failure is platform-independent) because check_escaped() calls the internal serializer::dump_escaped() directly and then reads the output stream. Since dump_escaped() now writes into the serializer's internal write buffer, the bytes were still buffered and the stream was empty. Expose flush() under JSON_PRIVATE_UNLESS_TESTED (same visibility as dump_escaped) and flush in check_escaped() before inspecting the output. Per-string flushing inside dump_escaped() was rejected on purpose: it would defeat the buffering that makes object/array-heavy dumps faster. Library behavior is unchanged (flush()'s body is identical; only its access label moved). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XAYM1qhSA2FDaDcGfPW3fG Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Avoid deep recursion in serialization write-buffer test The "many small structural writes exceed the write buffer" subcase built a 1100-deep nested array and dumped it to force >1024 consecutive single-character writes through put_char (exercising the write buffer's flush-when-full branch). dump() recurses per nesting level, so on MSVC debug builds (smaller default stack, larger frames) this overflowed the stack and crashed test-serialization; Linux/macOS have enough headroom to hide it. Replace the nesting with a flat array of 500 empty strings. Each element emits '"', '"', ',' via put_char, so the dump is a long run of single-character writes (1501 bytes > the 1024-byte buffer) at nesting depth two, hitting the same flush branch without deep recursion. Library code is unchanged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XAYM1qhSA2FDaDcGfPW3fG Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Split the write-buffer helpers and write indentation directly Follow-up to @gregmarr's review: put_chars() was doing four unrelated jobs, so give the two that can be made safe their own entry points. - put_literal(): takes the literal by reference and deduces the length from the array bound, so the 27 hand-counted lengths at the call sites can no longer drift from the literals they describe. A literal is checked at compile time to fit the buffer, so this path needs no write-through branch. - put_buffer(): takes the fixed-size buffer itself rather than a bare pointer, so the length can be checked against the buffer's own bound. - put_indent(): memsets the indentation into the write buffer, filling and flushing it as needed. This removes indent_string entirely, and with it both bugs of #5186: the indentation string was grown by doubling, which is not enough when indent_step more than doubles it (a heap over-read - dump(2000) read 2000 bytes out of a 1024-byte string), and the grown part was filled with a space instead of the configured indent_char. next_indent() keeps that PR's assertion against the unsigned indentation accumulation wrapping on deep nesting. put_chars() keeps the two cases that are genuinely a pointer and a count: the run-length copies out of the string being escaped, and to_chars() output. Tests cover an indent_step wider than the write buffer, a non-space indentation character past the old growth point, and nesting whose accumulated indentation spans several buffer-fulls. All three fail against develop. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Fill the indentation buffer once instead of once per flush @gregmarr's point on the fill-and-flush loop: flushing does not disturb what the write buffer holds, so an indentation spanning several buffer-fulls only has to be written into the buffer once and can then be handed to the adapter as many times as needed. The loop re-filled it every time, doing work it already knew was there. put_indent() now fills the room left in the buffer, and if anything remains, flushes, fills the buffer once, and re-flushes that same content. It also returns early for a zero-width indentation, which is what the closing brace of every outermost value asks for. Measured over a dump(), counting memset calls and bytes inside put_indent: indent before after 4 1 call / 4 B 1 call / 4 B 2000 2 calls / 2000 B 2 calls / 2046 B 100000 98 calls / 100000 B 2 calls / 2046 B The wide case is now constant work rather than proportional to the indentation width; ordinary widths are unchanged. Tests extended to cover several whole buffer-fulls and an exact multiple of the buffer size. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Tighten the write-buffer helpers after review More of @gregmarr's review on the put_* split: - Reattach the put_chars() doc comment, which the new helpers had been inserted in front of, leaving it describing put_indent(). - Compute the literal length once in put_literal() instead of spelling N - 1 at each use. - Add put_string(str, start, end), which keeps the pointer arithmetic and the bounds assertions inside the function instead of at the call site. With dump_float()'s to_chars() output moved onto put_buffer() as well, put_chars() now has no callers outside put_string()/put_buffer(): nothing passes a bare pointer and a count any more. - Carry the indentation as std::size_t rather than unsigned int. It is a size, it is compared and combined with buffer sizes throughout, and the casts in put_indent() disappear. next_indent() keeps its assertion, which is far harder to trip on a 64-bit size_t but still reachable where that is 32 bits. No output change: pretty and compact dumps, binary values included, are byte-identical to develop. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Silence avoid-c-arrays on put_literal's array reference clang-tidy flags the reference-to-array parameter under cppcoreguidelines/hicpp/modernize-avoid-c-arrays, and the CI treats warnings as errors. Binding to the array is the whole point here - it is what lets the length be deduced from the literal instead of hand-written at the call site - so suppress it the same way from_json(), to_json() and get_to() already suppress it for their own T (&arr)[N] parameters. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Bound the descent of dump() Serializing a container serializes its elements, so dump() descended into one call per nesting level. A value nested deeply enough exhausted the call stack and terminated the process with a segmentation fault - no exception, nothing the caller could catch. Parsing such a value works, as the parser is iterative, and so does destroying one, as #1436 made destruction iterative. Bound how far the descent goes rather than take the call stack away from it. The first 128 levels are written by exactly the code that always wrote them, and only below that does dump_iteratively write out what is left, keeping the containers it has entered on an explicit stack. Serializing can therefore no longer exhaust the stack, however deeply a value is nested, while a value nested less deeply than the bound pays only for one comparison per container. Writing every value that way instead measured between 2% and 20% slower - 20% on object-heavy documents - which is why the descent is kept for all but the values that cannot afford it. The bound costs nothing measurable: between -1.4% and +1.2% across compact and pretty output of number, integer, string, object-heavy, wide-object and deeply nested documents. The output is unchanged for every value. Both ways of writing a container emit the separator in front of every element but the first, rather than after every element but the last, which puts exactly one between each pair and none at the end. This fixes #5387 for dump(). The copy constructor is fixed in #5389. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Fold ensure_ascii into the escaper and write bytes without dump_integer Two hot spots that the write buffer and the bulk scanner left behind. dump_escaped took ensure_ascii as a runtime flag and tested it inside the loop, once per character run, although it cannot change while a string is written. It is now a template parameter, dispatched once per string, which folds the choice of scanner and lets each of the two be inlined into a loop of its own. This is the hottest loop in the serializer: it runs over every string and every object key. A binary value's bytes went through dump_integer, which counts digits and does 64-bit arithmetic for a number that is always in [0, 255]. dump_byte writes the three digits it takes at most straight into the write buffer instead. Any byte type that is not a plain unsigned byte is still left to dump_integer, whose representation of it may differ. Measured against the previous commit (medians of 9 interleaved runs, clang -O3): binary values -33.8%, dense CJK with ensure_ascii -20.6%, key-heavy objects -17.8%, deeply nested pretty output -17.9%, dense CJK without ensure_ascii -11.8%, object-heavy documents -9.3% compact and -9.5% pretty, a small value dumped in a loop -21.4%, wide objects -2.3%. Arrays of plain ASCII strings measured 3.5% to 4.2% slower, the one shape that loses; number and integer arrays are unchanged. Also tried and dropped: leaving the write and string buffers uninitialized rather than zeroing 1.5 KB per dump() call. It is worth -30% on small values, but two nearly identical string workloads moved 18% apart in opposite directions, so the measurements did not support it. The output is unchanged for every value: the differential now also covers every one of the 256 byte values, alone and together, in both binary layouts. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Write a byte without walking a pointer over the buffer clang-tidy's misc-const-correctness reads the pointer dump_byte advanced over the write buffer as one whose pointee could be const. Index the buffer instead, which says the same thing without a raw pointer at all. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Parenthesize the reserve arithmetic in the deep-nesting test clang-tidy's readability-math-missing-parentheses wants the multiplication spelled out in reserve(6 * depth + 1), and CI treats its warnings as errors. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Do not scan for a copyable run that cannot exist Under ensure_ascii, dump_escaped() calls find_ascii_copyable_run() at every character boundary. When the text is dense non-ASCII - CJK, where every byte is >= 0x80 - the scanner stops on its first byte and returns zero, so its SWAR block runs once per character and buys nothing, on top of the escaping that still has to happen afterwards. A run can only be non-empty when the first byte is one the scanner may copy, so test that single byte before calling it. Runs that do exist are found exactly as before, so the bulk-copy win is unchanged; only the calls that were always going to return zero are skipped. Output is unchanged: the dump digest over canada/citm/twitter, in compact, pretty and ensure_ascii form, matches develop byte for byte. dump(ensure_ascii=true) develop before after CJK text 3.54ms 4.25ms 3.36ms CJK, no ASCII at all 3.09ms 4.02ms 3.02ms Latin-1-ish text 4.39ms 3.04ms 2.93ms plain ASCII 3.92ms 0.80ms 0.79ms Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Address review of the write-buffer helpers Three points from @gregmarr's review: put_chars() is gone. It was the only entry point taking a bare pointer and a count, and it existed only so put_string() and put_buffer() had something to delegate to. Its body now lives in put_string(), and put_buffer() is put_string(buffer, 0, length) - std::array already carries data() and size(), so it satisfies the same interface a string does. Nothing appends characters without a bound any more. dump_escaped()'s documentation block was duplicated. The dispatcher was inserted between the original comment and the function it described, and the comment was copied rather than split. The worker now has its own short comment saying why ensure_ascii is a template parameter. The local in dump_byte() is deliberate, and is now documented as such: writing through write_buffer[] is a char write, which may alias any object, so with write_buffer_pos updated in place the compiler must reload and store it around every digit. Measured on a dump of a 4 MiB binary value, 18.0 ms without the local against 7.4 ms with it. Output is unchanged: byte-identical dumps across 77 files in compact, pretty, ensure_ascii, pretty+ascii, indent 600 and tab-indent form. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Address review: drop unneeded backslash-escapes and duplicate scan loop '"' does not need escaping in a char literal, unlike in a string literal. find_ascii_copyable_run() also duplicated the byte-at-a-time search that already exists as the loop's own scalar tail; break into it instead of re-deriving the offset in a second, near-identical loop. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Move pretty_print, ensure_ascii and indent_step into the serializer None of these change over the life of a serializer, unlike current_indent and depth, which do change on every recursive call. They are now captured once in the constructor - matching indent_char and error_handler - instead of being threaded through dump(), dump_internal(), dump_iteratively(), dump_value() and dump_escaped() on every call. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Stop the serializer from holding onto std::localeconv()'s pointer loc was only ever read twice, immediately, to seed thousands_sep and decimal_point; nothing else in the class used it. A local in the constructor body serves the same purpose without keeping the pointer around for the serializer's lifetime. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Keep thousands_sep/decimal_point const via a small locale_chars struct const members can't be assigned in a constructor body, so seeding them from std::localeconv() meant either dropping const or holding onto the lconv* for longer than needed. A sub-object computes both from the pointer in its own constructor and is itself initialized in serializer's mem-initializer-list, so the two chars stay const, std::localeconv() is still called exactly once, and nothing outlives the constructor. Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Niels Lohmann
|
d69fb8654d |
Return the parsed value by move from from_cbor() and friends (#5501)
The binary entry points end with
return res ? result : basic_json(value_t::discarded);
The condition operator's second operand is an lvalue, so this is not a case
where the return value can be elided or implicitly moved from: every
successful from_cbor(), from_msgpack(), from_ubjson(), from_bjdata() and
from_bson() call deep-copies the value it just parsed, and then destroys the
original.
The copy is not cheap, and it is not incidental: basic_json's copy
constructor walks the whole value. Parsing a 2 MB CBOR document with 60,000
objects, median of 25 runs, clang 17 -O3:
from_cbor 26.99 ms -> 14.65 ms
from_msgpack 26.82 ms -> 14.82 ms
Moving instead of copying is the entire change; the parsed value is not used
again after the return expression is evaluated.
There is a second reason to prefer the move. The copy constructor recurses
once per nesting level, so the copy is also a stack-overflow path on the
return side, on a value the reader has already accepted. That is currently
masked because the readers themselves recurse and overflow first (#5104), but
it has to be fixed for making them iterative to have any effect.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
|
||
|
Niels Lohmann
|
c9f1d2d646 |
Broaden JSON_HEDLEY_WARN_UNUSED_RESULT coverage to pure query functions (#5477)
* Broaden JSON_HEDLEY_WARN_UNUSED_RESULT coverage to pure query functions Add JSON_HEDLEY_WARN_UNUSED_RESULT to the unambiguous, const, side-effect-free observer functions whose return value is the entire purpose of the call: - dump() - type(), type_name() - all is_* predicates (is_primitive, is_structured, is_null, is_boolean, is_number, is_number_integer, is_number_unsigned, is_number_float, is_object, is_array, is_string, is_binary, is_discarded) - empty(), size(), max_size() - count(...) (both overloads) and contains(...) (all overloads, including the deprecated json_pointer<BasicJsonType> overload) This mirrors the direction the standard library has taken with [[nodiscard]] on the analogous std::vector/std::map members, and catches real bugs such as `j.empty();` (meant `j.clear();`) or `j.contains(k);` with the result thrown away. Deliberately out of scope (left for a separate, later policy decision, per the issue): at(), value(), get*(), flatten(), unflatten(), patch(), merge_patch(), begin()/end(), comparison operators, erase(), and emplace(). Compiling the full test suite (tests/src/unit-*.cpp) with -Wunused-result -Werror uncovered one real hit: a regression test in unit-regression2.cpp called dump() purely to check it does not throw, discarding the result. Fixed by explicitly casting to void, since the call is intentionally result-less there. Fixes #5410 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Fix discarded nodiscard results across the test suite for GCC's warn_unused_result A plain (void) cast on a call expression suppresses the C++17 [[nodiscard]] warning but not GCC's warning for functions annotated via the GNU __attribute__((warn_unused_result)) form -- which is what JSON_HEDLEY_WARN_UNUSED_RESULT expands to on GCC. Several existing tests that call a newly-annotated function (dump(), empty()) purely to check that it throws/does not throw, discarding the result via (void), newly warned (and failed -Werror builds) once the annotation was broadened. Route those discards through a small ignore_return_value() helper instead, which actually consumes the value and suppresses the warning on both attribute forms. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Use utils::ignore_return_value() for the issue #1445 dump() discard too Addresses review feedback from @gregmarr on PR #5477: this call site was still using the older "capture in a variable, then (void) it" pattern from before this PR introduced utils::ignore_return_value(), instead of the helper now used at every other discarded-nodiscard-result call site this PR touches. Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
a316738cfa |
Add JSON_HEDLEY_WARN_UNUSED_RESULT to the current accept() overloads (#5471)
accept() is a pure query whose only effect is the returned bool; both parse() overloads and the deprecated accept(span_input_adapter&&, ...) overload already carry JSON_HEDLEY_WARN_UNUSED_RESULT, but the two current, recommended accept() overloads were missing it. Add the annotation to match, so discarding accept()'s result now warns under -Wunused-result / [[nodiscard]], as it already does for parse(). Fixes #5407 Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
8c38e270b5 |
Reject JSON Patch move when from is a proper prefix of path (#5497)
* Reject JSON Patch move when from is a proper prefix of path RFC 6902 (section 4.4) forbids "from" from being a proper prefix of "path" for a "move" operation: "a location cannot be moved into one of its children." "move" is implemented as remove-then-add with no check for this. For object targets, the subsequent "add" happened to throw as a side effect of resolving through the now-removed parent, but for array targets, removing the "from" element shifts subsequent indices, so "path" silently re-resolves to a different element and the operation "succeeds" with a silently corrupted document. Add a check, before performing the remove/add, for whether "from" is a proper prefix of "path" at the reference-token level. This compares json_pointer's already-unescaped reference_tokens vectors (basic_json is a friend of json_pointer) rather than the raw pointer strings, so that tokens containing escaped '/' or '~' characters are compared correctly, and a token that merely looks like a string prefix (e.g. "/ab" vs "/abc/x") is not mistaken for a pointer-token prefix. When "from" is a proper prefix of "path", throw out_of_range.414. Fixes #5397. Stacked on top of the fix for #5396 (branch issue-5396-patch-remove-primitive-parent), since both touch the same patch_inplace move/remove handling in include/nlohmann/json.hpp. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Add root-pointer and array-append-token edge case tests for the move prefix check Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Replace std::equal with an explicitly-bounded loop in the move prefix check The three-iterator std::equal(first1, last1, first2) form has no explicit end iterator for the second range, which a static analyzer (Flawfinder, CWE-126) flags as a potential over-read even though the preceding size comparison already guarantees the second range is long enough. Rather than argue the point, make the bound visible in the code itself via an explicit loop -- every access to ptr.reference_tokens is now guarded by the same index the loop condition bounds against from_size. (The C++14 four-iterator std::equal(first1, last1, first2, last2) form was tried first as a more minimal fix, but this codebase targets C++11 and that overload is not safely usable under -std=c++11 with all supported standard library implementations.) Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Extract the move prefix check into a named helper lambda Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Account for JSON_DIAGNOSTIC_POSITIONS in the move-prefix-check error messages out_of_range::create() includes a "(bytes X-Y)" position annotation when JSON_DIAGNOSTIC_POSITIONS is enabled, which the ci_test_diagnostic_positions CI job builds the whole suite with. The five new out_of_range.414 assertions only checked the annotation-free message. Confirmed JSON_DIAGNOSTICS produces the same (annotation-free) message as the default build for this particular throw site (its path-based annotation is empty at the root, where &result always points here), so only two message variants are needed, not three. Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
c9477ccf91 |
Throw when JSON Patch remove's target path resolves through a primitive or null parent (#5496)
RFC 6902 (section 4.2) requires the target location of a "remove" operation to exist. operation_remove handled parent.is_object() and parent.is_array(), but had no final else branch: when the resolved parent was a primitive value or null, neither branch matched and the operation silently did nothing instead of failing. Add the missing else branch, throwing out_of_range.413 with wording that matches the existing out_of_range.411 thrown by the analogous "add" case (operation_add) for the same kind of invalid parent. Fixes #5396. Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
d6660cf718 |
Route hand-rolled diagnostic pragmas through Hedley (#5485)
* Route hand-rolled diagnostic pragmas through Hedley
Several places in the library hand-roll compiler diagnostic suppression
with raw `#pragma`/`#ifdef __GNUC__`/`#ifdef __clang__` guards instead of
using the Hedley primitives already bundled and used elsewhere
(JSON_HEDLEY_DIAGNOSTIC_PUSH/POP, JSON_HEDLEY_PRAGMA, ...). Converted six
of the seven listed push/pop pairs to use those primitives instead of
raw `#pragma GCC diagnostic`/`#pragma clang diagnostic` text:
- include/nlohmann/json.hpp (~3770, ~3863): -Wfloat-equal
- include/nlohmann/detail/conversions/to_chars.hpp (~1078): -Wfloat-equal
- include/nlohmann/detail/output/binary_writer.hpp (~1844): -Wfloat-equal
- include/nlohmann/detail/iterators/iteration_proxy.hpp (~211): -Wmismatched-tags
- include/nlohmann/detail/exceptions.hpp (~36): -Wweak-vtables
iteration_proxy.hpp did not previously include macro_scope.hpp itself
(it only compiled because some other header included earlier in
json.hpp happened to pull macro_scope.hpp in first); it now includes it
directly like the other detail headers that use Hedley macros, so it is
self-contained.
Each push/pop pair now uses JSON_HEDLEY_DIAGNOSTIC_PUSH/POP
unconditionally (a no-op on compilers that don't need it) and wraps the
actual `#pragma ... diagnostic ignored` text in JSON_HEDLEY_PRAGMA so it
goes through Hedley's _Pragma()-based emission instead of a raw #pragma
line, while keeping the original `#ifdef __GNUC__` / `#if
defined(__clang__)` guard around the ignored-pragma itself.
Deviation from the issue's suggested transformation: the issue's example
replaces the `#ifdef __GNUC__` guard with `#if
JSON_HEDLEY_HAS_WARNING("-Wfloat-equal")`. JSON_HEDLEY_HAS_WARNING is
implemented purely via Clang's `__has_warning` builtin and evaluates to
0 on real GCC (`#define JSON_HEDLEY_HAS_WARNING(warning) (0)` when
`__has_warning` is not defined), so adopting it verbatim would silently
stop suppressing -Wfloat-equal on GCC -- a real regression, not just a
style change. The existing `#ifdef __GNUC__` / `#if defined(__clang__)`
guards were kept for the ignored-pragma to stay behavior-preserving, and
only the push/pop/pragma-emission mechanism was routed through Hedley.
Two of the seven locations from the issue (the -Wignored-attributes
push at the very top of json.hpp and its matching pop after
`#include <nlohmann/detail/macro_unscope.hpp>`) were intentionally left
unconverted:
- The push, at the very top of json.hpp, runs before
`detail/macro_scope.hpp` (and therefore hedley.hpp) has been included
anywhere in the translation unit, so JSON_HEDLEY_DIAGNOSTIC_PUSH is not
yet defined at that point.
- The pop runs after `macro_unscope.hpp`, which -- via hedley_undef.hpp
-- has already #undef'd every JSON_HEDLEY_* macro (by design, see
#5408) precisely so they don't leak to users, so JSON_HEDLEY_DIAGNOSTIC_POP
is no longer defined by the time the pop is reached either.
Making this one pair work would require either hoisting the ~2000
line vendored hedley.hpp to the very top of the amalgamated single
header (a much bigger structural change to single_include than a pure
mechanism swap) or special-casing this one pop ahead of the general
macro cleanup. Both are riskier than the mechanical, behavior-preserving
change requested, so this pair was left as-is.
## Validation
- Compiled include/nlohmann/json.hpp and single_include/nlohmann/json.hpp
with `-Wall -Wextra -Wfloat-equal -Wmismatched-tags -Wweak-vtables`
(clang, which self-identifies as __GNUC__ too): no warnings, same as
before the change.
- Compiled and ran tests/src/unit-to_chars.cpp, unit-conversions.cpp,
unit-iterators1.cpp, unit-iterators2.cpp, and unit-class_parser.cpp
against the fixed include/: all pass.
- Compiled unit-msgpack.cpp, unit-bjdata.cpp, and unit-ubjson.cpp (which
exercise binary_writer.hpp's write_compact_float extensively): all
compile cleanly; the vast majority of assertions pass (the only
failures are pre-existing environment issues unrelated to this change
-- missing generated test-data files, not code correctness).
- Ran `make amalgamate`; the single_include diff is limited to exactly
the lines touched in include/, with no unrelated reordering.
- No real (non-Apple) GCC was available in this environment to test
directly; the `_Pragma("GCC diagnostic ...")` text emitted by
JSON_HEDLEY_PRAGMA is byte-identical to the prior `#pragma GCC
diagnostic ...` text, and the `#ifdef __GNUC__` guard is unchanged, so
GCC's behavior is expected to be identical. CI covers the GCC matrix.
This PR is stacked on top of #5475 (issue-5408-hedley-undef-leak) since
both touch the same files; only the last commit here is new.
Fixes #5409.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Guard JSON_HEDLEY_DIAGNOSTIC_PUSH/POP with the same compiler check as the pragma they bracket
Addresses review feedback from @gregmarr on PR #5485: the push/pop calls
were unconditional, so compilers other than the one the ignored-pragma
targets (e.g. MSVC, or GCC where the pair only applies under __clang__)
now did a needless push/pop with nothing suppressed in between. Move the
existing #ifdef __GNUC__ / #if defined(__clang__) guard to also cover the
push/pop, restoring the original zero-overhead behavior on other compilers
while still emitting the pragma itself through Hedley.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
|
||
|
Niels Lohmann
|
faa35cc647 |
Skip integer conversion in accept()/SAX validation when the value is unused (#5484)
* Skip integer conversion in accept()/SAX validation when the value is unused lexer::scan_number() always converted every numeric token with strtoull()/strtoll() before returning, even though accept() (and any consumer using json_sax_acceptor) immediately discards the converted value. For value_unsigned/value_integer tokens whose digit count already guarantees the value fits into 64 bits, the conversion cannot change the accept/reject decision (such tokens are always finite and unconditionally accepted), so scan_number() can skip strtoull()/ strtoll() entirely in that case when the caller signals it does not need the value. Numbers with more digits keep using the exact, unmodified conversion path, so overflow reclassification to value_float (and the finiteness check on it) is unaffected. parse() and value_float handling are completely unchanged. Fixes #5411 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Document why the digit-count fast path is safe regardless of number_unsigned_t/number_integer_t width Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Avoid temporary-string concatenation flagged by clang-tidy in the differential test Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
Avionic Harshit
|
3c875683a5 |
Make diff() linear when an array shrinks (#5461)
Signed-off-by: avionicharshit-byte <harshitavionic@gmail.com> |
||
Qatadaha Bin Matloob
|
137a40b9aa |
Compare integers with floats exactly instead of widening the integer (#5459)
The mixed number arms of JSON_IMPLEMENT_OPERATOR cast the integer to number_float_t before comparing. Past the float's mantissa that cast is lossy: 2^63-2 and 2^63-1 both round to 2^63, so each compares equal to that float while differing from each other. Equality is therefore intransitive and the ordering is not a strict weak ordering, which makes std::sort over such values, or using them as keys in std::set or std::map, undefined behavior. Compare the two exactly instead. The integer's range is a power of two the float represents exactly, so a float outside it is ordered by magnitude alone; inside it, truncating the float is exact, and the integer parts and then any fractional part decide. The helper hands back a pair whose comparison with the original operator reproduces that ordering, which keeps every operator's return type as it was, including partial_ordering for the spaceship. A NaN operand is returned in both members, so NaN stays false for the relational operators and unordered for <=>. Values a float represents exactly still compare equal, so json(1) == json(1.0) is unchanged. Signed-off-by: qatcod <79017227+qatcod@users.noreply.github.com> |
||
elix3r
|
35705d79d8 |
Fix update(merge_objects=true) throwing on primitive-to-object merge (#5414)
When merge_objects is true, recurse only if the existing value is an object. Otherwise overwrite, matching the documented "all other values are overwritten as usual" behavior. Fixes #5402 Signed-off-by: elix3r <157088510+22elix3r@users.noreply.github.com> |
||
ljcjclljc
|
6285225fd0 |
Fix integer comparison bug (#5211)
* Fix integer comparison bug Signed-off-by: ljccjlljc <939159710@qq.com> * commit Signed-off-by: ljccjlljc <939159710@qq.com> * Remove generated CI artifacts and update amalgamation Signed-off-by: ljccjlljc <939159710@qq.com> * Silence cpplint braces warning in comparison macro Signed-off-by: ljccjlljc <939159710@qq.com> * Update amalgamation after cpplint fix Signed-off-by: ljccjlljc <939159710@qq.com> * Add mixed signed and unsigned comparison regression test Signed-off-by: ljccjlljc <939159710@qq.com> * Clarify mixed signed and unsigned comparison handling Signed-off-by: ljccjlljc <939159710@qq.com> * Expand mixed signed and unsigned comparison tests Signed-off-by: ljccjlljc <939159710@qq.com> --------- Signed-off-by: ljccjlljc <939159710@qq.com> |
||
|
Niels Lohmann
|
ca76c37650 |
Add iterator+sentinel tests and docs for binary deserializers (#5265)
* Add iterator+sentinel tests and docs for binary deserializers This commit extends the C++20 ranges support (iterator+sentinel pairs) to the binary format deserializers from_cbor, from_msgpack, from_ubjson, from_bjdata, and from_bson, matching what was already done for parse(), accept(), and sax_parse(). Changes: - Add istreambuf_sentinel helper to test_utils.hpp for EOF detection in tests - Add 5 new test cases that read binary files directly via std::istreambuf_iterator<char> + sentinel, without pre-buffering - Update documentation for all 5 from_* functions to document overload (3) with SentinelType parameter - All tests pass; verified against existing test suite data - Fix potential buffer over-read warning in heterogeneous iterator test Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Merge iterator+sentinel overloads and fix ambiguity/CI issues Address PR review feedback and CI failures: - Merge the separate same-type and sentinel-type iterator overloads of parse(), accept(), sax_parse(), and the five from_* binary deserializers into a single overload with SentinelType defaulted to IteratorType, as suggested in review. Applied the same simplification to the detail::input_adapter() free functions. - Fix a latent ambiguity: some compilers (e.g. GCC 4.8) unreliably SFINAE the operator!= detection for std::nullptr_t against container/string types, making calls like parse(s, nullptr, ...) ambiguous with the compatible-input overload. can_compare_ne now explicitly excludes std::nullptr_t as a SentinelType. - Use a named enable_if_t template parameter instead of an unnamed function parameter for the SFINAE guard, fixing a clang-tidy hicpp-named-parameter/readability-named-parameter failure. - Update parse.md, accept.md, sax_parse.md, and the five from_*.md pages to document the merged overload instead of separate (2)/(3) overloads, also fixing an over-160-char line that broke the documentation style_check CI job. - Rework the BSON iterator+sentinel test to parse a BSON file already present in the test suite instead of writing/deleting a temp file. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Fix -Wunneeded-internal-declaration for CustomSentinel in test CustomSentinel lives in an anonymous namespace (internal linkage), and the library's parse loop only ever evaluates the iterator-first direction (it != last), so the reversed-order friend operator!= was never referenced. Clang's -Weverything flags such unused internal declarations as an error. Drop the unused overload; the used direction is enough to satisfy can_compare_ne's either-order detection. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Fix clang-tidy hicpp-named-parameter and misc-const-correctness - Drop the unused reversed-order operator!= overload from utils::istreambuf_sentinel (only iterator != sentinel is ever evaluated) and name the remaining friend's sentinel parameter, fixing hicpp-named-parameter/readability-named-parameter. - Mark the istreambuf_iterator first/last helper variable const in the five binary-format sentinel tests, fixing misc-const-correctness. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Fix clang-tidy misc-const-correctness in heterogeneous sentinel test json_str is only read via .data()/.size() and never reassigned, so clang-tidy correctly flags it as const-able. Verified against the exact CI job (silkeh/clang:dev, ci_clang_tidy target) by running clang-tidy directly on this file plus the five binary-format sentinel tests touched by prior commits; all are now clean. Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
d0a43141ea |
Fix #3868: Remove operator!= to enable P2468R2 rewritten candidate synthesis (#5253)
* Fix #3868: Remove operator!= to enable P2468R2 rewritten candidate synthesis Under C++20 P2468R2, a hand-written operator!= suppresses the compiler's rewritten-candidate synthesis for operator==, preventing heterogeneous comparisons like `std::string s; json j; s == j;` from compiling. Fix by removing the hand-written operator!=, allowing the compiler to synthesize != as !(a==b) in all language modes (C++20 member functions and pre-C++20 friend functions). Behavior change: operator!= now returns !(a==b) unconditionally, including for special values like NaN and discarded. This means: - NaN != NaN now returns true (matches IEEE-754 semantics) - discarded != x now returns true for any x (matches !(discarded == x)) This also fixes underlying defects in previously-working code: - Restores direct == comparison for views vs json (reverts std::ranges::equal workaround added in PR #3950 to dodge this bug) - Re-enables std::string == json comparisons (uncomments check in unit-constructor1.cpp) Fixes: #3868, #3979 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Signed-off-by: Niels Lohmann <mail@nlohmann.me> * 🚨 fix warning Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Niels Lohmann
|
31dd15b258 |
Fix ambiguous static_cast (#5221)
* 🐛 fix ambiguous static_cast Signed-off-by: Niels Lohmann <mail@nlohmann.me> * ✅ add regression test Signed-off-by: Niels Lohmann <mail@nlohmann.me> * 🐛 fix warning Signed-off-by: Niels Lohmann <mail@nlohmann.me> * 🚨 fix warning Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
8d7e0046f4 |
Add std::format and fmt support (#5224)
* ✨ add std::format and fmt support Signed-off-by: Niels Lohmann <mail@nlohmann.me> * ♻️ reorganize PR Signed-off-by: Niels Lohmann <mail@nlohmann.me> * 💚 fix build Signed-off-by: Niels Lohmann <mail@nlohmann.me> * 💚 fix build Signed-off-by: Niels Lohmann <mail@nlohmann.me> * 💚 fix build Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
ca49ab6123 |
Extend value to arrays when using JSON pointers (#5223)
* ✨ extend value to arrays when using JSON pointers Signed-off-by: Niels Lohmann <mail@nlohmann.me> * 💚 avoid exceptions Signed-off-by: Niels Lohmann <mail@nlohmann.me> * 💚 avoid exceptions Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
730b57775d | 🐛 avoid assertion in patch (#5222) | ||
|
Niels Lohmann
|
272411c5e6 |
Overwork project infrastructure (#5218)
* 📝 overwork project infrastructure Signed-off-by: Niels Lohmann <mail@nlohmann.me> * 🐛 fix GCC16 issue Signed-off-by: Niels Lohmann <mail@nlohmann.me> * 🐛 fix GCC16 issue Signed-off-by: Niels Lohmann <mail@nlohmann.me> * 🐛 only build module for GCC Signed-off-by: Niels Lohmann <mail@nlohmann.me> * 🐛 fix build Signed-off-by: Niels Lohmann <mail@nlohmann.me> * 📝 fix documentation Closes #5012: fix the error_handler_t::ignore wording Signed-off-by: Niels Lohmann <mail@nlohmann.me> * 📝 fix documentation Closes #4354: fix "Custom data source" example Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
Cosmin D.
|
adf78d3a76 |
Minor: unique_ptr template resolution workaround for MSVC (#5215)
Signed-off-by: drcosmin <cosmin.dr@pm.me> |
||
Swastik Bose
|
584e6b1cfb |
Fix: update() parent pointers not updated after recursive merge with JSON_DIAGNOSTICS (#5187)
* added fix for issue 4813 Signed-off-by: VasuBhakt <cpswastik31@gmail.com> * added regression test for 4813 Signed-off-by: VasuBhakt <cpswastik31@gmail.com> * moved test from unit-regression2 to unit-diagnostics Signed-off-by: VasuBhakt <cpswastik31@gmail.com> --------- Signed-off-by: VasuBhakt <cpswastik31@gmail.com> |
||
Hariom Phulre
|
bb5404bb86 |
Fix GCC C++20 modules compilation #5103 (#5164)
* Fix: Add GCC diagnostic pragmas for C++ modules support (issue #5103) Signed-off-by: hariomphulre <hariiomphullre@gmail.com> * Fix: GCC C++20 modules with __cplusplus >= 202002L check (#5103) Signed-off-by: hariomphulre <hariiomphullre@gmail.com> Signed-off-by: hariomphulre <hariiomphullre@gmail.com> * Fix: Remove indentation from nested preprocessor directives and disable astyle preprocessor indentation Signed-off-by: hariomphulre <hariiomphullre@gmail.com> * Update amalgamated files with correct preprocessor directive indentation Signed-off-by: hariomphulre <hariiomphullre@gmail.com> * Fix GCC build for issue #5103; refresh amalgamated header Signed-off-by: hariomphulre <hariiomphullre@gmail.com> --------- Signed-off-by: hariomphulre <hariiomphullre@gmail.com> |
||
Akhilesh Arora
|
93e49decbd |
Fix incomplete-type error in set_parents with ordered_json (#5167)
When iteration_proxy_value<iter_impl<ordered_json>> appears in a context
that requires it to be complete (function or lambda parameter), the
compiler instantiates basic_json<ordered_map> and walks into
set_parents(iterator, typename iterator::difference_type)
while iterator is still incomplete, failing with "invalid use of
incomplete type".
basic_json::difference_type is already std::ptrdiff_t, so just naming
the underlying type directly avoids the dependent lookup. Behavior and
ABI are unchanged. This was the approach suggested in the issue thread.
Added a regression case in unit-ordered_json.cpp using the same trigger
pattern (lambda parameter naming the proxy type).
Fixes #3732
Signed-off-by: Akhilesh Arora <akhildawra@gmail.com>
|
||
SamareshSingh
|
62f3b41b30 |
fix: treat single-element brace-init as copy/move instead of wrapping in array (#5074) (#5090)
* fix: treat single-element brace-init as copy/move
When passing a json value using brace initialization with a single element
(e.g., `json j{someObj}` or `foo({someJson})`), C++ always prefers the
initializer_list constructor over the copy/move constructor. This caused
the value to be unexpectedly wrapped in a single-element array.
This bug was previously compiler-dependent (GCC wrapped, Clang did not),
but Clang 20 started matching GCC behavior, making it a universal issue.
Fix: In the initializer_list constructor, when type deduction is enabled
and the list has exactly one element, copy/move it directly instead of
creating a single-element array.
Before:
json obj = {{"key", 1}};
json j{obj}; // -> [{"key":1}] (wrong: array)
foo({obj}); // -> [{"key":1}] (wrong: array)
After:
json j{obj}; // -> {"key":1} (correct: copy)
foo({obj}); // -> {"key":1} (correct: copy)
To explicitly create a single-element array, use json::array({value}).
Fixes the issue #5074
Signed-off-by: Samaresh Kumar Singh <ssam3003@gmail.com>
* fix: regenerate amalgamated single_include/nlohmann/json.hpp
- Add missing comment from include/nlohmann/json.hpp explaining the
single-element brace-init fix (issue #5074)
- Fix extra 4-space indentation in embedded json_fwd.hpp section
Regenerated by running: make amalgamate
Signed-off-by: Samaresh Kumar Singh <ssam3003@gmail.com>
* Revert brace-init semantics change and fix amalgamation
The single-element brace-init change was a breaking change that cannot be accepted upstream. Reverted all related source, test, and doc changes, then regenerated single_include with correct indentation to pass the amalgamation CI check.
Signed-off-by: Samaresh Kumar Singh <ssam3003@gmail.com>
* Fix: add JSON_BRACE_INIT_COPY_SEMANTICS opt-in macro for issue #5074
Single-element brace initialization wrapping in an array cannot be fixed without breaking existing code. Added JSON_BRACE_INIT_COPY_SEMANTICS as an opt-in macro (default 0) so users can enable copy/move semantics for single-element brace init without affecting anyone relying on the current behavior.
Signed-off-by: Samaresh Kumar Singh <ssam3003@gmail.com>
* docs: add dedicated macro page and CI test target for JSON_BRACE_INIT_COPY_SEMANTICS
Signed-off-by: Samaresh Kumar Singh <ssam3003@gmail.com>
* fix: remove compiler-dependent assertions from #5074 regression test
Signed-off-by: Samaresh Kumar Singh <ssam3003@gmail.com>
* fix: use defined() guard for JSON_BRACE_INIT_COPY_SEMANTICS to satisfy -Wundef
Signed-off-by: Samaresh Kumar Singh <ssam3003@gmail.com>
* docs: fix section name in json_brace_init_copy_semantics.md to pass style check
Signed-off-by: Samaresh Kumar Singh <ssam3003@gmail.com>
* docs: move Default definition section before Notes to fix style check order
Signed-off-by: Samaresh Kumar Singh <ssam3003@gmail.com>
---------
Signed-off-by: Samaresh Kumar Singh <ssam3003@gmail.com>
|
||
risa2000
|
fd17b0889e |
Remove nullptr safety check from sax_parse functions (#5139)
PR #4873 introduced a safety check in sax_parse functions to catch nullptr passed as SAX parser object, which had been already annotated by JSON_HEDLEY_NON_NULL macro. Compilers (e.g. clang) which respected the non-null annotation tended to eliminate the safety check completely in optimized builds, while compilers which did not, compiled the safety check in. This led to different behaviors accross different compilers/platforms and/or build types (debug, release). This commit reverts PR #4873 to remove this discrepancy. Passing null to non-null annotated parameter is considered to be undefined behavior. Fixes #5048 Signed-off-by: Richard Musil <risa2000x@gmail.com> Co-authored-by: Richard Musil <risa2000x@gmail.com> |
||
|
Niels Lohmann
|
515d994acb |
📄 adjust year (#5044)
Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
29913ca760 | Add char8_t* overload for _json and _json_pointer (#4963) | ||
|
Niels Lohmann
|
54be9b04f0 | 📄 update REUSE (#4960) | ||
|
Niels Lohmann
|
1cc56b2dcd | Address CWG issue 2521 (#4957) | ||
|
Niels Lohmann
|
4106af8d92 |
🚨 suppress cppcoreguidelines-c-copy-assignment-signature,misc-unconventional-assign-operator (#4896)
Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
Nikhil
|
efcf9efb4f |
Fixes #4854 Explicitly handle nullptr in sax_parse (#4873)
* handle nullptr explicitly Signed-off-by: Nikhil <nikhilreddydev@gmail.com> * add test Signed-off-by: Nikhil <nikhilreddydev@gmail.com> * make amalgamate Signed-off-by: Nikhil <nikhilreddydev@gmail.com> * Fix formatting Signed-off-by: Nikhil <nikhilreddydev@gmail.com> * move sax parse test to relevant unit test file Signed-off-by: Nikhil <nikhilreddydev@gmail.com> * extend exceptions.md to include other_error.502 Signed-off-by: Nikhil <nikhilreddydev@gmail.com> * Better exception messages Signed-off-by: Nikhil <nikhilreddydev@gmail.com> * link sax_parse function Signed-off-by: Nikhil <nikhilreddydev@gmail.com> * fix string Signed-off-by: Nikhil <nikhilreddydev@gmail.com> * amalgamate Signed-off-by: Nikhil <nikhilreddydev@gmail.com> * fix clang-tidy checks Signed-off-by: Nikhil <nikhilreddydev@gmail.com> * cover valid handler with no throw Signed-off-by: Nikhil <nikhilreddydev@gmail.com> * Add tests for other two overloads Signed-off-by: Nikhil <nikhilreddydev@gmail.com> * cover overload with valid sax handler Signed-off-by: Nikhil <nikhilreddydev@gmail.com> * pass an rvalue Signed-off-by: Nikhil <nikhilreddydev@gmail.com> * ignore -Wtautological-pointer-compare Signed-off-by: Nikhil <nikhilreddydev@gmail.com> * ignore clang-analyzer-core.NonNullParamChecker Signed-off-by: Nikhil <nikhilreddydev@gmail.com> * ignore gcc -Wnonnull-compare Signed-off-by: Nikhil <nikhilreddydev@gmail.com> * ignore undefined-behaviour-sanitizer Signed-off-by: Nikhil <nikhilreddydev@gmail.com> * nest directives to ignore sanitizer errors Signed-off-by: Nikhil <nikhilreddydev@gmail.com> * use elif Signed-off-by: Nikhil <nikhilreddydev@gmail.com> --------- Signed-off-by: Nikhil <nikhilreddydev@gmail.com> |
||
chirsz
|
4b17f90f65 |
Add ignore_trailing_commas option (#4609)
Added examples and modified the corresponding documents and unit tests. Signed-off-by: chirsz-ever <chirsz-ever@outlook.com> Co-authored-by: Niels Lohmann <niels.lohmann@gmail.com> |
||
|
Niels Lohmann
|
828c891427 |
Extend type_name() to invalid type (#4786)
* ✅ add regression test Signed-off-by: Niels Lohmann <mail@nlohmann.me> * 💚 fix build Signed-off-by: Niels Lohmann <mail@nlohmann.me> * 📝 add comment Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
9110918cf8 |
Fix typos (#4748)
* ✏️ fix typos Signed-off-by: Niels Lohmann <mail@nlohmann.me> * ✏️ address review comments Signed-off-by: Niels Lohmann <mail@nlohmann.me> * ✏️ address review comments Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
1705bfe914 |
🔖 set version to 3.12.0 (#4727)
Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
f06604fce0 |
Bump the copyright years (#4606)
* 📄 bump the copyright years Signed-off-by: Niels Lohmann <mail@nlohmann.me> * 📄 bump the copyright years Signed-off-by: Niels Lohmann <mail@nlohmann.me> * 📄 bump the copyright years Signed-off-by: Niels Lohmann <niels.lohmann@gmail.com> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> Signed-off-by: Niels Lohmann <niels.lohmann@gmail.com> |
||
Nebojša Cvetković
|
2e50d5b2f3 | BJData optimized binary array type (#4513) | ||
Michael Valladolid
|
4f64d8d0b4 |
Modernize integer comparison (#4577)
Replace static_cast<size_t>(-1) with std::numeric_limits<std::size_t>::max() via the detail::unknown_size() function |
||
|
Niels Lohmann
|
6057b31df7 |
Overwork astyle call (#4573)
* 🔧 overwork astyle call * 🔧 overwork astyle call * 🔧 overwork astyle call * 🔧 overwork astyle call * 🔧 overwork astyle call * 🔧 overwork astyle call * 🔧 overwork astyle call * 🔧 overwork astyle call * 🔧 overwork astyle call * 🔧 overwork astyle call * 🔧 overwork astyle call * Use ubuntu-latest image to run Valgrind (#4575) * 🔧 use Clang image to run valgrind * 🔧 use Clang image to run valgrind * 🔧 use Clang image to run valgrind * 🔧 use Ubuntu image to run valgrind * Use Clang image to run iwyu (#4574) * 🔧 use Clang image to run iwyu * 🔧 use Clang image to run iwyu * 🔧 overwork astyle call * 🔧 overwork astyle call * 🔧 overwork astyle call * 🔧 overwork astyle call * 🔧 overwork astyle call * 🔧 overwork astyle call * 🔧 overwork astyle call * 🔧 overwork astyle call * 🔧 overwork astyle call * 🔧 overwork astyle call * 🔧 overwork astyle call * 🎨 format code * 🔨 clean up |
||
Harinath Nampally
|
5ff90d9e02 | fix diagnostic positions related compilation errors (#4570) | ||
Sushrut Shringarputale
|
58f5f25968 |
json start/end position implementation (#4517)
* Add implementation to retrieve start and end positions of json during parse * Add more unit tests and add start/stop parsing for arrays * Add raw value for all types * Add more tests and fix compiler warning * Amalgamate * Fix CLang GCC warnings * Fix error in build * Style using astyle 3.1 * Fix whitespace changes * revert * more whitespace reverts * Address PR comments * Fix failing issues * More whitespace reverts * Address remaining PR comments * Address comments * Switch to using custom base class instead of default basic_json * Adding a basic using for a json using the new base class. Also address PR comments and fix CI failures * Address decltype comments * Diagnostic positions macro (#4) Co-authored-by: Sush Shringarputale <sushring@linux.microsoft.com> * Fix missed include deletion * Add docs and address other PR comments (#5) * Add docs and address other PR comments --------- Co-authored-by: Sush Shringarputale <sushring@linux.microsoft.com> * Address new PR comments and fix CI tests for documentation * Update documentation based on feedback (#6) --------- Co-authored-by: Sush Shringarputale <sushring@linux.microsoft.com> * Address std::size_t and other comments * Fix new CI issues * Fix lcov * Improve lcov case with update to handle_diagnostic_positions call for discarded values * Fix indentation of LCOV_EXCL_STOP comments * fix amalgamation astyle issue --------- Co-authored-by: Sush Shringarputale <sushring@linux.microsoft.com> |
||
|
Niels Lohmann
|
094bd2651b |
Set parents after insert call (#4537)
* 🐛 set parents after insert call * 🚨 fix warning |