mirror of
https://github.com/nlohmann/json.git
synced 2026-10-01 14:35:18 +00:00
18dd5663b0
* Diff deeply nested values without recursing per nesting level diff() descended into both values once per nesting level, and compared them with operator== on every level on the way, which recurses as well. Values nested deeply enough - 25,000 levels on an 8 MiB stack - exhausted the call stack and terminated the process, although parse() accepts them without complaint. On such a chain the per-level comparisons and path strings also made diff() quadratic in time and memory. Both the recursion and operator== only descend as far as the source is nested. So diff() first checks, recursing at most diff_depth_limit() (128) levels, whether the source is nested more deeply than that. If not - all but a vanishing minority of values - the recursive algorithm diffs it exactly as before, now as diff_recursively(). Otherwise diff_iteratively() walks the two values on an explicit stack, emitting the same operations in the same order. It does not compare arrays and objects with operator== up front (equal ones yield no operations anyway), keeps the path in one buffer instead of a new string per level, and hands every subtree that is not nested too deeply back to diff_recursively(), so equal parts are still skipped quickly. The check costs one pass over the source. On a 3,000-object document that is about 30% of diffing two equal values (which is just an operator== call), about 10% of diffing values that differ in a few places, and noise when arrays change length. Once operator== no longer recurses (#5390), the check can go. Tests check that the patch reproduces the target at every depth up to 300, for json and ordered_json, including reordered members. They also check the exact operation for a difference deep inside, and diff values nested 100,000 levels deep. Fixes #5393 for diff(). Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Make diff_frame a member struct that declares its special members GCC's -Weffc++ (an error in CI) asks a class with pointer members, a user constructor and a non-trivial destructor to declare its copy constructor and copy assignment; diff_frame's vector and basic_json members make its destructor non-trivial. Declare all five as defaulted, which also satisfies clang-tidy's special-member-functions check. Leave their exception specifications implicit: GCC 4.8 rejects an explicit one that differs from the implicit one, as it does for flatten_task in #5517. The converting constructor cannot throw, and is now declared noexcept for GCC's -Wnoexcept, which flags the emplace_back() under C++26 otherwise. The struct also moves from diff_iteratively() into the class, like dump_frame in the serializer. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Use the shared recursion limit in diff() diff_depth_limit() is gone in favor of detail::recursion_depth_limit(). Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Diff fewer nesting depths so the test does not time out under Valgrind Checking every depth up to 300 made test-json_patch exceed the 1500 s ctest timeout in ci_test_valgrind. Check the depths up to 16, those around the recursion limit of 128, and 300 instead. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Mark the diff frame's value-initialized members for clang-tidy The braces are kept for GCC's -Weffc++, as in json_sax.hpp. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Bound diff()'s descent with a depth count instead of scanning the source Now that operator== no longer recurses (#5390), diff() can keep its per-level equality shortcut all the way down. It diffs recursively for the first detail::recursion_depth_limit() levels, as merge_patch() does, and hands anything deeper to diff_iteratively(). The nesting_exceeds() scan, which cost about 30% on equal documents, is gone, and diff() is on par with develop again. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Note that the diff frame reference is invalidated by pop_back() too Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Keep diff()'s recursive levels small and its result elided diff_recursively built every patch operation in place from initializer lists. Unoptimized builds give each of those temporaries its own stack slot, so every level of the bounded descent cost kilobytes of stack (about 6 KB with clang -O0), and the 128 recursive levels overflowed the 1 MB stack of MSVC Debug in the "deeply nested values" test. The operations and the key comparison of two objects are now built by separate functions, which diff_iteratively shares, and both diff functions append to one result instead of returning a patch per level that the caller copies. With clang -O0, diffing values nested 300 levels deep now peaks at about 190 KB of stack instead of 880 KB. Since diff() now owns the only returned value, clang's -Wnrvo no longer reports the returns of diff_recursively, which alternated between the local patch and diff_iteratively's result. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Copy the diff frame's members instead of holding a reference to it The loop in diff_iteratively held a reference to the top frame, which enter() invalidates when it pushes and the end of the loop invalidates when it pops. Nothing used it afterwards, but a later change could. As in the other iterative walks, the members the loop reads are now copied out as constants and the ones it advances are changed through stack.back(). The frame as a whole is not copied: it holds the common keys and the "add" operations of an object. Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me>