Add tests for uncovered code paths (#5581)

* Add tests for uncovered code paths

Cover code the test suite did not reach, found from the Coveralls report
of develop and a local coverage run of HEAD:

- dump() of every kind of value below the bound of the recursive descent
  (pretty-printed objects, binary values, discarded values, scalars), and
  flushes of the escape and write buffers mid-string and mid-binary
- the iterative comparison: objects with different keys, containers that
  are a prefix of each other, and elements that cannot be ordered, each
  both at the top level and below the nesting bound
- SAX handlers that stop at any event, including the end of a nested
  container, in the BSON, CBOR, MessagePack, UBJSON and BJData readers
- from_bson/cbor/msgpack/ubjson/bjdata returning a discarded value
  through the iterator and pointer overloads
- JSON Patch, diff, merge_patch and update(..., true) on ordered_json
- smaller gaps: get_allocator(), to_ubjson/to_bjdata into a string,
  value() with an unresolvable JSON pointer, integer/float comparison
  below the integer range and with negative fractions, conversion to a
  custom binary type, std::formatter::parse on a spec without '}',
  unescape() of a lone '~', and the callback parser's start_array()

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Cover more paths that were thought unreachable

- parse_float_fast() declining malformed or inexact input, called
  directly since the lexer only passes well-formed numbers to it
- a UTF-16 high surrogate followed by a unit above the low surrogates
- self-assignment of a const_iterator
- a truncated CBOR string read through non-contiguous iterators
- serializing a long double under the de_DE locale, which undoes the
  locale's decimal point and thousands separator
- values read from a binary format carrying no diagnostic positions,
  with and without a parser callback

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix the CI failures of the new coverage tests

- declare the self-assignment reference const (misc-const-correctness)
- expect the (/path) prefix that JSON_DIAGNOSTICS adds to the messages
  of the failing ordered_json patch operations

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Expect the byte range JSON_DIAGNOSTIC_POSITIONS adds to the patch errors

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Build the expected dump of the nested-object test with +=

clang-tidy (performance-inefficient-string-concatenation) reported the
chain of operator+ calls that assembled the expected indented output.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Compare the BJData and UBJSON test outputs byte by byte

Building a std::string from the byte vector converts each byte
implicitly, which -fsanitize=integer reports for bytes of 0x80 and
above (ci_test_clang_sanitizer).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

tests/src/unit-ubjson.cpp

@@ -1640,6 +1640,29 @@ TEST_CASE("UBJSON")
                 });
                 CHECK_THROWS_AS(_ = json::sax_parse(v_ubjson, &scp, json::input_format_t::ubjson), json::out_of_range&);
             }
+
+            SECTION("array with a known size, read with a callback")
+            {
+                // a sized array announces its length to start_array()
+                std::vector<uint8_t> const v_ubjson = {'[', '#', 'i', 2, 'i', 1, 'i', 2};
+                json j;
+                nlohmann::detail::json_sax_dom_callback_parser<json, decltype(nlohmann::detail::input_adapter(v_ubjson))> scp(j, [](int /*unused*/, json::parse_event_t /*unused*/, const json& /*unused*/) noexcept
+                {
+                    return true;
+                });
+                CHECK(json::sax_parse(v_ubjson, &scp, json::input_format_t::ubjson));
+                CHECK(j == json({1, 2}));
+
+                // the readers reject a size this large before they announce
+                // it, so it can only reach start_array() directly (the largest
+                // value stands for an unknown size and is never checked)
+                json k;
+                nlohmann::detail::json_sax_dom_callback_parser<json, decltype(nlohmann::detail::input_adapter(v_ubjson))> scp2(k, [](int /*unused*/, json::parse_event_t /*unused*/, const json& /*unused*/) noexcept
+                {
+                    return true;
+                });
+                CHECK_THROWS_AS(scp2.start_array((std::numeric_limits<std::size_t>::max)() - 1), json::out_of_range&);
+            }
         }
     }
 
@@ -2255,6 +2278,46 @@ TEST_CASE("UBJSON nesting does not consume the call stack")
     }
 }
 
+TEST_CASE("UBJSON input that cannot be read is discarded by every overload")
+{
+    std::vector<std::uint8_t> input = json::to_ubjson(json({{"a", {1, 2}}}));
+    input.pop_back();
+
+    json _;
+    CHECK_THROWS_AS(_ = json::from_ubjson(input.begin(), input.end()), json::parse_error&);
+    CHECK(json::from_ubjson(input, true, false).is_discarded());
+    CHECK(json::from_ubjson(input.begin(), input.end(), true, false).is_discarded());
+    CHECK(json::from_ubjson(input.data(), input.size(), true, false).is_discarded());
+    CHECK(json::from_ubjson({input.data(), input.size()}, true, false).is_discarded());
+}
+
+TEST_CASE("UBJSON SAX parsing stops at every event")
+{
+    // Containers are opened and closed by the loop that reads them; a SAX
+    // handler that rejects any event - including the end of a nested
+    // container - must stop the parse right there.
+    const auto count_events = [](const std::vector<std::uint8_t>& input)
+    {
+        int events = 0;
+        while (true)
+        {
+            SaxCountdown scp(events);
+            if (json::sax_parse(input, &scp, json::input_format_t::ubjson))
+            {
+                return events;
+            }
+            ++events;
+            REQUIRE(events < 1000);
+        }
+    };
+
+    // 20 events: every container kind closes inside another one
+    const json j = json::parse(R"({"a": [1, {"b": []}], "c": {"d": [[2]]}})");
+    CHECK(count_events(json::to_ubjson(j)) == 20);
+    CHECK(count_events(json::to_ubjson(j, true)) == 20);
+    CHECK(count_events(json::to_ubjson(j, true, true)) == 20);
+}
+
 TEST_CASE("UBJSON optimized arrays of a valueless type are bounded")
 {
     // An element of type 'Z', 'T' or 'F' is encoded by its marker alone, so an