Scarfski/json
1
0
Fork 0
mirror of https://github.com/nlohmann/json.git synced 2026-10-12 20:05:23 +00:00
Code Issues Packages Projects Releases Wiki Activity

Write doubles with the shortest digits (Żmij), digits in registers

Browse Source 
Write doubles with the conversion of Zmij by Victor Zverovich (MIT),
ported to C++11 (detail/conversions/zmij.hpp). It finds the
shortest decimal that reads back as the same double, and the
closest one if there are several. Grisu2, used until now, is fast
but not always shortest: it sometimes writes a 17th digit where 16
suffice, or a last digit that is not the closest. The layout is
unchanged (1.5, 100.0, 1e+100, -0.0); float keeps Grisu2.

Digits are converted eight at a time with the BCD conversion of
Xiang JunBo, as in Zmij, and written with one byte swap per eight
digits and fixed-size moves instead of per-digit loops. Leading and
trailing zeros are counted from those bytes. to_chars() uses a
local buffer when the caller's is shorter than the 41 bytes this
may write. The powers of ten come from the number-parsing table,
adjusted where it holds values rounded up, and extended with Zmij's
compressed tables beyond 10^308.

write_shortest() converts its 16 digits in one vector register
(SSE2 on x86-64, NEON on 64-bit Arm, both baseline) and inserts the
decimal point inside the register, avoiding a store-forwarding
stall that cost about 25% of the time to write a double. dump()
writes floats and integers straight into the serializer's write
buffer instead of copying them from a member buffer, and small
integers eight digits at a time. read_eight_bytes() and
parse_eight_digits() are marked always-inline, which GCC had been
calling out of line in the number-parsing loops.

Of one million random doubles, about 0.14% are now written with
different digits, always to a value that still reads back as the
same double.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
This commit is contained in:
Niels Lohmann
2026-10-06 10:48:06 +02:00
parent 247b03b2de
commit f6895fb84e
14 changed files with 1054 additions and 63 deletions
Download Patch File Download Diff File Expand all files Collapse all files

22
tests/src/unit-binary_formats.cpp
View File

@@ -38,7 +38,7 @@ TEST_CASE("Binary Formats")
const auto ubjson_2_size = json::to_ubjson(j, true).size();
const auto ubjson_3_size = json::to_ubjson(j, true, true).size();
CHECK(json_size == 2090303);
CHECK(json_size == 2090234);
CHECK(bjdata_1_size == 1112030);
CHECK(bjdata_2_size == 1224148);
CHECK(bjdata_3_size == 1224148);
@@ -51,16 +51,16 @@ TEST_CASE("Binary Formats")
CHECK(ubjson_3_size == 1169069);
CHECK((100.0 * double(json_size) / double(json_size)) == Approx(100.0));
CHECK((100.0 * double(bjdata_1_size) / double(json_size)) == Approx(53.199));
CHECK((100.0 * double(bjdata_2_size) / double(json_size)) == Approx(58.563));
CHECK((100.0 * double(bjdata_3_size) / double(json_size)) == Approx(58.563));
CHECK((100.0 * double(bon8_size) / double(json_size)) == Approx(50.509));
CHECK((100.0 * double(bson_size) / double(json_size)) == Approx(85.849));
CHECK((100.0 * double(cbor_size) / double(json_size)) == Approx(50.497));
CHECK((100.0 * double(msgpack_size) / double(json_size)) == Approx(50.526));
CHECK((100.0 * double(ubjson_1_size) / double(json_size)) == Approx(53.199));
CHECK((100.0 * double(ubjson_2_size) / double(json_size)) == Approx(58.563));
CHECK((100.0 * double(ubjson_3_size) / double(json_size)) == Approx(55.928));
CHECK((100.0 * double(bjdata_1_size) / double(json_size)) == Approx(53.201));
CHECK((100.0 * double(bjdata_2_size) / double(json_size)) == Approx(58.565));
CHECK((100.0 * double(bjdata_3_size) / double(json_size)) == Approx(58.565));
CHECK((100.0 * double(bon8_size) / double(json_size)) == Approx(50.511));
CHECK((100.0 * double(bson_size) / double(json_size)) == Approx(85.853));
CHECK((100.0 * double(cbor_size) / double(json_size)) == Approx(50.499));
CHECK((100.0 * double(msgpack_size) / double(json_size)) == Approx(50.528));
CHECK((100.0 * double(ubjson_1_size) / double(json_size)) == Approx(53.201));
CHECK((100.0 * double(ubjson_2_size) / double(json_size)) == Approx(58.565));
CHECK((100.0 * double(ubjson_3_size) / double(json_size)) == Approx(55.930));
}
SECTION("twitter.json")