Skip the indentation after a newline 8 bytes at a time (#5810)

Pretty-printed JSON indents every line with a run of spaces, which
skip_whitespace() read one character at a time. For contiguous input
(bulk_scan) the lexer now skips the spaces after a newline 8 bytes at a
time directly in the input buffer, including the leading spaces of the
last, partial word. This bypasses get(), which is safe because
bulk_scan implies the lazy position from the previous commit: nothing
is counted per character.

Compared to lazy positions alone, pretty-printed input (twitter,
citm_catalog with indent 2 and 4) parses up to 13% faster and accept()
gets up to 30% faster (GCC 13 and clang 18 on x86-64, Apple clang 21 on
arm64). Minified input is unchanged within noise.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
This commit is contained in:
Niels Lohmann
2026-10-13 17:16:52 +02:00
committed by GitHub
parent f23830b65e
commit aa9620fb15
6 changed files with 227 additions and 25 deletions

View File

@@ -189,7 +189,7 @@ random-access input of single-byte characters:
`count_consumed_lines()`, the lexer also derives the read position from the adapter and counts lines only when an
error needs them, instead of counting characters and lines on every read.
- `supports_bulk_scan`, `bulk_data()`, `bulk_remaining()`, and `bulk_skip()` let the lexer scan strings directly in
contiguous memory, several bytes at a time.
contiguous memory, several bytes at a time, and skip the spaces that indent a line in 8-byte steps.
The function `input_adapter` picks the right adapter for the argument passed to `parse`, `accept`, `sax_parse`, or the
`from_*` functions: