Conversation
Similar to PyFloat_FromString(), but don't require a Python object. Use _PyFloat_FromString() in _json to avoid creating a temporary bytes objects. _Py_string_to_number_with_underscores() can now be called with NULL object and float_from_string_inner() can now be called with NULL data; if needed they create a temporary bytes objects to format the error message.
|
Benchmark: import pyperf
import json
import random
NUMBERS = 10 ** 4
ITEMS = [str(random.random()) for _ in range(NUMBERS)]
DATA = '[' + ', '.join(ITEMS) + ']'
runner = pyperf.Runner()
runner.bench_func('json.loads', json.loads, DATA)Result: |
Fix the compiler warning:
{'file': 'Objects/floatobject.c', 'line': '174', 'column': '5',
'message': 'label followed by a declaration is a C23 extension',
'option': '-Wc23-extensions'}
sprajs
left a comment
There was a problem hiding this comment.
Reviewed 68ecdf378bb7e403f5a9dcd935933c1dcd45293f against merge base 1620e0f1b59c2078e33d82bef75df3db3c651d1c.
No correctness regression reproduced in 5,252 C-versus-Python-scanner cases, 24 exact-token custom-hook checks, and 400 allocation-failure offsets. These cover 19/20-digit and signed/unsigned boundaries, negative zero, subnormal/overflow cases, exponent extremes, long buffers, 1/2/4-byte Unicode inputs and conversion limits. Exact debug base/head builds pass 328 JSON/float/complex tests (4 skipped); head also passes -R 3:3 on all three suites.
I compared the two overlapping proposals using one PGO/LTO core built from #156897's head. The baseline _json.c sources are identical; each of baseline, #156897 and #150639 was compiled with the same -O3 flags and headers, without module-specific PGO. This isolates the decoder components:
| Workload | base | 156897 | 150639 |
|---|---|---|---|
| float_array_10000 | 1.96 ms | 1.89 ms | 1.75 ms |
| numeric_records_2000 | 1.65 ms | 1.63 ms | 1.52 ms |
| pyperformance_json_loads | 20.00 µs | 19.88 µs | 19.00 µs |
Inputs: 10,000 successive random() calls on one Random(156897) instance (the float-array workload from #156897's discussion, with a fixed seed); 2,000 records with integer IDs, two floats and a boolean; and the three-object pyperformance json_loads fixture. The latter is timed with its original 20-call unrolling and normalization. #156897 still allocates the intermediate character buffer; #150639 additionally provides integer and stack-buffer fast paths.
Separately, exact native PGO/LTO builds of this PR's base and head gave:
| Workload | base | head |
|---|---|---|
| float_array_10000 | 1.94 ms | 1.92 ms |
| numeric_records_2000 | 1.72 ms | 1.69 ms |
| pyperformance_json_loads | 22.65 µs | 22.45 µs |
The native float-array medians give 1.01× here, with overlapping observed ranges (1.91 ms–2.00 ms base, 1.90 ms–1.92 ms head). I did not reproduce the reported 1.17× improvement in this setup.
Timings are medians from seven interleaved fresh processes on Linux x86-64, i9-9900K/GCC 16.2.1, pinned to one logical CPU; GC disabled during timed loops, hash seed 0, calibration to 65 ms followed by a separately timed batch. Builds and tests had finished before timing; the desktop was not frequency-isolated. These are workload-specific measurements, not application-wide speedups.
Review and testing performed with Codex (AI assistance).
Add internal _PyFloat_FromString(): similar to PyFloat_FromString(),
but don't require a Python object.
Use _PyFloat_FromString() in _json to avoid creating a temporary bytes objects.
_Py_string_to_number_with_underscores() can now be called with NULL object and float_from_string_inner() can now be called with NULL data; if needed they create a temporary bytes objects to format the error message.