Repository navigation
perf: reduce JSON v2 snapshot and normalization overhead - #253
Conversation
|
Author review completed against e300c95: checked bounded snapshot fallback, float32/integer conversion and invalid unrelated members; caller/result isolation and concurrent copies; no-callback root cycle equivalence and preserved replacer index keys; exact-number validation decoding and original diagnostic snapshots. Local four-module packaged acceptance, static amd64/arm64 Lambda builds and documentation checks pass. No outstanding defect identified in this review; no independent approval is claimed. Hosted CI is still running. |
|
Follow-up author review for ff134c1: reproduced a public-proxy build failure in the initial commit because released root v1.2.0 lacks Snapshot, then pinned the implementation commit and synchronized internal consumers/workspace metadata with tools/modules.py tidy. Four packaged modules/consumers pass again. A standalone consumer fetched ff134c1 Logger/JMESPath from the public proxy/checksum database and successfully executed correlation, query and stack Preview APIs with CGO disabled, GOWORK off and no replacements. Performance source hashes are unchanged; no redundant benchmark rerun. Also normalized PR-body line endings after the metadata editor caused the contribution-section parser to fail. Full hosted regression for the updated metadata is running. |
|
Final acceptance for ff134c1: full module/independent-consumer and dual-architecture artifact checks, native Lambda/streaming/Valkey simulation, KMS interoperability, DynamoDB Local, documentation, workflow metadata, release tooling, CI gate and Full regression all passed in https://github.com/rambow-cloud/powertools-lambda-go/actions/runs/38822206771 . Latest contribution policy passed after normalizing the body format. Public-proxy/checksum consumers also pass independently with GOWORK off, CGO disabled and no replacements. The implementation and internal dependency fix are ready for review; no merge, AWS test, tag or Go release was performed. |
Issue
Closes #250
Closes #251
Closes #252
Summary
Correlation extraction and JMESPath queries no longer encode/decode complete ordinary JSON-shaped inputs just to obtain private snapshots. A shared bounded copy preserves JSON v2 float64/default-container semantics; custom types, raw JSON, invalid values and deep/cyclic inputs retain the authoritative JSON v2 round trip. Unrelated invalid fields still fail instead of being bypassed by a direct lookup.
Logger combines root preparation/normalization only when callbacks cannot observe the intermediate root, traverses plain slices directly and avoids unused index strings. Validation directly decodes already serialized snapshots with exact-number options, retaining bytes for diagnostic ordering.
Identical-source local Windows amd64 benchmarks (Go 1.27.1, CGO disabled, GOMAXPROCS=2, one 500ms run) show correlation 4.41→0.75 ms, JMESPath lookup 4.21→0.79 ms, large event logging 5.08→4.31 ms and large string Validation 4.76→2.96 ms. Bound INFO allocations decrease from 53 to 48; its 404-byte output and the direct JSON control remain unchanged. Full results, fixture boundaries and source hashes are in
docs/PERFORMANCE_JSON_V2_250.json; these observations do not predict AWS latency.Testing
CGO_ENABLED=0 uv run --no-project python tools/modules.py check --only . --only logger --only jmespath --only validation: four packaged modules pass tests/vet/tidy/isolation and four independent consumers build with GOWORK off.Risks and compatibility
No public API, external dependency, parser, unsafe code or release-version change. Logger/JMESPath pin the internal root implementation to
v1.2.1-0.20261015225926-e300c9596f35; tidy propagates this internal requirement to affected consumers and go.work maps it locally. No filesystem replacements are added to module files. The fast snapshot path is deliberately restricted and bounded; the existing JSON v2 path remains for unsupported representations and authoritative errors. A late unsupported member can add speculative-copy work before fallback, so the measured gains apply to the ordinary JSON-shaped fixtures rather than every Go representation. Default no-callback Logger root handling is compared against the existing formatter path, including caller-owned root cycles. Snapshot/error normalization work can remain substantial for custom or replacer-heavy inputs; generic callback behavior is preserved.docs/CHECKLIST.mdrecords scoped local acceptance. Broader resource budgets and unrelated runtime/service gates remain open.Release notes
Checklist