Skip to content

perf(logger): reduce encoding allocations and filtered trace work - #249

Merged
jinxiao merged 1 commit into
mainfrom
perf/logger-encoding-hot-path
Oct 15, 2026
Merged

jinxiao merged 1 commit into
mainfrom
perf/logger-encoding-hot-path

Conversation

@jinxiao

@jinxiao jinxiao commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

Issue

Refs #248

Summary

Logger previously copied and reflectively traversed ordinary field maps, marshaled every root key and value separately, rebuilt duplicate ordering state, and looked up buffer trace data even when buffering was disabled. Use one JSON v2 encoder over an owned normalized root, direct map/scalar traversal, and conditional trace lookup. Keep full-record buffering before the single sink write and append its newline without an extra copy.

Preserve ordering and omissions, original formatter/replacer inputs, custom marshaler errors, dynamic trace precedence, invocation closure, buffering and the main-only stack Preview. Add warm-invocation/direct-JSON controls and targeted regression coverage; record the verified local milestone and measured scope.

Testing

  • Logger regression suite and native integration/loggerevent Lambda SDK fixture passed with CGO_ENABLED=0.
  • Packaged Logger and examples tests/vet/tidy/isolation passed with GOWORK=off, including one independent public Logger consumer.
  • Composed Lambda example cross-builds passed for Linux amd64 and arm64; both ELF headers have the expected machine and no PT_INTERP.
  • Same-source Windows amd64 benchmarks, CGO disabled, GOMAXPROCS=2, one 300ms observation each: emitted INFO 118 -> 34 allocations; invocation-bound INFO 193 -> 53; full wrapper/INFO 208 -> 68. Both filtered cases: zero allocations. Output sizes remain unchanged (208/404 bytes).
  • Targeted tests cover custom/duplicate root ordering and escaped names, formatter-owned input reuse and concrete root types, large streaming-marshaler failure atomicity, dynamic trace/OTel fallback, closed invocations, and zero-allocation filtering. Existing stack/panic, custom marshaler/replacer, buffering and concurrency tests pass.
  • git diff --check passed. Hosted CI run 38759475302 passed documentation and all three affected packaged modules (Logger, examples, integration), including one independent consumer. The change selector skipped the separate runtime-simulation job; no new Docker runtime acceptance is claimed, and local Docker was not retried after its earlier startup failure.

Risks and compatibility

No public API, dependency, unsafe/CGO, version tag or Go release changes. Source contains the existing main-only stack Preview; v1.2.0 remains unchanged. Only owned normalized root maps and encoded records are consumed; caller and hook inputs remain intact. Sink writers must follow io.Writer's existing ownership contract.

Local counting-writer measurements identify overhead, not CloudWatch/stdout latency or predicted AWS Lambda improvement. No AWS performance tests or race detector were run. Broader wrapper/cold-start and other-module optimizations remain outside this scoped fix.

Release notes

  • logger | maintenance | Reduce structured logging allocations and avoid unused trace lookups on filtered records.

Checklist

  • The issue records the scope and acceptance criteria
  • Tests and documentation match the change, or I explained why they are not applicable
  • I kept CGO disabled and did not run cloud AWS tests without authorization
  • I excluded credentials, personal configuration, and generated build artifacts

@jinxiao

jinxiao commented Oct 15, 2026

Copy link
Copy Markdown
Contributor Author

Author self-review of ab093bc completed; no additional defect identified.

  • Only normalized, private root maps are consumed during ordered encoding. Formatter/replacer inputs and nested caller containers retain their ownership.
  • JSON v2 still buffers the entire document before the sink write; failure after a large streaming value produces no partial log. Duplicate custom/standard ordering and invalid/escaped names retain coverage.
  • Filtering checks invocation closure first and preserves error-buffer flushing. Trace lookup remains dynamic for emitted records, and an active OTel span takes precedence over runtime fallback.
  • Output stays a single io.Writer call; encoded bytes are owned by the record and io.Writer's no-retention contract is unchanged.
  • Preview stacks remain main-only, public APIs/dependencies and published versions are unchanged.
  • The completed local packaged-module check covers Logger and examples plus one independent Logger consumer. Linux integration/runtime CI remains the merge gate. Local benchmark timings are scoped observations, with no AWS latency claim.

@jinxiao
jinxiao merged commit d991b5a into main Oct 15, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant