Skip to content

feat(tracing): add opt-in commit SHA stamping for SGP spans - #505

Open
cyntwang99 wants to merge 3 commits into
nextfrom
cynthiawang/agx1-969-commit-sha-optin
Open

feat(tracing): add opt-in commit SHA stamping for SGP spans#505
cyntwang99 wants to merge 3 commits into
nextfrom
cynthiawang/agx1-969-commit-sha-optin

Conversation

@cyntwang99

@cyntwang99 cyntwang99 commented Aug 28, 2026

Copy link
Copy Markdown

What

Adds opt-in stamping of the agent's source commit onto its spans, so a request
in the SGP Traces UI can be tied back to the code that served it:

search  __commit_sha__:<full-sha>

Part of AGX1-969 / the
Version Request Visibility
scope doc, Dimension 1 (Code Revision).

Why span metadata is the only option

Agent spans never reach the OTel collector — the SDK POSTs them straight to SGP,
agent pods set no OTEL_* env vars, and there is no auto-instrumentation
annotation. So the commit has to be attached in the agent's own process, and
span.data (→ operation_metadata, indexed, no key allowlist) is the carrier.

Opt-in

Nothing is emitted until an agent calls:

from agentex.lib import adk
adk.code_revision.enable()          # value from AGENT_COMMIT_SHA

Modelled on the lineage registry next door — the module global's default is
the off state, so commit_sha() returns None and the stamp site needs no flag,
no config field, and no env toggle. No agent inherits this field by upgrading
the SDK.
The existing test asserting span.data == {...} exactly still passes,
which proves that independently.

Why not just reuse __agent_version__

__agent_version__ (v0.25.0, #469) is automatic and carries the deployed image
tag verbatim. That tag is a real commit on some paths but not others:

Deploy path image.tag
CI → GCP / Azure b362b171… — a real SHA
CI → AWS ECR rocket_mock_agent-b362b171… — the gateway rewrites it to <image-name>-<tag>
Local agentex agents deploy latest — 129 of 133 fleet manifests declare it
Manual dispatch whatever was passed

A field named for a commit must not mirror that. Values that are not git object
names (7–64 hex) are refused with a warning, so __commit_sha__ only ever holds
one — absence is the honest answer where a commit isn't known.

Value precedence

enable() argument → AGENT_COMMIT_SHAAGENT_VERSION, the last only when
already SHA-shaped. So an opting-in agent on GCP/Azure needs no extra plumbing,
while a custom build path bakes the value at build time:

ARG GIT_SHA
ENV AGENT_COMMIT_SHA=${GIT_SHA}

A bad explicit value stops with a warning rather than silently falling through
to the image tag — setting it is a statement of intent.

Stamp site

_add_source_to_span in the SGP processor, not at span creation. Two consequences,
both deliberate:

  • Scoped to the SGP backend — the user story here, and 92 of the fleet's agents
    register SGPTracingProcessorConfig vs 3 on the Agentex one. The scope doc asked
    for both backends; that's the one point where this diverges, agreed as SGP-only.
  • Re-runs at span end. _build_sgp_span() is called from both on_span_start
    and on_span_end, and end-only ingest is the default — so the value survives an
    agent replacing span.data mid-span, which real agent code does today
    (eval_dashboard_agent/project/workflow.py:273,357).

Test plan

  • tests/lib/core/tracing/test_code_revision.py — 19 tests: disabled by default even
    with the env fully populated, precedence, disable(), and a parametrised set
    proving non-SHAs (latest, v1.2.3, the AWS composite, non-hex, too-short) are
    refused while git object names (40-hex, 64-hex, short, uppercase) are accepted.
  • test_sgp_tracing_processor.py — 2 tests pinning the opt-in gate at the stamp site.
  • Full tests/lib: 1058 passed, 3 skipped. ruff clean, pyright 0 errors.

Not in this PR

  • Populating AGENT_VERSION at deploy time (it is set nowhere today, so
    __agent_version__ is inert in production) — separate PR.
  • The Loki half of the scope doc: pod label + k8sattributes mapping in sgp.
  • Agentex-backend parity for this key.

🤖 Generated with Claude Code

Greptile Summary

This PR adds opt-in commit-SHA resolution and stamps the resolved revision into SGP metadata without mutating the shared span.

  • Exposes adk.code_revision with explicit enable and disable controls.
  • Validates explicit and environment-derived revisions as SHA-shaped values.
  • Copies dictionary metadata before adding __commit_sha__, resolving the previously reported cross-processor leak.
  • Adds coverage for opt-in behavior, precedence, validation, processor isolation, and non-dictionary span data.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains; the copied SGP metadata keeps the opt-in commit field off the shared span and therefore out of the AgentEx processor path.

Important Files Changed

Filename Overview
src/agentex/lib/core/tracing/code_revision.py Adds process-wide opt-in revision resolution with documented precedence, strict SHA-shape validation, and explicit reset behavior.
src/agentex/lib/core/tracing/processors/sgp_tracing_processor.py Adds the revision only to copied SGP metadata, preventing the previously reported mutation of shared span data.
tests/lib/core/tracing/processors/test_sgp_tracing_processor.py Verifies opt-in stamping, shared-span isolation, preservation of existing metadata, and list-shaped data behavior.
tests/lib/core/tracing/test_code_revision.py Covers default-off behavior, source precedence, disable semantics, accepted SHA forms, and invalid-value rejection.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
    A[Agent calls code_revision.enable] --> B[Resolve and validate commit SHA]
    B --> C[_build_sgp_span]
    C --> D[_sgp_metadata copies dictionary data]
    D --> E[Add __commit_sha__ to copied metadata]
    E --> F[SGP span]
    C --> G[Shared AgentEx Span remains unchanged]
    G --> H[AgentEx processor]
Loading

Reviews (2): Last reviewed commit: "fix(tracing): keep the commit SHA out of..." | Re-trigger Greptile

cyntwang99 and others added 2 commits August 28, 2026 13:25
Answers "which commit handled this request?" from the SGP Traces UI: search
`__commit_sha__:<sha>` and read it in the span-detail Metadata panel. Agent
spans never reach the OTel collector -- the SDK POSTs them straight to SGP --
so span metadata is the only carrier, and the value has to be present in the
agent's own process.

Opt-in, modelled on the lineage registry next door: the module global's default
is the off state, so `commit_sha()` returns None and the stamp site needs no
flag. Nothing is emitted until an agent calls `adk.code_revision.enable()`, and
no agent inherits the field by upgrading the SDK.

Deliberately separate from the automatic `__agent_version__`, which carries the
deployed image tag verbatim. That tag is a real commit on GCP/Azure CI but an
"<image-name>-<sha>" composite on AWS ECR, "latest" on local deploys, and an
arbitrary string on manual dispatch -- so a field named for a commit must not
mirror it. Values that are not git object names are refused with a warning,
and `__commit_sha__` therefore only ever holds one.

Value precedence: explicit argument, then AGENT_COMMIT_SHA, then AGENT_VERSION
only when it is already SHA-shaped -- so a platform-deployed agent that opts in
needs no extra plumbing, while a custom build path bakes AGENT_COMMIT_SHA into
the image.

Stamped in the SGP processor rather than at span creation. That scopes it to
the SGP backend, which is the user story here (and 92 of the fleet's agents),
and it re-runs at span end, so the value survives an agent replacing span.data
mid-span -- which real agent code does today.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`Span.data` is `dict | list[dict] | None`, so `in` and `[]` need an isinstance
guard to satisfy the project-wide pyright run in scripts/lint.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Comment thread src/agentex/lib/core/tracing/processors/sgp_tracing_processor.py Outdated
trace.py hands ONE Span instance to every registered processor, and
_add_source_to_span mutates span.data in place. So writing __commit_sha__ there
leaked it: a co-registered Agentex processor serialized it too, and it surfaced
in caller-visible span.data -- contradicting the claim that this field is
SGP-scoped.

Build the SGP write's metadata as a copy instead. Adds a regression test that
asserts the SGP metadata carries the key while the shared span and the Agentex
processor's payload do not, plus one for list-shaped data, which has nowhere to
put a metadata key and is now returned untouched.

The __source__ / __agent_* keys leak the same way today; left alone
deliberately, since changing five long-shipped fields is out of scope here.

Reported in review by Greptile.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant