Skip to content

fix(sdk): resolve all condition leaf selectors into control event input - #269

Open
jasmine-ab-tea wants to merge 3 commits into
mainfrom
fix/control-input-all-selector-values
Open

jasmine-ab-tea wants to merge 3 commits into
mainfrom
fix/control-input-all-selector-values

Conversation

@jasmine-ab-tea

@jasmine-ab-tea jasmine-ab-tea commented Sep 24, 2026 •

Copy link
Copy Markdown
Collaborator

What this changes

Agent Control controls can be simple (one selector + one evaluator) or composite trees built from and/or/not combinators, each combinator wrapping multiple {selector, evaluator} leaves. When a composite control's execution is reconstructed into a ControlExecutionEvent (and from there into the Console's control span), the existing _composite_metadata() logic in the engine picks one representative leaf to stand in for matched/action/condition_trace — that collapsing behavior is correct and unchanged.

The bug: the reconstructed event's input field inherited that same collapse. It was populated from a single leaf's selected_data_preview/engine_selected_data_preview (or, for simple controls, just whatever scalar the one selector resolved to). For a composite control, that meant the Console only ever showed the value for whichever leaf happened to be chosen as "representative" — every other selector path that the control actually evaluated (e.g. the other side of an and, or the untriggered branch of an or) was invisible in input, even though it was part of the decision.

The fix

_build_events_for_matches() (sdks/python/src/agent_control/evaluation_events.py) now walks the control's full condition tree via observability_identity().all_selector_paths (built from ConditionNode.iter_leaf_parts() — every leaf, in tree order, already deduplicated) and resolves each path against the real step payload with select_data(). input becomes {selector_path: resolved_value, ...} for every leaf, instead of one scalar. Falls back to the old preview-scalar behavior only when the control definition can't be resolved from control_lookup (e.g. genuinely missing).

This is entirely client-side SDK reconstruction — build_control_execution_events() runs in-process the same way regardless of whether a given control executed with execution="server" or execution="sdk", so the fix applies uniformly without touching the engine, the server, or orbit's ingestion/schema.

Before / after (real devstack traces)

Before — a composite control's span input (demo-steer-large-transfer-2fa, an and(amount >= $10k, not verified_2fa) control) showed only the one representative leaf, collapsed to a bare scalar:

15000

There's no way to tell from this that verified_2fa was even part of the decision.

After — the same kind of composite control (demo-deny-risky-transfer-composite, an or(destination sanctioned, fraud_score high) control) now shows every leaf that was actually evaluated:

{
  "input.destination_country": "United Kingdom",
  "input.fraud_score": 0.1
}

(Screenshots from Console attached below/in the PR thread — before: scalar input on demo-steer-large-transfer-2fa; after: full resolved map on demo-deny-risky-transfer-composite.)

Test plan

  • test_observability_updates.py: updated the stale single-scalar-fallback test to reflect the new resolved-map behavior, added coverage for the and composite (asserts input has both leaf values), the or composite, and the control-lookup-miss fallback path (control not in control_lookup still falls back to the debug preview scalar).
  • Validated live against a devstack: ran ace-demo's run_demo.py against a real Agent Control server, then fetched the resulting spans back via the Galileo API.
    • and+not composite (demo-steer-large-transfer-2fa, matched): input = {"input.amount": 15000.0, "input.verified_2fa": false}
    • or composite (demo-deny-risky-transfer-composite, non-match): input = {"input.destination_country": "United Kingdom", "input.fraud_score": 0.1}
    • Both show every leaf's real resolved value instead of one collapsed scalar.

🤖 Generated with Claude Code

Composite controls (and/or/not) previously collapsed control-execution
event input down to a single representative leaf's value via
_composite_metadata(), hiding the other selector values that fed the
decision. Reconstruct the full {selector_path: value} map by resolving
every leaf in the condition tree (via observability_identity's
all_selector_paths and select_data) against the original step, so the
emitted event/span input reflects everything the control actually
evaluated.

Co-Authored-By: Claude Code <noreply@anthropic.com>
@jasmine-ab-tea
jasmine-ab-tea force-pushed the fix/control-input-all-selector-values branch from be350e4 to 9de0cd0 Compare September 24, 2026 23:23
Yixin and others added 2 commits September 24, 2026 16:46
The untyped _apply_filters helper makes page_stmt infer as Any, which
cascades to Any for rows and leaves mypy unable to infer a concrete
element type for the list() call.

Co-Authored-By: Claude Code <noreply@anthropic.com>
identity_metadata is typed as dict[str, object], so .get(...) yields
object, which mypy correctly flags as not iterable in the dict
comprehension below.

Co-Authored-By: Claude Code <noreply@anthropic.com>
@codecov

codecov Bot commented Sep 24, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants