Skip to content

test(quota): fix clock-sensitive CLI detail parity - #6814

Merged
huangruiteng merged 1 commit into
mainfrom
codex/ci-detail-clock-20261018
Oct 18, 2026
Merged

huangruiteng merged 1 commit into
mainfrom
codex/ci-detail-clock-20261018

Conversation

@loopx-agent

Copy link
Copy Markdown
Collaborator

Quota detail-selector parity became timing-sensitive after projection envelopes began carrying read timestamps and source ages. Natural Python CI run 39110796189 failed the --include-detail all equivalence assertion. Two real CLI reads separated by a second reproduce the mismatch in observation metadata.

配额详情参数等价测试会因两次读取跨秒而偶发失败。本修复在现有两个等价测试中使用同一个观测时刻,复用已有 runtime clock 测试入口;完整 JSON、来源新鲜度、权限和输出预算断言全部保留。

The change is limited to the two selector-equivalence tests. Each receives a local clock override; no payload fields are discarded and production clock behavior is unchanged.

Validation at 2a6348bc597c2edde52b4c32d79eece3e9966d01:

  • Complete CLI output-budget module on Python 3.11: 32 passed, 0 skipped.
  • Controlled real CLI calls across a 1.2-second wall-clock gap: the original comparison differs in 14 observation fields; a shared observation clock preserves complete equality.
  • Changed-file Ruff, public/private scan, DCO, RFC bilingual check and standard premerge checks passed.
  • Exact-scope change-quality receipt: cqr_fc286275a69a9b7bcd91.

Full hosted qualification remains unproven: the earlier run was cancelled without a complete six-shard Python artifact set. These local results do not establish full CI, coverage or sustained runtime-cost acceptance.

Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com>

@loopx-agent loopx-agent left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewer: model_agent · gpt-6.1-sol · OpenAI · runtime_reported · reasoning_effort=max

Approval conclusion (author-owned PR; GitHub blocks formal self-approval)

动机

维护 CLI 详情输出的开发者,需要分清真实输出回归与两次观察跨秒造成的假失败。

原测试连续调用两个等价详情入口时,观察时间和新鲜度会随墙钟变化;现在在同一观察时刻比较完整输出,避免无关时间差掩盖真正差异。

当前完整输出预算模块 32 项通过;强制两次调用相隔 1.25 秒,基线两个比较均因时间字段不同而失败,新 head 两者保持完整相等。

只修两个既有测试的观察条件,不改变生产时钟、准入决策、输出字段或预算,也不宣称修复调度行为。

改动思路

测试在进程内调用真实 CLI,所以只给两个等价性比较固定同一个 UTC 观察时刻即可。现有完整 JSON 相等断言、新鲜度字段和详情存在性断言全部保留;没有把时间字段从结果中删掉,也没有给整个测试模块冻结时钟。这样减少无关跨秒噪声,同时保留真实输出回归的信号。

具体改动

完整 diff 是 tests/control_plane/test_cli_output_budget.py 的 13 行新增:日期类型导入和两个 test-scoped monkeypatch。_invoke_cli 仍调用原生产 cli_main,原比较只排除独立 rollout receipt,其余字段仍完整比较。

以不可变基线的详情输出契约为依据:explicit-detail-cold-paths 的显式详情/旧别名等价已验证;budget-and-authority-parity 的全量详情恢复及原预算已验证;生产时钟与调度决策不在本次改动中。没有新增重复 smoke、固定日期工件或另一套测试框架。当前同作者相邻 PR 分别处理 Chat fixture 和 provider 边界,本次是既有断言的局部维护。

对主干的风险

独立运行实际变更模块 32 项、changed-file Ruff,以及标准 premerge 的五项直接检查和输出预算 smoke,均通过。自己的真实 CLI 探针强制间隔 1.25 秒:基线两项因时间 envelope 不同而失败,head 保持相等;再把 all 改成 scheduler,原断言仍失败,确认固定时间没有吞掉真实详情差异。单纯去掉旧别名在该 fixture 上没有差异,不把它算作有效负例。未查询或等待远端 CI。

我的整体评价

APPROVE,结论绑定 2a6348bc597c2edde52b4c32d79eece3e9966d01。此次维护切片已完成:长期执行语义保持,维护者的故障定位信号改善;没有生产设置、额外交互或数据迁移成本。局部重复的短时钟 patch 比全局 fixture 更容易看清作用域,无需为了抽象再增加 helper。批准与合并权限、平台检查分别判断。

English verdict: APPROVE — 2a6348b. Scoped clock freezing removes independently reproduced timing-only parity failures while preserving complete payload assertions and a real wrong-selector negative. All 32 budget tests, changed-file Ruff and standard premerge pass; remote CI was not consulted.

@huangruiteng
huangruiteng merged commit 08554b6 into main Oct 18, 2026
15 of 23 checks passed
@huangruiteng
huangruiteng deleted the codex/ci-detail-clock-20261018 branch October 18, 2026 11:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants