fix(caching): robust trajectory auto-detection & resume; cache format 0.3

[philipph-askui]

Summary

Fixes the two reported trajectory-caching bugs and, per request, includes an adversarial review of the whole caching subsystem with the resulting logic bugs fixed. Cache format version is bumped 0.2 → 0.3.

Reported bugs

1. Crash when the last trajectory step is not cacheable

Resuming after the final (non-cacheable) step raised ValueError: Invalid start_from_step_index: 6. Trajectory has 6 steps.... Root cause: the executor rejected start_from_step_index == len(trajectory).

• start_from_step_index == len(trajectory) is now valid and means "already complete" — it flows into the COMPLETED/verification path instead of crashing. Only genuinely out-of-range values raise. Empty / all-non-cacheable trajectories are handled too.
• The non-cacheable pause message now tells the agent the exact start_from_step_index to resume with, or that the step was the final one (in which case: verify, don't resume).

2. Auto-detect the trajectory instead of asking the agent to list them

Previously the agent had to call retrieve_available_trajectories_tool first and be told in the prompt which caches to use.

• Added a top-level CachingSettings.filename (the test-case name). In execute/auto mode the SDK looks up <cache_dir>/<filename> automatically.
• If a usable trajectory exists, its details (path + required parameters) are injected into the first user message, and the CacheExecutor + verify_cache_execution are wired up, so the agent can hand off immediately.
• If none exists: execute runs normally; auto tells the agent none exists and records the run. An existing invalid cache is re-recorded (self-heal) rather than replayed.
• Removed retrieve_available_trajectories_tool and rewrote CACHE_USE_PROMPT accordingly.

Additional bugs fixed (surfaced by the adversarial review)

• CachingSettings.filename was silently ignored — the field didn't exist, so docstring/e2e examples that passed filename= never actually set it. Now a real top-level field used for both lookup and recording.
• execute-only mode crashed — it passed cache_manager=None, but CacheExecutor.handle_step raises if it's None. A CacheManager is now always created when executing.
• verify_cache_execution(success=False) was a no-op — it now actually invalidates the cache; success=True records the completion (execution_attempts / last_executed_at), which was never updated before.
• State leaked across act() calls — the CACHE_USE prompt (mutated on shared settings), the CacheExecutor speaker, and per-call caching tools all persisted into later runs. The wired VerifyCacheExecution leak could even persist a later run's result to a previous run's trajectory file. All now scoped per-call.
• Parameter identification could corrupt trajectories — LLM-identified params with an empty value shredded every string (str.replace("", ...)); invalid names silently broke validation/substitution. Such entries are now dropped.
• Empty recordings — a run with no cacheable steps no longer writes/overwrites a cache file (would be a silent no-op "cache hit").
• Teardown — finish_recording errors are logged instead of masking the real run outcome.

Known limitations (documented follow-ups, intentionally not in this PR)

These are deeper design changes with real regression risk; flagging rather than rushing them:

• Parameter identification uses substring matching and stringifies values, which can still mis-parameterize short/numeric values embedded in larger strings, and delivers substituted values as strings. A whole-field, type-preserving rewrite is the proper fix.
• Recording writes whatever happened regardless of run success (no success gate beyond the empty-trajectory guard added here).
• A cached action that raises AutomationError during replay still terminates the run (same as normal mode) rather than routing to the executor's self-heal/invalidate path; is_error tool results during replay aren't inspected.
• Replayed steps consume the max_steps budget one per action.
• Visual validation can run against a stale screenshot during a pure action replay.

Testing

• New unit tests: tests/unit/speaker/test_cache_executor.py, tests/unit/test_caching_agent_helpers.py, tests/unit/utils/caching/test_cache_manager.py, tests/unit/utils/caching/test_cache_parameter_handler.py; updated tests/unit/tools/test_caching_tools.py and the caching e2e test.
• pdm run qa:fix clean (typecheck + format + lint); full unit suite green (763 passed).
• Docs (docs/06_caching.md) rewritten for the new filename-based flow.

🤖 Generated with Claude Code

[philipph-askui]

Consolidated into a single PR: #311 now targets main and contains all of this branch's commits plus the parameter/observability work. Closing to avoid a stacked/duplicate PR.