[feat] Add agent skills for EBench usage

README.md

@@ -43,6 +43,7 @@ A new systematic diagnostic tool, **gmp analyse**, automatically produces a **fi
 
 | Date | Highlight |
 | :--- | :--- |
+| **2026-09** | 🔧 **Minor EBench fixes and improvements** — Adjusted the step limits for `bottle` and `shop` from 3,000 to 5,000, and refined the web interface of the online evaluation platform. |
 | **2026-09** | 🛠️ **OpenPI baseline update** — Updated π0 / π0.5 configs. Please use the updated baseline code for evaluation and reproducing results. See the [update notes](baselines/openpi/README.md#update-notes). |
 | **2026-07** | 🛠️ **Evaluation toolkit** — [`genmanip-client`](https://github.com/InternRobotics/genmanip-client) brings submission, monitoring, action/state plots, and interactive episode visualization together in the `gmp` CLI. |
 | **2026-06** | 🚀 **Public release** — EBench, reference baselines, training data, and held-out online evaluation are now available. |
@@ -115,6 +116,12 @@ gmp status
 
 A full validation pass takes roughly **30 minutes on 8× RTX 4090**. Detailed setup, asset download, and the complete `gmp` reference are in the [docs site](https://internrobotics.github.io/EBench-doc/).
 
+## Evaluate with a coding agent
+
+The repository includes five reusable [agent skills](skills/README.md) for environment setup, policy integration, evaluation runs, debugging, and result analysis. They guide agents through the checked-out baseline and client implementations, including real-policy launch commands and result coverage checks.
+
+To start, ask your agent: **“Read `skills/ebench-setup/SKILL.md` and check whether this environment is ready to evaluate my model.”** Then use [ebench-evaluate](skills/ebench-evaluate/SKILL.md) to run it and [ebench-analyze](skills/ebench-analyze/SKILL.md) to interpret the results. See the [skill catalog and usage guide](skills/README.md) for all workflows; automatic discovery depends on your agent's skill installation mechanism.
+
 ## Tasks
 
 **26 task types** across *Long-Horizon*, *Pick-and-Place*, and *Dexterous & Precise*, expanded with the four generalization axes and three splits into **794 evaluation task instances**. Browse the video gallery at [Task Showcase](https://internrobotics.github.io/EBench-doc/evaluation/task-showcase/).

README.zh-CN.md

@@ -45,6 +45,7 @@
 
 | 日期 | 重要更新 |
 | :--- | :--- |
+| **2026-09** | 🔧 **小修复与优化** — 将 `bottle` 和 `shop` 的步数上限从 3,000 调整为 5000，并优化了在线评测平台的网页界面。 |
 | **2026-09** | 🛠️ **OpenPI Baseline 更新** — 更新 π0 / π0.5 配置。评测与复现结果时，请使用更新后的 Baseline 代码。详见[更新说明](baselines/openpi/README.md#update-notes)。 |
 | **2026-07** | 🛠️ **评测工具链** — [`genmanip-client`](https://github.com/InternRobotics/genmanip-client) 通过 `gmp` CLI 统一提供评测提交与监控、动作/状态曲线绘制以及 episode 交互式查看。 |
 | **2026-06** | 🚀 **正式发布** — EBench、参考 Baseline、训练数据及训测隔离的在线评测现已公开。 |
@@ -117,6 +118,12 @@ gmp status
 
 8 卡 RTX 4090 上跑完整套验证集大约需要 **30 分钟**。完整的环境配置、资产下载与 `gmp` 命令参考请见[文档站点](https://internrobotics.github.io/EBench-doc/zh-cn/)。
 
+## 使用 coding agent 评测
+
+仓库提供五个可复用的 [agent skills](skills/README.md)，覆盖环境准备、模型接入、评测运行、故障排查和结果分析。它们引导 agent 按当前仓库的 baseline 与客户端实现完成操作，并核对真实模型入口及结果覆盖情况。
+
+可以先告诉 agent：**“请读取 `skills/ebench-setup/SKILL.md`，检查当前环境是否能评测我的模型。”** 随后使用 [ebench-evaluate](skills/ebench-evaluate/SKILL.md) 运行评测、[ebench-analyze](skills/ebench-analyze/SKILL.md) 解读结果。完整流程和调用示例见 [skill 目录与使用说明](skills/README.md)；是否自动发现取决于所用 agent 的技能安装机制。
+
 ## 任务总览
 
 **26 种任务**分布在*长程任务（Long-Horizon）*、*抓取放置（Pick-and-Place）*、*灵巧精细操作（Dexterous & Precise）*三个任务族；与四维泛化、三层划分组合后，共构成 **794 条评测任务**。完整视频演示见[任务展示](https://internrobotics.github.io/EBench-doc/zh-cn/evaluation/task-showcase/)。

skills/README.md

@@ -0,0 +1,60 @@
+# EBench coding agent skills
+
+These skills help coding agents evaluate **robot policies on EBench**: prepare environments, integrate models, run experiments, diagnose failures, and interpret results. They are not a benchmark for evaluating coding agents themselves.
+
+## Where to start
+
+| Skill | Priority | When to use it | Deliverables |
+| --- | --- | --- | --- |
+| [ebench-setup](ebench-setup/SKILL.md) | P0 | First-time setup, environment checks, baseline reproduction | Environment findings, missing prerequisites, executable launch commands |
+| [ebench-evaluate](ebench-evaluate/SKILL.md) | P0 | Local or online evaluation, launching multiple workers | Traceable runs, logs, and completion status |
+| [ebench-analyze](ebench-analyze/SKILL.md) | P0 | Report generation, model comparison, capability analysis | HTML reports, data coverage, and evidence-backed conclusions |
+| [ebench-integrate-policy](ebench-integrate-policy/SKILL.md) | P1 | Connecting a custom VLA to EBench | Observation/action adapter and contract validation |
+| [ebench-debug](ebench-debug/SKILL.md) | P1 | Stalled runs, errors, unexpected actions or success rates | Diagnosis, minimal fixes, and verification evidence |
+
+P0 skills cover the complete workflow for an existing model, from preparation to reporting. For custom models, start with policy integration; use debugging when failures occur. Training, data downloads, and leaderboard publication do not have separate skills yet because they are not required for every evaluation. Follow the baseline documentation for training and data preparation.
+
+## Usage
+
+Skills are versioned under `skills/`, with a standard `SKILL.md` containing `name` and `description` in each directory. **Not every agent automatically discovers this directory.** The simplest approach is to ask your agent to read a specific file from the EBench repository:
+
+```text
+Read skills/ebench-setup/SKILL.md, check whether this environment can run X-VLA online evaluation, and list any missing prerequisites.
+Read skills/ebench-evaluate/SKILL.md, submit an online evaluation using the configured token, wait for the endpoint and task_id, then launch my model.
+Read skills/ebench-evaluate/SKILL.md and launch my model using the existing endpoint and run_id, with worker 0 only.
+Read skills/ebench-integrate-policy/SKILL.md and connect my policy to EvalClient, validating the input/output contract first.
+Read skills/ebench-debug/SKILL.md and investigate this run's reset timeout while preserving existing results.
+Read skills/ebench-analyze/SKILL.md, compare these two result directories, generate a report, and explain whether their evaluation coverage matches.
+```
+
+You can also import these directories through your agent's skill installation mechanism. Once installed, agents that support `$skill-name` can invoke them by names such as `$ebench-evaluate`. Copying the entire collection into a global user directory is not a prerequisite for evaluation.
+
+Repository paths in all skills are relative to the **EBench root**, not the skill directory. Skill instructions are in English for international reuse; agents can respond in the user's language. For a first run, provide the model/checkpoint, online or local mode, split/track, and GPU/worker budget. For an existing task, also provide its run ID and endpoint. Supply tokens through the local environment or an existing credential mechanism, never through committed files or reports.
+
+## Online evaluation workflow
+
+[ebench-evaluate](ebench-evaluate/SKILL.md#online-evaluation-queue-obtain-endpoint-then-evaluate) includes complete command examples. The agent follows this sequence:
+
+1. Check the local model environment, checkpoint, platform URL, and token.
+2. Run `gmp online submit --print_endpoint` to join the queue and wait for resources.
+3. Parse `endpoint` and `task_id` from the returned JSON, using them as the evaluation URL and `run_id`, respectively.
+4. Launch the baseline or custom policy client for real model evaluation, then monitor progress and save results.
+
+A new task does not require the user to provide an evaluation URL or task ID in advance. After a queue timeout, query the existing task before retrying to avoid duplicate submissions. Receiving a URL does not mean evaluation is complete. The setup and policy integration skills also explain how to connect to this workflow.
+
+## Maintenance and acceptance checks
+
+Treat the baseline code and `third_party/genmanip-client` source in the current checkout as the source of truth. After updating submodules, review CLI arguments, action conversions, and result parsing. Do not assume one baseline's normalization or action layout applies to every model.
+
+When adding or changing a skill, check its frontmatter and file references, then review its behavior against these scenarios:
+
+| Scenario | Expected behavior |
+| --- | --- |
+| A fresh clone is missing submodules | Identify missing dependencies rather than treating import errors as model failures |
+| The user requests real model evaluation | Use a baseline/custom policy entry point; do not report `gmp eval` fake actions as model performance |
+| An existing online task is waiting for resources | Query the existing task instead of creating another one |
+| A step times out with unknown execution state | Discard the old chunk, use bounded recovery, and record the interruption rather than blindly resending actions |
+| Analysis inputs are empty | Report missing local results rather than presenting bundled reference data as the user's results |
+| Two runs have different splits or task coverage | Flag them as not directly comparable and explain denominators and missing data |
+
+These scenarios are maintenance acceptance criteria, not evidence that GPU or online platform end-to-end validation has been performed.

skills/ebench-analyze/SKILL.md

@@ -0,0 +1,40 @@
+---
+name: ebench-analyze
+description: Generate and interpret EBench evaluation reports, compare runs and baselines, and diagnose capability or generalization gaps with explicit data coverage and aggregation semantics.
+---
+
+# Analyze EBench evaluation results
+
+Work from the EBench root. Read `third_party/genmanip-client/src/genmanip_client/extensions/analyse_cli.py` and `analyse.py` for the pinned behavior; inspect `default_cluster_map.json` only when interpreting taxonomy or checking task coverage.
+
+## Verify inputs before rendering
+
+Identify each run's model/checkpoint, benchmark revision, track, split, task set, seed/episode coverage, and completion state from its manifest and result files. Keep incomplete runs labeled. Missing episodes are not automatic successes or failures; state the coverage and denominator instead of inventing outcomes.
+
+Default discovery scans `<project_root>/saved/eval_results/<benchmark>/<run_id>`. Client results commonly live under `client_results/<benchmark>/<run_id>` instead: pass the concrete run directory explicitly. Do not point to a single seed directory or assume EBench's root contains server outputs.
+
+The loader prefers per-episode `result_info.json`, then run-level `result.json`, then task-level `episode_result.json`. These formats retain different detail. Raw `result_info.json` can carry `metric_score` needed for atomic-skill aggregation; absent metrics cannot be recovered from a total success rate. Check loaded records and parse failures before drawing conclusions.
+
+## Generate the report
+
+With actual run paths already verified:
+
+```bash
+gmp analyse "$RUN_A_DIR" "$RUN_B_DIR" --no-reference -o "$REPORT_PATH"
+```
+
+Omit `--no-reference` when bundled baseline comparison is desired. The default includes bundled reference models; `--reference` renders only those reference data and skips local runs. Label their bundled version rather than claiming they are freshly measured or current leaderboard standings.
+
+**An HTML file is not proof that local results loaded.** The pinned CLI falls back to bundled reference data when no runs/records load, even when `--no-reference` was supplied. Check the CLI's loaded-record messages and report payload/run IDs against the requested inputs. If empty, report missing data; never describe the fallback as the user's model performance.
+
+Use `--group 'Label=pattern'` only to combine intended compatible runs. Grouping different checkpoints, splits or overlapping retries can conceal variation or double-count evidence. Explicitly identify grouping members. Choose a fresh output path so prior reports remain available.
+
+## Interpret with the right denominator
+
+- Report SR and score separately, per split, with coverage. In the current aggregator, top-line means are over loaded records; cluster summaries first average within each task and then across tasks. Run-level aggregated input is not equivalent to raw per-episode input for weighting or uncertainty.
+- Explain capability dimensions (Scene, Atomic Skill, Horizon, Precision, Mobility) and generalization dimensions (Object, Background, Instruction, Mixed) only where task labels and metrics support them. Mark absent axes as unavailable, not zero.
+- Compare models on aligned benchmark revisions, split/task coverage, and evaluation settings. If coverage differs, present that difference and, when raw data permits, a clearly labeled matched subset; do not silently compare partial scores as a full benchmark.
+- Do not present standard deviation across task/episode records as a confidence interval across independent runs. Separate repeated-seed variation from variation between tasks.
+- Ground failure hypotheses in task-level metrics and representative episode traces. Aggregate scores alone cannot establish a camera bug, planning failure, or causal explanation. Use validation splits for suggested tuning; keep held-out results for final assessment.
+
+Deliver a linked HTML report, input run paths/IDs, actual loaded coverage, aggregation/reference settings, key supported findings, and limitations. If only reference data or partial results exist, make that the main conclusion. Do not publish results to a leaderboard as a side effect of analysis.

skills/ebench-debug/SKILL.md

@@ -0,0 +1,33 @@
+---
+name: ebench-debug
+description: Diagnose EBench evaluation failures, stalled workers, transport errors, invalid actions, and unexpectedly low scores using logs and episode artifacts.
+---
+
+# Diagnose an EBench run
+
+Work from the EBench root. Obtain the failing command with credentials redacted, run ID, worker ID, first relevant traceback, checkpoint/config, and whether the failure occurs during loading, readiness, reset, inference, step, or result saving. Read existing logs before rerunning anything.
+
+## Isolate the failing layer
+
+| Evidence | Next useful check |
+| --- | --- |
+| Import/config/weight failure | Active interpreter, installed package path, pinned submodules, baseline overlays, checkpoint files/statistics; server restart will not fix a local import. |
+| Online task pending | Query the existing task with `gmp online ready`; distinguish scheduling from failure and retain its task ID. |
+| Authentication or route failure | Distinguish platform URL from evaluation endpoint; check run ID and credential presence without printing credentials. |
+| Reset timeout | Compare readiness, status and server logs if available. Asset loading and worker failure are hypotheses until supported. `EvalClient.reset()` kills/recreates its workers, so it is not a read-only probe. |
+| Step timeout/disconnect | Correlate worker/server timestamps. Server execution may have completed; inspect progress before recovery and discard stale observations/chunks. |
+| GPU OOM | Inspect model size, dtype and workers per GPU; reduce concurrency within scope before changing the model or evaluation protocol. |
+| Implausible motion/near-zero SR | Inspect camera order, normalization, joint/gripper order, relative/absolute base semantics, replan horizon, and history reset before attributing failure to model capability. |
+| Empty/missing report | Locate the actual result root, check completed episode files and writer errors; distinguish client traces from server result artifacts. |
+
+Read `baselines/X-VLA/run.py` for bounded recovery and `third_party/genmanip-client/src/genmanip_client/eval_client.py` for current reset/step/storage behavior. Do not turn every exception into an endless reconnect loop. Stop retries when the same failure persists without new evidence, and report the unresolved cause and retained artifacts.
+
+## Inspect one episode
+
+For client traces, `gmp plot "$EPISODE_DIR"` reads `steps.jsonl` and `merged*.mp4`, producing action/state plots and optional merged video. Inspect these against the adapter's transformations.
+
+For server-side results, `gmp visualize --project_root "$GENMANIP_ROOT"` uses the project's result tree. Its optional `visualize` dependencies and Rerun viewer cache are separate from model inference; the pinned README calls for Python 3.11+ for that viewer path. A viewer dependency failure does not establish that inference failed.
+
+Preserve logs, partial results, and task/seed identifiers. `gmp clean` deletes server evaluation results and logs; do not use it as a generic repair. Avoid resetting all workers to investigate one worker. Make the smallest justified fix and rerun only the needed verification within the requested evaluation scope.
+
+Deliver the observed symptom, evidence-backed cause (or remaining hypotheses), changed files/settings, validation outcome, and whether interrupted/partial episodes affect reported scores. Keep policy failures separate from infrastructure failures.