Conversation
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![macroscopeapp[bot]](https://avatars.githubusercontent.com/in/900172?s=60&v=4){kind=small}
| overlay["services"].setdefault(owner, {})["extra_hosts"] = { | ||
| "host.docker.internal": "host-gateway" | ||
| } |
There was a problem hiding this comment.
🟠 High harbor/runtime.py:69
When main uses network_mode: service:<sidecar>, the agent cannot resolve the host.docker.internal hostname in callback URLs, so model/interception requests fail. The mapping is added only to owner; add it to both main and the namespace owner.
- overlay["services"].setdefault(owner, {})["extra_hosts"] = {
- "host.docker.internal": "host-gateway"
- }
+ for service in {owner, "main"}:
+ overlay["services"].setdefault(service, {})["extra_hosts"] = {
+ "host.docker.internal": "host-gateway"
+ }🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @verifiers/v1/tasksets/harbor/runtime.py around lines 69-71:
When `main` uses `network_mode: service:<sidecar>`, the agent cannot resolve the `host.docker.internal` hostname in callback URLs, so model/interception requests fail. The mapping is added only to `owner`; add it to both `main` and the namespace owner.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 28dffe7255
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| overlay = {"services": {"main": main}} | ||
| overlay["services"].setdefault(owner, {})["extra_hosts"] = { | ||
| "host.docker.internal": "host-gateway" | ||
| } |
There was a problem hiding this comment.
When main uses network_mode: service:<sidecar>, owner becomes that sidecar and this installs host.docker.internal only in the sidecar. Although the containers share a network namespace, their /etc/hosts files are separate, so the agent running in main cannot resolve the hostname returned by host_url() and consequently cannot reach the interception/model or host-local MCP endpoints. Keep any owner-specific setup needed for forwarding, but add the alias to main as well.
Useful? React with 👍 / 👎.
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
ApprovabilityVerdict: Not approved Macroscope's review found this PR not approvable — This PR adds a substantial Harbor-to-local-Docker Compose execution path with new container ownership, networking, cleanup, and retry behavior. Unresolved comments identify concrete runtime failures and potential host-level security exposure, so the change warrants human review. Not approved because:
Adjust the Minimum Blocking Severity for this repo — including turning it Off — in Settings. You can add or adjust custom eligibility rules. Learn more. |
28dffe7 to
6bff76b
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6bff76b193
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
145f890 to
f589839
Compare
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
1 similar comment
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
f589839 to
141d878
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 141d878fba
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| await self._compose("config", "--quiet") | ||
| self._created = True | ||
| # Finish an interrupted Compose launch before deleting its partial project. | ||
| await run_shielded(self._compose("up", "--detach", "--wait")) |
There was a problem hiding this comment.
When a Harbor package comes from an untrusted or compromised registry, this starts its Compose file without validating host bind mounts, Docker-socket mounts, privileged, host PID namespaces, or devices. Unlike the existing constrained DockerRuntime, Compose honors those settings, and writable bind mounts can modify arbitrary host files, so merely loading such a task can yield host-level code execution before the agent starts (Docker documents this bind-mount security implication). Reject host-capability settings or require an explicit trusted-Compose opt-in before executing the file.
Useful? React with 👍 / 👎.
141d878 to
d66e109
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d66e109f55
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if "image" in services["main"] or "build" in services["main"]: | ||
| # A template default must not replace an authored image or skip its build. | ||
| base["services"]["main"].pop("image", None) |
There was a problem hiding this comment.
When a Compose task defines main.build with the conventional environment/Dockerfile and no [environment].docker_image, this build-preservation branch is unreachable: parse_task() first calls resolve_image(), which raises at taskset.py:531-539 under the default ignore_dockerfile=false. Consequently a common build-based Compose topology cannot be loaded even though this adapter explicitly preserves and runs its build; exempt Compose-managed builds from the legacy Dockerfile rejection.
Useful? React with 👍 / 👎.
| "--project-directory", | ||
| str(environment), |
There was a problem hiding this comment.
When an authored service uses a writable relative bind such as ./state:/state, this project directory resolves the host path inside the cached task.data.task_dir/environment, which dataset_dir() reuses for every episode. Distinct project names and down --volumes do not isolate bind-mounted files, so one solver or retry can mutate the starting state observed by later or concurrent rollouts; copy the environment into the runtime's temporary directory and use that copy as the Compose project directory.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
There are 2 total unresolved issues (including 1 from previous review).
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit d66e109. Configure here.
| from verifiers.v1.tasksets.harbor.taskset import HarborTask | ||
|
|
||
|
|
||
| class HarborComposeRuntime(DockerRuntime): |
There was a problem hiding this comment.
hmm why do we need a new runtime for this?
d66e109 to
41e30c8
Compare
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
| async with project as runtime: | ||
| await agents.agent.run( | ||
| task.defer_scoring() if separate else task, | ||
| runtime=runtime, |
There was a problem hiding this comment.
🟠 High harbor/env.py:54
Retryable rollout failures receive only one attempt when a Compose project is configured, even if --env.retries or agent retries are set. Passing the already-open project through runtime= makes it a borrowed runtime, so Agent.run disables its retry loop; create the project per attempt via a runtime factory instead of wrapping the entire run call.
🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @verifiers/v1/tasksets/harbor/env.py around line 54:
Retryable rollout failures receive only one attempt when a Compose project is configured, even if `--env.retries` or agent retries are set. Passing the already-open project through `runtime=` makes it a borrowed runtime, so `Agent.run` disables its retry loop; create the project per attempt via a runtime factory instead of wrapping the entire `run` call.
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
| if "image" in services["main"] or "build" in services["main"]: | ||
| # A template default must not replace an authored image or skip its build. | ||
| base["services"]["main"].pop("image", None) | ||
| base_file = directory / "base.json" |
There was a problem hiding this comment.
🟠 High harbor/compose.py:113
An authored main image or build runs the template's command: ["sh", "-c", "sleep infinity"] instead of its default CMD/ENTRYPOINT, so the application never starts and the declared topology can fail. Remove the template command alongside its image when preserving an authored image or build.
if "image" in services["main"] or "build" in services["main"]:
# A template default must not replace an authored image or skip its build.
base["services"]["main"].pop("image", None)
+ base["services"]["main"].pop("command", None)🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @verifiers/v1/tasksets/harbor/compose.py around lines 113-116:
An authored `main` image or build runs the template's `command: ["sh", "-c", "sleep infinity"]` instead of its default CMD/ENTRYPOINT, so the application never starts and the declared topology can fail. Remove the template `command` alongside its `image` when preserving an authored image or build.
2 participants
Harbor runs each task's authored Docker Compose project on local Docker. The Harbor environment owns startup, readiness, and cleanup; the agent borrows an ordinary
DockerRuntimeattached to the project'smaincontainer.DockerRuntime.attach()inspects an existing container and provides the shared command, process, file, and callback interfaces without taking ownership of container removal. Compose preserves authored images, working directories, entrypoints, dependencies, health checks, networks, and volumes, with explicit task/runtime overrides applied tomain.Projects are removed on completion, startup failure, or cancellation. Whole-episode retries create fresh projects through
env.retries. This supports public-network CPU tasks on local Docker; separate grading retains its ordinary fresh verifier runtime.Note
Add Harbor Compose, Podman, and Apptainer runtime support
HarborComposeRuntimeto run Harbor tasks using local Docker Compose. It validates the service topology, applies runtime resource overlays, starts the project, exposes the owner service port, and manages cleanup.ContainerRuntimein container.py and addsPodmanRuntimeandApptainerRuntimealongside the existingDockerRuntime.EgressProxyin egress.py to support tokenized HTTP(S) host callbacks, allowing loopback services inside local containers to be accessed via the proxy.runtime_factoryparameter toAgent.runandRollout.openin agent.py and rollout.py to enable per-attempt runtime construction for Harbor tasks.mcp.reachable_urlin launch.py to use runtime-native port exposure for non-colocated services, tunneling only host-local services accessed by remote consumers.HarborComposeRuntimein harbor/runtime.py rejects restricted-network or GPU configurations.DockerRuntime.host_urlin docker.py routes all loopback URLs through proxy callback URLs, changing behavior for unrestricted runtimes.Changes since #2547 opened
ComposeProjectclass inharbor.composeto own Compose project lifecycle [d9e6bd1]HarborEnv.runto own Compose projects and pass borrowed runtimes to agents [d9e6bd1]DockerRuntime.attachclassmethod async context manager to support borrowing existing containers [d9e6bd1]runtime_factoryparameter fromAgent.run,_EpisodeAgent.run, andRollout.__init__[d9e6bd1]DockerRuntime.cleanupto skip container removal for borrowed containers [d9e6bd1]ComposeProjectcompose attach/runtime step to always use localhost address [58a4790]ComposeProjectin favor of on-the-fly computation [58a4790]Macroscope summarized d66e109.
Note
Medium Risk
New Compose project lifecycle and borrowed-container semantics affect sandbox startup, networking, and cleanup; mis-cleanup or attach bugs could leak containers or break agent connectivity.
Overview
Adds local Docker Compose execution for Harbor tasks that ship
environment/docker-compose.yaml, withruntime.type = "docker"and public-network CPU only.ComposeProjectowns the Compose lifecycle: validates topology (nocontainer_name, no host networking), merges Harbor prebuilt/base files with task overrides and resource limits onmain, runsdocker compose up --wait, publishes the owner service on loopback, and tears the project down on exit, failure, or cancellation (withatexitbackstop). Retries get a fresh project via--env.retries.DockerRuntime.attach()lets the agent borrow themaincontainer without removing it on cleanup;cligains optional subprocessenvfor Compose infra variables.HarborEnv.runstarts Compose when the compose file exists, passes the attached runtime intoagents.agent.run, and keeps separate-verifier defer/collect behavior on the same path. Docs describe ownership, limits (no GPU/restricted network, no sidecar collect), and that entrypoint keepalive applies only outside Compose.Reviewed by Cursor Bugbot for commit 58a4790. Bugbot is set up for automated code reviews on this repo. Configure here.