Repository navigation
Say what ssh said when a host cannot be swept - #829
Merged
Merged
Conversation
ssh exits 255 for every failure of its own, so 'exit status 255' could not tell a refused session from an unreachable host or a remote command killed by a signal. The sweep's error now carries the end of ssh's stderr (last 300 bytes, one line), in the daemon warning and, once per run of failures, in each affected task's log. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
After #828 shipped, every sweep of
ivofailed withCould not end tasks' sessions on host; will retry host=ivo … error="exit status 255"and backed off to 15m. ssh exits 255 for every failure of its own: refused, unreachable, session lost, or the remote command killed by a signal. The bare status can't tell those apart.What
endRemoteTaskSessionsnow wraps the error with the end of ssh's stderr (last 300 bytes, whitespace folded to one line, valid UTF-8):exit status 255: <what ssh said>.Could not reach <host> to end this task's agent session; will retry: <error>. Later retries in the same run don't add more log lines.Investigating ivo's 255 (from Bruce)
I used the same ssh options and ControlPath,
sh -lcwrapping, andhostSweepScriptbuilt from ivo's 28 real blocked runs.echo) under my shell and under the daemon's exact environment (env -i, the daemon's PATH, noSSH_AUTH_SOCK) exits 0 in 0.2–0.6s. It matches no windows and no processes. Auth works with and without an agent.$$) and every ancestor are excluded;$(…)subshells andps) is held;killloops,readlinkandresolverun after the snapshot, so they are never candidates;tmux: servertitle, are excluded;$HOME(sshd-session's cwd is unreadable, so it is skipped).sleep 900with its cwd in one recorded worktree and ran the real script for that one task, with a session name that doesn't exist. It killed exactly thatsleep(P 5653 <pid>) and exited 0.So the cause was transient and is still unknown. If it comes back, this PR makes the warning and the task logs say why.
Tested
TestRemoteSweepFailureSaysWhatSSHSaid: a fake host whose ssh prints a reason on stderr and exits 255. The error carries both the status and the reason. Over two failed sweeps, the task log says so exactly once. Fails without the change.TestWithStderrKeepsTheEnd: only the end of a long stderr is kept, on one line, and other errors pass through unchanged.golangci-lintv2.13.2: 0 issues.go test -race -p 4 ./...: only the failures that also occur onmainon this Mac (the/private/varexecutor tests, reaperTestLivePanePIDs_HonorsAgentSocket, uiTestRemoteAttachDropsALeftOverViewPairing).🤖 Generated with Claude Code