Skip to content

test(harness): check fixture exit codes; fix exec signal exit codes and a ctrl-c panic - #50

Merged
tobert merged 5 commits into
mainfrom
test/harness-exit-codes
Oct 24, 2026
Merged

tobert merged 5 commits into
mainfrom
test/harness-exit-codes

Conversation

@tobert

@tobert tobert commented Oct 24, 2026 •

Copy link
Copy Markdown
Owner

The functional test harness never compared Expect.ExitCode. Nine fixtures set it, and none was tested. With the check on, three fixtures showed a real bug: otel-cli exec exits 255 when its child is killed by a signal.

Behavior change for review: exec now exits 128 + signal for a signaled child, which is the shell convention: 130 for Ctrl-C, 137 for --command-timeout (SIGKILL). Before, it exited 255. Scripts can see this change. The alternative is to keep 255 and pin it in the fixtures.

What changed:

  • main_test.go: checkProcess compares the exit code when the process exited on its own. A killed process reports -1, so it is skipped.
  • data_for_test.go: the #360 fixture uses ExitCode: 42 in place of a CheckFunc; "protocol: bad config" with --fail expects 1. The three signal fixtures expect 130 and 137. Their old values (2 and 0) were never checked.
  • otelcli/exec.go: 128 + status.Signal() when ProcessState.Sys() is a syscall.WaitStatus and Signaled().
  • otelcli/exec.go: signal.Stop(signals) before close(signals). Before, a Ctrl-C during the span export panicked with "send on closed channel" (exit 2). Now SIGINT has its default action. The new fixture "exec ctrl-c during export does not panic" was red before the fix.
  • CHANGELOG: two lines under 0.6.0 Fixed.

Commits are test-first: each fix follows a failing fixture.

Verification:

go vet ./... && go build && go test ./...                 # green
GOOS=windows|darwin|freebsd go build -o /dev/null .       # ok
./otel-cli exec --command-timeout 50ms -- sleep 1; echo $?   # 137
./otel-cli exec -- sh -c 'exit 3'; echo $?                   # 3

Found by the kaibo review of AGENTS.md on #49. After this merges, the AGENTS.md hazard "the harness ignores Expect.ExitCode" is no longer true; #49 drops it.

🤖 Generated with Claude Code

tobert added a commit that referenced this pull request Oct 24, 2026
🤖 Claude <claude@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
tobert and others added 2 commits October 24, 2026 13:24
Why: a kaibo review of AGENTS.md (#49) found that checkProcess never
compared Expect.ExitCode. Nine fixtures set it; none was tested. Amy: "yes,
go ahead on a test PR if we found some gaps."
Approach: checkProcess compares the exit code when the process exited on
its own (a killed process reports -1). Fixture fixes: #360 now uses
ExitCode: 42 in place of a CheckFunc; "protocol: bad config" with --fail
expects 1. The three exec signal fixtures expect the shell convention,
128 + signal: 130 for SIGINT, 137 for SIGKILL. Their old values (2 and 0)
were never checked and match no rule.
Learned: otel-cli exits 255 when the exec child dies from a signal.
ProcessState.ExitCode() is -1, and os.Exit(-1) is 255.
Next: this commit fails on purpose (3 signal fixtures); the fix follows.

🤖 Claude <claude@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Why: the harness now checks exit codes, and three exec fixtures showed
otel-cli exits 255 when its child dies from a signal (Ctrl-C forwarded,
or --command-timeout). ProcessState.ExitCode() is -1 for a signaled child,
and os.Exit(-1) is 255, so scripts lose the signal.
Approach: when ProcessState.Sys() is a syscall.WaitStatus and Signaled(),
exit 128 + signal, as bash does: SIGINT 130, SIGKILL 137. WaitStatus
exists on Windows too (Signaled is always false); windows, darwin and
freebsd cross-build.
Learned: --command-timeout kills with SIGKILL (exec.CommandContext), so a
timeout exits 137, not timeout(1)'s 124.
Next: Amy decides whether 128 + signal is the behavior to keep.

Verified: go vet, go test ./... green; `otel-cli exec --command-timeout
50ms -- sleep 1` exits 137; `exec -- sh -c 'exit 3'` exits 3.

🤖 Claude <claude@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@tobert

tobert commented Oct 24, 2026

Copy link
Copy Markdown
Owner Author

kaibo review (cast deepseek). Claude checked each applied finding against the code.

Applied:

  • checkProcess skipped the exit-code check whenever TimedOut was set. The server timer sets TimedOut even when otel-cli exits on its own (main_test.go:446-456), so a real exit code could be skipped. The guard is now !CommandFailed only.
  • The ExecExitCode: 1 expectation in "protocol: bad config" checked nothing. Diagnostics.ToStringMap omits exec_exit_code, and only exec writes it. Removed.
  • The CHANGELOG line now says Unix. On Windows, WaitStatus.Signaled() is always false.

Confirmed correct: 128 + signal on Linux, darwin and freebsd. Diag.ExecExitCode has one writer and is set before any export-time SoftFail. --fail precedence is unchanged from #360. Background fixtures do not race on ExitCode. No fixture passes by accident (all 11 ExitCode fields traced).

Not fixed here, verified and pre-existing:

  • exec.go calls close(signals) while signal.Notify is still registered. A second Ctrl-C during the export window sends on a closed channel and panics. The fix is signal.Stop(signals) before the close.
  • The signal fixtures have 50-200 ms budgets. This PR adds no new timing; the budgets were already enforced through TimedOut.
  • Lower priority: status always exits 0; start failures always exit 127 (shells use 126 for "not executable"); EventCount is never compared.

🤖 Generated with Claude Code

Why: kaibo (deepseek) review of #50. The server timer sets TimedOut even
when otel-cli exits on its own (main_test.go:446-456), so the TimedOut
guard could skip a real exit code. CommandFailed alone means "no exit code".
Approach: guard on !CommandFailed only. Removed the "protocol: bad config"
ExecExitCode: 1 expectation: Diagnostics.ToStringMap omits exec_exit_code
and only exec writes it, so it checked nothing. CHANGELOG notes the fix is
Unix-only (WaitStatus.Signaled is always false on Windows).
Learned: exec.go closes the signals channel while signal.Notify is still
registered; a second SIGINT during export can panic. Pre-existing, not
fixed here.
Next: Amy's call on the signal.Stop fix.

Verified: go vet, go test (3 consecutive runs green).

🔍 Reviewed-by: DeepSeek (kaibo, deepseek cast)
🤖 Claude <claude@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@tobert tobert changed the title test(harness): check fixture exit codes; exec exits 128 + signal, not 255 test(harness): check fixture exit codes; fix exec signal exit codes and a ctrl-c panic Oct 24, 2026
tobert and others added 2 commits October 24, 2026 13:36
…anicked

Why: the kaibo review of #50 found that exec.go closes the signals
channel while signal.Notify is still registered. Amy: "it can go in pr 50".
Approach: test first. New fixture "exec ctrl-c during export does not
panic": /bin/true exits, the export to a stopped server retries for up to
--timeout 1s, and the harness sends SIGINT at 200ms. Before the fix it
exits 2 with "panic: send on closed channel" (reproduced by hand too).
Fix: signal.Stop(signals) before close(signals). SIGINT during export now
has its default action, so otel-cli stops and the span is not sent.
Learned: a hand repro from a bash background job exits 0, not 130.
Background jobs start with SIGINT ignored, and after signal.Stop Go
returns to the inherited ignore.
Next: review.

Verified: go vet, go test ./... green 3 runs in a row; fixture red before
the fix.

🤖 Claude <claude@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@tobert
tobert merged commit cf11661 into main Oct 24, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant