Skip to content

[9.0] fix (WMS): clear the signal mask before starting the payload - #8838

Draft
chrisburr wants to merge 1 commit into
DIRACGrid:rel-v9r0from
chrisburr:fix/reset-signal-mask-for-payload
Draft

chrisburr wants to merge 1 commit into
DIRACGrid:rel-v9r0from
chrisburr:fix/reset-signal-mask-for-payload

Conversation

@chrisburr

Copy link
Copy Markdown
Member

Creating a gfal2 context (gfal2.creat_context()) leaves most signals blocked in the calling thread (SigBlk: fffffffe3ff9ea07: SIGHUP, SIGINT, SIGQUIT, SIGUSR1/2, SIGALRM, SIGTERM, ...). The signal mask is inherited across fork() and kept by exec(), so every payload started by the JobWrapper after a gfal2 context was created runs with these signals blocked:

  • the payload ignores SIGTERM, e.g. from the batch system before a hard kill;
  • Gaudi's EventWatchdogAlg can't abort a stuck event, because it uses raise(SIGQUIT): in LHCb productions it logged FATAL ... aborting process 17 times over ~27 hours without the job stopping (gaudi/Gaudi#428);
  • Ctrl-C doesn't stop a payload when running a job locally.

This clears the signal mask in ExecutionThread just before it starts the payload. The thread only exists to run the payload, so the rest of the JobWrapper is unaffected, and no preexec_fn is needed.

BEGINRELEASENOTES
*WorkloadManagement
FIX: the payload no longer inherits signals blocked in the JobWrapper (e.g. by gfal2), so it can be stopped with SIGTERM and abort itself with SIGQUIT
ENDRELEASENOTES

Creating a gfal2 context leaves most signals (SIGTERM, SIGINT, SIGQUIT,
SIGALRM, ...) blocked in the calling thread. The payload inherits the
signal mask of the thread that starts it, so it couldn't be stopped with
SIGTERM, and Gaudi's EventWatchdog could not abort stuck jobs with SIGQUIT
(gaudi/Gaudi#428).
@chrisburr
chrisburr force-pushed the fix/reset-signal-mask-for-payload branch from d89903c to 4d89c6b Compare October 20, 2026 12:43

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant