feat(task-runner): stop running tasks when the process is interrupted - #837
Open
ddeboer wants to merge 3 commits into
Open
feat(task-runner): stop running tasks when the process is interrupted#837ddeboer wants to merge 3 commits into
ddeboer wants to merge 3 commits into
Conversation
ddeboer
changed the title
…run is interrupted - Listen for SIGINT and SIGTERM for the duration of a run; on either, stop the processes still going, remove the run directory, drop the listeners and re-raise the signal so the process exits as it would have - Each run registers its own listeners and removes only those, so parallel runs do not clobber each other - Document the behaviour under “Converting several chunks at once”
- Add LiveTasks to @lde/task-runner: while any runner has a task going, the process listens for SIGINT and SIGTERM, stops every task the way the runner does, and then ends the process as it would have without the listening – with the signal’s exit status, or however another listener ends it. Listeners are removed as the signal arrives, so a second one ends the process at once. - Bind every process NativeTaskRunner spawns and every container DockerTaskRunner starts to the process’s lifetime, so QLever’s index build and server are covered as well. - Have SparqlAnythingConverter remove its run directory on process exit instead of listening for signals itself. - Document the behaviour on the task-runner docs page and link to it from the runner and converter pages.
ddeboer
force-pushed
the
fix/converter-stops-on-signal
branch
from
September 10, 2026 11:51
fb67550 to
dbf0eed
Compare
- ChildTasks says whose tasks they are, and that they end with the parent; LiveTasks only said they were running. - Keep it out of the task-runner docs page: it is for implementing a runner, and a runner’s users never see it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
NativeTaskRunnerspawns each task withdetached: true, in a process group of its own, so a Ctrl-C never reaches it; a container under the Docker daemon hears nothing of the process either. Nothing listened forSIGINT/SIGTERM, so interrupting the Node process – Ctrl-C during a long mapping, a cancelled CI job – left the JVMs running and thesparql-anything-*run directory behind.The runner made its tasks immune to the signal, so the runner is where they get bound to the process again. That fixes it for every consumer at once – QLever’s index build and server had the same gap – rather than in the one converter that happened to be reported.
@lde/task-runnergainsChildTasks: a runner adds a task once it has started and deletes it once it has ended. While any runner in the process has a task going, oneSIGINT/SIGTERMlistener stops every task the way its runner’sstop()does, then ends the process as it would have without the listening: with the signal’s exit status (130/143) when nothing else listens for the signal, and otherwise however that listener ends it. The listener is removed the moment the signal arrives, so a second Ctrl-C ends the process at once when a task refuses to stop. Idle runners leave the process’s signal handling as they found it.NativeTaskRunnerandDockerTaskRunnerbind each process/container to it. The native runner sends the groupSIGTERMsynchronously inside the listener, so even a host handler that callsprocess.exit(0)cannot orphan the processes.SparqlAnythingConverterno longer listens for signals. It registers a synchronous'exit'listener for the run’s duration that removes the run directory, which covers the runner ending the process as well as a host callingprocess.exit()mid-run.Fix #831