Escalating signal handling: kill child processes on repeated Ctrl-C

[CodeEnPlace]

Currently, Ctrl-C sets the done flag and waits for in-flight tests to finish naturally. This is safe (no orphans) but slow if a test is stuck.

Proposed escalation

1. 1st Ctrl-C: Set done, stop picking new work, let in-flight phases finish (current behavior)
2. 2nd Ctrl-C: Send SIGKILL to all child process groups, unblocking worker threads immediately
3. 3rd Ctrl-C: std::process::exit(1) as a last resort

This requires tracking spawned child PIDs in a global registry that the signal handler can access. The kill_process_tree function in phase.rs already uses SIGKILL on process groups, so the mechanism exists — it just needs to be callable from the handler.

Context

Children are spawned with process_group(0) so they're in their own process groups. SIGKILL to -pgid kills the entire group. This is already tested to work 3+ levels deep.

[FreddieGilbraith]

GH#40 Architecture Proposal: Cancellable Phase

The Problem

Phase::run() spawns a child, then blocks on child.wait(). To kill it from
the Ctrl-C handler we need to either (a) track the PID externally (the grotty
approach) or (b) get a signal into the Phase while it's blocking.

Call Chain (where the signal needs to travel)

run.rs (worker thread)
  → step_test_mutation.rs  ::  StepTestMutation::run()
    → bough_lib::run_test_in_workspace()            [dirs_ops.rs]
      → phase::run_phase_in_workspace()
        → Phase::run()      ← child lives & blocks here

Same chain for init and reset phases. About 10 function signatures to touch.

---

Option A — Thread Cancellation token through the call chain

Add a Cancellation type to bough-lib:

#[derive(Clone)]
pub struct Cancellation(Arc<AtomicBool>);

impl Cancellation {
    pub fn new() -> Self
    pub fn cancel(&self)
    pub fn is_cancelled(&self) -> bool
}

Phase::run() changes to poll it inside a 50ms wait_timeout loop:

pub fn run(&self, cancel: &Cancellation) -> Result<PhaseOutcome, Error> {
    // ...
    loop {
        if cancel.is_cancelled() {
            kill_process_tree(&mut child);
            child.wait()?;
            return Ok(PhaseOutcome::Cancelled);
        }
        match child.wait_timeout(Duration::from_millis(50))? {
            Some(_) => break,
            None => continue,
        }
    }

One Cancellation created in run.rs, cloned into each worker thread and the
ctrlc handler closure. 2nd Ctrl-C → cancel.cancel(). Every worker's in-flight
Phase sees it within 50ms and kills its own child.

Diff size: ~10 signatures change. Compiler guides every callsite.

Pros:

• No shared mutable collection, no PID tracking
• Typed, self-documenting
• Composable — can be cloned and passed independently
• Easy to add CancellationReason later
• Easy to unit-test (Cancellation::new(), call cancel(), check Phase returns
  PhaseOutcome::Cancelled)

Cons:

• PhaseOutcome gains a new variant (Cancelled)
• 50ms polling latency on kill (acceptable for mutation testing)
• All intermediate fns gain a parameter (but compiler tells you exactly where)

---

Option B — PhaseOptions context bag

Same as A but wrap the token in a struct so future concerns don't need new params:

#[derive(Clone, Default)]
pub struct PhaseOptions {
    pub cancel: Option<Cancellation>,
    // future: tracing span, env overrides, etc.
}

Phase::run(&self, opts: &PhaseOptions). Everything else identical.

vs A: One more struct, but future-proofs the API against further parameter
creep. Probably the right call if you expect more things to flow this way.

---

Option C — Split into spawn() + RunningPhase handle

Split Phase::run() into two steps:

impl Phase {
    pub fn spawn(self) -> Result<RunningPhase, Error>
}

pub struct RunningPhase {
    pid: u32,
    child: std::process::Child,
    start: Instant,
    timeout: Option<Duration>,
    stdout_thread: JoinHandle<io::Result<Vec<u8>>>,
    stderr_thread: JoinHandle<io::Result<Vec<u8>>>,
}

impl RunningPhase {
    pub fn pid(&self) -> u32
    pub fn wait(mut self) -> Result<PhaseOutcome, Error>
    pub fn wait_with_cancel(mut self, cancel: &Cancellation) -> Result<PhaseOutcome, Error>
    pub fn kill(mut self) -> Result<(), Error>
}

Workers call phase.spawn() then running.wait_with_cancel(&cancel).

vs A/B: RunningPhase is a valuable standalone abstraction:

• Can inspect pid() without a registry
• kill() is explicit and typed
• Could be collected into a Vec<RunningPhase> to implement a proper
  "drain and kill all" shutdown sequence
• Could select across multiple running phases with wait_timeout polling

More code to write, but arguably the right shape for a tool that has N concurrent
workers each running one phase at a time.

---

Recommendation

Option B (PhaseOptions) if you want the minimal-friction path and prefer to
keep Phase::run() as a single blocking call.

Option C + B (RunningPhase with PhaseOptions) if you're open to a slightly
larger refactor — RunningPhase is a genuinely useful abstraction for a parallel
mutation tester and positions the code better for future work (e.g. streaming
progress from individual phases, draining with outcome reports on Ctrl-C).

Either way, the escalating handler in run.rs becomes trivially simple:

let cancel = Cancellation::new();
let signal_count = Arc::new(AtomicUsize::new(0));
let cancel_for_handler = cancel.clone();

ctrlc::set_handler(move || {
    match signal_count.fetch_add(1, Ordering::SeqCst) {
        0 => { done.store(true, Ordering::SeqCst); eprintln!("...waiting"); }
        1 => { cancel_for_handler.cancel(); eprintln!("...killing"); }
        _ => std::process::exit(1),
    }
});

No PID registry anywhere.