← back to Exo
Fix graceful process shutdown in macOS app (#1372)
db79c350c1bc4af0c56c42d630d2b723d9a18a83 · 2026-02-17 09:03:54 -0800 · Alex Cheema
## Motivation
Fixes #1370
When the macOS app stops exo, GPU/system memory isn't released. This
happens because:
1. The macOS app calls `process.terminate()` (SIGTERM) but the Python
process only registers a graceful shutdown handler for SIGINT, not
SIGTERM. SIGTERM's default Python behavior raises `SystemExit` which
bypasses the cleanup cascade (runner subprocess MLX cleanup via
`mx.clear_cache()`, channel closing, etc.).
2. The app doesn't wait for the process to actually finish cleanup — it
immediately nils out the process reference.
## Changes
**`src/exo/main.py`**: Register SIGTERM handler alongside SIGINT so the
graceful shutdown cascade (`Node.shutdown()` → cancel task group →
worker/runner cleanup → `mx.clear_cache()` + `gc.collect()`) runs
regardless of which signal is received.
**`app/EXO/EXO/ExoProcessController.swift`**: Replace immediate
`process.terminate()` with escalating shutdown per @Evanev7's
suggestion:
1. Send SIGINT via `process.interrupt()` — triggers the registered
Python handler for graceful cleanup
2. Wait up to 5 seconds for the process to exit
3. If still running, escalate to SIGTERM via `process.terminate()`
4. Wait up to 3 seconds
5. If still running, force kill via SIGKILL
The escalation runs in a detached `Task` so the UI updates immediately
(status → stopped) without blocking.
## Why It Works
The root cause is that SIGTERM wasn't triggering the graceful shutdown
path. By registering a SIGTERM handler in Python and sending SIGINT
first from the macOS app, the process gets a chance to run the full
cleanup cascade: cancelling the task group, shutting down runners (which
call `del model; mx.clear_cache(); gc.collect()`), closing channels, and
flushing logs. The escalation to SIGTERM and SIGKILL ensures the process
always terminates even if graceful shutdown hangs.
## Test Plan
### Manual Testing
<!-- Hardware: Mac Studio M4 Max 128GB -->
- Start exo via macOS app, load a model, run inference
- Stop via the toggle switch, verify memory is released without
requiring a system restart
- Test rapid stop/start (restart) to ensure no race conditions
### Automated Testing
- `uv run basedpyright` — 0 errors
- `uv run ruff check` — passes
- `nix fmt` — no changes
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: Evan Quiney <evanev7@gmail.com>
Files touched
M app/EXO/EXO/ExoProcessController.swiftM src/exo/main.py
Diff
commit db79c350c1bc4af0c56c42d630d2b723d9a18a83
Author: Alex Cheema <41707476+AlexCheema@users.noreply.github.com>
Date: Tue Feb 17 09:03:54 2026 -0800
Fix graceful process shutdown in macOS app (#1372)
## Motivation
Fixes #1370
When the macOS app stops exo, GPU/system memory isn't released. This
happens because:
1. The macOS app calls `process.terminate()` (SIGTERM) but the Python
process only registers a graceful shutdown handler for SIGINT, not
SIGTERM. SIGTERM's default Python behavior raises `SystemExit` which
bypasses the cleanup cascade (runner subprocess MLX cleanup via
`mx.clear_cache()`, channel closing, etc.).
2. The app doesn't wait for the process to actually finish cleanup — it
immediately nils out the process reference.
## Changes
**`src/exo/main.py`**: Register SIGTERM handler alongside SIGINT so the
graceful shutdown cascade (`Node.shutdown()` → cancel task group →
worker/runner cleanup → `mx.clear_cache()` + `gc.collect()`) runs
regardless of which signal is received.
**`app/EXO/EXO/ExoProcessController.swift`**: Replace immediate
`process.terminate()` with escalating shutdown per @Evanev7's
suggestion:
1. Send SIGINT via `process.interrupt()` — triggers the registered
Python handler for graceful cleanup
2. Wait up to 5 seconds for the process to exit
3. If still running, escalate to SIGTERM via `process.terminate()`
4. Wait up to 3 seconds
5. If still running, force kill via SIGKILL
The escalation runs in a detached `Task` so the UI updates immediately
(status → stopped) without blocking.
## Why It Works
The root cause is that SIGTERM wasn't triggering the graceful shutdown
path. By registering a SIGTERM handler in Python and sending SIGINT
first from the macOS app, the process gets a chance to run the full
cleanup cascade: cancelling the task group, shutting down runners (which
call `del model; mx.clear_cache(); gc.collect()`), closing channels, and
flushing logs. The escalation to SIGTERM and SIGKILL ensures the process
always terminates even if graceful shutdown hangs.
## Test Plan
### Manual Testing
<!-- Hardware: Mac Studio M4 Max 128GB -->
- Start exo via macOS app, load a model, run inference
- Stop via the toggle switch, verify memory is released without
requiring a system restart
- Test rapid stop/start (restart) to ensure no race conditions
### Automated Testing
- `uv run basedpyright` — 0 errors
- `uv run ruff check` — passes
- `nix fmt` — no changes
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: Evan Quiney <evanev7@gmail.com>
---
app/EXO/EXO/ExoProcessController.swift | 32 +++++++++++++++++++++++++++++---
src/exo/main.py | 4 ++--
2 files changed, 31 insertions(+), 5 deletions(-)
diff --git a/app/EXO/EXO/ExoProcessController.swift b/app/EXO/EXO/ExoProcessController.swift
index 704b9d4f..7566674b 100644
--- a/app/EXO/EXO/ExoProcessController.swift
+++ b/app/EXO/EXO/ExoProcessController.swift
@@ -126,11 +126,37 @@ final class ExoProcessController: ObservableObject {
return
}
process.terminationHandler = nil
- if process.isRunning {
- process.terminate()
+ status = .stopped
+
+ guard process.isRunning else {
+ self.process = nil
+ return
}
+
+ let proc = process
self.process = nil
- status = .stopped
+
+ Task.detached {
+ proc.interrupt()
+
+ for _ in 0..<50 {
+ if !proc.isRunning { return }
+ try? await Task.sleep(nanoseconds: 100_000_000)
+ }
+
+ if proc.isRunning {
+ proc.terminate()
+ }
+
+ for _ in 0..<30 {
+ if !proc.isRunning { return }
+ try? await Task.sleep(nanoseconds: 100_000_000)
+ }
+
+ if proc.isRunning {
+ kill(proc.processIdentifier, SIGKILL)
+ }
+ }
}
func restart() {
diff --git a/src/exo/main.py b/src/exo/main.py
index e1ef3a70..1d358975 100644
--- a/src/exo/main.py
+++ b/src/exo/main.py
@@ -136,6 +136,8 @@ class Node:
async def run(self):
async with self._tg as tg:
+ signal.signal(signal.SIGINT, lambda _, __: self.shutdown())
+ signal.signal(signal.SIGTERM, lambda _, __: self.shutdown())
tg.start_soon(self.router.run)
tg.start_soon(self.election.run)
if self.download_coordinator:
@@ -147,8 +149,6 @@ class Node:
if self.api:
tg.start_soon(self.api.run)
tg.start_soon(self._elect_loop)
- signal.signal(signal.SIGINT, lambda _, __: self.shutdown())
- signal.signal(signal.SIGTERM, lambda _, __: self.shutdown())
def shutdown(self):
# if this is our second call to shutdown, just sys.exit
← d6301ed5 dashboard: redesign downloads page as model×node table (#146
·
back to Exo
·
Add MetaInstance declarative layer (#1447) a962a28a →