← back to Exo
fix: replace flaky internet checks with explicit offline mode (#1615)
7660893538dc0577cf5ed749d5e183168e7f6cd1 · 2026-02-24 13:36:19 -0800 · Alex Cheema
## Motivation
The socket-based internet connectivity probes (connecting to
1.1.1.1/8.8.8.8/1.0.0.1 every 10 seconds) are flaky — they produce false
negatives on some networks/ISPs, causing downloads to silently fail or
get stuck. Instead of dynamically detecting connectivity, this replaces
it with an explicit offline mode that the user opts into.
## Changes
- **Removed internet check infrastructure**: Deleted
`_test_internet_connection()` (socket probes) and
`_check_internet_connection()` (10s polling loop) from
`DownloadCoordinator`
- **Renamed `internet_connection` → `offline`** on `ShardDownloader`
base class and all subclasses (`SingletonShardDownloader`,
`CachedShardDownloader`, `ResumableShardDownloader`)
- **Removed `on_connection_lost` callbacks** from download calls — no
longer needed without dynamic connectivity detection
- **Added `EXO_OFFLINE` env var support** in `constants.py` and `Args`
default in `main.py`
- **Added Offline Mode toggle** to macOS app Settings → General tab,
which sets `EXO_OFFLINE=true` in the process environment and restarts
## Why It Works
The download behavior (`skip_internet` conditionals in
`download_utils.py`) is unchanged — we just changed what drives it.
Instead of a flaky socket probe setting
`internet_connection=True/False`, the user explicitly sets offline mode
via:
1. `--offline` CLI flag
2. `EXO_OFFLINE=true` environment variable
3. macOS app Settings → General → Offline Mode toggle
This is more reliable and predictable. Users on flaky networks no longer
get intermittent download failures.
## Test Plan
### Manual Testing
- `uv run exo --offline` starts without internet checks, rejects
downloads for unavailable models
- `EXO_OFFLINE=true uv run exo` behaves identically
- macOS app Settings toggle persists across restarts and passes env var
to the exo process
### Automated Testing
- All existing tests pass (222 passed), including 8 offline-mode
specific tests
- `uv run basedpyright` — 0 errors
- `uv run ruff check` — all checks passed
- `nix fmt` — 0 files changed
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Files touched
M app/EXO/EXO/ExoProcessController.swiftM app/EXO/EXO/Views/SettingsView.swiftM src/exo/download/coordinator.pyM src/exo/download/impl_shard_downloader.pyM src/exo/download/shard_downloader.pyM src/exo/main.pyM src/exo/shared/constants.py
Diff
commit 7660893538dc0577cf5ed749d5e183168e7f6cd1
Author: Alex Cheema <41707476+AlexCheema@users.noreply.github.com>
Date: Tue Feb 24 13:36:19 2026 -0800
fix: replace flaky internet checks with explicit offline mode (#1615)
## Motivation
The socket-based internet connectivity probes (connecting to
1.1.1.1/8.8.8.8/1.0.0.1 every 10 seconds) are flaky — they produce false
negatives on some networks/ISPs, causing downloads to silently fail or
get stuck. Instead of dynamically detecting connectivity, this replaces
it with an explicit offline mode that the user opts into.
## Changes
- **Removed internet check infrastructure**: Deleted
`_test_internet_connection()` (socket probes) and
`_check_internet_connection()` (10s polling loop) from
`DownloadCoordinator`
- **Renamed `internet_connection` → `offline`** on `ShardDownloader`
base class and all subclasses (`SingletonShardDownloader`,
`CachedShardDownloader`, `ResumableShardDownloader`)
- **Removed `on_connection_lost` callbacks** from download calls — no
longer needed without dynamic connectivity detection
- **Added `EXO_OFFLINE` env var support** in `constants.py` and `Args`
default in `main.py`
- **Added Offline Mode toggle** to macOS app Settings → General tab,
which sets `EXO_OFFLINE=true` in the process environment and restarts
## Why It Works
The download behavior (`skip_internet` conditionals in
`download_utils.py`) is unchanged — we just changed what drives it.
Instead of a flaky socket probe setting
`internet_connection=True/False`, the user explicitly sets offline mode
via:
1. `--offline` CLI flag
2. `EXO_OFFLINE=true` environment variable
3. macOS app Settings → General → Offline Mode toggle
This is more reliable and predictable. Users on flaky networks no longer
get intermittent download failures.
## Test Plan
### Manual Testing
- `uv run exo --offline` starts without internet checks, rejects
downloads for unavailable models
- `EXO_OFFLINE=true uv run exo` behaves identically
- macOS app Settings toggle persists across restarts and passes env var
to the exo process
### Automated Testing
- All existing tests pass (222 passed), including 8 offline-mode
specific tests
- `uv run basedpyright` — 0 errors
- `uv run ruff check` — all checks passed
- `nix fmt` — 0 files changed
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
---
app/EXO/EXO/ExoProcessController.swift | 12 +++++++++++
app/EXO/EXO/Views/SettingsView.swift | 11 ++++++++++
src/exo/download/coordinator.py | 35 -------------------------------
src/exo/download/impl_shard_downloader.py | 28 ++++++++++---------------
src/exo/download/shard_downloader.py | 5 -----
src/exo/main.py | 7 ++++---
src/exo/shared/constants.py | 2 ++
7 files changed, 40 insertions(+), 60 deletions(-)
diff --git a/app/EXO/EXO/ExoProcessController.swift b/app/EXO/EXO/ExoProcessController.swift
index 1c9bc12f..a97c0a06 100644
--- a/app/EXO/EXO/ExoProcessController.swift
+++ b/app/EXO/EXO/ExoProcessController.swift
@@ -5,6 +5,7 @@ import Foundation
private let customNamespaceKey = "EXOCustomNamespace"
private let hfTokenKey = "EXOHFToken"
private let enableImageModelsKey = "EXOEnableImageModels"
+private let offlineModeKey = "EXOOfflineMode"
private let onboardingCompletedKey = "EXOOnboardingCompleted"
@MainActor
@@ -60,6 +61,14 @@ final class ExoProcessController: ObservableObject {
UserDefaults.standard.set(enableImageModels, forKey: enableImageModelsKey)
}
}
+ @Published var offlineMode: Bool = {
+ return UserDefaults.standard.bool(forKey: offlineModeKey)
+ }()
+ {
+ didSet {
+ UserDefaults.standard.set(offlineMode, forKey: offlineModeKey)
+ }
+ }
/// Fires once when EXO transitions to `.running` for the very first time (fresh install).
@Published private(set) var isFirstLaunchReady = false
@@ -267,6 +276,9 @@ final class ExoProcessController: ObservableObject {
if enableImageModels {
environment["EXO_ENABLE_IMAGE_MODELS"] = "true"
}
+ if offlineMode {
+ environment["EXO_OFFLINE"] = "true"
+ }
var paths: [String] = []
if let existing = environment["PATH"], !existing.isEmpty {
diff --git a/app/EXO/EXO/Views/SettingsView.swift b/app/EXO/EXO/Views/SettingsView.swift
index ed221c89..bc600efe 100644
--- a/app/EXO/EXO/Views/SettingsView.swift
+++ b/app/EXO/EXO/Views/SettingsView.swift
@@ -13,6 +13,7 @@ struct SettingsView: View {
@State private var pendingNamespace: String = ""
@State private var pendingHFToken: String = ""
@State private var pendingEnableImageModels = false
+ @State private var pendingOfflineMode = false
@State private var needsRestart = false
@State private var bugReportInFlight = false
@State private var bugReportMessage: String?
@@ -42,6 +43,7 @@ struct SettingsView: View {
pendingNamespace = controller.customNamespace
pendingHFToken = controller.hfToken
pendingEnableImageModels = controller.enableImageModels
+ pendingOfflineMode = controller.offlineMode
needsRestart = false
}
}
@@ -72,6 +74,13 @@ struct SettingsView: View {
.foregroundColor(.secondary)
}
+ Section {
+ Toggle("Offline Mode", isOn: $pendingOfflineMode)
+ Text("Skip internet checks and use only locally available models.")
+ .font(.caption)
+ .foregroundColor(.secondary)
+ }
+
Section {
HStack {
Spacer()
@@ -445,6 +454,7 @@ struct SettingsView: View {
private var hasGeneralChanges: Bool {
pendingNamespace != controller.customNamespace || pendingHFToken != controller.hfToken
+ || pendingOfflineMode != controller.offlineMode
}
private var hasModelChanges: Bool {
@@ -454,6 +464,7 @@ struct SettingsView: View {
private func applyGeneralSettings() {
controller.customNamespace = pendingNamespace
controller.hfToken = pendingHFToken
+ controller.offlineMode = pendingOfflineMode
restartIfRunning()
}
diff --git a/src/exo/download/coordinator.py b/src/exo/download/coordinator.py
index f1988135..65a25e63 100644
--- a/src/exo/download/coordinator.py
+++ b/src/exo/download/coordinator.py
@@ -1,5 +1,4 @@
import asyncio
-import socket
from dataclasses import dataclass, field
from random import random
@@ -73,8 +72,6 @@ class DownloadCoordinator:
def __post_init__(self) -> None:
self.event_sender, self.event_receiver = channel[Event]()
- if self.offline:
- self.shard_downloader.set_internet_connection(False)
self.shard_downloader.on_progress(self._download_progress_callback)
def _model_dir(self, model_id: ModelId) -> str:
@@ -123,8 +120,6 @@ class DownloadCoordinator:
logger.info(
f"Starting DownloadCoordinator{' (offline mode)' if self.offline else ''}"
)
- if not self.offline:
- self._test_internet_connection()
try:
async with self._tg as tg:
tg.start_soon(self._command_processor)
@@ -132,40 +127,10 @@ class DownloadCoordinator:
tg.start_soon(self._emit_existing_download_progress)
tg.start_soon(self._resend_out_for_delivery)
tg.start_soon(self._clear_ofd)
- if not self.offline:
- tg.start_soon(self._check_internet_connection)
finally:
for task in self.active_downloads.values():
task.cancel()
- def _test_internet_connection(self) -> None:
- # Try multiple endpoints since some ISPs/networks block specific IPs
- for host in ("1.1.1.1", "8.8.8.8", "1.0.0.1"):
- try:
- socket.create_connection((host, 443), timeout=3).close()
- self.shard_downloader.set_internet_connection(True)
- logger.debug(f"Internet connectivity: True (via {host})")
- return
- except OSError:
- continue
- self.shard_downloader.set_internet_connection(False)
- logger.debug("Internet connectivity: False")
-
- async def _check_internet_connection(self) -> None:
- first_connection = True
- while True:
- await asyncio.sleep(10)
-
- # Assume that internet connection is set to False on 443 errors.
- if self.shard_downloader.internet_connection:
- continue
-
- self._test_internet_connection()
-
- if first_connection and self.shard_downloader.internet_connection:
- first_connection = False
- self._tg.start_soon(self._emit_existing_download_progress)
-
def shutdown(self) -> None:
self._tg.cancel_tasks()
diff --git a/src/exo/download/impl_shard_downloader.py b/src/exo/download/impl_shard_downloader.py
index 0f03074c..becd94f5 100644
--- a/src/exo/download/impl_shard_downloader.py
+++ b/src/exo/download/impl_shard_downloader.py
@@ -15,9 +15,13 @@ from exo.shared.types.worker.shards import (
)
-def exo_shard_downloader(max_parallel_downloads: int = 8) -> ShardDownloader:
+def exo_shard_downloader(
+ max_parallel_downloads: int = 8, offline: bool = False
+) -> ShardDownloader:
return SingletonShardDownloader(
- CachedShardDownloader(ResumableShardDownloader(max_parallel_downloads))
+ CachedShardDownloader(
+ ResumableShardDownloader(max_parallel_downloads, offline=offline)
+ )
)
@@ -50,10 +54,6 @@ class SingletonShardDownloader(ShardDownloader):
self.shard_downloader = shard_downloader
self.active_downloads: dict[ShardMetadata, asyncio.Task[Path]] = {}
- def set_internet_connection(self, value: bool) -> None:
- self.internet_connection = value
- self.shard_downloader.set_internet_connection(value)
-
def on_progress(
self,
callback: Callable[[ShardMetadata, RepoDownloadProgress], Awaitable[None]],
@@ -90,10 +90,6 @@ class CachedShardDownloader(ShardDownloader):
self.shard_downloader = shard_downloader
self.cache: dict[tuple[str, ShardMetadata], Path] = {}
- def set_internet_connection(self, value: bool) -> None:
- self.internet_connection = value
- self.shard_downloader.set_internet_connection(value)
-
def on_progress(
self,
callback: Callable[[ShardMetadata, RepoDownloadProgress], Awaitable[None]],
@@ -123,8 +119,9 @@ class CachedShardDownloader(ShardDownloader):
class ResumableShardDownloader(ShardDownloader):
- def __init__(self, max_parallel_downloads: int = 8):
+ def __init__(self, max_parallel_downloads: int = 8, offline: bool = False):
self.max_parallel_downloads = max_parallel_downloads
+ self.offline = offline
self.on_progress_callbacks: list[
Callable[[ShardMetadata, RepoDownloadProgress], Awaitable[None]]
] = []
@@ -151,8 +148,7 @@ class ResumableShardDownloader(ShardDownloader):
self.on_progress_wrapper,
max_parallel_downloads=self.max_parallel_downloads,
allow_patterns=allow_patterns,
- skip_internet=not self.internet_connection,
- on_connection_lost=lambda: self.set_internet_connection(False),
+ skip_internet=self.offline,
)
return target_dir
@@ -168,8 +164,7 @@ class ResumableShardDownloader(ShardDownloader):
shard,
self.on_progress_wrapper,
skip_download=True,
- skip_internet=not self.internet_connection,
- on_connection_lost=lambda: self.set_internet_connection(False),
+ skip_internet=self.offline,
)
semaphore = asyncio.Semaphore(self.max_parallel_downloads)
@@ -198,7 +193,6 @@ class ResumableShardDownloader(ShardDownloader):
shard,
self.on_progress_wrapper,
skip_download=True,
- skip_internet=not self.internet_connection,
- on_connection_lost=lambda: self.set_internet_connection(False),
+ skip_internet=self.offline,
)
return progress
diff --git a/src/exo/download/shard_downloader.py b/src/exo/download/shard_downloader.py
index 22e85643..2addda80 100644
--- a/src/exo/download/shard_downloader.py
+++ b/src/exo/download/shard_downloader.py
@@ -16,11 +16,6 @@ from exo.shared.types.worker.shards import (
# TODO: the PipelineShardMetadata getting reinstantiated is a bit messy. Should this be a classmethod?
class ShardDownloader(ABC):
- internet_connection: bool = False
-
- def set_internet_connection(self, value: bool) -> None:
- self.internet_connection = value
-
@abstractmethod
async def ensure_shard(
self, shard: ShardMetadata, config_only: bool = False
diff --git a/src/exo/main.py b/src/exo/main.py
index 85e27b29..1735a9eb 100644
--- a/src/exo/main.py
+++ b/src/exo/main.py
@@ -60,7 +60,7 @@ class Node:
download_coordinator = DownloadCoordinator(
node_id,
session_id,
- exo_shard_downloader(),
+ exo_shard_downloader(offline=args.offline),
download_command_receiver=router.receiver(topics.DOWNLOAD_COMMANDS),
local_event_sender=router.sender(topics.LOCAL_EVENTS),
offline=args.offline,
@@ -211,7 +211,7 @@ class Node:
self.download_coordinator = DownloadCoordinator(
self.node_id,
result.session_id,
- exo_shard_downloader(),
+ exo_shard_downloader(offline=self.offline),
download_command_receiver=self.router.receiver(
topics.DOWNLOAD_COMMANDS
),
@@ -283,7 +283,7 @@ class Args(CamelCaseModel):
tb_only: bool = False
no_worker: bool = False
no_downloads: bool = False
- offline: bool = False
+ offline: bool = os.getenv("EXO_OFFLINE", "false").lower() == "true"
fast_synch: bool | None = None # None = auto, True = force on, False = force off
@classmethod
@@ -334,6 +334,7 @@ class Args(CamelCaseModel):
parser.add_argument(
"--offline",
action="store_true",
+ default=os.getenv("EXO_OFFLINE", "false").lower() == "true",
help="Run in offline/air-gapped mode: skip internet checks, use only pre-staged local models",
)
fast_synch_group = parser.add_mutually_exclusive_group()
diff --git a/src/exo/shared/constants.py b/src/exo/shared/constants.py
index 795b1b84..e1c9ca19 100644
--- a/src/exo/shared/constants.py
+++ b/src/exo/shared/constants.py
@@ -78,4 +78,6 @@ EXO_ENABLE_IMAGE_MODELS = (
os.getenv("EXO_ENABLE_IMAGE_MODELS", "false").lower() == "true"
)
+EXO_OFFLINE = os.getenv("EXO_OFFLINE", "false").lower() == "true"
+
EXO_TRACING_ENABLED = os.getenv("EXO_TRACING_ENABLED", "false").lower() == "true"
← 9a2d2a4a bump (#1608)
·
back to Exo
·
fix: log exceptions causing silent node shutdown (#1621) 190e63e5 →