TB-bridge runbook: static 10.0.0.x on Thunderbolt Bridge across all 3 nodes

300130f8fe20b625179dc31d4f84ffca02032c00 · 2026-08-17 15:03:19 -0700 · steve
Files touched

A TB-BRIDGE-RUNBOOK.md
M data/latest.json
Diff

commit 300130f8fe20b625179dc31d4f84ffca02032c00
Author: steve <steve@designerwallcoverings.com>
Date:   Mon Aug 17 15:03:19 2026 -0700

    TB-bridge runbook: static 10.0.0.x on Thunderbolt Bridge across all 3 nodes
---
 TB-BRIDGE-RUNBOOK.md | 50 ++++++++++++++++++++++++++++++++++++++++++++++++++
 data/latest.json     |  2 +-
 2 files changed, 51 insertions(+), 1 deletion(-)

diff --git a/TB-BRIDGE-RUNBOOK.md b/TB-BRIDGE-RUNBOOK.md
new file mode 100644
index 0000000..7d49e55
--- /dev/null
+++ b/TB-BRIDGE-RUNBOOK.md
@@ -0,0 +1,50 @@
+# Put the exo cluster on the 40 Gb/s Thunderbolt fabric
+
+**Why it's not on TB now:** the TB links are set to DHCP; a direct Mac-to-Mac
+cable has no DHCP server, so they fall to `169.254` link-local (unroutable) and
+exo's zero-config discovery rides the regular LAN (`en0` / 192.168.1.x) instead.
+
+**Target:** a Thunderbolt Bridge on every node, each with a static IP on one
+`10.0.0.0/24` subnet. Bridging each Mac's TB ports turns the ring into one L2
+broadcast domain, which is exactly what exo's zero-config discovery expects.
+
+**Per-node config discovered:**
+- mac1 (.133): already has a "Thunderbolt Bridge" (bridge0). Just needs a static IP.
+- mac2 (.54): LOCKED — unknown, but almost certainly also default "Thunderbolt Bridge". Do it once logged in.
+- macstudio3 (this box): on 6 individual "EXO Thunderbolt N" services, NO bridge — must CREATE a bridge first (one GUI step), then set the IP.
+
+**Addressing:**  mac1 = 10.0.0.1 · mac2 = 10.0.0.2 · macstudio3 = 10.0.0.3  (mask 255.255.255.0, no router)
+
+---
+
+## Step 1 — macstudio3 (this box): create the bridge (GUI, once)
+System Settings → Network → ⋯ (bottom) → **Manage Virtual Interfaces…** → **＋** →
+**New Thunderbolt Bridge** → add all the `EXO Thunderbolt` ports → Create.
+(This replaces the 6 individual services with one bridge, matching mac1.)
+
+## Step 2 — set the static IP on each node
+Run AT each machine's terminal (or `ssh -t steve@<ip> '<cmd>'` so `sudo` gets a TTY).
+`sudo` will prompt for that machine's password — that's why you run these, not me.
+
+```bash
+# macstudio3 (this box) — after Step 1 created the bridge:
+sudo networksetup -setmanual "Thunderbolt Bridge" 10.0.0.3 255.255.255.0
+
+# mac1 (.133) — already has the bridge:
+ssh -t steve@192.168.1.133 'sudo networksetup -setmanual "Thunderbolt Bridge" 10.0.0.1 255.255.255.0'
+
+# mac2 (.54) — ONLY after it's logged in and reachable:
+ssh -t steve@192.168.1.54  'sudo networksetup -setmanual "Thunderbolt Bridge" 10.0.0.2 255.255.255.0'
+```
+
+Safe: this touches only the Thunderbolt Bridge, never `en0` — your LAN/SSH stays up.
+
+## Step 3 — verify (I do this part, no sudo needed)
+- `route get 10.0.0.1` on macstudio3 must resolve to a TB `en*` / `bridge0`, NOT `en0`.
+- `ping 10.0.0.1` / `10.0.0.2` succeed over TB.
+- exo `:52415/state` shows the peers, and node-to-node traffic rides the bridge.
+- Then a sharded model run should show the interconnect on TB, not gigabit LAN.
+
+## After this: RDMA (separate, later)
+Per the topology notes — match macOS 26.6.1 across nodes + `rdma_ctl enable` in
+Recovery. Only meaningful once the TB subnet above is live and exo shards across nodes.
diff --git a/data/latest.json b/data/latest.json
index bf134d7..aaaa18f 100644
--- a/data/latest.json
+++ b/data/latest.json
@@ -1 +1 @@
-{"generated_at":"2026-08-17T21:58:46Z","verdict":"PASS","status":"PASS","serving":3,"expected":3,"self_up":1,"down":"","healed":"","headline":"PASS: exo cluster 3/3 serving"}
\ No newline at end of file
+{"generated_at":"2026-08-17T22:02:59Z","verdict":"PASS","status":"PASS","serving":3,"expected":3,"self_up":1,"down":"","healed":"","headline":"PASS: exo cluster 3/3 serving"}
\ No newline at end of file
← c2a9282 auto-data-snapshot: 2026-08-17T14:59:21 (1 data files) — dat · back to Exo Cluster Watchdog · auto-data-snapshot: 2026-08-17T15:29:38 (1 data files) — dat 3833e41 →