← back to Exo
feat: seamless chat UX with auto model selection and smart recommendations (#1590)
5cd96b506089d0f2636e613a5ab1cac8c84a3a3f · 2026-02-23 09:32:55 -0800 · Alex Cheema
## Motivation
The current chat experience requires users to manually select a model
before they can start chatting. This creates unnecessary friction —
especially for new users who may not know which model to pick.
Additionally, models that are already running or downloading appear
greyed out in the picker if available memory is low, making them
impossible to select even though they're already loaded.
## Changes
### New: ChatModelSelector component
- Category-based model recommendations (Best for Coding, Writing,
Agentic, Biggest) shown on the idle chat screen
- Tiered model ranking system (`AUTO_TIERS`) that picks the best model
fitting available memory
- Fixed-position tooltips that escape overflow-hidden ancestors
### Auto model selection from chat
- Typing in an empty chat auto-picks the best model using tier rankings
- Prefers already-running models over launching smaller new ones (avoids
the "launched a big model but recommends a smaller one" bug)
- If a model is already downloading/loading, attaches to existing
progress instead of launching a duplicate
### Launch-from-chat state machine
- Full `idle → launching → downloading → loading → ready` flow with
inline progress display
- `pendingAutoMessage` queue: messages typed during model launch are
remembered and sent once model is ready
- Fixed `$effect` race condition where restore effect competed with
auto-advance effect, causing lost messages
### Model picker improvements
- Instance status badges: green dot (ready), blue pulsing (downloading),
yellow pulsing (loading) on both group and variant rows
- "Ready" availability filter in ModelFilterPopover
- **Models with active instances (ready, downloading, loading) are never
greyed out** regardless of memory — they're already loaded/in-progress
and must be selectable
### Home page model display
- Model picker button shows the best running model name instead of "—
SELECT MODEL —" when a model is already active
- Uses the same `bestRunningModelId` tier-based derived as the chat
auto-selection
### Chat sidebar & form
- ChatSidebar conversation management with rename, delete, "New Chat"
- `userForcedIdle` guard prevents reactive effects from overriding
explicit user actions (e.g., clicking "New Chat")
- ChatForm simplified: uses ModelPickerModal instead of inline dropdown,
`modelDisplayOverride` prop for showing auto-selected models
## Why It Works
The tier-based auto-selection (`getAutoTierIndex` + `pickAutoModel`)
ensures users always get the best model their cluster can run. By
checking running instances first and comparing tiers, we avoid the bug
where loading a large model reduces available memory and causes a
smaller model to be recommended. The `$effect` ordering fix (moving the
`chatLaunchState === "ready"` guard after the `hasRunningInstance`
check) ensures pending messages are always delivered. The instance
status override in ModelPickerGroup (`anyVariantHasInstance`) ensures
models with active instances bypass memory-based greying regardless of
available RAM.
## Test Plan
### Manual Testing
<!-- Hardware: (e.g., MacBook Pro M1 Max 32GB, Mac Mini M2 16GB,
connected via Thunderbolt 4) -->
<!-- What you did: -->
- Launch exo, navigate to Chat view
- Verify idle chat shows category recommendations (Coding, Writing,
Agentic, Biggest) based on available memory
- Type a message without selecting a model → verify auto-selection picks
the best tier model
- Launch a model, go to New Chat → verify the input bar shows the
running model name (not "SELECT MODEL")
- Launch a model, go to Home → verify model picker button shows the
running model
- While a model is downloading, send a message → verify download
progress appears and message is queued
- Open model picker with a model running → verify it is NOT greyed out
and is selectable
- Click "New Chat" while in a conversation → verify it resets to idle
with model selector
### Automated Testing
- Dashboard builds successfully (`cd dashboard && npm run build`)
- No TypeScript errors in the build output
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Files touched
M dashboard/src/lib/components/ChatForm.svelteA dashboard/src/lib/components/ChatModelSelector.svelteM dashboard/src/lib/components/ChatSidebar.svelteM dashboard/src/lib/components/ModelFilterPopover.svelteM dashboard/src/lib/components/ModelPickerGroup.svelteM dashboard/src/lib/components/ModelPickerModal.svelteM dashboard/src/lib/components/index.tsM dashboard/src/lib/stores/app.svelte.tsM dashboard/src/routes/+page.svelteM src/exo/master/placement.pyM src/exo/worker/engines/mlx/utils_mlx.py
Diff
commit 5cd96b506089d0f2636e613a5ab1cac8c84a3a3f
Author: Alex Cheema <41707476+AlexCheema@users.noreply.github.com>
Date: Mon Feb 23 09:32:55 2026 -0800
feat: seamless chat UX with auto model selection and smart recommendations (#1590)
## Motivation
The current chat experience requires users to manually select a model
before they can start chatting. This creates unnecessary friction —
especially for new users who may not know which model to pick.
Additionally, models that are already running or downloading appear
greyed out in the picker if available memory is low, making them
impossible to select even though they're already loaded.
## Changes
### New: ChatModelSelector component
- Category-based model recommendations (Best for Coding, Writing,
Agentic, Biggest) shown on the idle chat screen
- Tiered model ranking system (`AUTO_TIERS`) that picks the best model
fitting available memory
- Fixed-position tooltips that escape overflow-hidden ancestors
### Auto model selection from chat
- Typing in an empty chat auto-picks the best model using tier rankings
- Prefers already-running models over launching smaller new ones (avoids
the "launched a big model but recommends a smaller one" bug)
- If a model is already downloading/loading, attaches to existing
progress instead of launching a duplicate
### Launch-from-chat state machine
- Full `idle → launching → downloading → loading → ready` flow with
inline progress display
- `pendingAutoMessage` queue: messages typed during model launch are
remembered and sent once model is ready
- Fixed `$effect` race condition where restore effect competed with
auto-advance effect, causing lost messages
### Model picker improvements
- Instance status badges: green dot (ready), blue pulsing (downloading),
yellow pulsing (loading) on both group and variant rows
- "Ready" availability filter in ModelFilterPopover
- **Models with active instances (ready, downloading, loading) are never
greyed out** regardless of memory — they're already loaded/in-progress
and must be selectable
### Home page model display
- Model picker button shows the best running model name instead of "—
SELECT MODEL —" when a model is already active
- Uses the same `bestRunningModelId` tier-based derived as the chat
auto-selection
### Chat sidebar & form
- ChatSidebar conversation management with rename, delete, "New Chat"
- `userForcedIdle` guard prevents reactive effects from overriding
explicit user actions (e.g., clicking "New Chat")
- ChatForm simplified: uses ModelPickerModal instead of inline dropdown,
`modelDisplayOverride` prop for showing auto-selected models
## Why It Works
The tier-based auto-selection (`getAutoTierIndex` + `pickAutoModel`)
ensures users always get the best model their cluster can run. By
checking running instances first and comparing tiers, we avoid the bug
where loading a large model reduces available memory and causes a
smaller model to be recommended. The `$effect` ordering fix (moving the
`chatLaunchState === "ready"` guard after the `hasRunningInstance`
check) ensures pending messages are always delivered. The instance
status override in ModelPickerGroup (`anyVariantHasInstance`) ensures
models with active instances bypass memory-based greying regardless of
available RAM.
## Test Plan
### Manual Testing
<!-- Hardware: (e.g., MacBook Pro M1 Max 32GB, Mac Mini M2 16GB,
connected via Thunderbolt 4) -->
<!-- What you did: -->
- Launch exo, navigate to Chat view
- Verify idle chat shows category recommendations (Coding, Writing,
Agentic, Biggest) based on available memory
- Type a message without selecting a model → verify auto-selection picks
the best tier model
- Launch a model, go to New Chat → verify the input bar shows the
running model name (not "SELECT MODEL")
- Launch a model, go to Home → verify model picker button shows the
running model
- While a model is downloading, send a message → verify download
progress appears and message is queued
- Open model picker with a model running → verify it is NOT greyed out
and is selectable
- Click "New Chat" while in a conversation → verify it resets to idle
with model selector
### Automated Testing
- Dashboard builds successfully (`cd dashboard && npm run build`)
- No TypeScript errors in the build output
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
---
dashboard/src/lib/components/ChatForm.svelte | 213 +----
.../src/lib/components/ChatModelSelector.svelte | 401 +++++++++
dashboard/src/lib/components/ChatSidebar.svelte | 12 +-
.../src/lib/components/ModelFilterPopover.svelte | 75 +-
.../src/lib/components/ModelPickerGroup.svelte | 126 ++-
.../src/lib/components/ModelPickerModal.svelte | 37 +-
dashboard/src/lib/components/index.ts | 1 +
dashboard/src/lib/stores/app.svelte.ts | 4 +
dashboard/src/routes/+page.svelte | 916 ++++++++++++++++++++-
src/exo/master/placement.py | 8 +-
src/exo/worker/engines/mlx/utils_mlx.py | 2 +-
11 files changed, 1553 insertions(+), 242 deletions(-)
diff --git a/dashboard/src/lib/components/ChatForm.svelte b/dashboard/src/lib/components/ChatForm.svelte
index a7bf7673..b5cc7166 100644
--- a/dashboard/src/lib/components/ChatForm.svelte
+++ b/dashboard/src/lib/components/ChatForm.svelte
@@ -7,8 +7,6 @@
editingImage,
clearEditingImage,
selectedChatModel,
- setSelectedChatModel,
- instances,
ttftMs,
tps,
totalTokens,
@@ -30,6 +28,18 @@
modelTasks?: Record<string, string[]>;
modelCapabilities?: Record<string, string[]>;
onSend?: () => void;
+ onAutoSend?: (
+ content: string,
+ files?: {
+ id: string;
+ name: string;
+ type: string;
+ textContent?: string;
+ preview?: string;
+ }[],
+ ) => void;
+ onOpenModelPicker?: () => void;
+ modelDisplayOverride?: string;
}
let {
@@ -41,6 +51,9 @@
modelTasks = {},
modelCapabilities = {},
onSend,
+ onAutoSend,
+ onOpenModelPicker,
+ modelDisplayOverride,
}: Props = $props();
let message = $state("");
@@ -51,27 +64,12 @@
const thinkingEnabled = $derived(thinkingEnabledStore());
let loading = $derived(isLoading());
const currentModel = $derived(selectedChatModel());
- const instanceData = $derived(instances());
const currentTtft = $derived(ttftMs());
const currentTps = $derived(tps());
const currentTokens = $derived(totalTokens());
const currentEditingImage = $derived(editingImage());
const isEditMode = $derived(currentEditingImage !== null);
- // Custom dropdown state
- let isModelDropdownOpen = $state(false);
- let dropdownButtonRef: HTMLButtonElement | undefined = $state();
- let dropdownPosition = $derived(() => {
- if (!dropdownButtonRef || !isModelDropdownOpen)
- return { top: 0, left: 0, width: 0 };
- const rect = dropdownButtonRef.getBoundingClientRect();
- return {
- top: rect.top,
- left: rect.left,
- width: rect.width,
- };
- });
-
// Accept all supported file types
const acceptString = getAcceptString(["image", "text", "pdf"]);
@@ -124,73 +122,14 @@
uploadedFiles.length > 0),
);
- // Extract available models from running instances
- const availableModels = $derived(() => {
- const models: Array<{ id: string; label: string; isImageModel: boolean }> =
- [];
- for (const [, instance] of Object.entries(instanceData)) {
- const modelId = getInstanceModelId(instance);
- if (
- modelId &&
- modelId !== "Unknown" &&
- !models.some((m) => m.id === modelId)
- ) {
- models.push({
- id: modelId,
- label: modelId.split("/").pop() || modelId,
- isImageModel: modelSupportsImageGeneration(modelId),
- });
- }
- }
- return models;
- });
-
- // Track previous model IDs to detect newly added models (plain variable to avoid reactive loop)
- let previousModelIds: Set<string> = new Set();
-
- // Auto-select the first available model if none is selected, if current selection is stale, or if a new model is added
- $effect(() => {
- const models = availableModels();
- const currentModelIds = new Set(models.map((m) => m.id));
-
- if (models.length > 0) {
- // Find newly added models (in current but not in previous)
- const newModels = models.filter((m) => !previousModelIds.has(m.id));
-
- // If no model selected, select the first available
- if (!currentModel) {
- setSelectedChatModel(models[0].id);
- }
- // If current model is stale (no longer has a running instance), reset to first available
- else if (!models.some((m) => m.id === currentModel)) {
- setSelectedChatModel(models[0].id);
- }
- // If a new model was just added, select it
- else if (newModels.length > 0 && previousModelIds.size > 0) {
- setSelectedChatModel(newModels[0].id);
- }
- } else {
- // No instances running - clear the selected model
- if (currentModel) {
- setSelectedChatModel("");
- }
- }
-
- // Update previous model IDs for next comparison
- previousModelIds = currentModelIds;
- });
-
- function getInstanceModelId(instanceWrapped: unknown): string {
- if (!instanceWrapped || typeof instanceWrapped !== "object") return "";
- const keys = Object.keys(instanceWrapped as Record<string, unknown>);
- if (keys.length === 1) {
- const instance = (instanceWrapped as Record<string, unknown>)[
- keys[0]
- ] as { shardAssignments?: { modelId?: string } };
- return instance?.shardAssignments?.modelId || "";
- }
- return "";
- }
+ // Short label for the currently selected model
+ const currentModelLabel = $derived(
+ currentModel
+ ? currentModel.split("/").pop() || currentModel
+ : modelDisplayOverride
+ ? modelDisplayOverride.split("/").pop() || modelDisplayOverride
+ : "",
+ );
async function handleFiles(files: File[]) {
if (files.length === 0) return;
@@ -277,6 +216,15 @@
uploadedFiles = [];
resetTextareaHeight();
+ // When onAutoSend is provided, the parent controls all send logic
+ // (including launching non-running models before sending)
+ if (onAutoSend) {
+ onAutoSend(content, files);
+ onSend?.();
+ setTimeout(() => textareaRef?.focus(), 10);
+ return;
+ }
+
// Use image editing if in edit mode
if (isEditMode && currentEditingImage && content) {
editImage(content, currentEditingImage.imageDataUrl);
@@ -420,7 +368,7 @@
{/if}
<!-- Model selector (when enabled) -->
- {#if showModelSelector && availableModels().length > 0}
+ {#if showModelSelector}
<div
class="flex items-center justify-between gap-2 px-3 py-2 border-b border-exo-medium-gray/30"
>
@@ -429,33 +377,22 @@
class="text-xs text-exo-light-gray uppercase tracking-wider flex-shrink-0"
>MODEL:</span
>
- <!-- Custom dropdown -->
+ <!-- Model button — opens the full model picker -->
<div class="relative flex-1 max-w-xs">
<button
- bind:this={dropdownButtonRef}
type="button"
- onclick={() => (isModelDropdownOpen = !isModelDropdownOpen)}
- class="w-full bg-exo-medium-gray/50 border border-exo-yellow/30 rounded pl-3 pr-8 py-1.5 text-xs font-mono text-left tracking-wide cursor-pointer transition-all duration-200 hover:border-exo-yellow/50 focus:outline-none focus:border-exo-yellow/70 {isModelDropdownOpen
- ? 'border-exo-yellow/70'
- : ''}"
+ onclick={() => onOpenModelPicker?.()}
+ class="w-full bg-exo-medium-gray/50 border border-exo-yellow/30 rounded pl-3 pr-8 py-1.5 text-xs font-mono text-left tracking-wide cursor-pointer transition-all duration-200 hover:border-exo-yellow/50 focus:outline-none focus:border-exo-yellow/70"
>
- {#if availableModels().find((m) => m.id === currentModel)}
- <span class="text-exo-yellow truncate"
- >{availableModels().find((m) => m.id === currentModel)
- ?.label}</span
- >
- {:else if availableModels().length > 0}
- <span class="text-exo-yellow truncate"
- >{availableModels()[0].label}</span
+ {#if currentModelLabel}
+ <span class="text-exo-yellow truncate">{currentModelLabel}</span
>
{:else}
<span class="text-exo-light-gray/50">— SELECT MODEL —</span>
{/if}
</button>
<div
- class="absolute right-2 top-1/2 -translate-y-1/2 pointer-events-none transition-transform duration-200 {isModelDropdownOpen
- ? 'rotate-180'
- : ''}"
+ class="absolute right-2 top-1/2 -translate-y-1/2 pointer-events-none"
>
<svg
class="w-3 h-3 text-exo-yellow/60"
@@ -472,78 +409,6 @@
</svg>
</div>
</div>
-
- {#if isModelDropdownOpen}
- <!-- Backdrop to close dropdown -->
- <button
- type="button"
- class="fixed inset-0 z-[9998] cursor-default"
- onclick={() => (isModelDropdownOpen = false)}
- aria-label="Close dropdown"
- ></button>
-
- <!-- Dropdown Panel - fixed positioning to escape overflow:hidden -->
- <div
- class="fixed bg-exo-dark-gray border border-exo-yellow/30 rounded shadow-lg shadow-black/50 z-[9999] max-h-48 overflow-y-auto"
- style="bottom: calc(100vh - {dropdownPosition()
- .top}px + 4px); left: {dropdownPosition()
- .left}px; width: {dropdownPosition().width}px;"
- >
- <div class="py-1">
- {#each availableModels() as model}
- <button
- type="button"
- onclick={() => {
- setSelectedChatModel(model.id);
- isModelDropdownOpen = false;
- }}
- class="w-full px-3 py-2 text-left text-xs font-mono tracking-wide transition-colors duration-100 flex items-center gap-2 {currentModel ===
- model.id
- ? 'bg-transparent text-exo-yellow'
- : 'text-exo-light-gray hover:text-exo-yellow'}"
- >
- {#if currentModel === model.id}
- <svg
- class="w-3 h-3 flex-shrink-0"
- fill="currentColor"
- viewBox="0 0 20 20"
- >
- <path
- fill-rule="evenodd"
- d="M16.707 5.293a1 1 0 010 1.414l-8 8a1 1 0 01-1.414 0l-4-4a1 1 0 011.414-1.414L8 12.586l7.293-7.293a1 1 0 011.414 0z"
- clip-rule="evenodd"
- />
- </svg>
- {:else}
- <span class="w-3"></span>
- {/if}
- {#if model.isImageModel}
- <svg
- class="w-3.5 h-3.5 flex-shrink-0 text-exo-yellow"
- fill="none"
- viewBox="0 0 24 24"
- stroke="currentColor"
- stroke-width="2"
- aria-label="Image generation model"
- >
- <rect
- x="3"
- y="3"
- width="18"
- height="18"
- rx="2"
- ry="2"
- />
- <circle cx="8.5" cy="8.5" r="1.5" />
- <polyline points="21 15 16 10 5 21" />
- </svg>
- {/if}
- <span class="truncate flex-1">{model.label}</span>
- </button>
- {/each}
- </div>
- </div>
- {/if}
</div>
<!-- Thinking toggle -->
{#if modelSupportsThinking()}
diff --git a/dashboard/src/lib/components/ChatModelSelector.svelte b/dashboard/src/lib/components/ChatModelSelector.svelte
new file mode 100644
index 00000000..2d299abb
--- /dev/null
+++ b/dashboard/src/lib/components/ChatModelSelector.svelte
@@ -0,0 +1,401 @@
+<script lang="ts" module>
+ export interface ChatModelInfo {
+ id: string;
+ name: string;
+ base_model: string;
+ storage_size_megabytes: number;
+ capabilities: string[];
+ family: string;
+ quantization: string;
+ }
+
+ // Auto mode tier list (for when user just starts typing)
+ export const AUTO_TIERS: string[][] = [
+ // Tier 1 (frontier)
+ ["DeepSeek V3.1", "GLM-5", "Kimi K2.5", "Qwen3 Coder Next"],
+ // Tier 2 (excellent)
+ [
+ "Kimi K2",
+ "Qwen3 235B",
+ "MiniMax M2.5",
+ "Step 3.5 Flash",
+ "Qwen3 Next 80B",
+ ],
+ // Tier 3 (great)
+ [
+ "GLM 4.7",
+ "MiniMax M2.1",
+ "Qwen3 Coder 480B",
+ "GLM 4.5 Air",
+ "Llama 3.3 70B",
+ ],
+ // Tier 4 (good)
+ ["GPT-OSS 120B", "Qwen3 30B", "Llama 3.1 70B", "GLM 4.7 Flash"],
+ // Tier 5 (small/fast)
+ [
+ "Llama 3.1 8B",
+ "GPT-OSS 20B",
+ "Llama 3.2 3B",
+ "Qwen3 0.6B",
+ "Llama 3.2 1B",
+ ],
+ ];
+
+ /** Return the tier index (0 = best) for a base_model name. */
+ export function getAutoTierIndex(baseModel: string): number {
+ for (let i = 0; i < AUTO_TIERS.length; i++) {
+ if (AUTO_TIERS[i].includes(baseModel)) return i;
+ }
+ return AUTO_TIERS.length; // not in any tier → lowest priority
+ }
+
+ /** Auto mode: walk tiers top-down, pick biggest fitting variant from highest tier. */
+ export function pickAutoModel(
+ modelList: ChatModelInfo[],
+ memoryGB: number,
+ ): ChatModelInfo | null {
+ for (const tier of AUTO_TIERS) {
+ const candidates: ChatModelInfo[] = [];
+ for (const baseModel of tier) {
+ const variants = modelList
+ .filter(
+ (m) =>
+ m.base_model === baseModel &&
+ (m.storage_size_megabytes || 0) / 1024 <= memoryGB &&
+ (m.storage_size_megabytes || 0) > 0,
+ )
+ .sort(
+ (a, b) =>
+ (b.storage_size_megabytes || 0) - (a.storage_size_megabytes || 0),
+ );
+ if (variants[0]) candidates.push(variants[0]);
+ }
+ if (candidates.length > 0) {
+ candidates.sort(
+ (a, b) =>
+ (b.storage_size_megabytes || 0) - (a.storage_size_megabytes || 0),
+ );
+ return candidates[0];
+ }
+ }
+ return null;
+ }
+</script>
+
+<script lang="ts">
+ interface CategoryRecommendation {
+ category: "coding" | "writing" | "agentic" | "biggest";
+ label: string;
+ model: ChatModelInfo | null;
+ tooltip: string;
+ }
+
+ interface Props {
+ models: ChatModelInfo[];
+ clusterLabel: string;
+ totalMemoryGB: number;
+ onSelect: (modelId: string, category: string) => void;
+ onAddModel: () => void;
+ class?: string;
+ }
+
+ let {
+ models,
+ clusterLabel,
+ totalMemoryGB,
+ onSelect,
+ onAddModel,
+ class: className = "",
+ }: Props = $props();
+
+ // --- Hardcoded Rankings ---
+ const CODING_RANKING = [
+ "Qwen3 Coder Next",
+ "Qwen3 Coder 480B",
+ "Qwen3 30B",
+ "GPT-OSS 20B",
+ "Llama 3.1 8B",
+ "Llama 3.2 3B",
+ "Qwen3 0.6B",
+ ];
+
+ const WRITING_RANKING = [
+ "Kimi K2.5",
+ "Kimi K2",
+ "Qwen3 Next 80B",
+ "Llama 3.3 70B",
+ "MiniMax M2.5",
+ "GLM 4.5 Air",
+ "GLM 4.7 Flash",
+ "GPT-OSS 20B",
+ "Llama 3.1 8B",
+ "Llama 3.2 3B",
+ "Qwen3 0.6B",
+ ];
+
+ const AGENTIC_RANKING = [
+ "DeepSeek V3.1",
+ "GLM-5",
+ "Qwen3 235B",
+ "Step 3.5 Flash",
+ "GLM 4.7",
+ "MiniMax M2.1",
+ "GPT-OSS 120B",
+ "Llama 3.3 70B",
+ "Llama 3.1 70B",
+ "GLM 4.7 Flash",
+ "GPT-OSS 20B",
+ "Qwen3 30B",
+ "Llama 3.1 8B",
+ "Llama 3.2 3B",
+ "Qwen3 0.6B",
+ ];
+
+ function getModelSizeGB(m: ChatModelInfo): number {
+ return (m.storage_size_megabytes || 0) / 1024;
+ }
+
+ function fitsInMemory(m: ChatModelInfo): boolean {
+ return getModelSizeGB(m) <= totalMemoryGB && getModelSizeGB(m) > 0;
+ }
+
+ /** For a given base_model name, find the biggest quant variant that fits in memory. */
+ function pickBestVariant(baseModel: string): ChatModelInfo | null {
+ const variants = models
+ .filter((m) => m.base_model === baseModel && fitsInMemory(m))
+ .sort((a, b) => getModelSizeGB(b) - getModelSizeGB(a));
+ return variants[0] ?? null;
+ }
+
+ /** Walk a ranked list of base_model names, return the first that has a fitting variant. */
+ function pickFromRanking(ranking: string[]): ChatModelInfo | null {
+ for (const baseModel of ranking) {
+ const pick = pickBestVariant(baseModel);
+ if (pick) return pick;
+ }
+ return null;
+ }
+
+ /** Pick the single biggest model that fits. */
+ function pickBiggest(): ChatModelInfo | null {
+ const fitting = models
+ .filter((m) => fitsInMemory(m))
+ .sort((a, b) => getModelSizeGB(b) - getModelSizeGB(a));
+ return fitting[0] ?? null;
+ }
+
+ const recommendations = $derived.by((): CategoryRecommendation[] => {
+ return [
+ {
+ category: "coding",
+ label: "Best for Coding",
+ model: pickFromRanking(CODING_RANKING),
+ tooltip:
+ "Ranked by coding benchmark performance (LiveCodeBench, SWE-bench)",
+ },
+ {
+ category: "writing",
+ label: "Best for Writing",
+ model: pickFromRanking(WRITING_RANKING),
+ tooltip: "Ranked by creative writing quality and instruction following",
+ },
+ {
+ category: "agentic",
+ label: "Best Agentic",
+ model: pickFromRanking(AGENTIC_RANKING),
+ tooltip: "Ranked by reasoning, planning, and tool-use capability",
+ },
+ {
+ category: "biggest",
+ label: "Biggest",
+ model: pickBiggest(),
+ tooltip: "Largest model that fits in your available memory",
+ },
+ ];
+ });
+
+ function formatSize(mb: number): string {
+ const gb = mb / 1024;
+ if (gb >= 100) return `${Math.round(gb)} GB`;
+ return `${gb.toFixed(1)} GB`;
+ }
+
+ // Category icons (SVG paths)
+ const categoryIcons: Record<string, string> = {
+ coding:
+ "M8 9l3 3-3 3m5 0h3M5 20h14a2 2 0 002-2V6a2 2 0 00-2-2H5a2 2 0 00-2 2v12a2 2 0 002 2z",
+ writing:
+ "M11 5H6a2 2 0 00-2 2v11a2 2 0 002 2h11a2 2 0 002-2v-5m-1.414-9.414a2 2 0 112.828 2.828L11.828 15H9v-2.828l8.586-8.586z",
+ agentic:
+ "M9.663 17h4.673M12 3v1m6.364 1.636l-.707.707M21 12h-1M4 12H3m3.343-5.657l-.707-.707m2.828 9.9a5 5 0 117.072 0l-.548.547A3.374 3.374 0 0014 18.469V19a2 2 0 11-4 0v-.531c0-.895-.356-1.754-.988-2.386l-.548-.547z",
+ biggest:
+ "M19 11H5m14 0a2 2 0 012 2v6a2 2 0 01-2 2H5a2 2 0 01-2-2v-6a2 2 0 012-2m14 0V9a2 2 0 00-2-2M5 11V9a2 2 0 012-2m0 0V5a2 2 0 012-2h6a2 2 0 012 2v2M7 7h10",
+ };
+
+ let hoveredTooltip = $state<string | null>(null);
+ let tooltipAnchor = $state<{ x: number; y: number } | null>(null);
+
+ function showTooltip(category: string, e: MouseEvent | FocusEvent) {
+ hoveredTooltip = category;
+ const target = e.currentTarget as HTMLElement;
+ const rect = target.getBoundingClientRect();
+ tooltipAnchor = { x: rect.left + rect.width / 2, y: rect.top };
+ }
+
+ function hideTooltip() {
+ hoveredTooltip = null;
+ tooltipAnchor = null;
+ }
+</script>
+
+<div class="flex flex-col items-center justify-center gap-6 {className}">
+ <!-- Header -->
+ <div class="text-center">
+ <p class="text-xs text-exo-light-gray uppercase tracking-[0.2em] mb-1">
+ Recommended for your
+ </p>
+ <p class="text-sm text-white font-mono tracking-wide">{clusterLabel}</p>
+ </div>
+
+ <!-- Category Cards Grid -->
+ <div class="grid grid-cols-2 gap-3 w-full max-w-md">
+ {#each recommendations as rec}
+ {#if rec.model}
+ <button
+ type="button"
+ onclick={() => rec.model && onSelect(rec.model.id, rec.category)}
+ class="group relative flex flex-col items-start gap-2 p-4 rounded-lg border border-exo-medium-gray/50 bg-exo-dark-gray/50 hover:border-exo-yellow/40 hover:bg-exo-dark-gray transition-all duration-200 cursor-pointer text-left"
+ >
+ <!-- Category icon + label -->
+ <div class="flex items-center gap-2 w-full">
+ <svg
+ class="w-4 h-4 text-exo-yellow/70 group-hover:text-exo-yellow transition-colors flex-shrink-0"
+ fill="none"
+ viewBox="0 0 24 24"
+ stroke="currentColor"
+ stroke-width="1.5"
+ >
+ <path
+ stroke-linecap="round"
+ stroke-linejoin="round"
+ d={categoryIcons[rec.category]}
+ />
+ </svg>
+ <span
+ class="text-xs font-mono uppercase tracking-wider text-exo-light-gray group-hover:text-white transition-colors"
+ >
+ {rec.label}
+ </span>
+ <!-- Info tooltip -->
+ <div class="ml-auto flex-shrink-0">
+ <span
+ role="button"
+ tabindex="-1"
+ class="text-exo-light-gray/40 hover:text-exo-light-gray transition-colors cursor-help inline-flex"
+ onmouseenter={(e: MouseEvent) => showTooltip(rec.category, e)}
+ onmouseleave={() => hideTooltip()}
+ onclick={(e: MouseEvent) => {
+ e.stopPropagation();
+ if (hoveredTooltip === rec.category) {
+ hideTooltip();
+ } else {
+ showTooltip(rec.category, e);
+ }
+ }}
+ >
+ <svg
+ class="w-3.5 h-3.5"
+ fill="none"
+ viewBox="0 0 24 24"
+ stroke="currentColor"
+ stroke-width="2"
+ >
+ <circle cx="12" cy="12" r="10" />
+ <path d="M12 16v-4m0-4h.01" />
+ </svg>
+ </span>
+ </div>
+ </div>
+
+ <!-- Model name + size -->
+ <div class="w-full">
+ <p class="text-sm text-white font-mono truncate">
+ {rec.model.base_model}
+ </p>
+ <p class="text-xs text-exo-light-gray/60 font-mono mt-0.5">
+ {formatSize(rec.model.storage_size_megabytes)}
+ {#if rec.model.quantization}
+ <span class="text-exo-light-gray/40"
+ >· {rec.model.quantization}</span
+ >
+ {/if}
+ </p>
+ </div>
+ </button>
+ {:else}
+ <!-- No model fits for this category -->
+ <div
+ class="flex flex-col items-start gap-2 p-4 rounded-lg border border-exo-medium-gray/30 bg-exo-dark-gray/30 opacity-50"
+ >
+ <div class="flex items-center gap-2">
+ <svg
+ class="w-4 h-4 text-exo-light-gray/40 flex-shrink-0"
+ fill="none"
+ viewBox="0 0 24 24"
+ stroke="currentColor"
+ stroke-width="1.5"
+ >
+ <path
+ stroke-linecap="round"
+ stroke-linejoin="round"
+ d={categoryIcons[rec.category]}
+ />
+ </svg>
+ <span
+ class="text-xs font-mono uppercase tracking-wider text-exo-light-gray/50"
+ >{rec.label}</span
+ >
+ </div>
+ <p class="text-xs text-exo-light-gray/40 font-mono">No model fits</p>
+ </div>
+ {/if}
+ {/each}
+ </div>
+
+ <!-- Add Model Button -->
+ <button
+ type="button"
+ onclick={onAddModel}
+ class="flex items-center gap-2 px-4 py-2 text-xs font-mono uppercase tracking-wider text-exo-light-gray hover:text-exo-yellow border border-exo-medium-gray/30 hover:border-exo-yellow/30 rounded-lg transition-all duration-200 cursor-pointer"
+ >
+ <svg
+ class="w-3.5 h-3.5"
+ fill="none"
+ viewBox="0 0 24 24"
+ stroke="currentColor"
+ stroke-width="2"
+ >
+ <path stroke-linecap="round" stroke-linejoin="round" d="M12 4v16m8-8H4" />
+ </svg>
+ Add Model
+ </button>
+
+ <!-- Auto hint -->
+ <p class="text-xs text-exo-light-gray/40 font-mono tracking-wide text-center">
+ Or just start typing — we'll pick the best model automatically
+ </p>
+</div>
+
+<!-- Fixed-position tooltip (escapes overflow-hidden ancestors) -->
+{#if hoveredTooltip && tooltipAnchor}
+ {@const rec = recommendations.find((r) => r.category === hoveredTooltip)}
+ {#if rec}
+ <div
+ class="fixed z-[9999] px-3 py-2 bg-exo-black border border-exo-medium-gray/50 rounded text-xs text-exo-light-gray whitespace-nowrap shadow-lg pointer-events-none"
+ style="left: {tooltipAnchor.x}px; top: {tooltipAnchor.y -
+ 8}px; transform: translate(-50%, -100%);"
+ >
+ {rec.tooltip}
+ </div>
+ {/if}
+{/if}
diff --git a/dashboard/src/lib/components/ChatSidebar.svelte b/dashboard/src/lib/components/ChatSidebar.svelte
index e7dbe908..1f216498 100644
--- a/dashboard/src/lib/components/ChatSidebar.svelte
+++ b/dashboard/src/lib/components/ChatSidebar.svelte
@@ -2,7 +2,6 @@
import {
conversations,
activeConversationId,
- createConversation,
loadConversation,
deleteConversation,
deleteAllConversations,
@@ -17,9 +16,15 @@
interface Props {
class?: string;
+ onNewChat?: () => void;
+ onSelectConversation?: () => void;
}
- let { class: className = "" }: Props = $props();
+ let {
+ class: className = "",
+ onNewChat,
+ onSelectConversation,
+ }: Props = $props();
const conversationList = $derived(conversations());
const activeId = $derived(activeConversationId());
@@ -42,10 +47,11 @@
);
function handleNewChat() {
- createConversation();
+ onNewChat?.();
}
function handleSelectConversation(id: string) {
+ onSelectConversation?.();
loadConversation(id);
}
diff --git a/dashboard/src/lib/components/ModelFilterPopover.svelte b/dashboard/src/lib/components/ModelFilterPopover.svelte
index 452a1c65..8c36af03 100644
--- a/dashboard/src/lib/components/ModelFilterPopover.svelte
+++ b/dashboard/src/lib/components/ModelFilterPopover.svelte
@@ -6,6 +6,7 @@
capabilities: string[];
sizeRange: { min: number; max: number } | null;
downloadedOnly: boolean;
+ readyOnly: boolean;
}
type ModelFilterPopoverProps = {
@@ -190,34 +191,58 @@
</div>
</div>
- <!-- Downloaded only -->
+ <!-- Availability filters -->
<div>
<h4 class="text-xs font-mono text-white/50 mb-2">Availability</h4>
- <button
- type="button"
- class="px-2 py-1 text-xs font-mono rounded transition-colors {filters.downloadedOnly
- ? 'bg-green-500/20 text-green-400 border border-green-500/30'
- : 'bg-white/5 text-white/60 hover:bg-white/10 border border-transparent'}"
- onclick={() =>
- onChange({ ...filters, downloadedOnly: !filters.downloadedOnly })}
- >
- <svg
- class="w-3.5 h-3.5 inline-block"
- viewBox="0 0 24 24"
- fill="none"
- stroke="currentColor"
- stroke-width="2"
- stroke-linecap="round"
- stroke-linejoin="round"
+ <div class="flex flex-wrap gap-1.5">
+ <button
+ type="button"
+ class="px-2 py-1 text-xs font-mono rounded transition-colors {filters.downloadedOnly
+ ? 'bg-green-500/20 text-green-400 border border-green-500/30'
+ : 'bg-white/5 text-white/60 hover:bg-white/10 border border-transparent'}"
+ onclick={() =>
+ onChange({ ...filters, downloadedOnly: !filters.downloadedOnly })}
+ >
+ <svg
+ class="w-3.5 h-3.5 inline-block"
+ viewBox="0 0 24 24"
+ fill="none"
+ stroke="currentColor"
+ stroke-width="2"
+ stroke-linecap="round"
+ stroke-linejoin="round"
+ >
+ <path
+ class="text-white/40"
+ d="M20 20a2 2 0 0 0 2-2V8a2 2 0 0 0-2-2h-7.9a2 2 0 0 1-1.69-.9L9.6 3.9A2 2 0 0 0 7.93 3H4a2 2 0 0 0-2 2v13a2 2 0 0 0 2 2Z"
+ />
+ <path class="text-green-400" d="m9 13 2 2 4-4" />
+ </svg>
+ <span class="ml-1">Downloaded</span>
+ </button>
+ <button
+ type="button"
+ class="px-2 py-1 text-xs font-mono rounded transition-colors {filters.readyOnly
+ ? 'bg-green-500/20 text-green-400 border border-green-500/30'
+ : 'bg-white/5 text-white/60 hover:bg-white/10 border border-transparent'}"
+ onclick={() =>
+ onChange({ ...filters, readyOnly: !filters.readyOnly })}
>
- <path
- class="text-white/40"
- d="M20 20a2 2 0 0 0 2-2V8a2 2 0 0 0-2-2h-7.9a2 2 0 0 1-1.69-.9L9.6 3.9A2 2 0 0 0 7.93 3H4a2 2 0 0 0-2 2v13a2 2 0 0 0 2 2Z"
- />
- <path class="text-green-400" d="m9 13 2 2 4-4" />
- </svg>
- <span class="ml-1">Downloaded</span>
- </button>
+ <svg
+ class="w-3.5 h-3.5 inline-block"
+ viewBox="0 0 24 24"
+ fill="none"
+ stroke="currentColor"
+ stroke-width="2"
+ stroke-linecap="round"
+ stroke-linejoin="round"
+ >
+ <circle cx="12" cy="12" r="10" />
+ <path d="m9 12 2 2 4-4" />
+ </svg>
+ <span class="ml-1">Ready</span>
+ </button>
+ </div>
</div>
<!-- Size range -->
diff --git a/dashboard/src/lib/components/ModelPickerGroup.svelte b/dashboard/src/lib/components/ModelPickerGroup.svelte
index 1dbd18d9..e8e893d6 100644
--- a/dashboard/src/lib/components/ModelPickerGroup.svelte
+++ b/dashboard/src/lib/components/ModelPickerGroup.svelte
@@ -41,6 +41,7 @@
onShowInfo: (group: ModelGroup) => void;
downloadStatusMap?: Map<string, DownloadAvailability>;
launchedAt?: number;
+ instanceStatuses?: Record<string, { status: string; statusClass: string }>;
};
let {
@@ -56,6 +57,7 @@
onShowInfo,
downloadStatusMap,
launchedAt,
+ instanceStatuses = {},
}: ModelPickerGroupProps = $props();
// Group-level download status: show if any variant is downloaded
@@ -92,6 +94,12 @@
const anyVariantFits = $derived(
group.variants.some((v) => canModelFit(v.id)),
);
+ // Check if any variant has an active instance (ready, loading, downloading)
+ const anyVariantHasInstance = $derived(
+ instanceStatuses
+ ? group.variants.some((v) => instanceStatuses[v.id] != null)
+ : false,
+ );
const groupFitStatus = $derived.by((): ModelFitStatus => {
let hasClusterCapacityOnly = false;
for (const variant of group.variants) {
@@ -122,16 +130,35 @@
!group.hasMultipleVariants &&
group.variants.some((v) => v.id === selectedModelId),
);
+
+ // Group-level instance status: show the "best" status across all variants
+ const groupInstanceStatus = $derived.by(() => {
+ if (!instanceStatuses) return null;
+ const readyStatuses = ["READY", "LOADED", "RUNNING"];
+ const loadingStatuses = ["LOADING", "WARMING UP"];
+ let bestStatus: { status: string; statusClass: string } | null = null;
+ for (const variant of group.variants) {
+ const s = instanceStatuses[variant.id];
+ if (!s) continue;
+ if (readyStatuses.includes(s.status)) return s; // Ready is best
+ if (loadingStatuses.includes(s.status) || s.status === "DOWNLOADING") {
+ bestStatus = s;
+ }
+ }
+ return bestStatus;
+ });
</script>
<div
- class="border-b border-white/5 last:border-b-0 {!anyVariantFits
+ class="border-b border-white/5 last:border-b-0 {!anyVariantFits &&
+ !anyVariantHasInstance
? 'opacity-50'
: ''}"
>
<!-- Main row -->
<div
- class="flex items-center gap-2 px-3 py-2.5 transition-colors {anyVariantFits
+ class="flex items-center gap-2 px-3 py-2.5 transition-colors {anyVariantFits ||
+ anyVariantHasInstance
? 'hover:bg-white/5 cursor-pointer'
: 'cursor-not-allowed'} {isMainSelected
? 'bg-exo-yellow/10 border-l-2 border-exo-yellow'
@@ -141,7 +168,7 @@
onToggleExpand();
} else {
const modelId = group.variants[0]?.id;
- if (modelId && canModelFit(modelId)) {
+ if (modelId && (canModelFit(modelId) || instanceStatuses[modelId])) {
onSelectModel(modelId);
}
}
@@ -155,7 +182,7 @@
onToggleExpand();
} else {
const modelId = group.variants[0]?.id;
- if (modelId && canModelFit(modelId)) {
+ if (modelId && (canModelFit(modelId) || instanceStatuses[modelId])) {
onSelectModel(modelId);
}
}
@@ -346,6 +373,47 @@
</span>
{/if}
+ <!-- Instance status badge -->
+ {#if groupInstanceStatus}
+ {#if groupInstanceStatus.status === "READY" || groupInstanceStatus.status === "LOADED" || groupInstanceStatus.status === "RUNNING"}
+ <span class="flex-shrink-0" title="Running">
+ <svg
+ class="w-3 h-3 text-green-400"
+ viewBox="0 0 12 12"
+ fill="currentColor"
+ >
+ <circle cx="6" cy="6" r="5" />
+ </svg>
+ </span>
+ {:else if groupInstanceStatus.status === "DOWNLOADING"}
+ <span class="flex-shrink-0 animate-pulse" title="Downloading">
+ <svg
+ class="w-3.5 h-3.5 text-blue-400"
+ viewBox="0 0 24 24"
+ fill="none"
+ stroke="currentColor"
+ stroke-width="2"
+ stroke-linecap="round"
+ stroke-linejoin="round"
+ >
+ <path d="M21 15v4a2 2 0 01-2 2H5a2 2 0 01-2-2v-4" />
+ <polyline points="7 10 12 15 17 10" />
+ <line x1="12" y1="15" x2="12" y2="3" />
+ </svg>
+ </span>
+ {:else if groupInstanceStatus.status === "LOADING" || groupInstanceStatus.status === "WARMING UP"}
+ <span class="flex-shrink-0 animate-pulse" title="Loading">
+ <svg
+ class="w-3 h-3 text-yellow-400"
+ viewBox="0 0 12 12"
+ fill="currentColor"
+ >
+ <circle cx="6" cy="6" r="5" />
+ </svg>
+ </span>
+ {/if}
+ {/if}
+
<!-- Check mark if selected (single-variant) -->
{#if isMainSelected}
<svg
@@ -420,17 +488,19 @@
{#each group.variants as variant}
{@const fitStatus = getModelFitStatus(variant.id)}
{@const modelCanFit = canModelFit(variant.id)}
+ {@const variantHasInstance = instanceStatuses[variant.id] != null}
{@const isSelected = selectedModelId === variant.id}
<button
type="button"
- class="w-full flex items-center gap-3 px-3 py-2 pl-10 hover:bg-white/5 transition-colors text-left {!modelCanFit
+ class="w-full flex items-center gap-3 px-3 py-2 pl-10 hover:bg-white/5 transition-colors text-left {!modelCanFit &&
+ !variantHasInstance
? 'opacity-50 cursor-not-allowed'
: 'cursor-pointer'} {isSelected
? 'bg-exo-yellow/10 border-l-2 border-exo-yellow'
: 'border-l-2 border-transparent'}"
- disabled={!modelCanFit}
+ disabled={!modelCanFit && !variantHasInstance}
onclick={() => {
- if (modelCanFit) {
+ if (modelCanFit || variantHasInstance) {
onSelectModel(variant.id);
}
}}
@@ -478,6 +548,48 @@
{/if}
{/if}
+ <!-- Instance status badge -->
+ {#if instanceStatuses[variant.id]}
+ {@const instStatus = instanceStatuses[variant.id]}
+ {#if instStatus.status === "READY" || instStatus.status === "LOADED" || instStatus.status === "RUNNING"}
+ <span class="flex-shrink-0" title="Running">
+ <svg
+ class="w-3 h-3 text-green-400"
+ viewBox="0 0 12 12"
+ fill="currentColor"
+ >
+ <circle cx="6" cy="6" r="5" />
+ </svg>
+ </span>
+ {:else if instStatus.status === "DOWNLOADING"}
+ <span class="flex-shrink-0 animate-pulse" title="Downloading">
+ <svg
+ class="w-3.5 h-3.5 text-blue-400"
+ viewBox="0 0 24 24"
+ fill="none"
+ stroke="currentColor"
+ stroke-width="2"
+ stroke-linecap="round"
+ stroke-linejoin="round"
+ >
+ <path d="M21 15v4a2 2 0 01-2 2H5a2 2 0 01-2-2v-4" />
+ <polyline points="7 10 12 15 17 10" />
+ <line x1="12" y1="15" x2="12" y2="3" />
+ </svg>
+ </span>
+ {:else if instStatus.status === "LOADING" || instStatus.status === "WARMING UP"}
+ <span class="flex-shrink-0 animate-pulse" title="Loading">
+ <svg
+ class="w-3 h-3 text-yellow-400"
+ viewBox="0 0 12 12"
+ fill="currentColor"
+ >
+ <circle cx="6" cy="6" r="5" />
+ </svg>
+ </span>
+ {/if}
+ {/if}
+
<!-- Check mark if selected -->
{#if isSelected}
<svg
diff --git a/dashboard/src/lib/components/ModelPickerModal.svelte b/dashboard/src/lib/components/ModelPickerModal.svelte
index cf21c727..06ed4d24 100644
--- a/dashboard/src/lib/components/ModelPickerModal.svelte
+++ b/dashboard/src/lib/components/ModelPickerModal.svelte
@@ -36,6 +36,7 @@
capabilities: string[];
sizeRange: { min: number; max: number } | null;
downloadedOnly: boolean;
+ readyOnly: boolean;
}
interface HuggingFaceModel {
@@ -49,6 +50,11 @@
type ModelFitStatus = "fits_now" | "fits_cluster_capacity" | "too_large";
+ export type InstanceStatus = {
+ status: string;
+ statusClass: string;
+ };
+
type ModelPickerModalProps = {
isOpen: boolean;
models: ModelInfo[];
@@ -75,6 +81,7 @@
macmon_info?: { memory?: { ram_total?: number } };
}
>;
+ instanceStatuses?: Record<string, InstanceStatus>;
};
let {
@@ -96,6 +103,7 @@
usedMemoryGB,
downloadsData,
topologyNodes,
+ instanceStatuses = {},
}: ModelPickerModalProps = $props();
// Local state
@@ -107,6 +115,7 @@
capabilities: [],
sizeRange: null,
downloadedOnly: false,
+ readyOnly: false,
});
let infoGroup = $state<ModelGroup | null>(null);
@@ -440,6 +449,16 @@
);
}
+ // Filter to ready/running models only
+ if (filters.readyOnly) {
+ result = result.filter((g) =>
+ g.variants.some((v) => {
+ const s = instanceStatuses[v.id];
+ return s && s.statusClass === "ready";
+ }),
+ );
+ }
+
// Sort: fits-now first, then fits-cluster-capacity, then too-large
result.sort((a, b) => {
const getGroupFitRank = (group: ModelGroup): number => {
@@ -550,13 +569,19 @@
}
function clearFilters() {
- filters = { capabilities: [], sizeRange: null, downloadedOnly: false };
+ filters = {
+ capabilities: [],
+ sizeRange: null,
+ downloadedOnly: false,
+ readyOnly: false,
+ };
}
const hasActiveFilters = $derived(
filters.capabilities.length > 0 ||
filters.sizeRange !== null ||
- filters.downloadedOnly,
+ filters.downloadedOnly ||
+ filters.readyOnly,
);
</script>
@@ -825,6 +850,7 @@
onShowInfo={(g) => (infoGroup = g)}
downloadStatusMap={getVariantDownloadMap(group)}
launchedAt={recentTimestamps.get(group.variants[0]?.id ?? "")}
+ {instanceStatuses}
/>
{/each}
{/if}
@@ -892,6 +918,7 @@
{onToggleFavorite}
onShowInfo={(g) => (infoGroup = g)}
downloadStatusMap={getVariantDownloadMap(group)}
+ {instanceStatuses}
/>
{/each}
<!-- Other models -->
@@ -918,6 +945,7 @@
{onToggleFavorite}
onShowInfo={(g) => (infoGroup = g)}
downloadStatusMap={getVariantDownloadMap(group)}
+ {instanceStatuses}
/>
{/each}
{/if}
@@ -940,6 +968,11 @@
>Downloaded</span
>
{/if}
+ {#if filters.readyOnly}
+ <span class="px-1.5 py-0.5 bg-green-500/20 text-green-400 rounded"
+ >Ready</span
+ >
+ {/if}
{#if filters.sizeRange}
<span class="px-1.5 py-0.5 bg-exo-yellow/20 text-exo-yellow rounded">
{filters.sizeRange.min}GB - {filters.sizeRange.max}GB
diff --git a/dashboard/src/lib/components/index.ts b/dashboard/src/lib/components/index.ts
index b0424bf4..7efe0f87 100644
--- a/dashboard/src/lib/components/index.ts
+++ b/dashboard/src/lib/components/index.ts
@@ -12,3 +12,4 @@ export { default as HuggingFaceResultItem } from "./HuggingFaceResultItem.svelte
export { default as ModelFilterPopover } from "./ModelFilterPopover.svelte";
export { default as ModelPickerGroup } from "./ModelPickerGroup.svelte";
export { default as ModelPickerModal } from "./ModelPickerModal.svelte";
+export { default as ChatModelSelector } from "./ChatModelSelector.svelte";
diff --git a/dashboard/src/lib/stores/app.svelte.ts b/dashboard/src/lib/stores/app.svelte.ts
index 00c6669f..64f96fea 100644
--- a/dashboard/src/lib/stores/app.svelte.ts
+++ b/dashboard/src/lib/stores/app.svelte.ts
@@ -847,6 +847,10 @@ class AppStore {
this.thinkingEnabled = conversation.enableThinking ?? true;
this.refreshConversationModelFromInstances();
+ // Sync global selection to the loaded conversation's model so reactive
+ // effects in +page.svelte can determine the correct chat launch state.
+ this.selectedChatModel = conversation.modelId || "";
+
return true;
}
diff --git a/dashboard/src/routes/+page.svelte b/dashboard/src/routes/+page.svelte
index 7f374dd7..8d846f47 100644
--- a/dashboard/src/routes/+page.svelte
+++ b/dashboard/src/routes/+page.svelte
@@ -6,7 +6,12 @@
ChatSidebar,
ModelCard,
ModelPickerModal,
+ ChatModelSelector,
} from "$lib/components";
+ import {
+ pickAutoModel,
+ getAutoTierIndex,
+ } from "$lib/components/ChatModelSelector.svelte";
import {
favorites,
toggleFavorite,
@@ -678,9 +683,15 @@
selectPreviewModel(modelId);
onboardingStep = 7;
// Launch via standard placement API (same as main dashboard)
+ // Single-node: force Pipeline/Ring regardless of persisted defaults
+ const nodeCount = topologyData()
+ ? Object.keys(topologyData()!.nodes).length
+ : 1;
+ const sharding = nodeCount <= 1 ? "Pipeline" : selectedSharding;
+ const instanceType = nodeCount <= 1 ? "MlxRing" : selectedInstanceType;
try {
const placementResponse = await fetch(
- `/instance/placement?model_id=${encodeURIComponent(modelId)}&sharding=${selectedSharding}&instance_meta=${selectedInstanceType}&min_nodes=1`,
+ `/instance/placement?model_id=${encodeURIComponent(modelId)}&sharding=${sharding}&instance_meta=${instanceType}&min_nodes=1`,
);
if (!placementResponse.ok) {
const errorText = await placementResponse.text();
@@ -1992,6 +2003,39 @@
};
}
+ // Compute instance statuses by modelId for the model picker
+ const modelInstanceStatuses = $derived.by(() => {
+ const result: Record<string, { status: string; statusClass: string }> = {};
+ for (const [id, inst] of Object.entries(instanceData)) {
+ const modelId = getInstanceModelId(inst);
+ if (!modelId || modelId === "Unknown" || modelId === "Unknown Model")
+ continue;
+ const dlStatus = getInstanceDownloadStatus(id, inst);
+ const statusText = dlStatus.statusText;
+ let statusClass = "inactive";
+ if (
+ statusText === "READY" ||
+ statusText === "RUNNING" ||
+ statusText === "LOADED"
+ ) {
+ statusClass = "ready";
+ } else if (statusText === "DOWNLOADING") {
+ statusClass = "downloading";
+ } else if (statusText === "LOADING" || statusText === "WARMING UP") {
+ statusClass = "loading";
+ }
+ // Keep the best status per modelId (ready > loading > downloading > other)
+ const existing = result[modelId];
+ if (existing) {
+ const rank = (c: string) =>
+ c === "ready" ? 3 : c === "loading" ? 2 : c === "downloading" ? 1 : 0;
+ if (rank(statusClass) <= rank(existing.statusClass)) continue;
+ }
+ result[modelId] = { status: statusText, statusClass };
+ }
+ return result;
+ });
+
function formatLastUpdate(): string {
if (!update) return "ACQUIRING...";
const seconds = Math.floor((Date.now() - update) / 1000);
@@ -2225,10 +2269,22 @@
}
function handleNewChat() {
+ chatLaunchState = "idle";
+ pendingChatModelId = null;
+ selectedChatCategory = null;
+ pendingAutoMessage = null;
+ userForcedIdle = true;
+ setSelectedChatModel("");
createConversation();
}
function handleGoHome() {
+ chatLaunchState = "idle";
+ pendingChatModelId = null;
+ selectedChatCategory = null;
+ pendingAutoMessage = null;
+ userForcedIdle = true;
+ setSelectedChatModel("");
clearChat();
}
@@ -2392,6 +2448,586 @@
"Tell me a creative story",
];
+ // ── Seamless chat: launch models from chat view ──
+ type ChatLaunchState =
+ | "idle"
+ | "launching"
+ | "downloading"
+ | "loading"
+ | "ready";
+ let chatLaunchState = $state<ChatLaunchState>("idle");
+ let pendingChatModelId = $state<string | null>(null);
+ let selectedChatCategory = $state<string | null>(null);
+ // Guard: when true, the restore $effect must not override chatLaunchState.
+ // Set by handleNewChat/handleGoHome; cleared when the user picks a model.
+ let userForcedIdle = $state(false);
+
+ // Restore chat launch state when switching conversations
+ $effect(() => {
+ const currentModel = selectedChatModel();
+ // When the user explicitly requested the model selector (New Chat / Go Home),
+ // skip restoring state so the selector stays visible.
+ if (userForcedIdle) return;
+ if (!currentModel) {
+ if (chatStarted && chatLaunchState !== "idle") {
+ chatLaunchState = "idle";
+ pendingChatModelId = null;
+ selectedChatCategory = null;
+ }
+ return;
+ }
+
+ // Model is already running — no progress to show
+ if (hasRunningInstance(currentModel)) {
+ if (chatLaunchState !== "ready") {
+ chatLaunchState = "ready";
+ }
+ pendingChatModelId = currentModel;
+ return;
+ }
+
+ // Model is downloading
+ const dlStatus = getModelDownloadStatus(currentModel);
+ if (dlStatus.isDownloading) {
+ chatLaunchState = "downloading";
+ pendingChatModelId = currentModel;
+ return;
+ }
+
+ // Model is loading or in another pre-ready state
+ for (const [, inst] of Object.entries(instanceData)) {
+ if (getInstanceModelId(inst) !== currentModel) continue;
+ const status = deriveInstanceStatus(inst);
+ if (status.statusText === "LOADING") {
+ chatLaunchState = "loading";
+ pendingChatModelId = currentModel;
+ return;
+ }
+ if (
+ status.statusText === "WARMING UP" ||
+ status.statusText === "WAITING" ||
+ status.statusText === "INITIALIZING" ||
+ status.statusText === "PREPARING"
+ ) {
+ chatLaunchState = "launching";
+ pendingChatModelId = currentModel;
+ return;
+ }
+ }
+ });
+
+ // Suggested prompts per category
+ const categorySuggestedPrompts: Record<string, string[]> = {
+ coding: [
+ "Write a Snake game in Python",
+ "Build a REST API with FastAPI",
+ "Explain how async/await works",
+ "Help me write unit tests for my code",
+ ],
+ writing: [
+ "Write a short story about time travel",
+ "Draft a professional email to a client",
+ "Create a haiku about the ocean",
+ "Summarize the key ideas of stoicism",
+ ],
+ agentic: [
+ "Plan a weekend trip to Tokyo",
+ "Research and compare React vs Svelte",
+ "Create a step-by-step guide to learn ML",
+ "Analyze the pros and cons of remote work",
+ ],
+ biggest: [
+ "Explain quantum computing simply",
+ "Help me brainstorm startup ideas",
+ "What are the key differences between TCP and UDP?",
+ "Write a Python script to analyze a CSV file",
+ ],
+ auto: [
+ "Explain quantum computing simply",
+ "Help me brainstorm ideas for a side project",
+ "Write a Python function to sort a list",
+ "What makes a great technical interview?",
+ ],
+ };
+
+ // Cluster label for ChatModelSelector header
+ const chatClusterLabel = $derived.by(() => {
+ if (!data) return "your Mac";
+ const nodes = Object.values(data.nodes);
+ if (nodes.length === 0) return "your Mac";
+ if (nodes.length === 1) {
+ const node = nodes[0];
+ const name = node.system_info?.model_id || "your Mac";
+ const totalMem =
+ node.macmon_info?.memory?.ram_total ?? node.system_info?.memory ?? 0;
+ const memGB = Math.round(totalMem / (1024 * 1024 * 1024));
+ return `${name} ${memGB}GB`;
+ }
+ const totalMemGB = Math.round(clusterTotalMemoryGB());
+ return `cluster ${totalMemGB}GB`;
+ });
+
+ // Check if a model already has a running instance
+ function hasRunningInstance(modelId: string): boolean {
+ for (const [, inst] of Object.entries(instanceData)) {
+ const id = getInstanceModelId(inst);
+ if (id === modelId) {
+ const status = deriveInstanceStatus(inst);
+ if (
+ status.statusText === "READY" ||
+ status.statusText === "LOADED" ||
+ status.statusText === "RUNNING"
+ ) {
+ return true;
+ }
+ }
+ }
+ return false;
+ }
+
+ function hasExistingInstance(modelId: string): boolean {
+ for (const [, inst] of Object.entries(instanceData)) {
+ if (getInstanceModelId(inst) === modelId) return true;
+ }
+ return false;
+ }
+
+ // Pick optimal placement from previews (frontend logic)
+ // Rules: 1-node → Pipeline/Ring, multi-node with RDMA → Tensor/Jaccl (most nodes),
+ // multi-node without RDMA → 1-node Pipeline/Ring
+ function pickOptimalPlacement(
+ previews: PlacementPreview[],
+ ): PlacementPreview | null {
+ const valid = previews.filter((p) => p.instance && !p.error);
+
+ // Check if any valid placement uses multiple nodes (indicates multi-node cluster)
+ const hasMultiNode = valid.some((p) => getPreviewNodeCount(p) > 1);
+
+ if (hasMultiNode) {
+ // Multi-node with RDMA: prefer Jaccl + Tensor with most nodes (fastest TPS)
+ const jacclTensor = valid
+ .filter(
+ (p) => p.instance_meta === "MlxJaccl" && p.sharding === "Tensor",
+ )
+ .sort((a, b) => getPreviewNodeCount(b) - getPreviewNodeCount(a));
+ if (jacclTensor.length > 0) return jacclTensor[0];
+
+ // Multi-node without RDMA: fall back to single-node Pipeline/Ring
+ const singlePipeline = valid.filter(
+ (p) =>
+ p.instance_meta === "MlxRing" &&
+ p.sharding === "Pipeline" &&
+ getPreviewNodeCount(p) === 1,
+ );
+ if (singlePipeline.length > 0) return singlePipeline[0];
+ }
+
+ // Single node (or final fallback): Pipeline/Ring with fewest nodes
+ const ringPipeline = valid
+ .filter((p) => p.instance_meta === "MlxRing" && p.sharding === "Pipeline")
+ .sort((a, b) => getPreviewNodeCount(a) - getPreviewNodeCount(b));
+ if (ringPipeline.length > 0) return ringPipeline[0];
+
+ // Last resort: any valid placement, fewest nodes
+ return (
+ valid.sort(
+ (a, b) => getPreviewNodeCount(a) - getPreviewNodeCount(b),
+ )[0] ?? null
+ );
+ }
+
+ // Launch a model for seamless chat
+ async function launchModelForChat(modelId: string, category: string) {
+ userForcedIdle = false;
+ pendingChatModelId = modelId;
+ selectedChatCategory = category;
+
+ // Check if already running — skip straight to chat
+ if (hasRunningInstance(modelId)) {
+ setSelectedChatModel(modelId);
+ createConversation();
+ chatLaunchState = "ready";
+ return;
+ }
+
+ // Already has an instance (downloading/loading) — attach to its progress
+ if (hasExistingInstance(modelId)) {
+ setSelectedChatModel(modelId);
+ pendingChatModelId = modelId;
+ createConversation();
+ const dlStatus = getModelDownloadStatus(modelId);
+ if (dlStatus.isDownloading) {
+ chatLaunchState = "downloading";
+ } else {
+ chatLaunchState = "launching";
+ }
+ return;
+ }
+
+ chatLaunchState = "launching";
+
+ try {
+ // Fetch placement previews
+ const res = await fetch(
+ `/instance/previews?model_id=${encodeURIComponent(modelId)}`,
+ );
+ if (!res.ok) {
+ addToast({
+ type: "error",
+ message: `Failed to get placements: ${await res.text()}`,
+ });
+ chatLaunchState = "idle";
+ return;
+ }
+ const data: { previews: PlacementPreview[] } = await res.json();
+ const placement = pickOptimalPlacement(data.previews);
+ if (!placement) {
+ addToast({
+ type: "error",
+ message: "No valid placement found for this model",
+ });
+ chatLaunchState = "idle";
+ return;
+ }
+
+ // Launch the instance
+ const launchRes = await fetch("/instance", {
+ method: "POST",
+ headers: { "Content-Type": "application/json" },
+ body: JSON.stringify({ instance: placement.instance }),
+ });
+ if (!launchRes.ok) {
+ addToast({
+ type: "error",
+ message: `Failed to launch: ${await launchRes.text()}`,
+ });
+ chatLaunchState = "idle";
+ return;
+ }
+
+ setSelectedChatModel(modelId);
+ recordRecentLaunch(modelId);
+ createConversation();
+ chatLaunchState = "downloading";
+ } catch (error) {
+ addToast({ type: "error", message: `Network error: ${error}` });
+ chatLaunchState = "idle";
+ }
+ }
+
+ // Handle auto-send: user typed without selecting a model
+ async function handleAutoSend(
+ content: string,
+ files?: {
+ id: string;
+ name: string;
+ type: string;
+ textContent?: string;
+ preview?: string;
+ }[],
+ ) {
+ // Clear forced-idle so restore effect resumes normal operation
+ userForcedIdle = false;
+
+ // Find the best already-running model by tier
+ let bestRunning: { id: string; tierIndex: number } | null = null;
+ for (const [, inst] of Object.entries(instanceData)) {
+ const modelId = getInstanceModelId(inst);
+ if (modelId === "Unknown" || modelId === "Unknown Model") continue;
+ if (!hasRunningInstance(modelId)) continue;
+ const info = models.find((m) => m.id === modelId);
+ if (!info) continue;
+ const tierIndex = getAutoTierIndex(info.base_model ?? "");
+ if (!bestRunning || tierIndex < bestRunning.tierIndex) {
+ bestRunning = { id: modelId, tierIndex };
+ }
+ }
+
+ // Find the best auto model that fits in available memory
+ const totalMem = availableMemoryGB();
+ const modelInfos = models.map((m) => ({
+ id: m.id,
+ name: m.name ?? "",
+ base_model: m.base_model ?? "",
+ storage_size_megabytes: m.storage_size_megabytes ?? 0,
+ capabilities: m.capabilities ?? [],
+ family: m.family ?? "",
+ quantization: m.quantization ?? "",
+ }));
+ const autoModel = pickAutoModel(modelInfos, totalMem);
+
+ // Prefer running model unless auto-pick is a strictly better tier
+ if (bestRunning) {
+ const autoTier = autoModel
+ ? getAutoTierIndex(autoModel.base_model)
+ : Infinity;
+ if (autoTier >= bestRunning.tierIndex) {
+ // Running model is same or better tier — use it directly
+ setSelectedChatModel(bestRunning.id);
+ if (!chatStarted) createConversation();
+ sendMessage(content, files);
+ return;
+ }
+ }
+
+ if (!autoModel) {
+ addToast({
+ type: "error",
+ message: "No model fits in your available memory",
+ });
+ return;
+ }
+
+ // Check if the chosen auto model is already running
+ if (hasRunningInstance(autoModel.id)) {
+ setSelectedChatModel(autoModel.id);
+ if (!chatStarted) createConversation();
+ sendMessage(content, files);
+ return;
+ }
+
+ // Already has an instance (downloading/loading) — attach to its progress
+ if (hasExistingInstance(autoModel.id)) {
+ selectedChatCategory = "auto";
+ setSelectedChatModel(autoModel.id);
+ pendingChatModelId = autoModel.id;
+ if (!chatStarted) createConversation();
+ pendingAutoMessage = { content, files };
+ const dlStatus = getModelDownloadStatus(autoModel.id);
+ if (dlStatus.isDownloading) {
+ chatLaunchState = "downloading";
+ } else {
+ chatLaunchState = "launching";
+ }
+ return;
+ }
+
+ // Need to launch first, then send
+ selectedChatCategory = "auto";
+ pendingChatModelId = autoModel.id;
+ chatLaunchState = "launching";
+
+ try {
+ const res = await fetch(
+ `/instance/previews?model_id=${encodeURIComponent(autoModel.id)}`,
+ );
+ if (!res.ok) {
+ addToast({
+ type: "error",
+ message: `Failed to get placements: ${await res.text()}`,
+ });
+ chatLaunchState = "idle";
+ return;
+ }
+ const data: { previews: PlacementPreview[] } = await res.json();
+ const placement = pickOptimalPlacement(data.previews);
+ if (!placement) {
+ addToast({ type: "error", message: "No valid placement found" });
+ chatLaunchState = "idle";
+ return;
+ }
+
+ const launchRes = await fetch("/instance", {
+ method: "POST",
+ headers: { "Content-Type": "application/json" },
+ body: JSON.stringify({ instance: placement.instance }),
+ });
+ if (!launchRes.ok) {
+ addToast({
+ type: "error",
+ message: `Failed to launch: ${await launchRes.text()}`,
+ });
+ chatLaunchState = "idle";
+ return;
+ }
+
+ setSelectedChatModel(autoModel.id);
+ recordRecentLaunch(autoModel.id);
+ if (!chatStarted) createConversation();
+ chatLaunchState = "downloading";
+
+ // Queue the message to send once model is ready
+ pendingAutoMessage = { content, files };
+ } catch (error) {
+ addToast({ type: "error", message: `Network error: ${error}` });
+ chatLaunchState = "idle";
+ }
+ }
+
+ // Pending message to send after auto-launch completes
+ let pendingAutoMessage = $state<{
+ content: string;
+ files?: {
+ id: string;
+ name: string;
+ type: string;
+ textContent?: string;
+ preview?: string;
+ }[];
+ } | null>(null);
+
+ // Best running model by tier (for auto-pick display)
+ const bestRunningModelId = $derived.by(() => {
+ let best: { id: string; tierIndex: number } | null = null;
+ for (const [, inst] of Object.entries(instanceData)) {
+ const modelId = getInstanceModelId(inst);
+ if (modelId === "Unknown" || modelId === "Unknown Model") continue;
+ if (!hasRunningInstance(modelId)) continue;
+ const info = models.find((m) => m.id === modelId);
+ if (!info) continue;
+ const tierIndex = getAutoTierIndex(info.base_model ?? "");
+ if (!best || tierIndex < best.tierIndex) {
+ best = { id: modelId, tierIndex };
+ }
+ }
+ return best?.id ?? null;
+ });
+
+ // Track chat launch progress (download + loading)
+ const chatLaunchDownload = $derived.by(() => {
+ if (
+ !pendingChatModelId ||
+ (chatLaunchState !== "downloading" && chatLaunchState !== "launching")
+ )
+ return null;
+ const status = getModelDownloadStatus(pendingChatModelId);
+ if (status.isDownloading) return status.progress;
+ return null;
+ });
+
+ const chatLaunchLoadProgress = $derived.by(() => {
+ if (
+ !pendingChatModelId ||
+ chatLaunchState === "idle" ||
+ chatLaunchState === "ready"
+ )
+ return null;
+ let layersLoaded = 0,
+ totalLayers = 0;
+ for (const [, inst] of Object.entries(instanceData)) {
+ if (getInstanceModelId(inst) !== pendingChatModelId) continue;
+ const status = deriveInstanceStatus(inst);
+ if (
+ status.statusText === "LOADING" &&
+ status.totalLayers &&
+ status.totalLayers > 0
+ ) {
+ layersLoaded += status.layersLoaded ?? 0;
+ totalLayers += status.totalLayers;
+ }
+ }
+ if (totalLayers === 0) return null;
+ return {
+ layersLoaded,
+ totalLayers,
+ percentage: (layersLoaded / totalLayers) * 100,
+ };
+ });
+
+ // Auto-advance chat launch state based on instance status
+ $effect(() => {
+ if (!pendingChatModelId || chatLaunchState === "idle") return;
+
+ // Check if model is now ready
+ if (hasRunningInstance(pendingChatModelId)) {
+ chatLaunchState = "ready";
+ // Send pending auto message if any
+ if (pendingAutoMessage) {
+ const msg = pendingAutoMessage;
+ pendingAutoMessage = null;
+ sendMessage(msg.content, msg.files);
+ }
+ return;
+ }
+
+ // If already ready (set by restore effect), don't downgrade state
+ if (chatLaunchState === "ready") return;
+
+ // Check if currently loading
+ if (chatLaunchLoadProgress) {
+ chatLaunchState = "loading";
+ return;
+ }
+
+ // Check if currently downloading
+ if (chatLaunchDownload) {
+ chatLaunchState = "downloading";
+ }
+ });
+
+ // Check if any instance is running (for showing model selector vs chat)
+ const hasAnyRunningInstance = $derived(() => {
+ for (const [, inst] of Object.entries(instanceData)) {
+ const status = deriveInstanceStatus(inst);
+ if (
+ status.statusText === "READY" ||
+ status.statusText === "LOADED" ||
+ status.statusText === "RUNNING"
+ ) {
+ return true;
+ }
+ }
+ return false;
+ });
+
+ // Handle model selection from ChatModelSelector
+ function handleChatModelSelect(modelId: string, category: string) {
+ launchModelForChat(modelId, category);
+ }
+
+ // Handle "+ Add Model" from ChatModelSelector
+ function handleChatAddModel() {
+ modelPickerContext = "chat";
+ isModelPickerOpen = true;
+ }
+
+ // Track which context opened the model picker (dashboard launch vs chat selection)
+ let modelPickerContext = $state<"dashboard" | "chat">("dashboard");
+
+ // Open the model picker from a chat context (e.g. clicking the model button in ChatForm)
+ function openChatModelPicker() {
+ modelPickerContext = "chat";
+ isModelPickerOpen = true;
+ }
+
+ // Handle model selection from the picker when opened from chat context
+ function handleChatPickerSelect(modelId: string) {
+ setSelectedChatModel(modelId);
+ userForcedIdle = false;
+ isModelPickerOpen = false;
+ }
+
+ // Unified send handler: sends if model running, auto-launches if not
+ function handleChatSend(
+ content: string,
+ files?: {
+ id: string;
+ name: string;
+ type: string;
+ textContent?: string;
+ preview?: string;
+ }[],
+ ) {
+ const model = selectedChatModel();
+
+ // Model is selected and running — send directly
+ if (model && hasRunningInstance(model)) {
+ sendMessage(content, files, null);
+ return;
+ }
+
+ // Model is selected but NOT running — launch it, queue the message
+ if (model) {
+ pendingAutoMessage = { content, files };
+ userForcedIdle = false;
+ launchModelForChat(model, "picker");
+ return;
+ }
+
+ // No model selected — fall through to auto-pick
+ handleAutoSend(content, files);
+ }
+
// Helper to get the number of nodes in a placement preview
function getPreviewNodeCount(preview: PlacementPreview): number {
if (!preview.memory_delta_by_node) return 0;
@@ -3689,6 +4325,7 @@
<button
type="button"
onclick={() => {
+ modelPickerContext = "dashboard";
isModelPickerOpen = true;
}}
class="text-sm font-sans text-white/40 hover:text-exo-yellow transition-colors cursor-pointer underline underline-offset-4 decoration-white/20 hover:decoration-exo-yellow/50"
@@ -3845,6 +4482,8 @@
showModelSelector={true}
modelTasks={modelTasks()}
modelCapabilities={modelCapabilities()}
+ onOpenModelPicker={openChatModelPicker}
+ onAutoSend={handleChatSend}
/>
</div>
@@ -3963,7 +4602,13 @@
role="complementary"
aria-label="Conversation history"
>
- <ChatSidebar class="h-full" />
+ <ChatSidebar
+ class="h-full"
+ onNewChat={handleNewChat}
+ onSelectConversation={() => {
+ userForcedIdle = false;
+ }}
+ />
</div>
{/if}
@@ -4256,6 +4901,8 @@
showModelSelector={true}
modelTasks={modelTasks()}
modelCapabilities={modelCapabilities()}
+ onOpenModelPicker={openChatModelPicker}
+ onAutoSend={handleChatSend}
/>
</div>
</div>
@@ -4320,6 +4967,7 @@
instanceModelId !== "Unknown" &&
instanceModelId !== "Unknown Model"
) {
+ userForcedIdle = false;
setSelectedChatModel(instanceModelId);
}
}}
@@ -4757,7 +5405,7 @@
<p
class="text-[11px] text-green-400/70 leading-relaxed"
>
- Ready to chat! Type a message below.
+ Ready to chat!
</p>
{/if}
</div>
@@ -4799,7 +5447,10 @@
<div class="flex-shrink-0 mb-3">
<button
type="button"
- onclick={() => (isModelPickerOpen = true)}
+ onclick={() => {
+ modelPickerContext = "dashboard";
+ isModelPickerOpen = true;
+ }}
class="w-full bg-exo-medium-gray/50 border border-exo-yellow/30 rounded pl-3 pr-8 py-2.5 text-sm font-mono text-left tracking-wide cursor-pointer transition-all duration-200 hover:border-exo-yellow/50 focus:outline-none focus:border-exo-yellow/70 relative"
>
{#if selectedModelId}
@@ -4827,6 +5478,32 @@
{:else}
<span class="text-exo-light-gray">{selectedModelId}</span>
{/if}
+ {:else if bestRunningModelId}
+ {@const runModel = models.find(
+ (m) => m.id === bestRunningModelId,
+ )}
+ {#if runModel}
+ {@const sizeGB = getModelSizeGB(runModel)}
+ <span
+ class="flex items-center justify-between gap-2 w-full pr-4"
+ >
+ <span
+ class="flex items-center gap-2 text-exo-light-gray truncate"
+ >
+ <span class="truncate"
+ >{runModel.name || runModel.id}</span
+ >
+ </span>
+ <span class="text-white/50 text-xs flex-shrink-0"
+ >{sizeGB >= 1
+ ? sizeGB.toFixed(0)
+ : sizeGB.toFixed(1)}GB</span
+ >
+ </span>
+ {:else}
+ <span class="text-exo-light-gray">{bestRunningModelId}</span
+ >
+ {/if}
{:else}
<span class="text-white/50">— SELECT MODEL —</span>
{/if}
@@ -5127,30 +5804,207 @@
class="flex-1 flex flex-col min-w-0 overflow-hidden"
in:fade={{ duration: 300, delay: 100 }}
>
- <div
- class="flex-1 overflow-y-auto px-8 py-6"
- bind:this={chatScrollRef}
- role="log"
- aria-live="polite"
- aria-label="Chat messages"
- >
- <div class="max-w-7xl mx-auto">
- <ChatMessages scrollParent={chatScrollRef} />
+ {#if chatLaunchState === "idle"}
+ <!-- No running instance: show model selector -->
+ <div
+ class="flex-1 overflow-y-auto flex items-center justify-center px-8 py-6"
+ >
+ <ChatModelSelector
+ models={models.map((m) => ({
+ id: m.id,
+ name: m.name ?? "",
+ base_model: m.base_model ?? "",
+ storage_size_megabytes: m.storage_size_megabytes ?? 0,
+ capabilities: m.capabilities ?? [],
+ family: m.family ?? "",
+ quantization: m.quantization ?? "",
+ }))}
+ clusterLabel={chatClusterLabel}
+ totalMemoryGB={availableMemoryGB()}
+ onSelect={handleChatModelSelect}
+ onAddModel={handleChatAddModel}
+ />
</div>
- </div>
+ <div
+ class="flex-shrink-0 px-8 pb-6 pt-4 bg-gradient-to-t from-exo-black via-exo-black to-transparent"
+ >
+ <div class="max-w-7xl mx-auto">
+ <ChatForm
+ placeholder="Ask anything — we'll pick the best model automatically"
+ showModelSelector={!!bestRunningModelId}
+ modelDisplayOverride={bestRunningModelId ?? undefined}
+ modelTasks={modelTasks()}
+ modelCapabilities={modelCapabilities()}
+ onAutoSend={handleAutoSend}
+ onOpenModelPicker={openChatModelPicker}
+ />
+ </div>
+ </div>
+ {:else if chatLaunchState !== "idle" && chatLaunchState !== "ready"}
+ <!-- Model launching/downloading/loading: show progress -->
+ <div class="flex-1 flex items-center justify-center px-8 py-6">
+ <div class="flex flex-col items-center gap-6 max-w-md w-full">
+ <!-- Model name -->
+ {#if pendingChatModelId}
+ <p class="text-sm text-white font-mono tracking-wide">
+ {pendingChatModelId.split("/").pop()?.replace(/-/g, " ") ||
+ pendingChatModelId}
+ </p>
+ {/if}
- <div
- class="flex-shrink-0 px-8 pb-6 pt-4 bg-gradient-to-t from-exo-black via-exo-black to-transparent"
- >
- <div class="max-w-7xl mx-auto">
- <ChatForm
- placeholder="Ask anything"
- showModelSelector={true}
- modelTasks={modelTasks()}
- modelCapabilities={modelCapabilities()}
- />
+ {#if chatLaunchState === "launching"}
+ <div class="flex flex-col items-center gap-3">
+ <div
+ class="w-8 h-8 border-2 border-exo-yellow/30 border-t-exo-yellow rounded-full animate-spin"
+ ></div>
+ <p
+ class="text-xs text-exo-light-gray font-mono uppercase tracking-wider"
+ >
+ Preparing to launch…
+ </p>
+ </div>
+ {:else if chatLaunchState === "downloading"}
+ <div class="w-full flex flex-col gap-3">
+ <div
+ class="flex items-center justify-between text-xs font-mono"
+ >
+ <span class="text-exo-yellow uppercase tracking-wider"
+ >Downloading</span
+ >
+ {#if chatLaunchDownload}
+ <span class="text-exo-light-gray tabular-nums">
+ {chatLaunchDownload.percentage.toFixed(1)}%
+ </span>
+ {/if}
+ </div>
+ <div
+ class="w-full h-2 bg-exo-dark-gray rounded-full overflow-hidden border border-exo-medium-gray/30"
+ >
+ <div
+ class="h-full bg-gradient-to-r from-exo-yellow/80 to-exo-yellow rounded-full transition-all duration-300"
+ style="width: {chatLaunchDownload?.percentage ?? 0}%"
+ ></div>
+ </div>
+ {#if chatLaunchDownload}
+ <div
+ class="flex justify-between text-[10px] text-exo-light-gray/60 font-mono"
+ >
+ <span
+ >{formatBytes(chatLaunchDownload.downloadedBytes)} / {formatBytes(
+ chatLaunchDownload.totalBytes,
+ )}</span
+ >
+ <span>
+ {#if chatLaunchDownload.speed > 0}
+ {formatBytes(chatLaunchDownload.speed)}/s
+ {/if}
+ {#if chatLaunchDownload.etaMs > 0}
+ · {formatEta(chatLaunchDownload.etaMs)}
+ {/if}
+ </span>
+ </div>
+ {/if}
+ </div>
+ {:else if chatLaunchState === "loading"}
+ <div class="w-full flex flex-col gap-3">
+ <div
+ class="flex items-center justify-between text-xs font-mono"
+ >
+ <span class="text-exo-yellow uppercase tracking-wider"
+ >Loading model</span
+ >
+ {#if chatLaunchLoadProgress}
+ <span class="text-exo-light-gray tabular-nums">
+ {chatLaunchLoadProgress.layersLoaded}/{chatLaunchLoadProgress.totalLayers}
+ layers
+ </span>
+ {/if}
+ </div>
+ <div
+ class="w-full h-2 bg-exo-dark-gray rounded-full overflow-hidden border border-exo-medium-gray/30"
+ >
+ <div
+ class="h-full bg-gradient-to-r from-exo-yellow/80 to-exo-yellow rounded-full transition-all duration-300"
+ style="width: {chatLaunchLoadProgress?.percentage ??
+ 0}%"
+ ></div>
+ </div>
+ </div>
+ {/if}
+ </div>
</div>
- </div>
+ <div
+ class="flex-shrink-0 px-8 pb-6 pt-4 bg-gradient-to-t from-exo-black via-exo-black to-transparent"
+ >
+ <div class="max-w-7xl mx-auto">
+ <ChatForm
+ placeholder="Ask anything"
+ showModelSelector={true}
+ modelTasks={modelTasks()}
+ modelCapabilities={modelCapabilities()}
+ onAutoSend={handleChatSend}
+ onOpenModelPicker={openChatModelPicker}
+ />
+ </div>
+ </div>
+ {:else}
+ <!-- Normal chat: model is running -->
+ <div
+ class="flex-1 overflow-y-auto px-8 py-6"
+ bind:this={chatScrollRef}
+ role="log"
+ aria-live="polite"
+ aria-label="Chat messages"
+ >
+ <div class="max-w-7xl mx-auto">
+ <ChatMessages scrollParent={chatScrollRef} />
+ {#if chatLaunchState === "ready" && selectedChatCategory}
+ {@const prompts =
+ categorySuggestedPrompts[selectedChatCategory] ??
+ categorySuggestedPrompts.auto}
+ <div
+ class="flex flex-col items-center gap-4 mt-12"
+ in:fade={{ duration: 300 }}
+ >
+ <p
+ class="text-xs text-exo-light-gray/60 font-mono uppercase tracking-wider"
+ >
+ Try asking
+ </p>
+ <div class="grid grid-cols-2 gap-2 max-w-lg w-full">
+ {#each prompts as prompt}
+ <button
+ type="button"
+ onclick={() => {
+ chatLaunchState = "idle";
+ selectedChatCategory = null;
+ sendMessage(prompt);
+ }}
+ class="text-left px-3 py-2.5 text-xs text-exo-light-gray hover:text-white font-mono rounded-lg border border-exo-medium-gray/30 hover:border-exo-yellow/30 bg-exo-dark-gray/30 hover:bg-exo-dark-gray/60 transition-all duration-200 cursor-pointer"
+ >
+ {prompt}
+ </button>
+ {/each}
+ </div>
+ </div>
+ {/if}
+ </div>
+ </div>
+ <div
+ class="flex-shrink-0 px-8 pb-6 pt-4 bg-gradient-to-t from-exo-black via-exo-black to-transparent"
+ >
+ <div class="max-w-7xl mx-auto">
+ <ChatForm
+ placeholder="Ask anything"
+ showModelSelector={true}
+ modelTasks={modelTasks()}
+ modelCapabilities={modelCapabilities()}
+ onAutoSend={handleChatSend}
+ onOpenModelPicker={openChatModelPicker}
+ />
+ </div>
+ </div>
+ {/if}
</div>
<!-- Right: Mini-Map Sidebar -->
@@ -5244,6 +6098,7 @@
instanceModelId !== "Unknown" &&
instanceModelId !== "Unknown Model"
) {
+ userForcedIdle = false;
setSelectedChatModel(instanceModelId);
}
}}
@@ -5690,7 +6545,7 @@
<p
class="text-[11px] text-green-400/70 leading-relaxed"
>
- Ready to chat! Type a message below.
+ Ready to chat!
</p>
{/if}
</div>
@@ -5733,7 +6588,13 @@
const model = models.find((m) => m.id === modelId);
return model ? getModelMemoryFitStatus(model) : "too_large";
}}
- onSelect={handleModelPickerSelect}
+ onSelect={(modelId) => {
+ if (modelPickerContext === "chat") {
+ handleChatPickerSelect(modelId);
+ } else {
+ handleModelPickerSelect(modelId);
+ }
+ }}
onClose={() => (isModelPickerOpen = false)}
onToggleFavorite={toggleFavorite}
onAddModel={addModelFromPicker}
@@ -5742,5 +6603,6 @@
usedMemoryGB={clusterMemory().used / (1024 * 1024 * 1024)}
{downloadsData}
topologyNodes={data?.nodes}
+ instanceStatuses={modelInstanceStatuses}
/>
{/if}
diff --git a/src/exo/master/placement.py b/src/exo/master/placement.py
index 5f191b84..c7fc95e5 100644
--- a/src/exo/master/placement.py
+++ b/src/exo/master/placement.py
@@ -133,6 +133,11 @@ def place_instance(
),
)
+ # Single-node: force Pipeline/Ring (Tensor and Jaccl require multi-node)
+ if len(selected_cycle) == 1:
+ command.instance_meta = InstanceMeta.MlxRing
+ command.sharding = Sharding.Pipeline
+
shard_assignments = get_shard_assignments(
command.model_card, selected_cycle, command.sharding, node_memory
)
@@ -142,9 +147,6 @@ def place_instance(
instance_id = InstanceId()
target_instances = dict(deepcopy(current_instances))
- if len(selected_cycle) == 1:
- command.instance_meta = InstanceMeta.MlxRing
-
match command.instance_meta:
case InstanceMeta.MlxJaccl:
# TODO(evan): shard assignments should contain information about ranks, this is ugly
diff --git a/src/exo/worker/engines/mlx/utils_mlx.py b/src/exo/worker/engines/mlx/utils_mlx.py
index 30f9afb4..4c44ba65 100644
--- a/src/exo/worker/engines/mlx/utils_mlx.py
+++ b/src/exo/worker/engines/mlx/utils_mlx.py
@@ -193,7 +193,7 @@ def load_mlx_items(
on_layer_loaded(i, total)
except ValueError as e:
logger.opt(exception=e).debug(
- "Model architecture doesn't support layer-by-layer progress tracking"
+ "Model architecture doesn't support layer-by-layer progress tracking",
)
mx.eval(model)
end_time = time.perf_counter()
← 05986f77 add exo bench protobuf dependency (#1596)
·
back to Exo
·
feat: add info button to model picker variant rows (#1589) dc89ba66 →