Always load image model cards into cache (#1421)

ead19bea7483dbbda590b4b7a73530d5237e0019 · 2026-02-10 09:11:57 -0800 · Alex Cheema

## Motivation

Follows up on #1408. Image models (FLUX, Qwen-Image, etc.) don't have a
`config.json` on HuggingFace. Previously, image model TOML cards were
only loaded into `_card_cache` when `EXO_ENABLE_IMAGE_MODELS=true`. When
the flag was off but an image model was requested (e.g., via
`get_placement_previews`), `ModelCard.load()` fell through to
`fetch_from_hf()` which tried to download `config.json` — causing
`FileNotFoundError` spam. #1408 added defensive error handling; this PR
fixes the root cause.

## Changes

**`model_cards.py`**: Always include `image_model_cards/` in
`CARD_SEARCH_PATH` so image model TOML cards are always loaded into
`_card_cache`. `ModelCard.load()` then finds them directly and never
falls through to `fetch_from_hf()`. The `EXO_ENABLE_IMAGE_MODELS` flag
now controls whether image models appear in `get_model_cards()` (the
listing) rather than whether they're loaded at all.

## Why It Works

`fetch_from_hf()` is designed for text models only (it hardcodes
`tasks=[ModelTask.TextGeneration]` and requires `config.json`). Image
models should never reach that path. By always having them in the cache,
the lookup succeeds immediately and `fetch_from_hf()` is never called.

## Test Plan

### Automated Testing
- `uv run basedpyright` — 0 errors
- `uv run ruff check` — passes

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: rltakashige <rl.takashige@gmail.com>

Files touched

M src/exo/shared/models/model_cards.py

Diff

commit ead19bea7483dbbda590b4b7a73530d5237e0019
Author: Alex Cheema <41707476+AlexCheema@users.noreply.github.com>
Date:   Tue Feb 10 09:11:57 2026 -0800

    Always load image model cards into cache (#1421)
    
    ## Motivation
    
    Follows up on #1408. Image models (FLUX, Qwen-Image, etc.) don't have a
    `config.json` on HuggingFace. Previously, image model TOML cards were
    only loaded into `_card_cache` when `EXO_ENABLE_IMAGE_MODELS=true`. When
    the flag was off but an image model was requested (e.g., via
    `get_placement_previews`), `ModelCard.load()` fell through to
    `fetch_from_hf()` which tried to download `config.json` — causing
    `FileNotFoundError` spam. #1408 added defensive error handling; this PR
    fixes the root cause.
    
    ## Changes
    
    **`model_cards.py`**: Always include `image_model_cards/` in
    `CARD_SEARCH_PATH` so image model TOML cards are always loaded into
    `_card_cache`. `ModelCard.load()` then finds them directly and never
    falls through to `fetch_from_hf()`. The `EXO_ENABLE_IMAGE_MODELS` flag
    now controls whether image models appear in `get_model_cards()` (the
    listing) rather than whether they're loaded at all.
    
    ## Why It Works
    
    `fetch_from_hf()` is designed for text models only (it hardcodes
    `tasks=[ModelTask.TextGeneration]` and requires `config.json`). Image
    models should never reach that path. By always having them in the cache,
    the lookup succeeds immediately and `fetch_from_hf()` is never called.
    
    ## Test Plan
    
    ### Automated Testing
    - `uv run basedpyright` — 0 errors
    - `uv run ruff check` — passes
    
    Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
    Co-authored-by: rltakashige <rl.takashige@gmail.com>
---
 src/exo/shared/models/model_cards.py | 18 ++++++++++++------
 1 file changed, 12 insertions(+), 6 deletions(-)

diff --git a/src/exo/shared/models/model_cards.py b/src/exo/shared/models/model_cards.py
index f9448f11..e9d2cf1c 100644
--- a/src/exo/shared/models/model_cards.py
+++ b/src/exo/shared/models/model_cards.py
@@ -30,11 +30,11 @@ from exo.utils.pydantic_ext import CamelCaseModel
 # kinda ugly...
 # TODO: load search path from config.toml
 _custom_cards_dir = Path(str(EXO_CUSTOM_MODEL_CARDS_DIR))
-_csp = [Path(RESOURCES_DIR) / "inference_model_cards", _custom_cards_dir]
-if EXO_ENABLE_IMAGE_MODELS:
-    _csp.append(Path(RESOURCES_DIR) / "image_model_cards")
-
-CARD_SEARCH_PATH = _csp
+CARD_SEARCH_PATH = [
+    Path(RESOURCES_DIR) / "inference_model_cards",
+    Path(RESOURCES_DIR) / "image_model_cards",
+    _custom_cards_dir,
+]
 
 _card_cache: dict[ModelId, "ModelCard"] = {}
 
@@ -49,10 +49,16 @@ async def _refresh_card_cache():
                 pass
 
 
+def _is_image_card(card: "ModelCard") -> bool:
+    return any(t in (ModelTask.TextToImage, ModelTask.ImageToImage) for t in card.tasks)
+
+
 async def get_model_cards() -> list["ModelCard"]:
     if len(_card_cache) == 0:
         await _refresh_card_cache()
-    return list(_card_cache.values())
+    if EXO_ENABLE_IMAGE_MODELS:
+        return list(_card_cache.values())
+    return [c for c in _card_cache.values() if not _is_image_card(c)]
 
 
 class ModelTask(str, Enum):

← 5a83e591 dashboard: allow typing in chat input while response is gene · back to Exo · event_log: move event log from unbounded in-memory list to d 305a3c8b →