Fix exo bench for transformers 5.x (#1168)

4b3de6b9844a2f84e32bf648dec498a6c8af3670 · 2026-01-16 12:39:22 +0000 · rltakashige

## Motivation
Prompt Sizer was broken as transformers 5.x tokenizers create
BatchEncodings which are essentially a dictionary of {input_ids: []}
instead of the list of input ids.

## Test Plan

### Manual Testing
Tested that exo bench runs as expected.

### Automated Testing
<!-- Describe changes to automated tests, or how existing tests cover
this change -->
<!-- - -->

Files touched

M bench/exo_bench.py

Diff

commit 4b3de6b9844a2f84e32bf648dec498a6c8af3670
Author: rltakashige <rl.takashige@gmail.com>
Date:   Fri Jan 16 12:39:22 2026 +0000

    Fix exo bench for transformers 5.x (#1168)
    
    ## Motivation
    Prompt Sizer was broken as transformers 5.x tokenizers create
    BatchEncodings which are essentially a dictionary of {input_ids: []}
    instead of the list of input ids.
    
    ## Test Plan
    
    ### Manual Testing
    Tested that exo bench runs as expected.
    
    ### Automated Testing
    <!-- Describe changes to automated tests, or how existing tests cover
    this change -->
    <!-- - -->
---
 bench/exo_bench.py | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/bench/exo_bench.py b/bench/exo_bench.py
index 2443b03d..39d65553 100644
--- a/bench/exo_bench.py
+++ b/bench/exo_bench.py
@@ -241,6 +241,9 @@ class PromptSizer:
             ids = tokenizer.apply_chat_template(
                 messages, tokenize=True, add_generation_prompt=True
             )
+            # Fix for transformers 5.x
+            if hasattr(ids, "input_ids"):
+                ids = ids.input_ids
             return int(len(ids))
 
         return count_fn

← c8de3b90 quiet rust logs · back to Exo · fix local network warning (#1136) 991adfbd →