use AutoProcessor with use_fast=False since there's a bug with use_fast=True where whitespace is removed on single token decodes

2d2000096440c4783a33c8639192ff6d1c3125bc · 2024-07-30 22:40:02 +0100 · Alex Cheema

Files touched

M exo/api/chatgpt_api.py

Diff

commit 2d2000096440c4783a33c8639192ff6d1c3125bc
Author: Alex Cheema <alexcheema123@gmail.com>
Date:   Tue Jul 30 22:40:02 2024 +0100

    use AutoProcessor with use_fast=False since there's a bug with use_fast=True where whitespace is removed on single token decodes
---
 exo/api/chatgpt_api.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/exo/api/chatgpt_api.py b/exo/api/chatgpt_api.py
index 413b7f46..08e4cd76 100644
--- a/exo/api/chatgpt_api.py
+++ b/exo/api/chatgpt_api.py
@@ -73,8 +73,8 @@ def resolve_tinygrad_tokenizer(model_id: str):
 
 async def resolve_tokenizer(model_id: str):
   try:
-    if DEBUG >= 2: print(f"Trying to AutoProcessor for {model_id}")
-    processor = AutoProcessor.from_pretrained(model_id)
+    if DEBUG >= 2: print(f"Trying AutoProcessor for {model_id}")
+    processor = AutoProcessor.from_pretrained(model_id, use_fast=False)
     processor.eos_token_id = processor.tokenizer.eos_token_id
     processor.encode = processor.tokenizer.encode
     return processor

← af1c7ce3 add support for image upload to tinychat for vision models · back to Exo · fix image api prompt encoding 178fb75c →