trim off the eos_token_id from chatgpt api response

9759408a678f4522c7922344282f27ae32a53c0e · 2024-07-16 00:21:17 -0700 · Alex Cheema

Files touched

M exo/api/chatgpt_api.py

Diff

commit 9759408a678f4522c7922344282f27ae32a53c0e
Author: Alex Cheema <alexcheema123@gmail.com>
Date:   Tue Jul 16 00:21:17 2024 -0700

    trim off the eos_token_id from chatgpt api response
---
 exo/api/chatgpt_api.py | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/exo/api/chatgpt_api.py b/exo/api/chatgpt_api.py
index 36162728..4200b10c 100644
--- a/exo/api/chatgpt_api.py
+++ b/exo/api/chatgpt_api.py
@@ -64,6 +64,8 @@ class ChatGPTAPI:
                 continue
             await asyncio.sleep(0.1)
             if is_finished:
+                if result[-1] == tokenizer._tokenizer.eos_token_id:
+                    result = result[:-1]
                 return web.json_response({
                     "id": f"chatcmpl-{request_id}",
                     "object": "chat.completion",

← f2895cbc revive the chatgpt api endpoint on :8000 · back to Exo · clarify example in readme bdf105a6 →