Forward tools to the models chat template properly (#1106)

caafc4869384d4c310134f7f9021e0549dfc50d3 · 2026-01-09 13:28:41 +0000 · Evan Quiney

We did not properly forward tools to the chat template before. This is not a full tool calling impl - but it should improve things slightly.

## Changes made

Pass tools to the hf tokenizers chat template
Join message chunks into a larger message (opencode does this sometimes - we were ignoring before)

## Future work

We need to parse the model output and normalise the return format to be compatible with the openai api.

Files touched

M src/exo/worker/engines/mlx/utils_mlx.py

Diff

commit caafc4869384d4c310134f7f9021e0549dfc50d3
Author: Evan Quiney <evanev7@gmail.com>
Date:   Fri Jan 9 13:28:41 2026 +0000

     Forward tools to the models chat template properly (#1106)
    
    We did not properly forward tools to the chat template before. This is not a full tool calling impl - but it should improve things slightly.
    
    ## Changes made
    
    Pass tools to the hf tokenizers chat template
    Join message chunks into a larger message (opencode does this sometimes - we were ignoring before)
    
    ## Future work
    
    We need to parse the model output and normalise the return format to be compatible with the openai api.
---
 src/exo/worker/engines/mlx/utils_mlx.py | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/src/exo/worker/engines/mlx/utils_mlx.py b/src/exo/worker/engines/mlx/utils_mlx.py
index 5b2dfbaa..bbd0b38f 100644
--- a/src/exo/worker/engines/mlx/utils_mlx.py
+++ b/src/exo/worker/engines/mlx/utils_mlx.py
@@ -284,15 +284,15 @@ def apply_chat_template(
     messages = chat_task_data.messages
 
     formatted_messages: list[dict[str, Any]] = []
-    for _, message in enumerate(messages):
+    for message in messages:
         if isinstance(message.content, ChatCompletionMessageText):
             message.content = message.content.text
         if isinstance(message.content, list):
-            if len(message.content) != 1:
-                logger.warning("Received malformed prompt")
+            if len(message.content) == 0:
+                logger.warning("Received prompt with no content, skipping")
                 continue
 
-            message.content = message.content[0].text
+            message.content = "\n".join(c.text for c in message.content).strip()
         if message.content is None and message.thinking is None:
             continue
 
@@ -305,6 +305,7 @@ def apply_chat_template(
         formatted_messages,
         tokenize=False,
         add_generation_prompt=True,
+        tools=chat_task_data.tools,
     )
 
     return prompt  # type: ignore

← cca8c998 cleanup unused dependencies · back to Exo · feat: uninstall button (#1077) d85b5d37 →