Call no longer needs request_id

65fdc99cccd7cb807990ed6d7228872187cb4ccb · 2024-11-12 03:31:37 -0800 · Nel Nibcord

Files touched

M exo/inference/tinygrad/models/llama.py

Diff

commit 65fdc99cccd7cb807990ed6d7228872187cb4ccb
Author: Nel Nibcord <blindcrone@tuta.io>
Date:   Tue Nov 12 03:31:37 2024 -0800

    Call no longer needs request_id
---
 exo/inference/tinygrad/models/llama.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/exo/inference/tinygrad/models/llama.py b/exo/inference/tinygrad/models/llama.py
index 0dbde681..9928ab43 100644
--- a/exo/inference/tinygrad/models/llama.py
+++ b/exo/inference/tinygrad/models/llama.py
@@ -225,7 +225,7 @@ class Transformer:
       h = inputs
     return h
 
-  def __call__(self, tokens: Tensor, start_pos: Variable, request_id: str, cache: Optional[List[Tensor]] = None):
+  def __call__(self, tokens: Tensor, start_pos: Variable, cache: Optional[List[Tensor]] = None):
     # TODO: better way to handle the first call v.s. the rest?
     h = self.embed(x)
     if tokens.shape[0:2] == (1, 1) and self.forward_jit is not None:

← 90518a3b Hoisted caching to a wrapper class · back to Exo · Fixed unit tests 1cd3efbe →