make sure this is actually doing something

3a58576f8c3dd16a05344ddf9f1b52d54e3f3d0c · 2024-12-17 16:22:22 +0000 · Alex Cheema

Files touched

M exo/orchestration/node.py

Diff

commit 3a58576f8c3dd16a05344ddf9f1b52d54e3f3d0c
Author: Alex Cheema <alexcheema123@gmail.com>
Date:   Tue Dec 17 16:22:22 2024 +0000

    make sure this is actually doing something
---
 exo/orchestration/node.py | 1 +
 1 file changed, 1 insertion(+)

diff --git a/exo/orchestration/node.py b/exo/orchestration/node.py
index e55e96c5..68e996a1 100644
--- a/exo/orchestration/node.py
+++ b/exo/orchestration/node.py
@@ -122,6 +122,7 @@ class Node:
     if request_id not in self.buffered_token_output:
       self.buffered_token_output[request_id] = ([], False)
     is_finished = len(self.buffered_token_output[request_id][0]) >= self.max_generate_tokens
+    await asyncio.sleep(0.1)
     
     if shard.is_last_layer() and not is_finished:
       self.token_count += 1

← 0a072230 switch to uvloop (faster asyncio event loop) and optimise gr · back to Exo · remove test sleep 1f108a06 →