disable tinygrad test again, we need a smaller model or a machine with more memory otherwise we get Metal OOM

9d939b3703801a7989d8827d0bc91f7781c1d7f2 · 2024-07-20 12:42:56 -0700 · Alex Cheema

Files touched

M exo/inference/test_inference_engine.py

Diff

commit 9d939b3703801a7989d8827d0bc91f7781c1d7f2
Author: Alex Cheema <alexcheema123@gmail.com>
Date:   Sat Jul 20 12:42:56 2024 -0700

    disable tinygrad test again, we need a smaller model or a machine with more memory otherwise we get Metal OOM
---
 exo/inference/test_inference_engine.py | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/exo/inference/test_inference_engine.py b/exo/inference/test_inference_engine.py
index f5e7d863..393a2671 100644
--- a/exo/inference/test_inference_engine.py
+++ b/exo/inference/test_inference_engine.py
@@ -30,9 +30,9 @@ asyncio.run(test_inference_engine(
     "mlx-community/Meta-Llama-3-8B-Instruct-4bit",
 ))
 
-# TODO: Waiting on https://github.com/tinygrad/tinygrad/issues/5549
-asyncio.run(test_inference_engine(
-    TinygradDynamicShardInferenceEngine(),
-    TinygradDynamicShardInferenceEngine(),
-    "llama3-8b-sfr",
-))
+# TODO: Need more memory or a smaller model
+# asyncio.run(test_inference_engine(
+#     TinygradDynamicShardInferenceEngine(),
+#     TinygradDynamicShardInferenceEngine(),
+#     "llama3-8b-sfr",
+# ))

← 774e6209 add space between outputs in github workflow integration tes · back to Exo · test trigger workflow 719e149a →