Okay let's turn no_grad back on. We'll worry about that when tinygrad training works

bcf87e79b7fe34927b2f1567ea6b1091c0e91046 · 2024-12-06 04:49:46 -0800 · Nel Nibcord

Files touched

M exo/inference/tinygrad/inference.py

Diff

commit bcf87e79b7fe34927b2f1567ea6b1091c0e91046
Author: Nel Nibcord <blindcrone@tuta.io>
Date:   Fri Dec 6 04:49:46 2024 -0800

    Okay let's turn no_grad back on. We'll worry about that when tinygrad training works
---
 exo/inference/tinygrad/inference.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/exo/inference/tinygrad/inference.py b/exo/inference/tinygrad/inference.py
index aceb4614..f8503ba0 100644
--- a/exo/inference/tinygrad/inference.py
+++ b/exo/inference/tinygrad/inference.py
@@ -16,7 +16,7 @@ from .losses import length_masked_ce_loss
 from collections import OrderedDict
 import asyncio
 
-Tensor.no_grad = False
+Tensor.no_grad = True 
 # default settings
 TEMPERATURE = int(os.getenv("TEMPERATURE", 0.85))
 TOP_K = 25

← b7bbda33 Removed tinygrad StatefulModel class, as it's no longer used · back to Exo · Initialize inference engine session in base class 98edb393 →