clearer example docs and add 405b example

67f789b6ac682a77d9188520da373157edf265ef · 2024-10-01 01:37:06 +0400 · Alex Cheema

Files touched

M README.md

Diff

commit 67f789b6ac682a77d9188520da373157edf265ef
Author: Alex Cheema <alexcheema123@gmail.com>
Date:   Tue Oct 1 01:37:06 2024 +0400

    clearer example docs and add 405b example
---
 README.md | 18 +++++++++++++++++-
 1 file changed, 17 insertions(+), 1 deletion(-)

diff --git a/README.md b/README.md
index 53467c1b..c7fc7ee9 100644
--- a/README.md
+++ b/README.md
@@ -120,7 +120,9 @@ That's it! No configuration required - exo will automatically discover the other
 
 exo starts a ChatGPT-like WebUI (powered by [tinygrad tinychat](https://github.com/tinygrad/tinygrad/tree/master/examples/tinychat)) on http://localhost:8000
 
-For developers, exo also starts a ChatGPT-compatible API endpoint on http://localhost:8000/v1/chat/completions. Example with curls:
+For developers, exo also starts a ChatGPT-compatible API endpoint on http://localhost:8000/v1/chat/completions. Examples with curl:
+
+#### Llama 3.1 8B:
 
 ```sh
 curl http://localhost:8000/v1/chat/completions \
@@ -132,6 +134,20 @@ curl http://localhost:8000/v1/chat/completions \
    }'
 ```
 
+#### Llama 3.1 405B:
+
+```sh
+curl http://localhost:8000/v1/chat/completions \
+  -H "Content-Type: application/json" \
+  -d '{
+     "model": "llama-3.1-405b",
+     "messages": [{"role": "user", "content": "What is the meaning of exo?"}],
+     "temperature": 0.7
+   }'
+```
+
+#### Llava 1.5 7B (Vision Language Model):
+
 ```sh
 curl http://localhost:8000/v1/chat/completions \
   -H "Content-Type: application/json" \

← abca3bfa add support for qwen2.5 coder 1.5b and 7b · back to Exo · update readme with PyTorch inference engine and llama.cpp li 0120891c →