list exo processes before test, warmup req in bench

87865f0cd97b99e2f04952305153fe4aa8eb54ed · 2024-12-08 20:56:37 +0000 · Alex Cheema

Files touched

M .github/bench.py
M .github/workflows/bench_job.yml

Diff

commit 87865f0cd97b99e2f04952305153fe4aa8eb54ed
Author: Alex Cheema <alexcheema123@gmail.com>
Date:   Sun Dec 8 20:56:37 2024 +0000

    list exo processes before test, warmup req in bench
---
 .github/bench.py                | 11 ++++++++++-
 .github/workflows/bench_job.yml |  4 ++++
 2 files changed, 14 insertions(+), 1 deletion(-)

diff --git a/.github/bench.py b/.github/bench.py
index 734eaf2a..1f056b0b 100644
--- a/.github/bench.py
+++ b/.github/bench.py
@@ -97,9 +97,18 @@ async def main() -> None:
     api_endpoint = "http://localhost:52415/v1/chat/completions"
 
     # Define prompts
+    prompt_warmup = "what is the capital of France?"
     prompt_essay = "write an essay about cats"
 
-    # Measure performance for the essay prompt, which depends on the first measurement
+    # Warmup request
+    print("\nPerforming warmup request...", flush=True)
+    try:
+        warmup_results = await measure_performance(api_endpoint, prompt_warmup)
+        print("Warmup completed successfully", flush=True)
+    except Exception as e:
+        print(f"Warmup request failed: {e}", flush=True)
+
+    # Measure performance for the essay prompt
     print("\nMeasuring performance for the essay prompt...", flush=True)
     results = await measure_performance(api_endpoint, prompt_essay)
 
diff --git a/.github/workflows/bench_job.yml b/.github/workflows/bench_job.yml
index be918ecf..79eea80a 100644
--- a/.github/workflows/bench_job.yml
+++ b/.github/workflows/bench_job.yml
@@ -57,6 +57,10 @@ jobs:
           aws_access_key_id: ${{ secrets.S3_EXO_BENCHMARKS_AWS_ACCESS_KEY_ID }}
           aws_secret_key: ${{ secrets.S3_EXO_BENCHMARKS_AWS_SECRET_ACCESS_KEY }}
         run: |
+          # List existing exo processes
+          echo "Existing exo processes:"
+          ps aux | grep exo || true
+
           CALLING_JOB="${{ inputs.calling_job_name }}"
           UNIQUE_JOB_ID="${CALLING_JOB}_${GITHUB_RUN_ID}"
           ALL_NODE_IDS=$(for i in $(seq ${{ strategy.job-total }} -1 0); do echo -n "${UNIQUE_JOB_ID}_${i},"; done | sed 's/,$//')

← 755dd477 jobname · back to Exo · t fb8d8702 →