ETL: flag-gated cache push to Kamatera (keeps analytics.agentabrams.com fresh every 30min, no restart)

c1f48fbf719d29c8a295bf854533c6ca6e136e58 · 2026-08-05 07:44:15 -0700 · Steve Abrams

Files touched

M .gitignore
M etl.err
M etl.py

Diff

commit c1f48fbf719d29c8a295bf854533c6ca6e136e58
Author: Steve Abrams <steve@designerwallcoverings.com>
Date:   Wed Aug 5 07:44:15 2026 -0700

    ETL: flag-gated cache push to Kamatera (keeps analytics.agentabrams.com fresh every 30min, no restart)
---
 .gitignore |  1 +
 etl.err    |  2 ++
 etl.py     | 15 +++++++++++++++
 3 files changed, 18 insertions(+)

diff --git a/.gitignore b/.gitignore
index 39aa936..19564c6 100644
--- a/.gitignore
+++ b/.gitignore
@@ -11,3 +11,4 @@ __pycache__/
 *.bak*
 dashboard-preview.png
 preview-*.png
+.push-kamatera
diff --git a/etl.err b/etl.err
index d5d9611..c58d7ed 100644
--- a/etl.err
+++ b/etl.err
@@ -4,3 +4,5 @@ Traceback (most recent call last):
   File "/Users/macstudio3/Projects/ga-allsites/etl.py", line 44, in main
     from google.analytics.admin import AnalyticsAdminServiceClient
 ModuleNotFoundError: No module named 'google'
+I0805 07:44:14.997547 484048221 ev_poll_posix.cc:593] FD from fork parent still in poll list: fd(12, generation: 1)
+I0805 07:44:14.997736 484048221 ev_poll_posix.cc:593] FD from fork parent still in poll list: fd(7, generation: 1)
diff --git a/etl.py b/etl.py
index 908d543..b45833f 100644
--- a/etl.py
+++ b/etl.py
@@ -169,6 +169,21 @@ def main() -> int:
         "rows": crows,
     }, indent=2))
     print(f"Wrote {cpath} · {len(crows)} countries")
+
+    # Freshness push: if the flag file exists, rsync the fresh caches to the public
+    # Kamatera instance (analytics.agentabrams.com). The server reads cache files per
+    # request, so no restart is needed. Flag-gated so local dev runs never push.
+    if (Path(__file__).parent / ".push-kamatera").exists():
+        import subprocess
+        try:
+            r = subprocess.run(
+                ["rsync", "-az", str(CACHE), str(cpath),
+                 "root@45.61.58.125:/root/Projects/ga-allsites/cache/"],
+                capture_output=True, timeout=90)
+            print("Pushed cache → Kamatera" if r.returncode == 0
+                  else f"Kamatera push FAILED: {r.stderr.decode()[:120]}")
+        except Exception as e:
+            print(f"Kamatera push error: {e}")
     return 0

← 531a1ab Add Global tab: animated liquid-fill gauge orbs (zero-dep SV · back to Ga Allsites · Add 'exclude main store' toggle — hides bot-inflated DW stor a252fa0 →