← back to Last30 Skill
last30: multi-source last-30-days research skill (own-stack, no-key)
ea5183dfeb9379b3a7f3b2a86cf770f4975c4bab · 2026-07-13 15:11:44 -0700 · Steve Abrams
Engagement-ranked, recency-filtered research over free no-key APIs
(Hacker News, arXiv, GitHub, Polymarket) with a zero-dependency Python
engine; social platforms (Reddit, X, YouTube) covered via the host's own
web search. Inspired by mvanhorn/last30days-skill (MIT).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Files touched
A .gitignoreA LICENSEA README.mdA skills/last30/SKILL.mdA skills/last30/references/sources.mdA skills/last30/scripts/last30.py
Diff
commit ea5183dfeb9379b3a7f3b2a86cf770f4975c4bab
Author: Steve Abrams <steve@designerwallcoverings.com>
Date: Mon Jul 13 15:11:44 2026 -0700
last30: multi-source last-30-days research skill (own-stack, no-key)
Engagement-ranked, recency-filtered research over free no-key APIs
(Hacker News, arXiv, GitHub, Polymarket) with a zero-dependency Python
engine; social platforms (Reddit, X, YouTube) covered via the host's own
web search. Inspired by mvanhorn/last30days-skill (MIT).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---
.gitignore | 15 +
LICENSE | 23 ++
README.md | 72 +++++
skills/last30/SKILL.md | 130 +++++++++
skills/last30/references/sources.md | 87 ++++++
skills/last30/scripts/last30.py | 531 ++++++++++++++++++++++++++++++++++++
6 files changed, 858 insertions(+)
diff --git a/.gitignore b/.gitignore
new file mode 100644
index 0000000..514709f
--- /dev/null
+++ b/.gitignore
@@ -0,0 +1,15 @@
+# Steve standing .gitignore
+node_modules/
+.env*
+tmp/
+*.log
+.DS_Store
+dist/
+build/
+.next/
+
+# Python
+__pycache__/
+*.pyc
+.venv/
+venv/
diff --git a/LICENSE b/LICENSE
new file mode 100644
index 0000000..ca9b5d4
--- /dev/null
+++ b/LICENSE
@@ -0,0 +1,23 @@
+MIT License
+
+Copyright (c) 2026 Steve Abrams
+
+Portions inspired by mvanhorn/last30days-skill (MIT).
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
diff --git a/README.md b/README.md
new file mode 100644
index 0000000..dd471da
--- /dev/null
+++ b/README.md
@@ -0,0 +1,72 @@
+# /last30
+
+**An agent skill that tells you what people actually said about a topic in the last
+30 days — ranked by real engagement, not editors.**
+
+Runs a zero-dependency Python engine over free, no-key public APIs (Hacker News,
+arXiv, GitHub, Polymarket), layers in social platforms (Reddit, X, YouTube) via the
+host agent's own web search, scores everything by **engagement × recency**, and
+synthesizes one grounded, cited brief.
+
+Inspired by [mvanhorn/last30days-skill](https://github.com/mvanhorn/last30days-skill)
+(MIT), surfaced by [@Saboo_Shubham_](https://x.com/saboo_shubham_) — whose pro tip was
+literally *"use this open-source skill … to make it your own."* This is that: an
+independent, **no-key, $0-local** reimplementation for a host that already has web
+search (e.g. Claude Code + Exa MCP).
+
+## What's different from the original
+
+| | Original `/last30days` | This `/last30` |
+|---|---|---|
+| Social platforms | ScrapeCreators paid key + vendored X client | Host's existing web search (Exa/WebSearch) |
+| Dependencies | Python + Node + yt-dlp + CLIs | **Python stdlib only** |
+| Default cost | PAYG after free tier | **$0** (free APIs + already-present web search) |
+| Engine sources | many | HN · arXiv · GitHub · Polymarket (all no-key) |
+
+## Layout
+
+```
+last30-skill/
+├── README.md
+├── LICENSE # MIT
+└── skills/
+ └── last30/ # the installable skill
+ ├── SKILL.md # runtime contract (intent → engine → synth)
+ ├── scripts/
+ │ └── last30.py # zero-dependency engine + `doctor`
+ └── references/
+ └── sources.md # source matrix, scoring, extending
+```
+
+## Quick start
+
+```bash
+# health-check every free source
+python3 skills/last30/scripts/last30.py doctor
+
+# research a topic (markdown brief)
+python3 skills/last30/scripts/last30.py "nvidia earnings reaction" --emit markdown
+
+# tighter window, more items, JSON for downstream synthesis
+python3 skills/last30/scripts/last30.py "AI video tools" --days 14 --limit 8 --emit json
+```
+
+## Install as a skill
+
+Symlink (or copy) `skills/last30/` into your agent's skills directory:
+
+```bash
+ln -s "$(pwd)/skills/last30" ~/.claude/skills/last30
+```
+
+Then invoke `/last30 <topic>` in your agent.
+
+## Requirements
+
+- Python 3.9+ (standard library only — no `pip install`)
+- A host web-search tool for the social-platform layer (optional but recommended):
+ an Exa MCP connection, or the agent's built-in WebSearch.
+
+## License
+
+MIT — see [LICENSE](LICENSE). Credit to mvanhorn for the original idea.
diff --git a/skills/last30/SKILL.md b/skills/last30/SKILL.md
new file mode 100644
index 0000000..cdf1033
--- /dev/null
+++ b/skills/last30/SKILL.md
@@ -0,0 +1,130 @@
+---
+name: last30
+description: 'Research what people actually said about any topic in the LAST 30 DAYS, ranked by real engagement (upvotes, points, GitHub stars, prediction-market money) and recency, then synthesize one grounded, cited brief. Runs a zero-dependency Python engine over free no-key APIs (Hacker News, arXiv, GitHub, Polymarket) and supplements social platforms (Reddit, X, YouTube) through the host web search. This skill should be used when the user asks what people are saying about a topic, the latest or recent buzz or current state of something, what is trending in a domain, wants a recency-scoped multi-source scan, or invokes /last30. Not for evergreen facts, code in the current repo, or single-site lookups.'
+argument-hint: 'last30 nvidia earnings reaction | last30 AI video tools --days 7 | last30 doctor'
+allowed-tools: Bash, Read, WebSearch, AskUserQuestion
+homepage: https://github.com/mvanhorn/last30days-skill
+license: MIT
+user-invocable: true
+metadata:
+ type: reference
+ inspired-by: mvanhorn/last30days-skill (MIT)
+ engine: scripts/last30.py
+ tags:
+ - research
+ - recency
+ - trends
+ - multi-source
+ - engagement-ranked
+ - hacker-news
+ - arxiv
+ - github
+ - polymarket
+ - reddit
+ - citations
+---
+
+# /last30
+
+## Overview
+
+`/last30` answers **"what have people actually been saying/betting/shipping about
+this topic in the last N days?"** — not an editor's summary, but the real signal,
+ranked by what people engaged with. It runs a deterministic Python engine over
+free no-key APIs, layers in social platforms via the host's web search, and
+synthesizes a single grounded, cited brief.
+
+This is a **specific research contract**, not a keyword to improvise against.
+Always run the engine (Step 2). Do not answer from web search or memory alone.
+
+## When to use
+
+- "What are people saying about `<topic>`?" / "latest on `<topic>`" / "recent buzz"
+- "What's trending in `<domain>`?" / "current state of `<X>`"
+- Any request scoped to **recency** ("this month", "past week", "right now") that
+ benefits from multiple engagement signals converging.
+
+Do **not** use for: evergreen/definitional facts, code in the current repo, or a
+single known URL (use WebFetch for that).
+
+## The pipeline
+
+### Step 0 — Resolve host web search
+Confirm a web-search tool exists this session. Prefer, in order: the **Exa MCP**
+(`mcp__exa__web_search_exa`) if connected, else the built-in **WebSearch** tool.
+This is what covers Reddit/X/YouTube in Step 3. If neither exists, proceed with
+the engine-only sources and say so in the brief.
+
+### Step 1 — Parse intent
+- **Topic run** (default): a subject was named → Steps 2–4.
+- **Doctor**: user said "doctor"/"is it working" → run `python3 scripts/last30.py doctor`
+ and relay verbatim. Stop.
+- Pick the window: default **30 days**; honor "past week" → `--days 7`, etc.
+
+### Step 2 — Run the engine (REQUIRED)
+From the skill directory, run the engine via Bash. Use `--emit markdown` for a
+quick read, or `--emit json` when you want to re-rank/synthesize yourself:
+
+```bash
+python3 scripts/last30.py "<topic>" --days 30 --emit markdown
+```
+
+Common flags:
+- `--days N` — lookback window (default 30)
+- `--sources hn,arxiv,github,polymarket` — default set (all free, no keys)
+- `--limit N` — items per source (default 12)
+- `--emit json|markdown`
+- add `reddit` to `--sources` only on networks where Reddit isn't IP-blocked
+
+The engine is fault-isolated: a source that 429s/403s returns an `error` marker,
+never crashes the run. Relay a **"nothing solid in the window"** result honestly —
+do not pad it with fabricated items.
+
+### Step 3 — Supplement social platforms via host web search
+The engine deliberately omits key-gated / IP-blocked platforms. Fill the gaps
+with the Step-0 tool, scoped to recency and domain. Run these **in parallel**:
+
+- **Reddit** — Exa/WebSearch `allowed_domains: ["reddit.com"]`, query `<topic>`.
+- **X / Twitter** — `allowed_domains: ["x.com","twitter.com","nitter.net"]`.
+- **YouTube** — `allowed_domains: ["youtube.com"]` (look for reaction/deep-dive).
+- Optionally **news/web** — a broad Exa search for the freshest reporting.
+
+Keep only results plausibly inside the window. Capture the engagement signal when
+visible (upvotes, view counts). See `references/sources.md` for the full source
+matrix and the reasoning behind the engine/web-search split.
+
+### Step 4 — Synthesize one grounded brief
+Merge engine output + web-search supplements into a single brief:
+
+1. **Headline** — the 1–2 sentence "here's what matters" (not "here's what I found").
+2. **What people are saying** — 4–8 bullets, each **grounded in a specific item**
+ with its engagement number and an inline citation `([source](url))`.
+3. **Signals & convergence** — where multiple sources agree; note Polymarket odds
+ as *money-backed probability*, not opinion.
+4. **New research / tools** — notable arXiv papers and fast-rising GitHub repos.
+5. **Caveats** — sources that were unavailable this run, and thin windows.
+
+Rank by the engine's `score` (engagement × recency) and cite every claim. For
+long syntheses on Steve's stack, a local Ollama model (hermes3:8b / qwen3:14b)
+can do the merge at **$0** — but the agent doing it directly is fine.
+
+## Cost
+
+Engine + host web search over free sources = **$0 (local/free)**. If the host
+web search is the paid **Exa MCP**, note it: roughly a few cents per run for the
+Step-3 supplements. Surface the cost line per Steve's standing rule.
+
+## Examples
+
+```bash
+python3 scripts/last30.py "nvidia earnings reaction" --emit markdown
+python3 scripts/last30.py "AI video tools" --days 14 --limit 8 --emit markdown
+python3 scripts/last30.py "rust vs zig" --emit json # then synthesize
+python3 scripts/last30.py doctor # health check
+```
+
+## Files
+
+- `scripts/last30.py` — the zero-dependency engine (stdlib only; Python 3.9+).
+- `references/sources.md` — per-source config, the engine/web-search split
+ rationale, optional keyed sources, and scoring details.
diff --git a/skills/last30/references/sources.md b/skills/last30/references/sources.md
new file mode 100644
index 0000000..1350b0b
--- /dev/null
+++ b/skills/last30/references/sources.md
@@ -0,0 +1,87 @@
+# /last30 — source matrix & design notes
+
+Detailed reference for the `/last30` skill. Loaded by the agent only when it needs
+the full source config, the engine/web-search split rationale, or scoring detail.
+Keep procedural instructions in `SKILL.md`; keep depth here.
+
+## The two-layer design
+
+`/last30` splits sources into two layers by what they cost and how they're reached:
+
+| Layer | Sources | Reached by | Why |
+|---|---|---|---|
+| **Engine** (`scripts/last30.py`) | Hacker News, arXiv, GitHub, Polymarket | Free public JSON/Atom APIs, no key | Clean, deterministic, parallel, structured engagement numbers |
+| **Web-search** (agent) | Reddit, X/Twitter, YouTube, TikTok, general news | Host tool (Exa MCP → WebSearch) | These are IP-blocked (Reddit) or key-gated (X/TikTok); the host already indexes them |
+
+The original `mvanhorn/last30days-skill` reaches the second layer with a paid
+**ScrapeCreators** key and a vendored X client. This reimplementation deliberately
+avoids new paid keys — it uses the web-search tool the host **already has** (on
+Steve's stack, the Exa MCP + built-in WebSearch), honoring the standing
+"generation/enrichment runs local/free, paid infra is opt-in" rule.
+
+## Engine sources (free, no key)
+
+| Source | Endpoint | Engagement signal | Notes |
+|---|---|---|---|
+| **Hacker News** | `hn.algolia.com/api/v1/search` | points | `numericFilters=created_at_i>` enforces the window server-side |
+| **arXiv** | `export.arxiv.org/api/query` (Atom) | none (recency-led) | sorted by `submittedDate` desc; window filtered client-side |
+| **GitHub** | `api.github.com/search/repositories` | stars | `pushed:>=<date>` qualifier; unauth = 10 req/min (fine for one run). If the `gh` CLI is authed, higher limits are available but not required. |
+| **Polymarket** | `gamma-api.polymarket.com/public-search` | $ volume + live odds | odds are money-backed probability, not opinion |
+
+All four confirmed working from a clean IP (`scripts/last30.py doctor`).
+
+### Reddit (engine source, opt-in, best-effort)
+Reddit's unauthenticated JSON (`www.reddit.com/search.json`, `/r/<sub>/top.json`)
+returns **403 Blocked** or **429** on many networks — it's an IP/policy block, not
+a User-Agent issue (a desktop-browser UA does not fix it). So Reddit is:
+- **kept in the engine** (`--sources reddit,...`) for networks where it works;
+- **off by default** (not in `DEFAULT_SOURCES`) so a default run is all-green;
+- a **WARN, never a failure** in `doctor`;
+- **covered by the web-search layer** in `SKILL.md` Step 3 regardless.
+
+If you need Reddit's real upvote/comment structure and the engine is blocked, the
+web-search supplement (Exa/WebSearch scoped to `reddit.com`) is the fallback.
+
+## Optional keyed sources (documented, not required)
+
+Not wired into this engine — enable only if the user explicitly wants them and
+accepts the cost. Add as a new fetcher following the fault-isolated pattern
+(return `{"source","items"}` or `{"source","error"}`; never raise past the boundary):
+
+| Platform | Key / access | Cost |
+|---|---|---|
+| X / Twitter | `XAI_API_KEY` (xAI) or browser cookies (`AUTH_TOKEN`+`CT0`) | key = PAYG; cookies = free but account-risk |
+| YouTube transcripts | `yt-dlp` CLI on PATH | free |
+| TikTok / Instagram | ScrapeCreators key | 10k free calls, then PAYG |
+| Bluesky | app password from bsky.app | free |
+
+Cookie-scraping X carries account-suspension risk and is off by design.
+
+## Scoring model (`score_items`)
+
+Each item gets a normalized `score` in **0–100**:
+
+1. **Per-source engagement normalization** — divide by that source's max, so
+ 50k Reddit upvotes and 5k HN points are comparable within the run.
+2. **Recency weight** — linear decay from `1.0` (now) to `~0.15` at the window edge.
+3. **Blend** — engagement-led sources: `score = 100·(0.7·eng_norm + 0.3·recency)`.
+ Discovery sources with no engagement signal (arXiv, thin Polymarket): recency-only.
+
+`top_across()` flattens every source and returns the global top-N by `score` —
+that's the "Top across all sources" block the brief leads with.
+
+## Extending
+
+- **New free source** → add a `fetch_<name>()` returning the standard shape, add it
+ to `FETCHERS`, and (if it should run by default) to `DEFAULT_SOURCES`.
+- **Pin a fixed source set** → pass `--sources` explicitly on every call.
+- **Faster runs** → `--no-comments` (skips Reddit comment enrichment) and a lower
+ `--limit`.
+
+## Lineage / credit
+
+Idea and command shape from **[@mvanhorn](https://github.com/mvanhorn/last30days-skill)**'s
+`/last30days` (MIT), surfaced by [@Saboo_Shubham_](https://x.com/saboo_shubham_) — whose
+own pro tip was "use this open-source skill … to make it your own." This is that:
+an independent, no-key, $0-local reimplementation tuned to a host that already has
+web search. MIT-licensed in kind.
diff --git a/skills/last30/scripts/last30.py b/skills/last30/scripts/last30.py
new file mode 100755
index 0000000..5e94a87
--- /dev/null
+++ b/skills/last30/scripts/last30.py
@@ -0,0 +1,531 @@
+#!/usr/bin/env python3
+"""
+last30 — engagement-ranked, recency-filtered multi-source research engine.
+
+Researches what people actually said about a topic in the last N days across
+FREE, no-key public APIs, scores each item by engagement x recency, and emits
+a structured, cited result set for an agent to synthesize.
+
+Zero third-party dependencies — Python 3.9+ standard library only. This is the
+deterministic core of the /last30 skill; the SKILL.md contract decides intent,
+runs this engine, and writes the human brief.
+
+Sources (all free, no API key):
+ reddit Reddit search + top comments (upvotes)
+ hn Hacker News via Algolia (points)
+ arxiv arXiv Atom API (recency)
+ github GitHub repo search (stars)
+ polymarket Polymarket Gamma public-search (odds / $ volume)
+
+Usage:
+ last30.py "<topic>" research the topic (all free sources)
+ last30.py "<topic>" --days 7 narrow the window to 7 days
+ last30.py "<topic>" --sources reddit,hn only these sources
+ last30.py "<topic>" --emit markdown human-readable instead of JSON
+ last30.py "<topic>" --no-comments skip Reddit comment fetching (faster)
+ last30.py doctor health-check every source
+
+Inspired by mvanhorn/last30days-skill (MIT). This is an independent, no-key,
+$0-local reimplementation — see references/sources.md for the lineage.
+"""
+
+from __future__ import annotations
+
+import argparse
+import concurrent.futures
+import json
+import re
+import sys
+import time
+import urllib.parse
+import urllib.request
+import xml.etree.ElementTree as ET
+from datetime import datetime, timezone
+
+# Reddit blocklists descriptive scraper UAs (403 Blocked), so present as a
+# standard desktop browser. From a residential IP this passes the public
+# JSON endpoints for every source below.
+USER_AGENT = (
+ "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) "
+ "AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
+)
+# Engine default = the clean no-key JSON APIs that work from any IP.
+# Reddit is available (--sources reddit,...) but ships OFF by default: its
+# unauthenticated JSON is IP-blocked on many networks (403 Blocked), so the
+# SKILL.md contract covers Reddit + other social platforms via the agent's
+# own host web-search tools instead. See references/sources.md.
+DEFAULT_SOURCES = ["hn", "arxiv", "github", "polymarket"]
+ALL_SOURCES = ["reddit", "hn", "arxiv", "github", "polymarket"]
+HTTP_TIMEOUT = 20 # seconds per request
+
+
+# --------------------------------------------------------------------------- #
+# HTTP helpers #
+# --------------------------------------------------------------------------- #
+def _get(url: str, headers: dict | None = None, timeout: int = HTTP_TIMEOUT) -> bytes:
+ """GET a URL with a browser-ish UA. Raises on network/HTTP error."""
+ req = urllib.request.Request(url, headers={"User-Agent": USER_AGENT, **(headers or {})})
+ with urllib.request.urlopen(req, timeout=timeout) as resp:
+ return resp.read()
+
+
+def _get_json(url: str, headers: dict | None = None, timeout: int = HTTP_TIMEOUT):
+ return json.loads(_get(url, headers, timeout).decode("utf-8", "replace"))
+
+
+def _now() -> datetime:
+ return datetime.now(timezone.utc)
+
+
+def _since_ts(days: int) -> int:
+ return int(_now().timestamp()) - days * 86400
+
+
+def _age_days(ts: float) -> float:
+ return max(0.0, (_now().timestamp() - ts) / 86400.0)
+
+
+def _recency_weight(age_days: float, window_days: int) -> float:
+ """Linear decay from 1.0 (now) to ~0.15 at the edge of the window."""
+ if window_days <= 0:
+ return 1.0
+ return max(0.15, 1.0 - 0.85 * (age_days / window_days))
+
+
+def _clip(text: str, n: int = 280) -> str:
+ text = re.sub(r"\s+", " ", (text or "")).strip()
+ return text if len(text) <= n else text[: n - 1] + "…"
+
+
+# --------------------------------------------------------------------------- #
+# Source fetchers — each returns {"source", "items": [...]} or {"error": ...} #
+# Every fetcher is fault-isolated: it must never raise past its own boundary. #
+# --------------------------------------------------------------------------- #
+def fetch_reddit(topic: str, days: int, limit: int, comments: bool) -> dict:
+ """Reddit search (top posts in window) + a couple of top comments each."""
+ try:
+ q = urllib.parse.quote(topic)
+ window = "week" if days <= 7 else ("month" if days <= 31 else "year")
+ url = (
+ f"https://www.reddit.com/search.json?q={q}"
+ f"&sort=top&t={window}&limit={min(limit * 2, 50)}"
+ )
+ data = _get_json(url)
+ cutoff = _since_ts(days)
+ items = []
+ for child in data.get("data", {}).get("children", []):
+ p = child.get("data", {})
+ created = p.get("created_utc", 0)
+ if created < cutoff:
+ continue
+ ups = int(p.get("ups", 0) or 0)
+ permalink = "https://www.reddit.com" + p.get("permalink", "")
+ item = {
+ "title": _clip(p.get("title", ""), 200),
+ "url": permalink,
+ "external_url": p.get("url_overridden_by_dest") or p.get("url"),
+ "subreddit": "r/" + p.get("subreddit", ""),
+ "engagement": ups,
+ "engagement_label": f"{ups:,} upvotes",
+ "comment_count": int(p.get("num_comments", 0) or 0),
+ "created_utc": created,
+ "age_days": round(_age_days(created), 1),
+ "snippet": _clip(p.get("selftext", ""), 300),
+ "top_comments": [],
+ }
+ items.append(item)
+ items.sort(key=lambda x: x["engagement"], reverse=True)
+ items = items[:limit]
+
+ if comments:
+ for it in items[: min(len(items), 6)]: # only enrich the strongest
+ try:
+ c_url = it["url"].rstrip("/") + ".json?limit=3&sort=top"
+ cdata = _get_json(c_url)
+ if len(cdata) > 1:
+ for cc in cdata[1].get("data", {}).get("children", [])[:2]:
+ cd = cc.get("data", {})
+ body = cd.get("body")
+ if body:
+ it["top_comments"].append(
+ {
+ "text": _clip(body, 240),
+ "upvotes": int(cd.get("ups", 0) or 0),
+ }
+ )
+ except Exception:
+ pass # comment enrichment is best-effort
+ return {"source": "reddit", "items": items}
+ except Exception as e:
+ hint = ""
+ if "403" in str(e):
+ hint = " (Reddit blocks unauthenticated JSON on many IPs — " \
+ "the agent should cover Reddit via host web search instead)"
+ return {"source": "reddit", "error": f"{type(e).__name__}: {e}{hint}"}
+
+
+def fetch_hn(topic: str, days: int, limit: int) -> dict:
+ """Hacker News stories in the window, ranked by points, via Algolia."""
+ try:
+ q = urllib.parse.quote(topic)
+ cutoff = _since_ts(days)
+ url = (
+ f"https://hn.algolia.com/api/v1/search?query={q}"
+ f"&tags=story&numericFilters=created_at_i>{cutoff}&hitsPerPage={limit}"
+ )
+ data = _get_json(url)
+ items = []
+ for h in data.get("hits", []):
+ created = h.get("created_at_i", 0) or 0
+ points = int(h.get("points", 0) or 0)
+ obj_id = h.get("objectID")
+ items.append(
+ {
+ "title": _clip(h.get("title") or h.get("story_title") or "", 200),
+ "url": h.get("url") or f"https://news.ycombinator.com/item?id={obj_id}",
+ "discussion_url": f"https://news.ycombinator.com/item?id={obj_id}",
+ "engagement": points,
+ "engagement_label": f"{points:,} points",
+ "comment_count": int(h.get("num_comments", 0) or 0),
+ "author": h.get("author"),
+ "created_utc": created,
+ "age_days": round(_age_days(created), 1),
+ "snippet": _clip(h.get("story_text") or "", 300),
+ }
+ )
+ items.sort(key=lambda x: x["engagement"], reverse=True)
+ return {"source": "hn", "items": items[:limit]}
+ except Exception as e:
+ return {"source": "hn", "error": f"{type(e).__name__}: {e}"}
+
+
+def fetch_arxiv(topic: str, days: int, limit: int) -> dict:
+ """Recent arXiv papers matching the topic (Atom API)."""
+ try:
+ q = urllib.parse.quote(f"all:{topic}")
+ url = (
+ f"http://export.arxiv.org/api/query?search_query={q}"
+ f"&sortBy=submittedDate&sortOrder=descending&max_results={limit * 2}"
+ )
+ raw = _get(url)
+ ns = {"a": "http://www.w3.org/2005/Atom"}
+ root = ET.fromstring(raw)
+ cutoff = _since_ts(days)
+ items = []
+ for entry in root.findall("a:entry", ns):
+ pub = entry.findtext("a:published", default="", namespaces=ns)
+ try:
+ ts = datetime.strptime(pub, "%Y-%m-%dT%H:%M:%SZ").replace(
+ tzinfo=timezone.utc
+ ).timestamp()
+ except ValueError:
+ continue
+ if ts < cutoff:
+ continue
+ title = entry.findtext("a:title", default="", namespaces=ns)
+ link = entry.findtext("a:id", default="", namespaces=ns)
+ summary = entry.findtext("a:summary", default="", namespaces=ns)
+ authors = [
+ a.findtext("a:name", default="", namespaces=ns)
+ for a in entry.findall("a:author", ns)
+ ]
+ items.append(
+ {
+ "title": _clip(title, 220),
+ "url": link,
+ "engagement": 0,
+ "engagement_label": "new paper",
+ "authors": authors[:4],
+ "created_utc": ts,
+ "age_days": round(_age_days(ts), 1),
+ "snippet": _clip(summary, 320),
+ }
+ )
+ items.sort(key=lambda x: x["created_utc"], reverse=True)
+ return {"source": "arxiv", "items": items[:limit]}
+ except Exception as e:
+ return {"source": "arxiv", "error": f"{type(e).__name__}: {e}"}
+
+
+def fetch_github(topic: str, days: int, limit: int) -> dict:
+ """GitHub repos pushed within the window, ranked by stars (unauth search)."""
+ try:
+ since = datetime.fromtimestamp(_since_ts(days), timezone.utc).strftime("%Y-%m-%d")
+ q = urllib.parse.quote(f"{topic} pushed:>={since}")
+ url = (
+ f"https://api.github.com/search/repositories?q={q}"
+ f"&sort=stars&order=desc&per_page={limit}"
+ )
+ data = _get_json(url, headers={"Accept": "application/vnd.github+json"})
+ items = []
+ for r in data.get("items", []):
+ stars = int(r.get("stargazers_count", 0) or 0)
+ pushed = r.get("pushed_at", "")
+ try:
+ ts = datetime.strptime(pushed, "%Y-%m-%dT%H:%M:%SZ").replace(
+ tzinfo=timezone.utc
+ ).timestamp()
+ except ValueError:
+ ts = _now().timestamp()
+ items.append(
+ {
+ "title": _clip(r.get("full_name", ""), 120),
+ "url": r.get("html_url"),
+ "engagement": stars,
+ "engagement_label": f"{stars:,} stars",
+ "language": r.get("language"),
+ "created_utc": ts,
+ "age_days": round(_age_days(ts), 1),
+ "snippet": _clip(r.get("description") or "", 280),
+ }
+ )
+ return {"source": "github", "items": items[:limit]}
+ except Exception as e:
+ return {"source": "github", "error": f"{type(e).__name__}: {e}"}
+
+
+def fetch_polymarket(topic: str, days: int, limit: int) -> dict:
+ """Active Polymarket markets matching the topic — odds backed by real money."""
+ try:
+ q = urllib.parse.quote(topic)
+ url = (
+ f"https://gamma-api.polymarket.com/public-search?q={q}"
+ f"&limit_per_type={limit}&events_status=active"
+ )
+ data = _get_json(url)
+ events = data.get("events", []) if isinstance(data, dict) else []
+ items = []
+ for ev in events[:limit]:
+ vol = float(ev.get("volume", 0) or 0)
+ markets = ev.get("markets", []) or []
+ odds = []
+ for m in markets[:4]:
+ outcome = m.get("groupItemTitle") or m.get("question") or ""
+ price = m.get("outcomePrices") or m.get("lastTradePrice")
+ if isinstance(price, str):
+ try:
+ price = json.loads(price)
+ except Exception:
+ price = None
+ if isinstance(price, list) and price:
+ try:
+ odds.append(f"{outcome}: {round(float(price[0]) * 100)}%")
+ except Exception:
+ pass
+ elif isinstance(price, (int, float)):
+ odds.append(f"{outcome}: {round(float(price) * 100)}%")
+ items.append(
+ {
+ "title": _clip(ev.get("title", ""), 200),
+ "url": "https://polymarket.com/event/" + (ev.get("slug", "")),
+ "engagement": int(vol),
+ "engagement_label": f"${vol:,.0f} volume",
+ "odds": odds,
+ "created_utc": _now().timestamp(),
+ "age_days": 0.0,
+ "snippet": _clip(ev.get("description") or "", 240),
+ }
+ )
+ items.sort(key=lambda x: x["engagement"], reverse=True)
+ return {"source": "polymarket", "items": items[:limit]}
+ except Exception as e:
+ return {"source": "polymarket", "error": f"{type(e).__name__}: {e}"}
+
+
+FETCHERS = {
+ "reddit": lambda t, d, l, c: fetch_reddit(t, d, l, c),
+ "hn": lambda t, d, l, c: fetch_hn(t, d, l),
+ "arxiv": lambda t, d, l, c: fetch_arxiv(t, d, l),
+ "github": lambda t, d, l, c: fetch_github(t, d, l),
+ "polymarket": lambda t, d, l, c: fetch_polymarket(t, d, l),
+}
+
+
+# --------------------------------------------------------------------------- #
+# Cross-source scoring #
+# --------------------------------------------------------------------------- #
+def score_items(results: list[dict], window_days: int) -> None:
+ """Attach a normalized 0-100 `score` to every item in-place.
+
+ Per source we normalize engagement to its own max (so 5k HN points and
+ 50k reddit upvotes are comparable), then multiply by a recency weight.
+ arXiv/polymarket carry engagement of 0/volume — recency dominates there.
+ """
+ for r in results:
+ items = r.get("items", [])
+ if not items:
+ continue
+ peak = max((it.get("engagement", 0) for it in items), default=0) or 1
+ for it in items:
+ eng_norm = (it.get("engagement", 0) / peak) if peak else 0.0
+ rec = _recency_weight(it.get("age_days", 0.0), window_days)
+ # engagement-led sources weight engagement 0.7 / recency 0.3;
+ # discovery sources (no engagement signal) lean fully on recency.
+ if peak > 1:
+ score = 100 * (0.7 * eng_norm + 0.3 * rec)
+ else:
+ score = 100 * rec
+ it["score"] = round(score, 1)
+
+
+def top_across(results: list[dict], n: int) -> list[dict]:
+ """Flatten every source's items and return the global top-N by score."""
+ flat = []
+ for r in results:
+ for it in r.get("items", []):
+ flat.append({**it, "source": r["source"]})
+ flat.sort(key=lambda x: x.get("score", 0), reverse=True)
+ return flat[:n]
+
+
+# --------------------------------------------------------------------------- #
+# Emitters #
+# --------------------------------------------------------------------------- #
+def emit_json(topic, days, sources, results, elapsed):
+ scored_sources = {}
+ for r in results:
+ if "error" in r:
+ scored_sources[r["source"]] = {"error": r["error"], "items": []}
+ else:
+ scored_sources[r["source"]] = {"count": len(r["items"]), "items": r["items"]}
+ payload = {
+ "topic": topic,
+ "window_days": days,
+ "generated_at": _now().isoformat(),
+ "elapsed_seconds": round(elapsed, 1),
+ "sources_requested": sources,
+ "sources": scored_sources,
+ "top_overall": top_across(results, 15),
+ }
+ print(json.dumps(payload, indent=2, ensure_ascii=False))
+
+
+def emit_markdown(topic, days, sources, results, elapsed):
+ out = [f"# /last30 — “{topic}” (last {days} days)", ""]
+ out.append(f"_generated {_now().strftime('%Y-%m-%d %H:%M UTC')} · {elapsed:.1f}s · "
+ f"sources: {', '.join(sources)}_")
+ out.append("")
+ top = top_across(results, 12)
+ if top:
+ out.append("## Top across all sources")
+ out.append("")
+ for i, it in enumerate(top, 1):
+ out.append(
+ f"{i}. **[{it['source']}·{it.get('score',0)}]** "
+ f"[{it['title']}]({it.get('url','')}) — "
+ f"{it.get('engagement_label','')}, {it.get('age_days','?')}d ago"
+ )
+ if it.get("odds"):
+ out.append(f" - odds: {', '.join(it['odds'])}")
+ if it.get("snippet"):
+ out.append(f" - {it['snippet']}")
+ out.append("")
+ for r in results:
+ out.append(f"## {r['source']}")
+ if "error" in r:
+ out.append(f"> ⚠️ unavailable — {r['error']}")
+ out.append("")
+ continue
+ if not r["items"]:
+ out.append("> (nothing in the window)")
+ out.append("")
+ continue
+ for it in r["items"]:
+ out.append(
+ f"- [{it['title']}]({it.get('url','')}) · "
+ f"{it.get('engagement_label','')} · {it.get('age_days','?')}d"
+ )
+ for c in it.get("top_comments", []):
+ out.append(f" - 💬 ({c['upvotes']}▲) {c['text']}")
+ out.append("")
+ print("\n".join(out))
+
+
+# --------------------------------------------------------------------------- #
+# Doctor #
+# --------------------------------------------------------------------------- #
+def doctor():
+ print("last30 doctor — probing free sources (no keys required)\n")
+ probes = {
+ "reddit": lambda: fetch_reddit("openai", 30, 3, False),
+ "hn": lambda: fetch_hn("openai", 30, 3),
+ "arxiv": lambda: fetch_arxiv("language model", 30, 3),
+ "github": lambda: fetch_github("agent", 30, 3),
+ "polymarket": lambda: fetch_polymarket("election", 90, 3),
+ }
+ hard_fail = False
+ for name, fn in probes.items():
+ t0 = time.time()
+ r = fn()
+ dt = time.time() - t0
+ if "error" in r:
+ # Reddit is opt-in/best-effort (IP-blocked on many networks) — its
+ # 403/429 is expected and never fails the doctor; the agent covers
+ # Reddit via host web search instead.
+ if name == "reddit":
+ code = "403" if "403" in r["error"] else ("429" if "429" in r["error"] else "err")
+ print(f" ⚠ {name:11s} {dt:5.1f}s blocked ({code}) — "
+ f"agent covers Reddit via host web search")
+ else:
+ hard_fail = True
+ print(f" ✗ {name:11s} {dt:5.1f}s {r['error']}")
+ else:
+ print(f" ✓ {name:11s} {dt:5.1f}s {len(r['items'])} items")
+ print("\nAll default sources healthy." if not hard_fail else
+ "\nA default source failed — likely a transient rate-limit; retry in a minute.")
+ return 0 if not hard_fail else 1
+
+
+# --------------------------------------------------------------------------- #
+# CLI #
+# --------------------------------------------------------------------------- #
+def main() -> int:
+ ap = argparse.ArgumentParser(
+ prog="last30",
+ description="Engagement-ranked, recency-filtered multi-source research.",
+ )
+ ap.add_argument("topic", nargs="?", help='research topic, or "doctor"')
+ ap.add_argument("--days", type=int, default=30, help="lookback window (default 30)")
+ ap.add_argument("--sources", default=",".join(DEFAULT_SOURCES),
+ help="comma-separated: " + ",".join(DEFAULT_SOURCES))
+ ap.add_argument("--limit", type=int, default=12, help="items per source (default 12)")
+ ap.add_argument("--emit", choices=["json", "markdown"], default="json")
+ ap.add_argument("--no-comments", action="store_true",
+ help="skip Reddit comment enrichment (faster)")
+ args = ap.parse_args()
+
+ if not args.topic or args.topic == "doctor":
+ return doctor() if args.topic == "doctor" else (ap.print_help() or 1)
+
+ sources = [s.strip() for s in args.sources.split(",") if s.strip() in FETCHERS]
+ if not sources:
+ print(f"No valid sources in '{args.sources}'. Valid: {list(FETCHERS)}",
+ file=sys.stderr)
+ return 1
+
+ t0 = time.time()
+ results: list[dict] = []
+ with concurrent.futures.ThreadPoolExecutor(max_workers=len(sources)) as ex:
+ futs = {
+ ex.submit(FETCHERS[s], args.topic, args.days, args.limit,
+ not args.no_comments): s
+ for s in sources
+ }
+ by_source = {}
+ for fut in concurrent.futures.as_completed(futs):
+ r = fut.result()
+ by_source[r["source"]] = r
+ results = [by_source[s] for s in sources if s in by_source]
+
+ score_items(results, args.days)
+ elapsed = time.time() - t0
+
+ if args.emit == "markdown":
+ emit_markdown(args.topic, args.days, sources, results, elapsed)
+ else:
+ emit_json(args.topic, args.days, sources, results, elapsed)
+ return 0
+
+
+if __name__ == "__main__":
+ sys.exit(main())
(oldest)
·
back to Last30 Skill
·
last30: add relevance gate (topic-token coverage) to kill ke 248dc83 →