`bankrupt-leads/review.md`

33 lines
## Snapshot
- MVP pipeline for bankruptcy asset lead digests from CourtListener/RECAP + local LLM extraction.
- Stack: Python, httpx, psycopg 3, Ollama models, PostgreSQL.
- How it's run: manual CLI scripts; no scheduler/service config observed.
- Status: stale/scaffold — README says first poll not run.

## Top Risks (P0 — fix this week)
- [src/poller.py](/Users/stevestudio2/Projects/bankrupt-leads/src/poller.py:37): only requests first 100 dockets and never follows pagination; busy courts silently lose leads. Fix: loop CourtListener `next` pages until exhausted.
- [src/poller.py](/Users/stevestudio2/Projects/bankrupt-leads/src/poller.py:45): poller inserts cases only; no code populates `filings`, downloads PDFs, or extracts text, so classify/extract/digest have no real input. Fix: implement filing/document ingestion or mark current scripts as nonfunctional scaffold.
- [src/extract.py](/Users/stevestudio2/Projects/bankrupt-leads/src/extract.py:77): extraction is not idempotent; rerunning a filing duplicates assets every time. Fix: add a natural key/unique constraint or delete/upsert assets per `filing_id` in one transaction.
- Secrets/auth/SQL injection: none observed.

## Notable Issues (P1 — fix this month)
- [src/llm.py](/Users/stevestudio2/Projects/bankrupt-leads/src/llm.py:34): assumes Ollama returns valid JSON content; malformed/empty model output crashes the job with no filing-level failure record. Fix: catch parse errors and persist retry/error state.
- [src/classify.py](/Users/stevestudio2/Projects/bankrupt-leads/src/classify.py:70): one missing/unreadable `text_path` aborts the whole batch after prior commits. Fix: handle per-file read/classify errors and continue.
- [src/extract.py](/Users/stevestudio2/Projects/bankrupt-leads/src/extract.py:91): inserts trust arbitrary LLM keys; missing `description` or invalid dates/numerics can fail the run mid-stream. Fix: validate/coerce asset objects before DB writes.
- [src/poller.py](/Users/stevestudio2/Projects/bankrupt-leads/src/poller.py:76): if one court request fails, remaining courts are skipped. Fix: isolate court-level failures and log retryable errors.
- [src/schema.sql](/Users/stevestudio2/Projects/bankrupt-leads/src/schema.sql:27): `filings.case_id` allows null and lacks `ON DELETE` behavior; orphan/invalid workflow states are easy. Fix: make core relationships `NOT NULL` once ingestion shape is known.

## Polish (P2 — when convenient)
- [requirements.txt](/Users/stevestudio2/Projects/bankrupt-leads/requirements.txt:1): dependency ranges are broad and no lockfile exists; reproducibility is weak.
- No tests observed for classifier normalization, extraction validation, or digest filters.
- [README.md](/Users/stevestudio2/Projects/bankrupt-leads/README.md:8): claims extraction goes to Postgres, but operational setup and script order are incomplete.

## Strengths
- Uses parameterized SQL consistently where external values enter queries.
- Local LLM boundary is isolated in [src/llm.py](/Users/stevestudio2/Projects/bankrupt-leads/src/llm.py:17), making model/provider changes straightforward.
- Schema is simple and oriented around the actual business entities: cases, filings, assets, subscribers.

## Open questions for the owner
- Is there unpublished code for PDF download/text extraction, or is this repo the whole pipeline?
- Should duplicate extraction overwrite prior assets, append revisions, or preserve history?
- Is this intended to run manually, from cron, or as a long-running service?