`README.md`

98 lines
# LA Lawyer Directory — Office Map

Compliance-first PostgreSQL database of **law firm offices in LA County**, focused on
geographic clustering (Beverly Hills, Century City, DTLA, etc.). Powers profile-site
generation, SEO landing pages, directory pages, and CRM enrichment.

## Status

| | |
|---|---|
| DB | `lawyer_professional_directory` on `127.0.0.1:5432` (Unix socket) |
| API | `http://localhost:9701` |
| Dashboard | `http://localhost:9701/` (Leaflet map of every geocoded firm) |

Sources currently wired (free, no auth, robots-clean):
- **OpenStreetMap Overpass API** — every `office=lawyer` / `amenity=lawyer` node in LA County.
- **Wikidata SPARQL** — notable firms with HQ/office in LA County (Latham, Gibson Dunn, Paul Hastings, Manatt, Lewis Brisbois, Littler Mendelson, Irell, etc.).

Sources blocked by anti-bot — see `docs/blocked_sources.md`:
- CA State Bar attorney search (SPA, JS-only render)
- CA SOS bizfile API (recaptcha-gated)

Both are reachable with a Playwright pass — that's the next milestone, not tonight's.

## Run

```bash
npm install                      # one-time
npm run migrate                  # create / update tables (idempotent)
npx tsx src/server/index.ts &    # start API + dashboard on :9701
npx tsx src/scripts/run_overnight.ts   # one-shot import pass + summary
```

Logs land in `logs/overnight-YYYY-MM-DD.log`.

## Schedule it nightly

Pick one:

**cron** (macOS):
```cron
# nightly at 02:00 local
0 2 * * *  cd /Users/stevestudio2/Projects/lawyer-directory-builder && /usr/local/bin/npx tsx src/scripts/run_overnight.ts >> logs/cron.log 2>&1
```

**pm2** (if installed):
```bash
pm2 start npx --name lawyer-server -- tsx src/server/index.ts
pm2 start --cron-restart "0 2 * * *" --no-autorestart \
  npx --name lawyer-overnight -- tsx src/scripts/run_overnight.ts
pm2 save
```

## Useful queries

```sql
-- top neighborhoods by firm count
SELECT COALESCE(neighborhood, city) AS area, COUNT(*) AS firms
FROM organizations WHERE type='law_firm'
GROUP BY area ORDER BY firms DESC LIMIT 20;

-- multi-firm buildings (true clusters)
SELECT address, city, COUNT(*) AS firms,
       ARRAY_AGG(name ORDER BY name) AS firms_at_address
FROM organizations
WHERE type='law_firm' AND address IS NOT NULL
GROUP BY address, city HAVING COUNT(*) > 1
ORDER BY firms DESC;

-- 90067 Century City roster
SELECT name, address, phone, website
FROM organizations
WHERE type='law_firm' AND zip='90067'
ORDER BY name;
```

## API endpoints

| | |
|---|---|
| `GET /api/health` | liveness |
| `GET /api/stats` | totals + coverage |
| `GET /api/firms?city=Beverly+Hills&limit=200` | firms list |
| `GET /api/firms/:id` | one firm |
| `GET /api/cities` | firm count + centroid by area |
| `GET /api/buildings` | multi-firm addresses |
| `GET /api/heatmap` | `{lat,lng,weight}` for map overlay |
| `GET /api/jobs` | last 30 scrape_jobs runs |

## Schema

Mirrored on the `organizations` + `professionals` + `professional_locations`
join model. See `migrations/001_initial_schema.sql` (base) and
`migrations/002_firm_office_columns.sql` (clustering: `neighborhood`, `lat`,
`lng`, `address_norm`, `attorney_count`, `firm_size_band`).

Every fact has a `source_url` traceable in `raw_records`.