`README.md`

90 lines
# LABJ CRE + Banking Public-Source Dataset

A research dataset of Los Angeles commercial-real-estate, developer, property-management,
construction, architecture, and banking/finance companies — built as a **prospect/advertiser
database for the RE builds** (RENTV, usrealestate, CRCP, HomesOnSpec).

Ticket: **TK-10411**. Owner area: `re-props`.

## What this is (and is not)

The LA Business Journal's "The Lists" are copyright-gated and sold as a paid Excel product.
This project uses LABJ **only as a discovery/ranking pointer** — every company record is
**rebuilt from independent public sources** (each firm's official website, press/media pages,
public directories) with source attribution. **No LABJ paid product is reproduced**, and
**no field is fabricated** — a value that couldn't be publicly verified is left blank.

## Data flow

```
data/seed.csv                 15 LABJ public-seed rows (top-3 of 5 lists)
   +
data/enriched/*.json          researched public company records, one file per lane:
   brokerage · developers · property_mgmt · banks · contractors_architects
        │
        ▼   scripts/build.py  (stdlib only — no pandas)
   normalize → dedupe by domain → score RENTV-advertiser-fit → emit outputs
```

Run it:

```bash
python3 scripts/build.py
```

## Outputs

| File | What |
|------|------|
| `data/labj_master.csv` / `.json` | the full deduped, scored dataset (**RE builds consume the `.json`**) |
| `data/labj_companies.csv` | firmographic subset (identity/address/site/score) |
| `data/labj_contacts.csv` | rows that have a real, verified public contact |
| `data/labj_list_memberships.csv` | company × LABJ-list join (preserves multi-list membership) |
| `data/research_queue.csv` | records still missing key fields (next-pass work list) |
| `reports/top_100_rentv_prospects.csv` | ranked advertiser targets |
| `reports/source_audit.csv` | per-record provenance (LABJ pointer + company sources) |
| `reports/domain_check.csv` | live HTTP reachability check of every company domain |

## RENTV advertiser-fit score (0–100)

Per-company, from the prompt's rubric:

| Points | Signal |
|--------|--------|
| +25 | CRE core business (brokerage / developer / property-mgmt / GC / architecture) |
| +20 | active LA presence |
| +15 | a verified marketing/PR contact or public email exists |
| +15 | appears on more than one relevant list |
| +10 | lender/bank serving CRE |
| +10 | a named public transaction/development/portfolio footprint |
| +5  | public advertising/sponsorship evidence |

## Coverage (current build)

- **99** deduped companies across **17** categories · **124** list memberships · **63** with a verified public contact · **15** multi-list firms.
- Coverage now spans the **full target category list**: CRE brokerage, CRE + residential developers, property management (office/retail/industrial), banks + financial institutions + credit unions, general contractors + architecture, **SBA 7(a)/504 lenders, securities brokerages, private equity, and venture capital** (v2 lane).
- A **prospect viewer** ships in `public/index.html` (loads `public/labj_master.json`): search-all, sortable +
  drag-reorder + toggle + resize columns, density slider, category filter chips, expandable rows, every datum
  drillable, choices persisted to `localStorage`. Serve it with the `/viewer` skill (basic-auth admin/DW2024!).
- Lanes built: CRE brokerage, CRE + residential developers, property management (office/retail/industrial),
  banks + financial institutions + credit unions, general contractors + architecture firms.
- **Fit-scoring note:** PE / VC / securities firms score lower by design (not CRE-core, not CRE lenders) —
  they're in the DB for completeness. The proptech-focused VCs (Fifth Wall, TenOneTen) are the standout
  RENTV-relevant names in that lane, flagged in `la_presence`.

## Known limitations

- **Rankings are approximate.** LABJ's exact rank order sits behind their paywall; `labj_rank` is
  populated only where a defensible *public* ranking basis exists, else blank.
- **Marketing/PR contacts are thin by design.** Most of these firms don't publish a dedicated media
  contact; the never-fabricate rule means those cells stay blank rather than guessed. A cross-company
  duplicated-name guard (in the build verify step) nulls any contact name that appears at >1 firm.
- **`domain_check.csv` `403`/`405` = reachable** (the host answered but blocks bot HEAD requests), not dead.

## Refresh process

1. Re-run the 5 lane research passes (fresh public data) → overwrite `data/enriched/*.json`.
2. `python3 scripts/build.py` regenerates every output deterministically.
3. Re-run the domain reachability check and the duplicated-contact guard.
4. Commit.