← back to La Permits School
LADBS+SOS free permit puller (school project): stdlib Socrata pull, contractor/applicant/principal names, APN, CA SOS lookup URL
5f33d188b27dcc7864025be95559f34f949cb722 · 2026-08-10 12:58:59 -0700 · Steve Abrams
Files touched
A .gitignoreA README.mdA pull_permits.py
Diff
commit 5f33d188b27dcc7864025be95559f34f949cb722
Author: Steve Abrams <steve@designerwallcoverings.com>
Date: Mon Aug 10 12:58:59 2026 -0700
LADBS+SOS free permit puller (school project): stdlib Socrata pull, contractor/applicant/principal names, APN, CA SOS lookup URL
---
.gitignore | 6 ++
README.md | 54 ++++++++++++++++++
pull_permits.py | 168 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 228 insertions(+)
diff --git a/.gitignore b/.gitignore
new file mode 100644
index 0000000..b3c69ee
--- /dev/null
+++ b/.gitignore
@@ -0,0 +1,6 @@
+*.csv
+__pycache__/
+*.pyc
+.DS_Store
+.env*
+*.log
diff --git a/README.md b/README.md
new file mode 100644
index 0000000..61491f6
--- /dev/null
+++ b/README.md
@@ -0,0 +1,54 @@
+# LA Permits — free data puller (school project)
+
+A $0, no-API-key tool that pulls **City of Los Angeles building permits** from the
+public LA Open Data Portal and writes a clean CSV — including **contractor,
+applicant, and principal names**, project **valuation**, address, **APN**, and a
+ready-made **California Secretary of State** lookup URL per contractor.
+
+Two free public data sources, no signup, no cost:
+
+| Source | What it gives | Access |
+|---|---|---|
+| **LADBS building permits** (`data.lacity.org`, dataset `d9aa-v8bm`) | Permit type, valuation, address, APN, **contractor / applicant / principal names**, license, stories, dwelling units | Socrata API — free, no key |
+| **CA Secretary of State** (`bizfileonline.sos.ca.gov`) | Legal entity + agent for service of process behind a business name | Free web lookup (the tool generates the URL per contractor) |
+
+## Quick start
+No install needed — Python 3 standard library only.
+
+```bash
+python3 pull_permits.py # new builds + additions > $1M -> permits.csv
+python3 pull_permits.py --min-valuation 5000000 # only projects over $5M
+python3 pull_permits.py --permit-types "Bldg-New" --since 2020-01-01
+python3 pull_permits.py --limit 500 --out sample.csv
+```
+
+### Options
+- `--min-valuation N` — minimum project $ (default `1000000`; `0` = no filter)
+- `--permit-types "A,B"` — e.g. `Bldg-New`, `Bldg-Addition`, `Bldg-Alter/Repair`,
+ `Bldg-Demolition`, `Sign` (`""` = all types)
+- `--since YYYY-MM-DD` — only permits issued on/after this date
+- `--limit N` — cap rows written (`0` = all matches)
+- `--out FILE.csv` — output path
+- `--app-token TOKEN` — optional free Socrata token (only if you hit rate limits)
+
+## The pipeline
+1. **LADBS** (this tool) → bulk CSV of significant permits with the party names.
+2. **CA SOS** (free lookup) → open any row's `ca_sos_lookup_url` to resolve the
+ contractor's legal entity + agent of record. Join on the business name (or use
+ the `apn` column to tie back to the county assessor).
+
+## Data notes
+- `d9aa-v8bm` is a **stable historical snapshot: 2013-01 → 2023-05, ~317k permits**.
+ Great for a reproducible course project. (The *live* daily permit feed exists but
+ the name-bearing fields are login-gated; the open live feed `pi9x-tg5x` omits
+ contractor/applicant names.)
+- `apn` is the full LA County parcel id (book-page-parcel), e.g. `4129-027-902`.
+- Real examples in a default run: the **$554M LAX addition** (Hensel Phelps) and
+ the **$476M tower at 100 S Grand Ave** (Tishman).
+
+## Compliance
+Both sources are **public / open data**, free to reuse. This project uses only
+public records — it does **not** touch paid or license-restricted databases
+(e.g. Data Axle / Reference Solutions library edition, which is academic-use-only
+and must not be redistributed). If this work is ever used beyond coursework,
+re-check terms before republishing compiled lists.
diff --git a/pull_permits.py b/pull_permits.py
new file mode 100644
index 0000000..3648c17
--- /dev/null
+++ b/pull_permits.py
@@ -0,0 +1,168 @@
+#!/usr/bin/env python3
+"""
+pull_permits.py — free, $0, no-key LADBS building-permit puller (school project).
+
+Pulls City of Los Angeles "Building Permit Information Data" from the LA Open Data
+Portal (Socrata API), filters to significant permits (by type + valuation), and
+writes a clean CSV — including contractor / applicant / principal names and a
+ready-made California Secretary of State lookup URL per contractor for enrichment.
+
+Data source (public, free, no signup):
+ https://data.lacity.org/resource/d9aa-v8bm.json (2013-01 .. 2023-05, ~317k rows)
+
+Standard library only — no pip installs, no API key required. A free Socrata
+"app token" only raises rate limits; not needed for this volume.
+
+Examples:
+ python3 pull_permits.py # new + additions over $1M -> permits.csv
+ python3 pull_permits.py --min-valuation 5000000 # only >$5M projects
+ python3 pull_permits.py --permit-types "Bldg-New" --since 2020-01-01
+ python3 pull_permits.py --limit 500 --out sample.csv
+"""
+
+import argparse
+import csv
+import sys
+import time
+import urllib.parse
+import urllib.request
+
+DATASET_DEFAULT = "d9aa-v8bm" # richest PUBLIC dataset (names + valuation)
+BASE = "https://data.lacity.org/resource/{dataset}.json"
+PAGE = 50000 # Socrata max rows per request
+CA_SOS_SEARCH = "https://bizfileonline.sos.ca.gov/search/business"
+
+# Columns kept in the output CSV, in order. (dataset field -> stays same name)
+OUT_FIELDS = [
+ "pcis_permit", "issue_date", "permit_type", "permit_sub_type", "permit_category",
+ "valuation", "work_description",
+ "address", "zip_code", "zone", "council_district",
+ "of_stories", "of_residential_dwelling_units", "apn",
+ "contractors_business_name", "license", "license_type",
+ "contractor_city", "contractor_state",
+ "applicant_first_name", "applicant_last_name", "applicant_relationship",
+ "principal_first_name", "principal_middle_name", "principal_last_name",
+ "latest_status", "status_date",
+ "ca_sos_lookup_url",
+]
+
+
+def build_where(min_valuation, permit_types, since):
+ clauses = []
+ if min_valuation is not None:
+ clauses.append(f"valuation > {int(min_valuation)}")
+ if permit_types:
+ quoted = ",".join("'" + t.replace("'", "''") + "'" for t in permit_types)
+ clauses.append(f"permit_type in({quoted})")
+ if since:
+ clauses.append(f"issue_date >= '{since}T00:00:00'")
+ return " and ".join(clauses)
+
+
+def fetch_page(dataset, where, offset, app_token=None):
+ params = {
+ "$where": where,
+ "$order": "valuation DESC",
+ "$limit": PAGE,
+ "$offset": offset,
+ }
+ if app_token:
+ params["$$app_token"] = app_token
+ url = BASE.format(dataset=dataset) + "?" + urllib.parse.urlencode(params)
+ req = urllib.request.Request(url, headers={"Accept": "application/json"})
+ import json
+ for attempt in range(4):
+ try:
+ with urllib.request.urlopen(req, timeout=60) as r:
+ return json.load(r)
+ except Exception as e: # transient network / rate limit -> back off
+ if attempt == 3:
+ print(f" ! giving up on offset {offset}: {e}", file=sys.stderr)
+ return []
+ time.sleep(2 * (attempt + 1))
+ return []
+
+
+def address_of(row):
+ parts = [
+ row.get("address_start", ""),
+ row.get("street_direction", ""),
+ row.get("street_name", ""),
+ row.get("street_suffix", ""),
+ ]
+ return " ".join(p for p in parts if p).strip()
+
+
+def sos_url(business_name):
+ if not business_name:
+ return ""
+ return CA_SOS_SEARCH + "?" + urllib.parse.urlencode({"searchType": "BUSINESS", "q": business_name})
+
+
+def apn_of(row):
+ # Full LA County APN = book (4) + page (3) + parcel (3), e.g. 5144-021-024
+ b, p, parcel = row.get("assessor_book", ""), row.get("assessor_page", ""), row.get("assessor_parcel", "")
+ if b and p and parcel:
+ return f"{b}-{p}-{parcel}"
+ return "-".join(x for x in (b, p, parcel) if x)
+
+
+def shape(row):
+ derived = ("address", "ca_sos_lookup_url", "apn")
+ out = {k: row.get(k, "") for k in OUT_FIELDS if k not in derived}
+ out["address"] = address_of(row)
+ out["apn"] = apn_of(row)
+ out["ca_sos_lookup_url"] = sos_url(row.get("contractors_business_name", ""))
+ return out
+
+
+def main():
+ ap = argparse.ArgumentParser(description="Free LADBS permit puller (school project).")
+ ap.add_argument("--dataset", default=DATASET_DEFAULT, help="Socrata dataset id (default d9aa-v8bm)")
+ ap.add_argument("--min-valuation", type=int, default=1_000_000, help="min project valuation $ (default 1000000; 0 = no filter)")
+ ap.add_argument("--permit-types", default="Bldg-New,Bldg-Addition",
+ help="comma list; '' = all. Options: Bldg-New, Bldg-Addition, Bldg-Alter/Repair, Bldg-Demolition, Sign, ...")
+ ap.add_argument("--since", default="", help="only permits issued on/after YYYY-MM-DD")
+ ap.add_argument("--limit", type=int, default=0, help="max rows to write (0 = all matches)")
+ ap.add_argument("--out", default="permits.csv", help="output CSV path")
+ ap.add_argument("--app-token", default="", help="optional free Socrata app token (raises rate limit)")
+ args = ap.parse_args()
+
+ permit_types = [t.strip() for t in args.permit_types.split(",") if t.strip()]
+ min_val = args.min_valuation if args.min_valuation > 0 else None
+ where = build_where(min_val, permit_types, args.since) or "valuation IS NOT NULL"
+
+ print("LADBS permit puller — cost: $0 (public LA Open Data API, no key)")
+ print(f" dataset : {args.dataset}")
+ print(f" filter : {where}")
+ print(f" output : {args.out}")
+
+ written = 0
+ offset = 0
+ with open(args.out, "w", newline="", encoding="utf-8") as f:
+ w = csv.DictWriter(f, fieldnames=OUT_FIELDS)
+ w.writeheader()
+ while True:
+ rows = fetch_page(args.dataset, where, offset, args.app_token or None)
+ if not rows:
+ break
+ for row in rows:
+ w.writerow(shape(row))
+ written += 1
+ if args.limit and written >= args.limit:
+ break
+ print(f" ... {written} rows", flush=True)
+ if args.limit and written >= args.limit:
+ break
+ if len(rows) < PAGE:
+ break
+ offset += PAGE
+ time.sleep(0.3) # be polite to the free API
+
+ print(f"\nDone. Wrote {written} permits to {args.out}")
+ print("Next step (free): open the CSV, and for any contractor, click its")
+ print("ca_sos_lookup_url to pull the legal entity + agent from CA SOS bizfile.")
+
+
+if __name__ == "__main__":
+ main()
(oldest)
·
back to La Permits School
·
Add enrich_contractors.py: dedupe permits to unique contract 4e1bc46 →