ingest pipeline: ict-cst + local-portals + meic-pymes + google-places + run-all; ict-cst yielded 9 entities (page is AJAX-rendered, needs Browserbase upgrade for full ~130)

e517d11c82f2c83fc15e04422c1f28615893dfb4 · 2026-05-07 23:33:16 -0700 · Steve
Files touched

M .env.example
A PLAN.md
A scripts/ingest/_lib.js
A scripts/ingest/google-places.js
A scripts/ingest/ict-cst.js
A scripts/ingest/local-portals.js
A scripts/ingest/meic-pymes.js
A scripts/ingest/run-all.js
Diff

commit e517d11c82f2c83fc15e04422c1f28615893dfb4
Author: Steve <steve@designerwallcoverings.com>
Date:   Thu May 7 23:33:16 2026 -0700

    ingest pipeline: ict-cst + local-portals + meic-pymes + google-places + run-all; ict-cst yielded 9 entities (page is AJAX-rendered, needs Browserbase upgrade for full ~130)
---
 .env.example                    |   5 +-
 PLAN.md                         |  98 +++++++++++++++++++++++++++++++++
 scripts/ingest/_lib.js          |  83 ++++++++++++++++++++++++++++
 scripts/ingest/google-places.js | 104 +++++++++++++++++++++++++++++++++++
 scripts/ingest/ict-cst.js       | 117 ++++++++++++++++++++++++++++++++++++++++
 scripts/ingest/local-portals.js |  84 +++++++++++++++++++++++++++++
 scripts/ingest/meic-pymes.js    | 112 ++++++++++++++++++++++++++++++++++++++
 scripts/ingest/run-all.js       |  32 +++++++++++
 8 files changed, 634 insertions(+), 1 deletion(-)

diff --git a/.env.example b/.env.example
index d461296..2afb3f4 100644
--- a/.env.example
+++ b/.env.example
@@ -1,5 +1,8 @@
 PORT=9791
-DATABASE_URL=postgresql://stevestudio2@localhost:5432/costa_rica_directory
+DATABASE_URL=postgresql:///costa_rica_directory?host=/tmp&user=stevestudio2
+PGHOST=/tmp
+PGUSER=stevestudio2
+PGDATABASE=costa_rica_directory
 BASIC_AUTH_USER=costarica@agentabrams.com
 BASIC_AUTH_PASS=__set_in_local_env__
 SITE_NAME=Costa Rica Directory
diff --git a/PLAN.md b/PLAN.md
new file mode 100644
index 0000000..5e924ab
--- /dev/null
+++ b/PLAN.md
@@ -0,0 +1,98 @@
+# Costa Rica Directory — Plan
+
+**What:** A multi-vertical Costa Rica business directory — tourism + rentals + local services — across all 84 cantones, in the Ventura-Corridor / Ventura-Claw pattern.
+
+**Where:** `~/Projects/costa-rica/` · pm2 process `costa-rica` on `:9791` · standalone PG database `costa_rica_directory` · gated behind HTTP Basic Auth `costarica@agentabrams.com` (password in `secrets-manager` as `COSTARICA_AGENTABRAMS_PASSWORD`).
+
+**Public domain (planned):** `costarica.agentabrams.com` — DNS not yet pointed; deploy step not yet run.
+
+---
+
+## 1. Verticals (16 total)
+
+| Category | Verticals |
+|---|---|
+| **tourism** | hotel · tour · beach · restaurant · surf |
+| **rentals** | short_term · long_term · realestate |
+| **service** | food · beauty · retail · fitness · pet · auto · cleaning · creative |
+
+These mirror the Ventura-Claw 8-vertical green list for `service`, plus tourism + rentals on top.
+
+## 2. Regions (33 seeded)
+
+San José · Escazú · Santa Ana · Alajuela · Atenas · Grecia · La Fortuna/Arenal · Heredia · Cartago · Liberia · Tamarindo · Nosara · Santa Teresa · Montezuma · Jacó · Manuel Antonio · Quepos · Uvita · Dominical · Puerto Viejo · Cahuita · Monteverde · Santa Elena · Limón · Puntarenas · Papagayo · Playas del Coco · Flamingo · Sámara · Nicoya · plus 3 catch-all province buckets.
+
+---
+
+## 3. Data sources (ranked by usefulness, per EXA research 2026-05-07)
+
+### 3.1 Government (authoritative)
+| Source | Type | Bulk? | Use |
+|---|---|---|---|
+| **`api.hacienda.go.cr/fe/ae?identificacion={cedula}`** | Lookup-by-cédula | No (rate-limited) | Per-cédula enrichment (situación tributaria + actividades económicas) |
+| **MEIC PYMEs Activas (datos.go.cr CKAN)** | XLSX | Yes | National PYME-registered firms list |
+| **ICT CST (turismo-sostenible.co.cr)** | HTML scrape | ~130 entities | Tourism vertical seed (hotels, tours, rent-a-car, gastro, spa) |
+| **Heredia CKAN — patentes por actividad/distrito** | XLSX | Yes | Heredia cantón ground-truth (pattern for other cantones) |
+| **Registro Nacional — Personas Jurídicas** | Search-only | No | Cédula format validation (`3-XXX-XXXXXX`); no bulk export |
+| **CGR (Contraloría) RDF/JSON/CSV** | Bulk | Yes (procurement-only) | Government vendors list |
+
+### 3.2 Local / web directories (broad coverage)
+- **Google Places API** — Nearby Search around each region's lat/lng (key already in routes.json for the Ad-Signals Rolodex)
+- **TripAdvisor** — by town/category (Browserbase headless)
+- **Booking.com** — accommodations (Browserbase)
+- **Airbnb** — short-term rentals (existing CR pattern from `jill-website` is reference, not source)
+- **Yelp Costa Rica** — local services
+- **Tico Times classifieds** — `ticotimes.net/classifieds`
+- **AM Costa Rica classifieds** — `amcostarica.com`
+- **Town-specific portals** — `nosaranow.com`, `tamarindo.com`, `quepos-info.com`, `monteverdeinfo.com`, etc.
+
+### 3.3 Per-cantón patente CKAN (where available)
+- Heredia ✅ CKAN
+- Most others (Santa Cruz/Tamarindo, Nicoya/Nosara, Aguirre/Manuel Antonio, Garabito/Jacó, Talamanca/Puerto Viejo, San Carlos/La Fortuna, Cóbano/Santa Teresa) require per-cantón check; many are PDF/search-only.
+- **Fallback:** Ley 8968 / Ley 8220 transparency request to each municipalidad for patente list.
+
+---
+
+## 4. Ingest pipeline (planned scripts)
+
+```
+scripts/ingest/
+  _lib.js              # PG upsert, slugify, fetch helpers, region resolver
+  ict-cst.js           # ICT sustainable-tourism scrape (~130 rows)
+  meic-pymes.js        # CKAN download + parse XLSX
+  hacienda-enrich.js   # Per-cédula enrichment via api.hacienda.go.cr
+  heredia-patentes.js  # Heredia CKAN download
+  google-places.js     # Nearby Search per region centroid
+  tripadvisor-cr.js    # Browserbase scrape per town
+  booking-cr.js        # Browserbase scrape per town
+  local-portals.js     # nosaranow / tamarindo / monteverdeinfo / ticotimes / etc
+  run-all.js           # orchestrator → ingest_runs table
+```
+
+Each writes to `places` (UPSERT on slug), bumps `ingest_runs`, and respects per-source backoff. None run inside the Express process — they execute via launchd.
+
+## 5. Schedule (planned launchd)
+
+| Job | Frequency | Source |
+|---|---|---|
+| `com.steve.cr.ict-cst` | weekly Sun 02:00 | ICT CST |
+| `com.steve.cr.meic-pymes` | monthly 1st 03:00 | MEIC XLSX |
+| `com.steve.cr.google-places` | weekly Mon 03:00 | Google Places |
+| `com.steve.cr.local-portals` | daily 04:00 | Web directories |
+| `com.steve.cr.hacienda-enrich` | nightly 05:00 | Hacienda (only for `places` w/o `cedula_juridica` → backfill) |
+
+## 6. Compliance / safety rules
+
+- **No headless browser locally** unless Browserbase is unavailable (per MEMORY rule `feedback_browserbase_first`).
+- **Respect robots.txt** for every web-directory scrape; rate-limit at 1 req / 2s minimum.
+- **No phone-number exfiltration for SMS / DNC scrub**: phone numbers ingested are display-only, not for outreach.
+- **No CR DNC list exists nationally** (TCPA is US-only); CR has Ley 8968 (privacy) — display public-record info only, never email-blast scraped owners.
+- **Image rule** (per `feedback_no_stock_images`): no Unsplash/Pexels stock; only photos directly from the source listing or public-domain (Wikimedia, Wikicommons CR, etc.).
+- **info@ rule**: contact email is `info@agentabrams.com` until `info@<final-domain>` is provisioned.
+
+## 7. Open decisions for Steve
+
+1. Final domain — `costarica.agentabrams.com` (subdomain, fastest) vs. a registered `.cr` / `.com` (longer; needs TLD work).
+2. Public vs. gated — is the basic-auth gate temporary (drop when populated) or permanent (admin-only product)?
+3. Is `jill-website` (nosarabeachfrontrentals.com) supposed to xref into the Nosara region's `rentals_short` listings, or stay completely independent? (Currently independent per Steve's instruction.)
+4. Lead routing — same Stripe-backed `service_*` subscription pattern as Ventura-Claw, or skipped for v1?
diff --git a/scripts/ingest/_lib.js b/scripts/ingest/_lib.js
new file mode 100644
index 0000000..578fa9d
--- /dev/null
+++ b/scripts/ingest/_lib.js
@@ -0,0 +1,83 @@
+'use strict';
+
+require('dotenv').config({ path: require('path').join(__dirname, '..', '..', '.env') });
+const { Pool } = require('pg');
+
+const pool = new Pool({ connectionString: process.env.DATABASE_URL });
+
+const UA = process.env.SCRAPER_UA || 'Mozilla/5.0 (Macintosh; Intel Mac OS X 14_5) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.5 Safari/605.1.15 (CR-Directory bot; contact info@agentabrams.com)';
+
+async function fetchText(url, opts = {}) {
+  const res = await fetch(url, {
+    headers: { 'User-Agent': UA, 'Accept': 'text/html,application/xhtml+xml,application/json;q=0.9,*/*;q=0.8', 'Accept-Language': 'es-CR,es;q=0.9,en;q=0.7', ...(opts.headers||{}) },
+    redirect: 'follow',
+    signal: opts.signal,
+  });
+  if (!res.ok) throw new Error(`HTTP ${res.status} on ${url}`);
+  return await res.text();
+}
+
+async function fetchJson(url, opts = {}) {
+  const res = await fetch(url, { headers: { 'User-Agent': UA, 'Accept': 'application/json', ...(opts.headers||{}) }, redirect: 'follow', signal: opts.signal });
+  if (!res.ok) throw new Error(`HTTP ${res.status} on ${url}`);
+  return await res.json();
+}
+
+const slug = (s, maxLen = 96) => String(s||'')
+  .normalize('NFD').replace(/[̀-ͯ]/g,'')
+  .toLowerCase()
+  .replace(/&/g,' and ')
+  .replace(/[^a-z0-9]+/g,'-')
+  .replace(/^-+|-+$/g,'')
+  .slice(0, maxLen);
+
+const sleep = ms => new Promise(r => setTimeout(r, ms));
+
+async function regionMap() {
+  const { rows } = await pool.query('SELECT id, slug, name, lat, lng FROM regions');
+  const bySlug = new Map(rows.map(r => [r.slug, r]));
+  const byName = new Map(rows.map(r => [r.name.toLowerCase(), r]));
+  return { rows, bySlug, byName, fallback: bySlug.get('cr-other') };
+}
+
+function resolveRegion(rmap, hint) {
+  if (!hint) return rmap.fallback;
+  const h = String(hint).toLowerCase().trim();
+  if (rmap.bySlug.has(h)) return rmap.bySlug.get(h);
+  if (rmap.byName.has(h)) return rmap.byName.get(h);
+  for (const r of rmap.rows) {
+    if (h.includes(r.name.toLowerCase()) || h.includes(r.slug.replace(/-/g,' '))) return r;
+  }
+  return rmap.fallback;
+}
+
+async function upsertPlace(p) {
+  const cols = ['slug','name','category','vertical','region_id','description','address','phone','email','website','price_range','rating','image_url','tags','lat','lng','cedula_juridica','source','source_url'];
+  const vals = cols.map(c => p[c] ?? null);
+  const placeholders = cols.map((_, i) => `$${i+1}`).join(',');
+  const updates = cols.filter(c => c!=='slug').map(c => `${c} = COALESCE(EXCLUDED.${c}, places.${c})`).join(', ');
+  const sql = `
+    INSERT INTO places (${cols.join(',')}) VALUES (${placeholders})
+    ON CONFLICT (slug) DO UPDATE SET ${updates}, updated_at = NOW()
+    RETURNING (xmax = 0) AS inserted, id
+  `;
+  const { rows } = await pool.query(sql, vals);
+  return rows[0]; // { inserted: bool, id }
+}
+
+async function startRun(source, notes='') {
+  const { rows } = await pool.query(
+    `INSERT INTO ingest_runs (source, status, notes) VALUES ($1, 'running', $2) RETURNING id`,
+    [source, notes]
+  );
+  return rows[0].id;
+}
+
+async function finishRun(id, { rows_in=0, rows_added=0, rows_updated=0, status='ok', notes='' } = {}) {
+  await pool.query(
+    `UPDATE ingest_runs SET finished_at=NOW(), rows_in=$2, rows_added=$3, rows_updated=$4, status=$5, notes=COALESCE($6, notes) WHERE id=$1`,
+    [id, rows_in, rows_added, rows_updated, status, notes]
+  );
+}
+
+module.exports = { pool, UA, fetchText, fetchJson, slug, sleep, regionMap, resolveRegion, upsertPlace, startRun, finishRun };
diff --git a/scripts/ingest/google-places.js b/scripts/ingest/google-places.js
new file mode 100644
index 0000000..5c0d35b
--- /dev/null
+++ b/scripts/ingest/google-places.js
@@ -0,0 +1,104 @@
+'use strict';
+
+// Google Places Nearby Search — sweep each region centroid for businesses.
+// Reads GOOGLE_PLACES_API_KEY from env; if missing, exits with notes.
+
+const { fetchJson, slug, sleep, regionMap, upsertPlace, startRun, finishRun, pool } = require('./_lib');
+
+const KEY = process.env.GOOGLE_PLACES_API_KEY;
+const RADIUS_M = 5000;
+
+const TYPE_TO_VERTICAL = {
+  lodging:           ['tourism', 'tourism_hotel'],
+  restaurant:        ['tourism', 'tourism_restaurant'],
+  cafe:              ['service', 'service_food'],
+  bakery:            ['service', 'service_food'],
+  bar:               ['service', 'service_food'],
+  meal_takeaway:     ['service', 'service_food'],
+  spa:               ['service', 'service_beauty'],
+  beauty_salon:      ['service', 'service_beauty'],
+  hair_care:         ['service', 'service_beauty'],
+  gym:               ['service', 'service_fitness'],
+  pet_store:         ['service', 'service_pet'],
+  veterinary_care:   ['service', 'service_pet'],
+  car_repair:        ['service', 'service_auto'],
+  car_wash:          ['service', 'service_auto'],
+  car_rental:        ['tourism', 'tourism_tour'],
+  travel_agency:     ['tourism', 'tourism_tour'],
+  tourist_attraction:['tourism', 'tourism_tour'],
+  shopping_mall:     ['service', 'service_retail'],
+  store:             ['service', 'service_retail'],
+  real_estate_agency:['rentals', 'rentals_realestate'],
+};
+
+const TYPES = Object.keys(TYPE_TO_VERTICAL);
+
+async function nearby(lat, lng, type, pageToken = null) {
+  const url = new URL('https://maps.googleapis.com/maps/api/place/nearbysearch/json');
+  url.searchParams.set('location', `${lat},${lng}`);
+  url.searchParams.set('radius', RADIUS_M);
+  url.searchParams.set('type', type);
+  url.searchParams.set('key', KEY);
+  if (pageToken) url.searchParams.set('pagetoken', pageToken);
+  return await fetchJson(url.toString());
+}
+
+(async () => {
+  if (!KEY) { console.warn('[google-places] GOOGLE_PLACES_API_KEY missing — skipping'); process.exit(0); }
+  const runId = await startRun('google-places', 'Sweep CR regions via Places Nearby Search');
+  const rmap = await regionMap();
+  let total = 0, added = 0, updated = 0, errors = 0;
+
+  try {
+    const targets = rmap.rows.filter(r => r.lat && r.lng && r.region_type !== 'region');
+    for (const region of targets) {
+      for (const type of TYPES) {
+        try {
+          let pageToken = null, pages = 0;
+          do {
+            const res = await nearby(region.lat, region.lng, type, pageToken);
+            if (res.status !== 'OK' && res.status !== 'ZERO_RESULTS') {
+              console.warn(`[google-places] ${region.slug}/${type} status=${res.status}`);
+              break;
+            }
+            const [category, vertical] = TYPE_TO_VERTICAL[type];
+            for (const p of (res.results || [])) {
+              const name = p.name; if (!name) continue;
+              const placeSlug = slug(`gp-${p.place_id}`);
+              const r = await upsertPlace({
+                slug: placeSlug,
+                name,
+                category, vertical,
+                region_id: region.id,
+                description: `${vertical.replace(/_/g,' ')} in ${region.name}.`,
+                address: p.vicinity || null,
+                lat: p.geometry?.location?.lat ?? null,
+                lng: p.geometry?.location?.lng ?? null,
+                rating: p.rating ?? null,
+                source: 'google-places',
+                source_url: `https://www.google.com/maps/place/?q=place_id:${p.place_id}`,
+              });
+              if (r.inserted) added++; else updated++;
+              total++;
+            }
+            pageToken = res.next_page_token || null;
+            pages++;
+            if (pageToken) await sleep(2200); // Google needs ~2s before next_page_token is valid
+          } while (pageToken && pages < 3);
+        } catch (e) {
+          errors++;
+          console.warn(`[google-places] ${region.slug}/${type} err:`, e.message);
+        }
+        await sleep(120);
+      }
+    }
+    await finishRun(runId, { rows_in: total, rows_added: added, rows_updated: updated, status: errors ? 'partial' : 'ok', notes: `errors=${errors}` });
+    console.log(`[google-places] in=${total} added=${added} updated=${updated} errors=${errors}`);
+  } catch (e) {
+    await finishRun(runId, { status: 'error', notes: e.message });
+    console.error('[google-places] FAIL', e.message);
+    process.exitCode = 1;
+  } finally {
+    await pool.end();
+  }
+})();
diff --git a/scripts/ingest/ict-cst.js b/scripts/ingest/ict-cst.js
new file mode 100644
index 0000000..d2f70b0
--- /dev/null
+++ b/scripts/ingest/ict-cst.js
@@ -0,0 +1,117 @@
+'use strict';
+
+// ICT CST — Certificación para la Sostenibilidad Turística
+// Public directory of CR-government-certified sustainable-tourism businesses.
+// Source: https://turismo-sostenible.co.cr/entidades-certificadas/
+
+const { fetchText, slug, sleep, regionMap, resolveRegion, upsertPlace, startRun, finishRun, pool } = require('./_lib');
+
+const ROOT = 'https://turismo-sostenible.co.cr/entidades-certificadas/';
+
+const VERTICAL_BY_CATEGORY = {
+  'hospedaje':           'tourism_hotel',
+  'alojamiento':         'tourism_hotel',
+  'tour operadores':     'tourism_tour',
+  'tour-operadores':     'tourism_tour',
+  'tour operador':       'tourism_tour',
+  'agencia de viajes':   'tourism_tour',
+  'gastronomia':         'tourism_restaurant',
+  'gastronomía':         'tourism_restaurant',
+  'restaurante':         'tourism_restaurant',
+  'transporte':          'tourism_tour',
+  'rent a car':          'tourism_tour',
+  'spa':                 'service_beauty',
+  'wellness':            'service_beauty',
+};
+
+function pickVertical(category) {
+  const c = (category||'').toLowerCase().trim();
+  for (const [k,v] of Object.entries(VERTICAL_BY_CATEGORY)) {
+    if (c.includes(k)) return v;
+  }
+  return 'tourism_hotel';
+}
+
+function extractListings(html) {
+  // Conservative regex parse; the page renders entity cards as anchor tags.
+  // We avoid heavy DOM parsing to keep the script dep-light.
+  const out = [];
+  const cardRe = /<article[^>]*class="[^"]*entidad[^"]*"[^>]*>([\s\S]*?)<\/article>/gi;
+  let m;
+  while ((m = cardRe.exec(html)) !== null) {
+    const block = m[1];
+    const name      = (block.match(/<h2[^>]*>\s*([^<]+?)\s*<\/h2>/i)||[])[1] || (block.match(/<h3[^>]*>\s*([^<]+?)\s*<\/h3>/i)||[])[1];
+    const province  = (block.match(/Provincia[^:]*:\s*<\/?[a-z]+>?\s*([^<]+?)\s*</i)||[])[1];
+    const canton    = (block.match(/Cant[oó]n[^:]*:\s*<\/?[a-z]+>?\s*([^<]+?)\s*</i)||[])[1];
+    const distrito  = (block.match(/Distrito[^:]*:\s*<\/?[a-z]+>?\s*([^<]+?)\s*</i)||[])[1];
+    const categoria = (block.match(/Categor[ií]a[^:]*:\s*<\/?[a-z]+>?\s*([^<]+?)\s*</i)||[])[1];
+    const link      = (block.match(/<a[^>]+href="([^"]+)"/i)||[])[1];
+    const img       = (block.match(/<img[^>]+src="([^"]+)"/i)||[])[1];
+    if (name) {
+      out.push({
+        name: name.trim(),
+        category: 'tourism',
+        vertical: pickVertical(categoria),
+        province: (province||'').trim(),
+        canton: (canton||'').trim(),
+        distrito: (distrito||'').trim(),
+        source: 'ict-cst',
+        source_url: link ? new URL(link, ROOT).toString() : ROOT,
+        image_url: img ? new URL(img, ROOT).toString() : null,
+        address: [distrito, canton, province].filter(Boolean).join(', '),
+      });
+    }
+  }
+  return out;
+}
+
+(async () => {
+  const runId = await startRun('ict-cst', 'Scrape ICT CST sustainable-tourism directory');
+  const rmap = await regionMap();
+  let html, listings = [], inserted = 0, updated = 0;
+
+  try {
+    html = await fetchText(ROOT);
+    listings = extractListings(html);
+    if (!listings.length) {
+      // Fallback: many WP sites use AJAX; if direct HTML had no <article class=entidad>, log markup hint
+      const hint = html.match(/class="[^"]*entidad[^"]*"/i)?.[0] || '(no entidad class found)';
+      console.warn(`[ict-cst] no listings parsed from root HTML; hint=${hint}`);
+    }
+
+    for (const l of listings) {
+      const region = resolveRegion(rmap, l.canton || l.distrito || l.province);
+      const placeSlug = slug(`ict-${l.name}-${region.slug}`);
+      const r = await upsertPlace({
+        slug: placeSlug,
+        name: l.name,
+        category: 'tourism',
+        vertical: l.vertical,
+        region_id: region.id,
+        description: `ICT CST-certified ${l.vertical.replace('tourism_','')} in ${l.canton||l.province||'Costa Rica'}.`,
+        address: l.address,
+        image_url: l.image_url,
+        source: l.source,
+        source_url: l.source_url,
+      });
+      if (r.inserted) inserted++; else updated++;
+      await sleep(50);
+    }
+
+    await finishRun(runId, {
+      rows_in: listings.length,
+      rows_added: inserted,
+      rows_updated: updated,
+      status: listings.length ? 'ok' : 'empty',
+      notes: listings.length ? '' : 'parser found no <article class=entidad> blocks; ICT page likely AJAX-rendered, switch to Browserbase'
+    });
+
+    console.log(`[ict-cst] in=${listings.length} added=${inserted} updated=${updated}`);
+  } catch (e) {
+    await finishRun(runId, { status: 'error', notes: e.message });
+    console.error('[ict-cst] FAIL', e.message);
+    process.exitCode = 1;
+  } finally {
+    await pool.end();
+  }
+})();
diff --git a/scripts/ingest/local-portals.js b/scripts/ingest/local-portals.js
new file mode 100644
index 0000000..a5a4d55
--- /dev/null
+++ b/scripts/ingest/local-portals.js
@@ -0,0 +1,84 @@
+'use strict';
+
+// Local town-portal scrapers. Each portal has its own listing layout; this
+// runner is a generic "fetch + extract anchor names" pass that tags origin.
+// Browserbase-required portals (TripAdvisor, Booking, Yelp) live in their
+// own scripts; this one targets static-HTML town directories.
+
+const { fetchText, slug, sleep, regionMap, resolveRegion, upsertPlace, startRun, finishRun, pool } = require('./_lib');
+
+const PORTALS = [
+  // { url, region_hint, vertical_hint, name_selector_regex }
+  { url: 'https://nosaranow.com/business-directory/', region: 'nosara',        vertical: 'service_retail', label: 'nosaranow' },
+  { url: 'https://www.tamarindo.com/directory/',      region: 'tamarindo',     vertical: 'service_retail', label: 'tamarindo-com' },
+  { url: 'https://monteverdeinfo.com/businesses/',    region: 'monteverde',    vertical: 'service_retail', label: 'monteverdeinfo' },
+  { url: 'https://www.quepos.com/business-directory', region: 'quepos',        vertical: 'service_retail', label: 'quepos-com' },
+  { url: 'https://thehowlermag.com/category/directory/', region: 'tamarindo',  vertical: 'service_retail', label: 'howler-tamarindo' },
+];
+
+function extractAnchors(html) {
+  const out = [];
+  const re = /<a[^>]+href="([^"]+)"[^>]*>([^<]{2,80})<\/a>/gi;
+  let m;
+  while ((m = re.exec(html)) !== null) {
+    const href = m[1].trim();
+    const text = m[2].replace(/\s+/g,' ').trim();
+    if (!text) continue;
+    if (/(^#|mailto:|tel:|javascript:|^\/$|^\/$|\bhome\b|\bcontact\b|\babout\b|\blogin\b|\bregister\b)/i.test(href + ' ' + text)) continue;
+    if (/^\d+$/.test(text) || text.length < 3) continue;
+    out.push({ href, text });
+  }
+  return out;
+}
+
+(async () => {
+  const runId = await startRun('local-portals', 'Static-HTML CR town-portal directory scrape');
+  const rmap = await regionMap();
+  let total = 0, added = 0, updated = 0, ok = 0, fail = 0;
+
+  for (const p of PORTALS) {
+    try {
+      const html = await fetchText(p.url);
+      const anchors = extractAnchors(html);
+      const seen = new Set();
+      const region = resolveRegion(rmap, p.region);
+      // Heuristic: keep only anchors whose href looks like a deeper listing page on the same domain
+      const host = new URL(p.url).host;
+      const candidates = anchors.filter(a => {
+        try {
+          const u = new URL(a.href, p.url);
+          if (u.host !== host) return false;
+          if (u.pathname === '/' || u.pathname === new URL(p.url).pathname) return false;
+          return /(business|directory|listing|company|empresa|negocio|hotel|tour|restaurant|salon|spa|fitness|surf|yoga|store|tienda)/i.test(u.pathname + ' ' + a.text);
+        } catch { return false; }
+      });
+
+      for (const c of candidates.slice(0, 200)) {
+        const placeSlug = slug(`${p.label}-${c.text}-${region.slug}`);
+        if (seen.has(placeSlug)) continue; seen.add(placeSlug);
+        const r = await upsertPlace({
+          slug: placeSlug,
+          name: c.text,
+          category: p.vertical.startsWith('tourism_') ? 'tourism' : 'service',
+          vertical: p.vertical,
+          region_id: region.id,
+          description: `Listed on ${p.label} (${region.name}).`,
+          source: p.label,
+          source_url: new URL(c.href, p.url).toString(),
+        });
+        if (r.inserted) added++; else updated++;
+        total++;
+      }
+      ok++;
+      console.log(`[local-portals] ${p.label} in=${candidates.length}`);
+      await sleep(800);
+    } catch (e) {
+      fail++;
+      console.warn(`[local-portals] ${p.label} err:`, e.message);
+    }
+  }
+
+  await finishRun(runId, { rows_in: total, rows_added: added, rows_updated: updated, status: fail ? 'partial' : 'ok', notes: `portals_ok=${ok} portals_fail=${fail}` });
+  console.log(`[local-portals] DONE in=${total} added=${added} updated=${updated} portals_ok=${ok} portals_fail=${fail}`);
+  await pool.end();
+})();
diff --git a/scripts/ingest/meic-pymes.js b/scripts/ingest/meic-pymes.js
new file mode 100644
index 0000000..3af1a46
--- /dev/null
+++ b/scripts/ingest/meic-pymes.js
@@ -0,0 +1,112 @@
+'use strict';
+
+// MEIC PYMEs Activas — bulk national list of registered SMEs.
+// Source: datos.go.cr CKAN dataset "Empresas Activas" (XLSX).
+// Discovers latest CSV/XLSX resource via CKAN API and ingests rows.
+
+const { fetchJson, fetchText, slug, regionMap, resolveRegion, upsertPlace, startRun, finishRun, pool } = require('./_lib');
+const fs = require('fs');
+const path = require('path');
+
+const CKAN_BASE = process.env.CKAN_BASE || 'https://datosabiertos.presidencia.go.cr/api/3/action';
+const SEARCH_QS = 'q=pymes+activas+OR+empresas+activas&rows=10';
+const CACHE_DIR = path.join(__dirname, '..', '..', 'data', 'cache');
+
+function findResource(pkg) {
+  const resources = pkg.result?.resources || [];
+  const csv = resources.find(r => /csv$/i.test(r.format||r.url||''));
+  const xlsx = resources.find(r => /xlsx?$/i.test(r.format||r.url||''));
+  return csv || xlsx || resources[0];
+}
+
+(async () => {
+  const runId = await startRun('meic-pymes', 'Fetch MEIC PYMEs Activas via datos.go.cr CKAN');
+  let added = 0, updated = 0, total = 0;
+  try {
+    fs.mkdirSync(CACHE_DIR, { recursive: true });
+
+    const search = await fetchJson(`${CKAN_BASE}/package_search?${SEARCH_QS}`);
+    const pkgs = search.result?.results || [];
+    if (!pkgs.length) throw new Error('no PYMEs package found in datos.go.cr CKAN search');
+
+    const pkg = pkgs[0];
+    const pkgFull = await fetchJson(`${CKAN_BASE}/package_show?id=${encodeURIComponent(pkg.id)}`);
+    const res = findResource(pkgFull);
+    if (!res?.url) throw new Error('no downloadable resource on package');
+
+    console.log(`[meic-pymes] package=${pkg.title} resource=${res.format} url=${res.url}`);
+
+    const cachePath = path.join(CACHE_DIR, `meic-pymes${path.extname(res.url)||'.csv'}`);
+    const body = await fetchText(res.url);
+    fs.writeFileSync(cachePath, body);
+    console.log(`[meic-pymes] cached → ${cachePath} (${(body.length/1024).toFixed(1)} KB)`);
+
+    if (!/csv$/i.test(res.format||res.url||'')) {
+      await finishRun(runId, { status: 'partial', notes: `XLSX cached; CSV-parsing path only — XLSX requires xlsx pkg (not in deps)` });
+      console.log('[meic-pymes] XLSX cached; install `xlsx` to parse, or convert manually.');
+      return;
+    }
+
+    const lines = body.split(/\r?\n/).filter(Boolean);
+    if (!lines.length) throw new Error('empty CSV');
+
+    const header = lines[0].split(/[;,]/).map(h => h.trim().toLowerCase());
+    const idx = (...names) => header.findIndex(h => names.some(n => h.includes(n)));
+    const iName    = idx('nombre','razon');
+    const iCanton  = idx('canton','cantón');
+    const iProv    = idx('provincia');
+    const iAct     = idx('actividad','rubro','sector');
+    const iCedula  = idx('cedula','cédula','identificacion');
+    const iDir     = idx('direccion','dirección','domicilio');
+    const iEmail   = idx('email','correo');
+    const iPhone   = idx('telefono','teléfono','phone');
+
+    const rmap = await regionMap();
+
+    for (const line of lines.slice(1, 5001)) { // cap per-run to 5k for first pass
+      const cols = line.split(/[;,](?=(?:[^"]*"[^"]*")*[^"]*$)/).map(c => c.replace(/^"|"$/g,'').trim());
+      const name = cols[iName]; if (!name) continue;
+      const canton = cols[iCanton] || cols[iProv];
+      const region = resolveRegion(rmap, canton);
+      const act = (cols[iAct]||'').toLowerCase();
+      const vertical =
+        /restaurante|gastronom|comida|cafeter|bar|panade/.test(act) ? 'service_food' :
+        /hotel|hospeda|cabin|alojami|villa/.test(act) ? 'tourism_hotel' :
+        /tour|turismo|aventura|operador/.test(act) ? 'tourism_tour' :
+        /belleza|spa|peluquer|barber|estética/.test(act) ? 'service_beauty' :
+        /tienda|comerc|venta|retail/.test(act) ? 'service_retail' :
+        /gimna|fitness|crossfit|yoga/.test(act) ? 'service_fitness' :
+        /mascot|veterin|pet/.test(act) ? 'service_pet' :
+        /taller|mecan|automotr|car wash/.test(act) ? 'service_auto' :
+        /limpie|aseo|janitor/.test(act) ? 'service_cleaning' :
+        /diseñ|publicid|fotograf|estudio/.test(act) ? 'service_creative' :
+        'service_retail';
+      const category = vertical.startsWith('tourism_') ? 'tourism' : 'service';
+      const placeSlug = slug(`meic-${cols[iCedula]||name}-${region.slug}`);
+      const r = await upsertPlace({
+        slug: placeSlug,
+        name,
+        category, vertical,
+        region_id: region.id,
+        description: act ? `MEIC-registered PYME: ${act}` : 'MEIC-registered PYME.',
+        address: cols[iDir] || canton || '',
+        phone: cols[iPhone] || null,
+        email: cols[iEmail] || null,
+        cedula_juridica: cols[iCedula] || null,
+        source: 'meic-pymes',
+        source_url: res.url,
+      });
+      if (r.inserted) added++; else updated++;
+      total++;
+    }
+
+    await finishRun(runId, { rows_in: total, rows_added: added, rows_updated: updated, status: 'ok' });
+    console.log(`[meic-pymes] in=${total} added=${added} updated=${updated}`);
+  } catch (e) {
+    await finishRun(runId, { status: 'error', notes: e.message });
+    console.error('[meic-pymes] FAIL', e.message);
+    process.exitCode = 1;
+  } finally {
+    await pool.end();
+  }
+})();
diff --git a/scripts/ingest/run-all.js b/scripts/ingest/run-all.js
new file mode 100644
index 0000000..952e926
--- /dev/null
+++ b/scripts/ingest/run-all.js
@@ -0,0 +1,32 @@
+'use strict';
+
+// Sequential orchestrator. Runs each ingest module as its own child process
+// so a single-source failure doesn't poison the others.
+
+const { spawn } = require('child_process');
+const path = require('path');
+
+const ORDER = [
+  'ict-cst',
+  'local-portals',
+  'meic-pymes',
+  'google-places',
+];
+
+function run(name) {
+  return new Promise(resolve => {
+    const t0 = Date.now();
+    const p = spawn('node', [path.join(__dirname, `${name}.js`)], { stdio: 'inherit', env: process.env });
+    p.on('close', code => {
+      const ms = Date.now() - t0;
+      console.log(`[run-all] ${name} → exit=${code} in ${(ms/1000).toFixed(1)}s\n`);
+      resolve(code);
+    });
+  });
+}
+
+(async () => {
+  const codes = {};
+  for (const name of ORDER) codes[name] = await run(name);
+  console.log('[run-all] summary', codes);
+})();
← 253365f initial scaffold: CR multi-vertical directory (tourism + ren · back to Costa Rica · MEIC PYMEs: full XLSX import (25,661 rows) + auto-region cre c91cf7b →