← back to Email Deliverability Agent

README.md

291 lines

# email-deliverability-agent

Fleet email-deliverability monitor for the Designer Wallcoverings
sister-site network (~58 customer-facing wallpaper/wallcovering
storefronts). It answers one question Steve actually cares about:

> **When a customer fills out a contact form / clicks a `mailto:` link on
> one of our sister sites, does that lead actually reach a human?**

Today, for 42 of 58 sister sites, the answer is **no**.

---

## Purpose & mental model (read this first)

A self-sent test email loop is a **deliverability monitor**, not a
reputation or "warm-up" tool. It is important to be honest about what
this agent does and does not do:

- **What `canary.js` proves:** that the *receive + route* path for a
  domain works end-to-end — i.e. a message sent to `info@<domain>` is
  accepted by the MX, routed by the Purelymail rule, and lands in a
  mailbox a human reads. That's it. It is a smoke test.
- **What it does NOT do:** it does not build "Gmail credibility" or
  sender reputation. Sending mail to yourself on a 3-day loop teaches
  Gmail nothing — there is no recipient engagement, no varied
  destinations, no opens/replies from real people. Inbox-provider trust
  is earned by:
  1. **Valid DKIM** — your sends are cryptographically signed and the
     signature verifies. (Currently failing on **all 211 domains**.)
  2. **Valid DMARC** — you publish an alignment policy so receivers know
     unsigned/unaligned mail claiming to be you is fake. (Currently
     failing on **all 211 domains**.)
  3. **Real, engaged sends** — actual humans opening and replying to
     real mail from the domain over time.
- **The fix for "our mail goes to spam" is not a canary loop.** It is:
  publish DKIM + DMARC DNS records, keep SPF valid (already passing),
  and send real mail people want. The canary just *tells you when the
  receive path silently breaks* so a lead-loss outage like the current
  one is caught in 3 days instead of never.

---

## What's in this repo

| File                 | Purpose |
|----------------------|---------|
| `audit.js`           | Pulls every Purelymail domain, flags MX / DKIM / DMARC / routing failures. Writes `output/domain-health.json`. |
| `contact-audit.js`   | Walks every `~/Projects/*` with a `site.config.json`, greps `mailto:`/`tel:` literals, cross-references each contact domain against the health data. Writes `output/lead-loss-report.json` — the list of sites losing leads. |
| `canary.js`          | Deliverability canary — sends one tokenised test email and confirms receipt via IMAP. **Dry-run by default**; `--live` needs creds. Cron/launchd-ready but not scheduled. |
| `lib/data.js`        | Shared fixture loaders + per-domain health logic. |
| `data/*.json`        | Snapshots captured from the Purelymail MCP (see below). |
| `output/*.json`      | Generated reports. |

### Run it

```sh
node audit.js          # domain-level health
node contact-audit.js  # sister-site lead-loss report
node canary.js         # dry-run canary (sends nothing)
npm run all            # audit + contact-audit
```

### Refreshing the Purelymail snapshot

`audit.js` / `contact-audit.js` read two fixture files, not the live API,
because the Purelymail MCP tools can only be invoked from inside a Claude
session, not from a plain Node script. To refresh:

1. Call `mcp__purelymail__purelymail_list_domains` → save the `domains`
   array into `data/purelymail-snapshot.json` (shape: `{capturedAt,
   source, domains:[...]}`).
2. Call `mcp__purelymail__purelymail_list_routing_rules` (any
   `domainName` — note the MCP returns **all** account-wide rules
   regardless of the argument) → save into `data/purelymail-routing.json`
   (shape: `{capturedAt, source, rules:[...]}`).

`audit.js` prints a staleness warning if the snapshot is older than 7 days.

---

## Current findings (snapshot 2026-05-18)

### Domain audit — 211 Purelymail domains

| Metric                       | Count |
|------------------------------|-------|
| Total domains                | 211   |
| **Fail MX** (cannot receive) | **110** |
| No routing rule              | 2 (`venturaclaw.com`, `venturaclaw.agentabrams.com`) |
| Undeliverable (no MX OR no routing) | 112 |
| **Fail DKIM**                | **211 (all)** |
| **Fail DMARC**               | **211 (all)** |
| Fail SPF                     | 0 (SPF is fine fleet-wide) |
| Fully healthy                | **0** |

**Reading:** 110 domains physically cannot receive mail. Worse, even the
101 domains that *can* receive have **no DKIM and no DMARC**, so any mail
they *send* looks unauthenticated and lands in spam. Zero domains are
fully healthy.

### Sister-site lead-loss audit — 58 sites scanned

- **42 sites are LOSING LEADS** — their `mailto:` contact address is on a
  domain that fails MX. Every contact-form / mailto lead is silently
  dropped.
- 14 sites OK (contact domain has MX + routing).
- 2 unverified (`bleachresistantfabrics`, `fliepaper`) — contact domain
  not present on the Purelymail account at all, so it can't be checked
  here; verify their DNS/host separately.

#### The 42 sister sites currently losing leads

Every one fails for the same reason: **the contact domain fails MX —
the domain cannot receive email**.

| Site | Contact address |
|------|-----------------|
| 1800swallpaper | info@1800swallpaper.com |
| 1890swallpaper | info@1890swallpaper.com |
| 1920swallpaper | info@1920swallpaper.com |
| 1930swallpaper | info@1930swallpaper.com |
| 1940swallpaper | info@1940swallpaper.com |
| 1950swallpaper | info@1950swallpaper.com |
| 1960swallpaper | info@1960swallpaper.com |
| 1970swallpaper | info@1970swallpaper.com |
| agedwallpaper | info@agedwallpaper.com |
| apartmentwallpaper | info@apartmentwallpaper.com |
| blockprintedwallpaper | info@blockprintedwallpaper.com |
| contractwallpaper | info@contractwallpaper.com |
| embroideredwallpaper | info@embroideredwallpaper.com |
| fabricwallpaper | info@fabricwallpaper.com |
| ffepurchasing | info@ffepurchasing.com |
| glitterwalls | info@glitterwalls.com |
| greenwallcoverings | info@greenwallcoverings.com |
| handcraftedwallpaper | info@handcraftedwallpaper.com |
| healthcarewallpaper | info@healthcarewallpaper.com |
| hospitalitywallcoverings | info@hospitalitywallcoverings.com |
| jutewallpaper | info@jutewallpaper.com |
| linenwallpaper | info@linenwallpaper.com |
| madagascarwallpaper | info@madagascarwallpaper.com |
| metallicwallpaper | info@metallicwallpaper.com |
| micawallpaper | info@micawallpaper.com |
| museumwallpaper | info@museumwallpaper.com |
| mylarwallpaper | info@mylarwallpaper.com |
| naturalwallcoverings | info@naturalwallcoverings.com |
| raffiawallcoverings | info@raffiawallcoverings.com |
| raffiawalls | info@raffiawalls.com |
| recycledwallpaper | info@recycledwallpaper.com |
| restaurantwallpaper | info@restaurantwallpaper.com |
| retrowalls | info@retrowalls.com |
| saloonwallpaper | info@saloonwallpaper.com |
| screenprintedwallpaper | info@screenprintedwallpaper.com |
| selfadhesivewallpaper | info@selfadhesivewallpaper.com |
| silkwallcoverings | info@silkwallcoverings.com |
| silkwallpaper | info@silkwallpaper.com |
| stringwallpaper | info@stringwallpaper.com |
| suedewallpaper | info@suedewallpaper.com |
| textilewallpaper | info@textilewallpaper.com |
| vinylwallpaper | info@vinylwallpaper.com |

> **Note on `ffepurchasing` / `wallpapercanada`:** their `site.config.json`
> `domain` field points at `hospitalitywallpaper.com`, but the `mailto:`
> literals in their own source use their *own* domains. `contact-audit.js`
> trusts the actual `mailto:` literal (what a customer really clicks), so
> `ffepurchasing` is correctly flagged on `info@ffepurchasing.com` and
> `wallpapercanada` lands OK on `info@hospitalitywallpaper.com`.

### Recommended remediation (not performed by this agent — diagnose-only)

1. **Restore MX** for the 110 failing domains in Purelymail's DNS (the
   42 with live sister sites are the priority — those are losing leads
   *right now*).
2. **Publish DKIM** records for all 211 domains (Purelymail provides the
   selector record).
3. **Publish DMARC** (`_dmarc` TXT, start at `p=none` for monitoring,
   move to `p=quarantine`).
4. Re-run `node audit.js` to confirm, then stand up `canary.js` on a
   3-day schedule so the next outage is caught fast.

---

## Scheduling the canary (manual — not installed by this agent)

This agent intentionally **does not register any cron or launchd job**.
When you're ready, install one of these yourself.

**cron (every 3 days, 9am):**
```cron
0 9 */3 * * cd ~/Projects/email-deliverability-agent && /usr/bin/env node canary.js --live >> output/canary-cron.log 2>&1
```

**launchd** — create `~/Library/LaunchAgents/com.steve.deliverability-canary.plist`
with a `StartCalendarInterval` and `ProgramArguments` invoking
`node canary.js --live`, then `launchctl load` it.

`canary.js --live` needs `CANARY_SMTP_*` / `CANARY_IMAP_*` env vars and
the optional deps: `npm install nodemailer imapflow`.

---

## George status (outbound Gmail mailer) — diagnosis only

**George is the outbound Gmail agent**, exposed to Claude via the
`george-mcp` MCP wrapper. The reported symptom is `Cannot POST
/george/api/send` and `Cannot GET /george/api/health` (HTTP 404).

### Root cause: a base-path prefix mismatch in the MCP wrapper config

George itself is an Express app at `~/Projects/george-gmail/server.js`.
It mounts its routes **at the application root** — `app.post('/api/send', ...)`
(line 966) and `app.get('/api/health', ...)` (line 687). There is **no
`/george` route prefix anywhere** in `server.js` — no `express.Router()`
mounted at `/george`, no `app.use('/george', ...)`. It listens on port
`9850`.

The MCP wrapper, `~/Projects/george-mcp/index.js`, builds request URLs as:

```js
// index.js line 13
const BASE_PATH_PREFIX = process.env.GEORGE_PATH_PREFIX ?? "/george";
// index.js line 28
const fullPath = BASE_PATH_PREFIX + (path.startsWith("/") ? path : "/" + path);
```

So the wrapper **defaults to a `/george` prefix** unless
`GEORGE_PATH_PREFIX` is set in its environment.

In `~/.claude.json` the `george` MCP server's `env` block sets:

```jsonc
"george": {
  "env": {
    "GEORGE_URL": "http://100.107.67.67:9850",
    "GEORGE_BASIC_AUTH": "..."
    // GEORGE_PATH_PREFIX is NOT set
  }
}
```

Because `GEORGE_PATH_PREFIX` is absent, the wrapper falls back to
`/george`, and every call resolves to:

```
http://100.107.67.67:9850/george/api/send   -> 404 (no such route)
http://100.107.67.67:9850/george/api/health -> 404 (no such route)
```

The route George actually serves is `http://100.107.67.67:9850/api/send`.
The comment at `index.js:10-12` shows the default `BASE_URL` was designed
for a Tailscale-Serve front-end that *did* expose George under a
`/george` path. The current config points `GEORGE_URL` straight at the
raw port `9850` (no reverse proxy, no `/george` mount) — so the prefix
must be cleared to match.

### Fix (not applied — Steve to action; do not restart prod)

Set `GEORGE_PATH_PREFIX` to empty in the `george` MCP `env` block in
`~/.claude.json`, so the wrapper hits the routes George actually serves:

```jsonc
"george": {
  "env": {
    "GEORGE_URL": "http://100.107.67.67:9850",
    "GEORGE_PATH_PREFIX": "",
    "GEORGE_BASIC_AUTH": "..."
  }
}
```

(The `secrets`/`update-config` workflow is the right tool to make that
edit, since it backs up `~/.claude.json` first. A running Claude session
needs `/clear` or a restart to pick up the changed MCP env.)

**Alternative** if you'd rather keep the `/george` prefix in the MCP:
re-front George behind the Tailscale-Serve proxy that mounts it at
`/george`, and point `GEORGE_URL` at that proxy origin instead of the
raw `:9850` port.

### Notes from the investigation

- `pm2 list` on this machine (Mac2) is **empty** — George is not running
  here. Its `server.js` logs `Dashboard: http://45.61.58.125:.../` and
  `AUDIT_LOG = /root/DW-Agents/logs/...`, i.e. George runs on **Kamatera**
  (`45.61.58.125`, reachable over Tailscale at `100.107.67.67:9850`), not
  locally. This is a config-only bug — George's process is fine; the MCP
  is just calling the wrong path.
- No George process was restarted or modified. No production service was
  touched. This section is diagnosis only.
```