Data pipeline
One state adapter (src/adapters/ca.ts) joins four sources into two output files. Only the government sources ever create or move a record — Google and human overrides only enrich or annotate one.
Dropped bucketname, lat, or lng.Google matching
Runs only on the published-status candidates from step 1 — never the full inventory. Every number below is read live from GOOGLE_TUNING, not typed by hand, so retuning a threshold updates this page automatically.
Showing built-in fallback values — could not reach /api/constants (is the review server running?).
Commands
Every script in package.json, and what each flag costs you. The default is deliberately free and offline — network access is something you opt into.
data/cache/ and re-applies the committed Google cache. Writes both output files. Byte-identical on a re-run.data/cache/. Still no Google calls.MAX_DETAILS_CALLS. A missing GOOGLE_MAPS_API_KEY logs and continues.google block reaches the output.http://localhost:4400. Local-only, zero dependencies, never deployed.tsc --noEmit.data/publish/*.json → sandy/data/master-v3.json. Restart the app's dev server after. Runs in the sibling repo, not this one.--state=ca; Refresh Google data appends --refresh-google and nothing else can add it.Endpoints
Everything this project talks to, upstream and local. Adding a state adds rows to the first table only — the local API is state-agnostic.
data/cache/ (gitignored); re-hit only by --refresh. Free, unkeyed. Carries the licence and indemnity text every record must render.prohibited or upgrade to leashed. Free, unkeyed.X-Goog-Api-Key + X-Goog-FieldMask. Nearest candidate wins, not top relevance.--refresh-google only, 30-day licence, capped at 1,000 calls per run. Cached in data/google-cache/, which is committed — that's why a plain seed needs no key.STATIC_FILES.publishStatuses, dogStatuses, googleTtlDays.GOOGLE_TUNING + GOOGLE_TTL_DAYS, read live from the pipeline code — this is what keeps section 02 honest.{ id, status, note } or { id, remove:true } → data/overrides/ca.json. Unknown id throws.{ ids[], status, note } — the bulk bar. Replaces existing overrides after a confirm.{ id, skip?, placeId?, textQuery?, trustCoordinates?, note } → data/overrides/ca-google.json.{ ids[], skip:true, note } — bulk "skip Google matching".{ id, flagSignature, note? } or { id, remove:true } → data/overrides/ca-reviewed.json. Backs "Match is right — clear flag"; a record falls back into Needs attention the moment its flag set changes.&refreshGoogle=1 is the single place --refresh-google can enter the system — the only request here that can spend money.Tuning constants
Read live from GOOGLE_TUNING in src/sources/googlePlaces.ts via /api/constants — the numbers here and in section 02 can never drift apart.
| Constant | Value | What it controls |
|---|---|---|
| DISTANCE_CONFIDENT_M | 300 | Auto-accept a match at or under this distance from the government coordinate. |
| DISTANCE_MAX_M | 800 | Accept on distance alone up to here; flagged for review above the confident threshold. |
| DISTANCE_EXTENDED_M | 2000 | Accept out to here when the name also agrees — recovers large parks/trails whose Google centroid sits far from the access point. |
| EXTENDED_NAME_SIMILARITY | 0.5 | Minimum name-token overlap required to accept a match beyond DISTANCE_MAX_M. Never a gate on its own — only ever widens the distance gate. |
| UNMATCHED_RETRY_DAYS | 14 | How long an unmatched record is left alone before it's searched again. |
| MAX_DETAILS_CALLS | 1000 | Hard cap on Details calls in a single run — the cost guardrail. A run stops rather than silently exceeding it. |
| GOOGLE_TTL_DAYS | 30 | Licence window. Enforced on every seed run, with or without an API key — an expired entry drops from the output even if it's still sitting in the cache. |
Review-tool guide
Needs attention is the default view — the union of the five reasons in section 08, sorted worst-first. Re-run seed re-joins the government sources (using cached raw responses) and re-applies Google's already-cached data — zero external calls. Refresh Google data is the one action that spends quota; it's gated behind a confirmation that reports the live candidate count and free-tier headroom.
Filters compose: OR within a group (Pipeline or Google), AND across groups. They echo as removable pills below the controls, and survive a reload along with your search, sort, selection and scroll position.
Match is right — clear flag records a per-record review (data/overrides/ca-reviewed.json) keyed to the exact flags you saw. If those flags change on a later run, the record returns to Needs attention automatically.
Audit-only fields: a Google match's matchedName and its coordinate describe what Google thinks the place is called and where it is — shown so you can sanity-check the match. Neither ever overwrites the record's real name, lat, or lng; those stay the government's, always, unless a human explicitly sets trustCoordinates.
Bulk selection persists across filter and search changes — picking records in one view, then narrowing the list to check a couple more, won't lose your selection. The bulk bar tells you when some of your selection is currently hidden by the active filter.
Keyboard shortcuts
The review tool is a queue you walk, not a page you browse — everything below works from anywhere on the page except while you're typing in a field.
Your view, filters, search, sort, selection, and scroll position all survive a reload.
Chips and buckets
Needs attention is a computed queue, not a pipeline bucket of its own — every record matching at least one of five reasons a human should look at it, ranked worst-first. A record is never double-counted for matching more than one reason.
| Reason | What it means |
|---|
| Chip | Meaning |
|---|
src/adapters/, one registry line in adapters/index.ts, and widen SourceId + LICENSES in types.ts. seed.ts needs no changes — and this guide needs only new rows in the upstream table above.