Local-First Migration: Pathfinder's Biggest Release Yet

Most migration tools ask you to trust them with your data. Pathfinder's answer is that they shouldn't have to.

Today's release is the largest we've cut, and the one I'd defend hardest in front of a CISO. Local-first architecture all the way down. A tamper-evident audit chain your auditor verifies with a public key and a protocol we publish in the open. Pre-flight gates that turn migration disasters into Friday-afternoon catches. Twenty named instruments, fourteen alert seeds, and a dashboard — in the repo, not gated behind an SKU.

The substrate matters. The bet underneath it matters more.

The bet: local-first migration

Sit in the CDO chair at a $30M nonprofit. 1.4 million donor records, fifteen years of grant history, PHI on the program side, a board that reads breach headlines on the weekend. You've migrated CRMs three times, and each time the vendor in the middle held a working copy of everything that mattered for the duration of the project — usually longer.

What you actually price is vendor concentration risk. Every party holding a queryable copy of your donor PII is another breach blast radius you're insuring against. Cyber liability priced this in years ago; the last quote I negotiated for a comparable nonprofit moved $18K when a fourth processor showed up on the SIG. You don't insure against the breach you can't have. The question is what it costs to not have it.

Pathfinder runs on the laptop of the person doing the work, under their account, with credentials gated by the macOS Keychain. The Keychain ACL boundary isn't new to anyone in IT — every Mac in the building already uses it for Wi-Fi PSKs and cert private keys. We reuse that trust boundary instead of asking you to bless a new SaaS vendor's KMS posture. One less SOC 2 to read, one less DPA, one less subprocessor on the donor-facing privacy notice.

The audit demo writes itself. Pull up the verifier in front of your auditor. Hand them the public key. Watch them re-derive a Merkle root from raw rows on their own laptop. The auditor stops asking about the vendor's controls because the vendor isn't in the chain of custody. That conversation used to eat six weeks of questionnaire back-and-forth; now it's forty minutes and a laptop.

Lock-in collapses into something procurement actually likes: a workspace is a folder on disk. Zip it, hand it to the next firm, stop paying us. The work product travels because we never built a retention hook — the data was never on our infrastructure to retain.

The bet is that customers who've been burned three times will pay a premium for an architecture that makes the fourth burn structurally impossible. They do. I've watched them sign.

The Data Cloud question

Three Data Cloud pitches walked into rooms I was in last year, same deck more or less. The argument is convenience: your CRM data is already in Salesforce, your service data is already in Salesforce, why not unify everything else there so Einstein and Agentforce can reason across it? It's a good pitch. It works on a lot of buyers.

It shouldn't work on you.

The competitive frame

Convenience is a price you pay later. Data Cloud is metered — Data Service credits for ingestion, segmentation, activation, queries. Pricing sheets I've seen for comparable nonprofit deployments start around $108K/year, and the rep will tell you straight that real workloads land at three to five times that inside eighteen months. The roadmap you're buying is Salesforce's roadmap. The day they decide the pricing model needs another tier, you're not negotiating — you're renewing. Ask anyone who lived through the Marketing Cloud Engagement repricing in 2024.

The audit log story is the part nobody pushes hard enough. Data Cloud's audit log is Salesforce's audit log, on Salesforce's infrastructure, gated by Salesforce's access controls. Fine — until the question you need to answer is "did Salesforce mishandle this record." At that point the log you need is owned by the entity you need to audit. I've watched a compliance officer realize this in real time during an RFP. It does not go well for the vendor.

The verifier, compared honestly

A typical SaaS audit log is a table the vendor owns, rotated on the vendor's schedule, queryable through the vendor's UI, signed — if at all — by the vendor's key. You're trusting four things you can't independently check: that no privileged employee edited a row, that no truncation happened during a rotation, that no rotation gap silently dropped events, that the export you pulled last quarter still matches the table today.

Pathfinder's chain dissolves all four. A vendor edit fails verify_chain at the offending row index — including an edit by us, which is the entire point of asymmetric signing. Truncation fails because the next signed Merkle root won't cover the missing leaves. Rotation gaps don't exist; the chain is append-only and the key change itself is a row in the chain. Insider tampering, ours or yours, gets treated the same as outsider tampering. The math doesn't care who held the editor.

The chain is signed by a key in your Keychain, on your hardware, verified by a protocol we publish and source we share with authorized reviewers. We can't forge a row. We can't quietly delete one. If our build pipeline gets compromised tomorrow, the chain on your machine still verifies against the public key you printed and put in your fireproof. That is a different category of guarantee than inheriting Salesforce's posture wholesale.

The AI angle

Data Cloud's AI is opt-out. Einstein Trust Layer ingests your unified records by default; your admins go find the toggles to exclude specific objects. The burden of proof sits on the customer.

Pathfinder Fabric is opt-in. The LLM pattern engine ships off. You bring your own Anthropic or OpenAI key, or you point at a local Ollama instance and stay fully offline. Nothing — not a record, not a field name, not a row count — leaves your machine unless you click the thing that sends it. That is the posture a regulator expects when you tell them you handle donor PII responsibly.

The "we're already a Salesforce shop" trap

So is everyone Salesforce wants to sell Data Cloud to. The moment your donor unification logic lives in Data Cloud, leaving Salesforce isn't a CRM swap anymore — it's CRM plus warehouse plus activation layer plus the institutional muscle memory of the team that learned the SQL dialect. You become the account that gets sold to the next acquirer at a multiple Salesforce's board will love and your board will not.

Pathfinder is the inverse bet. Move data between systems you own. Keep the connective tissue local. Stay portable.

What an audit-ready migration actually looks like

State AG opens an inquiry. A donor has filed a complaint that her 2024 pledge was visible to a volunteer who shouldn't have had access during your migration from Raiser's Edge NXT to NPC. The AG's letter gives you fourteen days. CFO in your office, insurance carrier on the phone, counsel wants a number by end of day.

You open Pathfinder. Workspace search on the donor's record ID returns three audit chain rows — capture from RENXT on March 14, mapping transform on March 18, write to NPC on March 21. You run audit_chain_status against the workspace public key. Chain verifies clean. Last-sealed Merkle root: March 22. Signature valid.

Drill into March 18. The mapping checksum matches the version pinned in the replay manifest. The conflict resolution table shows the address conflict between RENXT and NPC resolved MostRecent by survivorship rule, the resolution row signed and timestamped. The volunteer's user ID doesn't appear in any access log because the migration ran under your service account, on your hardware. There is no shared vendor tenant for it to appear in.

Export the workspace archive: 380MB zip. Inside, the signed chain, mapping config, conflict rows, replay manifest, schema snapshots from both sides. Generate the regulator CSV from the verifier — one row per audit event, signature column, hash chain column, human-readable diff.

You hand the AG four files and the public key. Inquiry closes in six days. No subpoena to Salesforce. No subpoena to us. The evidence belonged to you the whole time.

What shipped

Every claim above has a code path behind it. Every code path has a test. Most have a runbook.

Trust

Ed25519 + Merkle audit chain. Every mutation in audit_log is bound to the previous one by row_hash = SHA-256(FRAME_PREFIX || prev_hash || len(payload) || payload). Frame and length prefixes enforce domain separation and defeat the concatenation attack. Genesis mixes the workspace ID into the tree's root so signed windows can't be transplanted between chains. Every 256 leaves the writer seals a Merkle root and signs with the workspace's Ed25519 key — deterministic, constant-time, no nonce-reuse footgun, 32-byte public key your auditor pins by fingerprint. verify_chain fails at the first broken row index; verify_signed_root recomputes the Merkle root over the sealed window. Sealer crosses 100ms at ~10K depth — tracked as an incremental-tree follow-up, not a soundness gap. Key rotation is itself a row in the chain; historical roots verify forever under their original fingerprint.

Keychain-only credentials, secrets-lint CI gate. Production reads from src-tauri/src/keychain.rs and nowhere else. The from_env() constructors that used to exist are not reachable from any binary entry point. scripts/secrets-lint.sh runs in pre-commit and CI against AWS keys, JWT literals, GitHub PATs, Slack tokens, OAuth refresh tokens, generic hex secrets, Bearer literals, PEM private keys, and credentialed URLs. Suppression takes pf:allow-secret plus a reviewer-approved reason.

Per-connector base-URL allow-list. Each connector pins a small set of host-suffix patterns. Paste salesforce.com.evil.example into the Connector Wizard and it bounces. Frontend previews, backend enforces.

Four threat models, one per trust boundary — IPC, MCP, HTTP, Connector. STRIDE-aligned, named owners, controls and gaps cited. The artifact a customer's security review actually asked for.

Safety

Transform typechecker. The transform layer is where bad migrations are born quietly. Every team has shipped the TEXT(NumericField) that silently lost two trailing zeros across forty thousand donation records, or the DateValue cast against a non-ISO string that detonated three hours into a Sunday load. The expression lexes, parses, evaluates, returns a value, corrupts the target column, and never raises anything you can grep for. The typechecker refuses that whole category at save. An HM-lite checker walks every formula, knows the arity and types of every built-in (BLANKVALUE, NULLIF, CASE, TEXT, VALUE, DOUBLE, IF, YEAR, the lot), and resolves $Field against the live source schema. UPPER($Revenue) doesn't save. $Amount + $Name doesn't save. Unknown functions come back with a Levenshtein suggestion in the same finding. Strict typing rolls in on a ninety-day warn-then-strict window — warn emits findings and inline badges while saves still commit; day ninety they block.

Mapping validation, dry-run, golden-dataset diff. Migrations don't go wrong in staging. They go wrong on the third Tuesday after go-live, with a Director of Operations watching the loader spin while a query against the target says the phone numbers are gone. mapping_validate is the cheap gate. The dry-run engine runs every formula against real source rows via reservoir sampling on a deterministic seed (Vitter Algorithm R, splitmix64), traces success or failure of every expression, never touches a target connector. A uniform sample of 200 rows out of 1.4 million still catches the one constituent whose phone field is the literal string null rather than SQL NULL — the row that takes down the load. Golden datasets pin (input, expected_output) pairs and shout when an operator's "small tweak" to an eighteen-month-old mapping changes them. CI for mappings.

Schema drift detection. Every migration engineer has lived this. Admin adds a picklist value Friday at 4:30. Marketing drops a field's max length from 80 to 40 at 4:45. Monday's run fires, the loader truncates twelve thousand records, nobody notices until the gift officer pulls a report Wednesday and the donor narratives are cut mid-sentence. The audit trail says "Success." The detector classifies diffs into three tiers — info for a new nullable field nobody maps, warn for changed nullability on an unused field, block for a removed field, a changed type, or a max-length reduction on a column a live mapping targets. It walks every active ObjectMapping, intersects targeted fields against the drifted set, and flags the mappings that just became unsafe. Scheduled snapshots surface drift before the next migration finds it for you, in a conference room.

Fellegi–Sunter identity scoring + survivorship. Each candidate pair gets a log-likelihood score from per-field m/u weights; the score routes to AutoMerge, ManualReview, or Discard against configurable thresholds. AutoMerge picks via MostRecent, Longest, NonNullPreferred, or HigherTrustScore. The math is on the page.

MCP risk-tier registry. Every tool exposed to an LLM is tagged Read | Write | Admin. Write tools refuse to fire without confirmed=true in the payload; Admin tools require a UI prompt on top. migrate_run is pinned Admin. Seven invariant tests keep the roster honest.

Observability

Twenty instruments treated as a contract. pathfinder-core::observability::instruments defines twenty InstrumentDef constants — name, kind, unit, description, label set — and every call site references them by symbol. Stringly-typed metrics are how you end up with 47 minor variants of the same counter six months in (audit.append, audit_append_total, pf.audit.appends, three with workspace and one with workspace_id, none aggregating in the dashboard). The catalog forecloses that at compile time. Eight tests enforce label discipline.

Alerts and the dashboard ship with the binary. Most vendors hand you a metrics endpoint and call observability "supported" — the runbook is your homework, the thresholds are your weekend, the dashboard is a screenshot in a sales deck. Pathfinder ships alerts/pathfinder.yml and dashboards/pathfinder-overview.json in the repo, and every alert's runbook: annotation points at a docs/runbooks/ file with Symptom → Diagnose → Mitigate → Rollback → Hand-off already written. AuditChainVerificationFailure pages at for: 0m, severity critical — tamper isn't something you batch for fifteen minutes. CdcLagClimbing warns at 60s over fifteen minutes; CdcLagCritical pages at ten minutes held for five. ConnectorRateLimitNearExhaustion watches headroom drop below 100 — early enough to throttle before the SaaS starts returning 429s and the retry storm makes it worse.

The honest gap

The substrate is in place; the pipe to your scraper isn't, yet. The catalog renders # HELP/# TYPE blocks and the recording adapters know the schema, but per-sample emission still needs to land at the call sites. Until then, alert expressions scrape clean type stanzas and no data; the Grafana dashboard renders axes and no series. OTel span propagation across the Tauri IPC boundary is queued behind the Prometheus work. Real wiremock fixtures for Salesforce, RENXT, and HubSpot land next — the contract harness has the JSON and XML connectors today and the structure is right for the network ones. Replay v2 adds the "play" button and divergence diff. Mapping Studio v2 reaches feature parity with the legacy page and the route swaps.

Known gaps with owners, not surprises waiting to bite. Naming them on the way out is a stronger signal than naming features.

Sync, UX, and process

At-least-once, idempotent CDC. Each ChangeEvent carries a source_event_id; cdc_committed_events has a UNIQUE on (connection_id, source_event_id) with ON CONFLICT DO NOTHING. Duplicates absorbed silently; cdc.duplicate_suppressions tells you when the absorption rate spikes. The CSV adapter is the demonstrable case; the pattern generalizes.

Replay manifests. Every migration writes a manifest pinning source snapshot, mapping config, conflict override, engine config, and engine version hashes. The Replay Player opens any historical manifest and runs a compatibility check against the current engine.

Connector Wizard, Mapping Studio v2, Replay Player. Five-step connector flow with live URL allow-list preview. Mapping Studio v2 wires typechecker, dry-run, and golden-dataset onto one page; legacy stays one click away.

Process artifacts. Six ADRs document the load-bearing choices. The compliance crosswalk maps controls to SOC 2 Type II, HIPAA, and GDPR Article 32 with code paths cited. The DPA template and pre-filled security questionnaire are written in plain English with real answers. SECURITY.md is real: seventy-two hours to acknowledge, seven days to triage, thirty for high-severity fix, ninety-day coordinated disclosure, safe harbor for good-faith researchers. Bus-factor and triage rotation docs put continuity in writing.

Why this matters

Migration tooling has been bad for a long time. The default story: buy an ETL platform that promises everything, watch it become a multi-tenant SaaS that ingests your data and rents it back, discover the audit story isn't quite what you thought, acquiesce to the vendor's roadmap because moving off is harder than staying. The customers hit hardest are the ones who care most about the data — nonprofits with donor records, healthcare orgs with PHI, financial services firms with client information — and who can least afford vendor concentration risk.

Local-first migration is the way this should have worked the whole time. We finally got there. The rest of the industry can follow when they're ready.