December 11, 2024 Data Protection • Disaster Recovery

Recovery-Ready or Just Backed Up?

You pay for backups. But can you actually recover? Most companies discover the answer at 3 AM during a disaster—when it's too late. Here's the difference between having backups and being recovery-ready.

By Tyler Colby

The Question Nobody Asks

VP of IT to backup vendor salesperson: "Can you back up our Salesforce org?"

Vendor: "Absolutely. Daily snapshots, encrypted storage, 99.9% SLA. $15K/year."

VP: "Great. We're covered."

Wrong.

The question nobody asks: "Can you recover our Salesforce org?"

Because having backups ≠ being able to recover.

The Shared Responsibility Model

Backup vendors will tell you what they do:

  • Extract data from Salesforce (nightly snapshots)
  • Store it securely (encrypted, redundant, compliant)
  • Retain it per your policy (30 days, 90 days, 7 years)

What they don't tell you: Recovery is your responsibility.

What Backup Vendors Provide

  • Data extraction and storage
  • Self-service restore interface (usually)
  • Documentation on how to use their tool

What You're Responsible For

  • Knowing how to use the restore interface
  • Understanding relationship dependencies (which objects to restore first)
  • Handling schema drift (fields added/removed since backup)
  • Resolving conflicts (records changed since backup)
  • Validating data integrity post-recovery
  • Testing recovery procedures before disaster strikes

The gap: Most companies pay for backups but never test recovery. When disaster strikes, they discover recovery is far harder than expected.

Architect's Note: The Salesforce Well-Architected framework emphasizes Trusted systems must be resilient. Resilience means designing for failure—not hoping it doesn't happen. Salesforce architects recommend quarterly recovery drills where you actually restore data to a sandbox and validate integrity. If you haven't tested recovery in the past 90 days, you don't know if you can recover. And if you don't know, assume you can't.

Why Recovery Fails: The 5 Gotchas

1. Relationship Reconstruction

Scenario: You delete 10,000 Accounts. Need to restore from backup.

The problem: Accounts have Contacts. Contacts have Opportunities. Opportunities have OpportunityLineItems.

When you restore Accounts, Salesforce assigns new Record IDs. But Contacts still reference the old Account IDs (from before deletion). Relationships broken.

What recovery requires:

  1. Restore Accounts with External IDs (Global_ID__c)
  2. Update Contact.AccountId to reference Global_ID__c instead of Salesforce ID
  3. Upsert Contacts using updated references
  4. Repeat for Opportunities, Cases, and all related objects

If your backup doesn't preserve External IDs: Manual reconciliation. For 10,000 Accounts with 50,000 Contacts and 30,000 Opportunities? Days of work.

2. Schema Drift

Scenario: Backup from 30 days ago. Need to restore after ransomware.

The problem: In the past 30 days, you added 5 custom fields to Account. Backup doesn't include those fields (they didn't exist yet).

When you restore, 30-day-old data loads into current schema. New fields: null for all restored records.

What recovery requires:

  • Identify schema changes since backup
  • Determine default values for new required fields
  • Backfill new fields after restore
  • Validate formula fields recalculate correctly

If your backup doesn't include metadata versioning: You're guessing what the schema looked like 30 days ago.

3. Incremental Changes

Scenario: Accidental deletion at 11 PM. Backup runs at midnight. You catch it at 9 AM.

The problem: Backup is from midnight. But users made changes from midnight to 9 AM (9 hours of work).

If you restore from midnight backup, you lose 9 hours of changes.

What recovery requires:

  • Identify records modified between backup time and deletion time
  • Extract those changes from audit logs or Field History
  • Merge backup data with incremental changes
  • Resolve conflicts (same field updated in backup and post-backup)

If your backup is daily (not hourly): Potential data loss window is 24 hours. For high-transaction orgs, that's unacceptable.

4. Validation Rules and Required Fields

Scenario: Restoring 20,000 Opportunities from 60-day-old backup.

The problem: Since backup, you added validation rule: "Close Date must be within 90 days of Create Date."

Opportunities in backup are 60+ days old. Close Dates are 150 days from Create Date. Validation rule blocks restore.

What recovery requires:

  • Temporarily disable validation rules
  • Restore data
  • Fix validation failures
  • Re-enable validation rules
  • Accept that some data may violate current business logic

If you restore without disabling validations: Bulk restore fails. Have to fix data manually or modify restore process.

5. Integration State Synchronization

Scenario: Salesforce integrated with ERP. Restore Salesforce from backup.

The problem: ERP has Orders from the past week. Salesforce backup is 7 days old. Restoring Salesforce creates state mismatch.

ERP thinks Orders exist in Salesforce. Salesforce has no record of them (backup predates Order creation).

What recovery requires:

  • Pause integrations during restore
  • Identify data created in external systems during backup-to-now window
  • Manually sync external data to restored Salesforce
  • Re-enable integrations
  • Monitor for sync conflicts

If you restore without integration coordination: Data corruption, duplicate records, broken workflows.

Real Recovery Failure: Healthcare SaaS

Company: $40M ARR healthcare SaaS
Backup vendor: Major player, $18K/year contract
Disaster: Flow automation deleted 12,000 Patient records (Friday 6 PM)

Recovery Attempt #1: Self-Service Restore

Friday 6:30 PM: Admin logs into backup vendor portal. Selects "Restore Patient records from last backup."

Friday 6:45 PM: Restore starts. Progress bar: 2 hours estimated.

Friday 8:47 PM: Restore completes. Admin checks Salesforce.

Problem: 12,000 Patients restored. But 8,400 are orphaned—no relationship to Treatment_Plan__c records. Relationships broken.

Recovery Attempt #2: Call Backup Vendor Support

Friday 8:52 PM: Admin calls backup vendor support. Gets voicemail: "Our support team is available Monday-Friday, 9 AM - 5 PM EST."

It's Friday 8:52 PM. No support until Monday.

Recovery Attempt #3: Call Salesforce Rescue

Friday 9:15 PM: VP of Engineering calls us. "We restored from backup but relationships are broken. Patients have no treatment plans. Operations start Monday 8 AM. We need this fixed."

Our Response Timeline

  • 9:22 PM: Connected to org, assessed damage: 8,400 Patients orphaned from Treatment Plans
  • 9:45 PM: Root cause: Backup restore used Salesforce Record IDs instead of External IDs. New IDs assigned on restore, breaking relationships
  • 10:10 PM: Recovery plan: Re-restore with External ID matching, rebuild Patient-to-Treatment_Plan relationships
  • 10:30 PM: Extracted backup data from vendor (they provided raw CSV export)
  • 11:45 PM: Built relationship mapping table (Patient External ID → Treatment Plan External ID)
  • 12:30 AM: Re-imported Patients using External ID upsert (preserved existing IDs where possible)
  • 1:45 AM: Rebuilt Patient_Treatment_Plan__c junction object relationships
  • 2:30 AM: Validation complete: All 12,000 Patients restored with intact relationships

Total recovery time: 5 hours 8 minutes
Cost to company: $12K emergency recovery fee
Alternative cost: Operations down Monday, estimated $200K+ revenue impact

The Lesson

Their backup vendor did exactly what was promised: stored data, provided restore interface.

But the company wasn't recovery-ready. They didn't know:

  • How to preserve relationships during restore
  • That External IDs were critical for recovery
  • That backup vendor support was business-hours only
  • How long recovery would actually take

They discovered these gaps during a disaster. At 9 PM on a Friday.

What Recovery-Ready Looks Like

1. External IDs on All Major Objects

Create a Global_ID__c field (External ID, Unique) on every object you back up:

  • Account, Contact, Lead, Opportunity, Case
  • All custom objects

Populate with UUID or sequential ID on record creation. This enables relationship-preserving upserts during recovery.

2. Metadata Versioning

Store metadata snapshots alongside data backups. When you restore data from 30 days ago, you know what the schema looked like.

Many backup vendors don't include this. Build it yourself:

  • Weekly metadata backup via Metadata API
  • Store object definitions, field definitions, validation rules
  • Version with timestamp

3. Documented Recovery Procedures

Create step-by-step playbooks for common disaster scenarios:

  • Accidental mass deletion: Which objects to restore first, relationship rebuild order, validation steps
  • Corrupted data: How to identify scope, selective field restore, merge strategies
  • Full org restore: Integration pause procedures, schema drift handling, post-restore validation checklist

Playbooks should include: vendor-specific commands, expected timelines, rollback procedures.

4. Quarterly Recovery Drills

Schedule recovery drills every 90 days. Not theoretical—actual restore to sandbox.

Drill scenario examples:

  • Restore 1,000 Accounts from 30-day-old backup
  • Restore all Opportunities from last week's backup
  • Full org restore to sandbox (test disaster recovery RTO)

Measure and document:

  • Time to complete restore
  • Data integrity score (% of records with intact relationships)
  • Issues encountered
  • Process improvements needed

If drills consistently fail or take too long, escalate. Don't wait for real disaster to discover problems.

5. Defined RTO and RPO

RTO (Recovery Time Objective): How long can you be down?

  • 4 hours? Daily backups might work.
  • 1 hour? Need hourly backups + fast restore process.
  • 15 minutes? Need real-time replication.

RPO (Recovery Point Objective): How much data loss is acceptable?

  • 24 hours? Daily backups acceptable.
  • 1 hour? Hourly backups required.
  • Zero? Need Change Data Capture streaming to external storage.

Get RTO/RPO requirements from business stakeholders. Then validate via recovery drills that you can meet them.

Architect's Note: Salesforce architects recommend implementing a 3-2-1 backup strategy: 3 copies of data, 2 different storage media, 1 offsite. For Salesforce, this means: production org (copy 1), backup vendor storage (copy 2), secondary archive to S3/Azure (copy 3). The Well-Architected principle of Trusted means redundancy and testability. One backup vendor is a single point of failure—consider dual vendors for critical orgs.

The Backup Vendor Questions You Should Ask

Before signing a backup contract, ask:

1. Relationship Preservation

  • "How do you preserve parent-child relationships during restore?"
  • "Do you support External ID-based upserts?"
  • "Can I restore just Accounts without breaking Contact relationships?"

2. Schema Drift Handling

  • "Do you version metadata alongside data backups?"
  • "What happens if I restore from a backup where field X didn't exist yet?"
  • "Can you transform data to match current schema during restore?"

3. Incremental Recovery

  • "If I restore from midnight backup at 9 AM, can you merge changes made 12 AM - 9 AM?"
  • "Do you provide point-in-time recovery (restore to any moment, not just backup snapshots)?"

4. Support Availability

  • "Is support available 24/7 or business hours only?"
  • "What's your SLA for critical recovery scenarios?"
  • "Do you provide hands-on recovery assistance or just self-service tools?"

5. Testing and Validation

  • "Do you offer recovery drill services?"
  • "Can I restore to sandbox for testing without affecting production?"
  • "What validation checks do you run post-restore?"

If vendor can't answer these questions confidently, consider alternatives.

The Bottom Line

Most companies have backups. Few companies are recovery-ready.

Having backups means: Data is stored somewhere, theoretically retrievable.

Being recovery-ready means:

  • You've tested recovery and it works
  • You know exactly how long it takes
  • You have documented procedures
  • Your team is trained
  • Relationships and integrations survive recovery

The gap between the two is discovered during disasters. At 3 AM. When leadership is asking "How long until we're operational?"

Don't discover the gap during a disaster. Test recovery now. Before you need it.

Because "I thought we were backed up" is not a recovery plan.

Are You Recovery-Ready?

We offer free recovery readiness assessments. We'll review your backup strategy, test restore procedures, validate External ID implementation, and give you an honest grade. Find out if you can actually recover—before disaster strikes.