Resilience and recovery
Test whether critical services can be restored, identify fragile dependencies and leave recovery evidence that operators can use during an outage.
What we review
- Business service and dependency mapping: Applications, data, identity, secrets, DNS, certificates, infrastructure, network paths, suppliers, facilities and required staff.
- Recovery objectives and priorities: Recovery time and recovery point needs, service tiers, restoration order, minimum viable operation and the assumptions behind each target.
- Backup and replication controls: Coverage, schedules, retention, immutability or offline copies, encryption, administration, monitoring, failed jobs and restore history.
- Recovery access and security: Emergency identities, privileged access, key and secret availability, separation from production compromise, logging and approval paths.
- Runbooks and decision authority: Activation criteria, roles, contacts, technical steps, escalation, external support, communications and authority to make time-sensitive decisions.
- Tabletop and technical exercises: Scenario-based walkthroughs, isolated restores, timed recovery tasks, dependency tests and functional service validation.
- Evidence and improvement tracking: Test artifacts, timings, failed steps, deviations, assigned actions and retest criteria after material changes.
Deliverables
- Service recovery map: Critical components, dependency order, owners, external services, recovery sources and known single points of failure.
- Objective and capability comparison: Documented recovery targets compared with backup frequency, restoration evidence, staffing, access and supplier constraints.
- Recovery control findings: Evidence-based findings covering gaps in coverage, protection, administration, monitoring, documentation and testing.
- Usable recovery runbook: Activation, roles, prerequisites, ordered technical steps, decision points, validation, communications and exit criteria.
- Exercise plan and facilitation: A bounded scenario with objectives, participants, safety controls, injects or restore steps and agreed evidence to capture.
- After-action record: What was tested, elapsed times, evidence, failed or unclear steps, decisions, assigned improvements and retest conditions.
Engagement boundaries
- Recovery testing reduces uncertainty; it does not remove it. Results apply to the versions, data, people, dependencies and scenario tested at that time.
- Included: Review of the named services, recovery systems and evidence.
- Included: Tabletop or technical exercise design with documented safety controls.
- Included: Honest comparison between recovery objectives and demonstrated capability.
- Included: Findings, ownership and retest criteria for material gaps.
- Not included: A guarantee that future recovery time or data loss will match a target during every incident.
- Not included: Destructive production failover, data overwrite or crisis declaration without explicit written scope and authority.
- Not included: Business continuity planning for facilities, workforce and non-technology processes unless added.
- Not included: Replacement for backup operations, incident command, cyber insurance, legal advice or supplier support.
- Not included: Validation of dependencies that are unavailable or excluded from the exercise.
Questions before the work starts
Our backup dashboard is green. Why do we need a recovery review?
A green job shows that the backup platform completed a task. It may not show whether the right systems and data were captured, credentials remain available, dependencies can be rebuilt or users can complete a real transaction after restoration.
Does testing require an outage?
Not necessarily. Many restore tests can use an isolated environment, and decision paths can be examined in a tabletop exercise. A live failover needs separate approval, detailed safety controls and a clear reason that lower-risk testing cannot answer the question.
What is the difference between a tabletop exercise and a restore test?
A tabletop walks participants through decisions, roles and information under a scenario. A restore test uses backup or replication data to recover technical components and validate function. They answer different questions and are often most useful together.
Can you define our recovery time and recovery point objectives?
HAI Consulting can facilitate the work and show the technical and operating cost of different targets. Business owners must approve the acceptable outage and data-loss limits because those are business risk decisions.
How often should recovery be tested?
Frequency should follow service criticality, rate of change, prior test results and external requirements. Material architecture, identity, backup or supplier changes are reasons to retest even if the normal calendar date has not arrived.
Can the engagement cover ransomware recovery?
Yes. The scenario can examine loss of production identity, compromised administrative access, unavailable systems, trusted restore points and the need to rebuild before reconnecting services. Incident response and legal work remain separate unless specifically scoped.
The platforms this work covers
Recovery testing covers the platforms the priority services depend on, including the ones people forget to test.
- On-premises and colocation data centre
- Virtualization platforms
- Microsoft Azure
- Amazon Web Services
- Microsoft 365 and Entra ID
- Cisco and F5 dependencies in the recovery path
Audits available
- Backup and recovery audit: coverage, isolation, retention and whether a restore has actually been proven.
- Identity recovery audit: whether administrative access survives an identity-platform outage.
Other ways we can help
Engagements are often scoped together; these are the closest neighbours to this one.
Need evidence that a critical service can be recovered?
Share the service, current backup or recovery method, target recovery time and the last test completed. If an audit, migration or renewal date matters, include it.