The Backups Were Encrypted Too: How to Build Real Cyber Resilience Before the Big Attack
The views expressed are my own and do not represent those of my employer.
Picture a CISO at a mid sized manufacturer. It is 2:47 on a Saturday morning and her phone will not stop buzzing. The on call engineer says the file servers are acting strange. Then production scheduling goes down. Then email. By 3:30 she is on a bridge call with eleven people, and someone says the sentence the whole company has been counting on for years. "Don't worry. We have backups."
Twenty minutes later, a different voice ends the call. "The backup server is encrypted too."
I have watched versions of this story play out at multiple organizations. Sometimes the attacker reaches the backup console. Sometimes it is the domain controllers or the virtualization layer. The details change but the pattern holds. The company believed it had a recovery plan. What it had was a document that nobody had ever tested against an adversary who knew exactly where to look.
I covered the basics of recovery in The Recover Function in NIST CSF 2.0 back in January 2026. If you are new to the topic, start there. This post picks up where that one left off. The question has moved on. Boards used to ask whether you had a recovery plan. Now they ask how fast the business can actually come back and how you know. This post is about building that answer, and proving it before an attacker tests it for you.
Prevention Fails Eventually, So Build for the Day It Does
Every CISO I respect says some version of the same thing in private. You will be breached. Not might. Will. Attackers need one path and you need to cover every path. That math does not stay in your favor forever.
None of this makes prevention pointless. Strong identity controls, patching, segmentation, and detection still stop most attacks and make the rest harder. But a security program that puts everything into keeping attackers out has a single point of failure. The day someone gets through, the program has nothing left to offer the business.
Resilience is the other half of the job. It means the business keeps operating while under attack, restores what was lost, and learns fast enough that the next attempt costs the attacker more. The measure is not how many vulnerabilities you closed. The measure is how many days the company spends unable to ship product, take orders, or pay people.
The numbers show where the bill lands. The Sophos State of Ransomware 2024 report put the average recovery cost at $2.73 million for organizations hit by ransomware, and that figure excluded any ransom. IBM's 2026 Cost of a Data Breach report puts the average U.S. breach at $11.5 million. Your own number will differ. But notice what the Sophos figure measures. It is the cost of getting back to normal, not the cost of the break in.
Start With the Minimum Viable Business
Technology teams love to start with systems. Start with the business instead. Sit down with the leaders who run revenue and operations and ask one question. If every system went dark tomorrow, what must run again within 72 hours for this company to survive?
The answer is rarely "everything." For a manufacturer it might be production scheduling, order entry, shipping, and payroll. For a hospital or a retailer the list looks completely different. The point is to force a ranking. Not every application deserves the same recovery investment, and the business owners, not IT, should decide the order.
Then set a recovery time objective and a recovery point objective for each item on the list, and get the business owner to sign them. This step exposes the gap. Imagine an IT team that publishes a four hour recovery time for a system that has never once been restored in under three days. That is not an objective. It is a wish. Treat every number as unproven until a timed exercise says otherwise.
Ask every system owner on your list one question: what happens on day three? Day one is annoying. Day three is when customers go elsewhere and suppliers stop shipping. The answer sets your real recovery objective, and it is usually shorter than the one IT wrote down.
Map the Dependencies That Stop Everything Else
This is where most recovery plans quietly fall apart. Applications do not restore themselves into thin air. They need identity, name resolution, network services, certificates, and a working virtualization layer underneath them. If those foundations are gone, the applications on top are just files on a disk. I walked through the infrastructure side of this in the PR.IR deep dive, and the dependency map is where that thinking becomes practical.
Identity comes first. If an attacker controls your directory, you cannot trust any account, any group, or any authentication decision. Rebuilding that trust is slow. Most teams have read about a full directory forest recovery and almost none have practiced one. The teams that try it for the first time during a live incident find it takes far longer than anyone planned.
Build a dependency map for the minimum viable business. Start from the applications on your 72 hour list and trace every layer beneath them. Include the services people forget: DNS, DHCP, the certificate authority, the hypervisor management console, the privileged access vault, and your own security tooling. Then set the restore order from the bottom up. Identity and network foundations first, core platforms second, applications third.
One more item belongs on that map. Your backup system has dependencies too. If it authenticates through the same directory the attacker just took over, it is not an independent recovery path. It is another target.
Build Backups That Survive an Attacker Who Is Hunting for Them
Modern ransomware operators go after backups early. They know a company with clean backups does not pay. So they look for the backup console, the credentials, and the storage, and they delete or encrypt them before they trigger anything visible.
The data backs this up. In the Sophos State of Ransomware 2024 report, 94 percent of organizations hit by ransomware said attackers tried to compromise their backups, and 57 percent of those attempts succeeded. Veeam's 2023 Ransomware Trends report found attackers targeted backup repositories in 93 percent of the attacks it studied, and in 75 percent of those cases the attack impaired the victim's ability to recover. Your backup design has to assume that hunt is already underway.
Ask your backup administrator one question. If I stole your credentials tonight, what could I delete? The honest answer is your real exposure, and it is almost always bigger than the architecture diagram suggests.
Immutable and offline copies. The classic rule calls for three copies on two media types with one offsite. Today the rule has grown. Add one copy that is immutable or offline, and require zero errors after restore testing. Immutable means no one can change or delete the data for a set retention period, not even an administrator account. If a single stolen credential can erase your recovery, you do not have a recovery capability.
A separate identity plane. Backup servers should not be joined to your production directory. Give the backup platform its own credentials, protect the console with multifactor authentication, and require two people to approve any deletion or retention change.
Clean room restores. Detection is slow. IBM's 2026 Cost of a Data Breach report puts the mean time to identify and contain a breach at 247 days. Attackers often sit in an environment for days or weeks before they detonate, which means your most recent restore points may contain their tools. Restore into an isolated environment, scan and validate, and only then reconnect to production. Build that environment before you need it.
Test Restores Until They Bore Everyone
The fastest way to learn what is broken in your recovery plan is to run it. Pick your tier one systems and do a full, timed restore from backup into your isolated environment. Write down how long each step took and what went wrong. Something will go wrong. A missing license key. An undocumented configuration. A restore that succeeds and produces a database nobody can connect to.
Run these tests on a schedule, not once. Quarterly works for the most critical systems. Track one metric above all others: the actual time to restore minimum viable operations, measured in a real exercise. It is the same logic behind why MTTR matters, applied to the whole business instead of a single incident. Compare that number to the one the business signed. The gap between them is the most honest risk figure your security program will ever produce, and it is the one your board needs to see.
Do not forget the people side. Systems restore faster than habits change. Ask business teams to practice working without their applications for half a day. Paper order forms, phone calls to customers, manual shipping documents. Teams that have rehearsed this lose far less during a real outage than teams that improvise.
Plan for the Day Your Normal Tools Are Gone
During a major attack, assume email is unavailable or unsafe. Assume your chat platform and single sign on are compromised or offline. If your incident response plan lives on a file share, you may not be able to open it.
Build an out of band communication path before the incident. Keep printed copies of the response plan and contact lists with the people who will lead the response. Set up a separate messaging channel on accounts that do not depend on corporate identity. Write down the phone numbers of your executives, your legal team, your response firm, and your key vendors.
Pre draft your messages too. Within hours you will need to talk to employees, customers, suppliers, regulators, and the board, and you will be exhausted when you do it. A legal clock also starts. For public companies in the United States, the SEC requires disclosure of a material cybersecurity incident within four business days of determining that it is material. That determination has to happen without unreasonable delay after discovery, so a company cannot stall the decision to hold the clock off. If you operate in Europe, other clocks run alongside it. GDPR gives you 72 hours to notify a regulator of a personal data breach, and NIS2 expects an early warning within 24 hours for organizations in scope. Your legal team needs to be inside the response from the first hour, not briefed after the fact.
Decide Who Decides Before the Crisis Starts
The worst delays in major incidents are rarely technical. They are decisions nobody knew who was allowed to make. Can the incident commander cut the connection to a plant or a business unit? Who authorizes shutting down a revenue system to stop the spread? Who speaks to a negotiator if the board wants to explore that path? Who calls the insurance carrier, and in what order?
Answer these questions in writing and get the executive team to approve them in advance. I made a similar argument about operational technology in Isolation Is Not Failure. Pre authorizing isolation actions means your responders are not waiting on a committee while the encryption spreads. Put a response firm and outside counsel on retainer before you need them. Read your cyber insurance policy first, because many carriers require you to use vendors from an approved panel, and finding that out mid incident is expensive.
Rehearse the Bad Day With the People Who Will Live It
A tabletop exercise with only the security team is a training session. A tabletop with the CEO, CFO, general counsel, head of operations, and communications lead is a rehearsal. Run the second kind at least once a year.
Make the scenario hurt. Remove identity and email at the start. Force a choice between restoring fast and preserving evidence for the investigation. Introduce a supplier who cannot ship because your ordering system is down. You are not trying to embarrass anyone. You are trying to find the broken handoffs while the price is a few hours of discomfort instead of a few weeks of lost production.
After each exercise, assign an owner and a date to every gap, and report progress to the executive team. A tabletop that produces no action items was just a meeting.
Getting the Organization Behind You: From Frontline Teams to the Board
Resilience spending competes with every other priority, and it is hard to sell because success looks like nothing happening. The way through is to stop talking about security and start talking about downtime.
With the board, speak in days and dollars. Skip the tool inventory. Say something like this: "In a major attack, the tested time to restore core operations is nine days. The business can tolerate three. Closing that gap takes this investment." That sentence is a business risk statement. It has a number, a tolerance, and a decision attached.
With the CFO, build the cost of a day of downtime together. Use lost revenue, idle labor, contractual penalties, and recovery expenses. That number turns an abstract risk into something the board can compare against the cost of fixing it. With the COO and business unit leaders, ask them to own their manual workarounds. Those are continuity decisions, not IT decisions.
With Legal, Risk, and Insurance, make them partners from the start. Legal should co own decision rights and disclosure planning. Your risk and insurance leaders can tell you how recovery capability affects coverage terms and premiums. Do not treat resilience as a security project in isolation. Build the coalition before the incident forces you to.
With managers and frontline teams, keep the message small and practical. They do not need the architecture. They need to know where the printed contact list lives, what to do if systems go dark, and who to call. Then start with a win. One successful, timed restore of a critical system is a better pitch than any slide deck. It proves the program works and shows exactly where the next dollar should go.
Key Points
- Resilience is measured in days of business disruption. Controls deployed is the wrong scoreboard.
- Define the minimum viable business with the business owners, then set recovery objectives they sign.
- Map the foundation layer: identity, DNS, network services, virtualization, and the backup system itself.
- Make backups immutable, separate from production identity, and validated in a clean room before you restore.
- Run timed restores on a schedule and report the real number to the board.
- Pre authorize decisions, pre draft communications, and build an out of band channel before you need them.
Pro Tips
- Write the first 24 hours as a separate plan from the first 30 days. Hour one needs a single page checklist that a tired person can follow at 3 a.m.
- Hold a kickoff call with your response firm before an incident. A firm that already understands your environment saves you a full day when it counts.
- Keep known good images and configurations for your critical systems in offline storage. Rebuilding from memory is slow and error prone.
- Ask your most important suppliers how they would operate if you went down. Then ask what happens to you if they go down. Your recovery depends on theirs.
- Make one executive, not you, the owner of the restore time metric. Recovery becomes a business commitment instead of a security request.
- Rehearse with executives, not just the security team. The decisions that stall a real response belong to people who are rarely in the room.
Pitfalls to Avoid
- Do not treat the backup dashboard as proof. Green checkmarks mean the job ran. They do not mean the restore works.
- Do not join backup infrastructure to the production directory. One compromised domain admin then reaches everything, including your way back.
- Do not let IT set recovery objectives alone. Numbers the business never signed will not survive a real incident.
- Do not restore from the newest backup without scanning it. You may be reinstalling the attacker.
- Do not let the plan age. Every cloud migration, acquisition, or major platform change can invalidate your recovery assumptions. Retest after each one.
Final Thought
Nobody gets credit for the attack that did not shut down the plant because the restore worked in two days. That is the strange part of this job. But the CISOs who build real resilience change what a worst day looks like for their company. A breach becomes an expensive bad week instead of an existential event. You will not stop every attack. Your job is to make sure that when one lands, the business already knows the way back, has practiced it, and can prove it. Build the capability now, test it until it embarrasses you, and fix what it shows you. The attack will come. Make sure it finds a company that already knows how to get back up.
If this gave you something useful, subscribe to InfoSec Made Easy for more straight talk on what it takes to lead in security today. Share it with a peer who is building a recovery program. They need this conversation before the bridge call, not during it. And drop a comment below: what surprised you most the first time you ran a real restore test? I read every one.
Comments ()