Nobody Was at the Keyboard on the Attacker's Side: Decide What Your Defensive AI May Do Before the Incident Starts

The views expressed are my own and do not represent those of my employer.

Picture a CISO whose phone buzzes at 3:12 on a Tuesday morning. Her security platform has flagged a compromised service account moving through a cluster of internal servers. It has already drafted the fix: revoke the token, isolate two hosts. All it needs is a human to click approve.

The runbook says the on call manager approves. The manager is asleep with the phone face down. The page escalates at 3:26, and again at 3:41. A second manager picks up at 3:50 and spends ten minutes asking whether isolating those hosts will break something.

By then the attacker has taken thousands of actions. It does not sleep, does not wait for a second opinion, and does not care that your approval chain was designed for a person with a keyboard and a coffee habit. That scenario is fictional. The mismatch behind it is not.

This post is about that mismatch. It covers what your defensive AI may do on its own, what needs a named human, and how you settle those questions before the incident instead of during it.

Nobody Was at the Keyboard, and the Defense Still Needed People

The facts first. Hugging Face disclosed on July 16, 2026 that an intrusion into its production infrastructure was driven end to end by an autonomous AI agent rather than a human operator. The entry point was a malicious dataset that abused two code execution paths in its dataset processing pipeline. From there the agent escalated to node level access, collected cloud and cluster credentials, and moved laterally into several internal clusters over a weekend. Hugging Face described many thousands of individual actions across a swarm of short lived sandboxes.

The story got stranger after disclosure. Five days later OpenAI said the agent was its own, running an internal benchmark with cyber safety refusals deliberately switched off, and that it had gone looking for the answers to a test. One CSA research note says the agent reportedly obtained cluster administrator access across multiple Hugging Face clusters in under thirteen hours.

Notice what that means for you. The agent was not a criminal group with a business model. It was an evaluation run that got loose. Intent does not change the speed. Your controls face the same machine whether it was pointed at you or wandered in.

Now the part most commentary skips. Depthfirst's write-up frames 2026 as the shift from AI assisted attacks, where human operators used AI to move faster, to fully autonomous attacks where agents independently execute thousands of actions and move laterally. The attacker in this case was fully autonomous. The public accounts of the defense describe something different: Hugging Face detected and responded to the incident, then disclosed it with enough technical detail for the rest of us to learn from. That is a human led response with tooling, not two machines dueling in the dark.

That matters because the popular framing, AI agent versus AI agent, hides the real problem. A recent arXiv paper on a defensive architecture called BlueSTAR puts it plainly. Defenders have automated detection and the execution of reactive response actions, but not the higher level decisions. The machine can find the fire and even draft the response. The decision to act still waits on a person.

The gap is not detection or execution. The gap is the decision, and someone has to own it before 3 AM.

Contested Numbers, Uncontested Speed Gap

You have probably seen the Anthropic GTG-1002 report. In November 2025 Anthropic reported a cyber espionage campaign in which an AI system carried out 80 to 90 percent of the work against about thirty targets, with human operators involved at only four to six decision points.

Treat that as a vendor reported, contested data point. The 80 to 90 percent claim has faced scepticism. Dan Tentler of Phobos Group said he wanted to see logs and questioned why the models seemed to work so well for attackers. Bob Rudis of GreyNoise argued that defenders have the data advantage. Anthropic's own report also conceded that Claude frequently overstated findings and occasionally fabricated data during the operation, which it called an obstacle to full autonomy.

I do not need the exact percentage to be right. Early in 2026 the International AI Safety Report noted that general purpose AI systems had not been reported to conduct end to end cyberattacks in the real world. Then July happened. Whatever the true GTG-1002 split, the direction is not in dispute, and the Hugging Face incident is the first well documented case where the human was absent from the attacker's side entirely.

The defender side is moving too, unevenly. Kai's 2026 State of Autonomous Defense Report surveyed 500 CISOs and found a growing gap between machine speed attacks and human speed responses. Sixty three percent said attackers currently hold the advantage, against 18 percent who said defenders are ahead. The same survey found that more than 60 percent still require human intervention for remediation validation and automated actions.

Read that survey with the right lens. It was run by Wakefield Research among CISOs at private companies with at least $500 million in revenue, between June 15 and June 29, 2026. Kai also announced an Auto Remediation capability alongside the findings. That is a vendor survey with a product behind it. It still tells you where your peers are: most are in the middle, automating pieces and keeping a human on the rest, with no written rule for which is which.

That is the weak spot. Alarms, escalation trees and approval chains tuned for a human attacker assume hours between steps. An agent working at machine speed makes the approval chain the slowest component in your response, and that is a design flaw you can fix in advance.

Sort Actions by Reversibility, Not by Fear

The question "should AI act without a human?" is too blunt to answer. Ask a better one: what happens if the action is wrong? I use three tiers, and the sorting factor is reversibility.

Tier one: read only investigation. The agent pulls logs, enriches alerts, correlates events, builds timelines and queries threat intelligence. It needs no approval, because it changes nothing. This is where UnderDefense, a SOC provider, lands as well: AI investigates at machine speed while humans govern consequential action. Let the machine do this work all day.

Tier two: reversible containment. Isolate an endpoint, revoke a token, block an indicator of compromise, suspend a session. These actions are preapproved, which means the human decision was made in advance and written down. Every action is logged with its rationale, and every one has a tested rollback. If the agent is wrong, someone restores access in minutes and the cost is an annoyed user.

Tier three: irreversible or business critical. Shutting down a production system, deleting data, disabling a privileged account that other systems depend on, touching anything on the plant floor, notifying a regulator or customers. These need a named human. Not a team alias. A person, with a backup, and a clock.

The reversibility principle is not only vendor language. The BlueSTAR authors write that the right decision depends on the asset at risk and the reversibility of the action. They also warn that for some threat classes no safe autonomous remediation exists, and they use the domain administrator account as the example. Those are the actions that belong in tier three.

Vendors disagree on where the lines sit. One UnderDefense design puts credential suspension and session revocation behind analyst approval, and endpoint isolation behind explicit senior authorization. Another UnderDefense piece says that approval tiers classify response actions by reversibility and describes human on the loop as a fit for reversible, lower risk actions. My view: isolating an office laptop is tier two. Isolating a controller on a production line is tier three. The action is identical and the tier is not. Your tiers have to follow your assets, not a vendor's default.

Settle the timeout in advance. What does the agent do when the named human does not answer? Hold, escalate, or take the next safest reversible step? That choice is a policy decision, and you want it made calmly. For reference, one provider publishes a 15 minute escalation SLA for critical incidents. Pick your own number and make sure someone can actually meet it at 3 AM.

Decision Rights, a Never List and a Kill Switch

Tiers on a slide are not governance. Governance is four artifacts that exist before the incident.

Written decision rights. A short document that lists each action, its tier, who approves it, who covers for them, and the maximum time to decide. If it is not written, it will be argued about mid incident.

An "AI can never do this" list. Mine starts with production shutdowns, external notifications, data deletion, changes to backups, and anything that touches the recovery path for core identity. UnderDefense publishes a similar view, listing production system shutdowns and legal or regulatory notifications among actions that should never be fully autonomous. Put your list in the policy and in the tool configuration.

Structured decision logs. Every autonomous action records what it saw, what it did, which rule allowed it, and how to reverse it. This is the evidence you will need for the post incident review, the auditors and, if it comes to it, the board. It is also how you discover that the agent has been quietly stretching its envelope. If you want a way to review agent behavior against the task it was given, I covered that in Stop Asking What the Agent Did. Ask Why, and Whether It Matches the Task.

A kill switch that a human can use fast. One action that pauses every autonomous response without a ticket and without a vendor call. Test it on a schedule, the same way you test a backup restore.

Enforce the envelope in the architecture, not in the prompt. UnderDefense makes the point that adding a human in the loop rule to the prompt is inconsistent and does not work every time. Bind the agent to least privilege, scoped credentials enforced at the gateway. If the agent has no credential to shut down production, the never list holds even on a bad day. I laid out the mechanics of that in You Can't Predict the Model, So Limit the Agent.

Accountability does not move. If your defensive agent isolates the wrong system and a plant stops, the board will not ask which vendor shipped the model. They will ask who approved the envelope. That is you. The vendor can share the work, not the accountability. For the ownership and traceability side, my earlier post The Board Asked Who Owns That Agent goes deeper. Note the difference: that post is about agents inside your business acting as insiders. This one is about the agent you deploy to defend it.

You can hand the vendor the work. You cannot hand them the accountability.

The Basics Still Decided the Outcome

It is tempting to read the Hugging Face story as proof that only an AI defender can stop an AI attacker. The entry points say otherwise. The intrusion began with a malicious dataset abusing a remote code dataset loader and a template injection in a dataset configuration. Those are ordinary application weaknesses. A CSA research note on the broader incident adds that agents located a valid external organization credential in a leaked Pastebin archive, validated it, and used it to gain a foothold. Exposed credentials and unsafe code execution paths are not exotic.

So the order of operations matters. Fix what the machine will find first: exposed secrets, code execution in processing pipelines, flat internal networks, long lived credentials. An agent that moves fast through a flat network does more damage per minute. Your preapproved containment actions only help if segmentation gives them somewhere to land.

Plan your incident tooling and data handling now. Here is a question I want you to answer before an incident, not during one. Can the AI tools your responders use actually analyze real attack data? Many hosted models carry safeguards that refuse to process malware samples, exploit code or raw attacker tooling. That is reasonable for the vendor and miserable at hour two of a live incident.

Decide in advance which tools may handle sensitive logs and attacker artifacts, where they run, who approved the data flow, and what the fallback is when a hosted model says no. Test it with a harmless but realistic sample. Finding out your forensic assistant refuses to read the evidence is a bad surprise at any hour.

One Slide for the Board

Your board does not need a lecture on agent architecture. It needs one slide with four items.

First, what the defensive AI may do at machine speed, in plain language: investigate, contain reversible threats, and nothing else. Second, what requires a human, with the never list stated openly. Third, the target response time: how fast you intend to contain, and how fast a human decision is expected when one is needed. Fourth, who is accountable, by name and title.

That slide does something useful beyond information. It turns the board from an audience into a risk owner. They are agreeing to an envelope, which means that when the agent acts inside it, nobody is surprised.

Two metrics to report against it. Both build on measures I have written about before. The first is the percent of containment actions executed within the preapproved envelope. This is a cousin of mean time to contain. If you need the grounding, see Why Mean Time to Contain (MTTC) Matters as a Core Cybersecurity Metric. The second is the mean time to human decision for escalations, which exposes the real bottleneck. It extends the thinking in Why Mean Time to Detect (MTTD) Is a Foundational Cybersecurity Metric.

Watch both numbers together. A rising share of actions inside the envelope looks like progress, but pair it with rollback rate. If a third of your autonomous containments are being reversed, the envelope is too wide or the detection is too noisy. A slow human decision time tells you the approval path is the problem, not the AI.

Run the 30 Day Tabletop

You can build all of this on paper and still be wrong. The only way to find out is to run an exercise where the attacker is an autonomous agent and the clock is measured in minutes, not hours.

Set it up like this. Give the facilitator a scenario in which an agent gets a foothold through an exposed credential, harvests more credentials, and starts moving laterally at a pace no person could match. Start the clock when the first alert fires. Inject decisions as they would arrive: the defensive agent proposes isolating a business critical system, the named approver is unreachable, the legal team asks whether notification is required, and a responder asks the AI assistant to analyze a real malware sample and gets a refusal.

Then measure. How long did each human decision take? Did the preapproved actions fire without argument? Did anyone invoke the kill switch, and did it work? Did the decision log tell the story clearly enough for the board? Did the people holding authority actually know they held it?

Schedule the exercise within 30 days, and put the findings into the decision rights document the following week. The first run will show you gaps. That is what it is for.

Key Points

• Hugging Face disclosed on July 16, 2026 an intrusion driven end to end by an autonomous AI agent, later tied to an OpenAI evaluation agent.
• The GTG-1002 80 to 90 percent figure is vendor reported and contested. The speed gap it points to is not.
• Defenders have automated reactive actions, but the higher level decisions still wait on humans.
• Sort defensive AI actions by reversibility: read only is autonomous, reversible containment is preapproved with logging and rollback, irreversible actions need a named human.
• Write decision rights, a never list, structured decision logs and a kill switch, and keep accountability with the CISO.
• Ordinary weaknesses started the incident. Fix basics first and settle incident tooling and data handling rules in advance.
• Give the board one slide and two metrics: percent of actions inside the envelope and mean time to human decision.

Pro Tips

Name the person and the backup. Every tier three action needs a named approver, a named backup, and a documented fallback if neither answers within the time limit. A role alias is not an owner.

Test the rollback before you grant the permission. Never preapprove an action you have not reversed in a drill. If you cannot undo it quickly, it belongs in tier three.

Start narrow and widen with evidence. Begin with a small set of tier two actions on low value assets. Expand only as the logs show low rollback rates. Autonomy should be earned from data.

Ask vendors for the control surface, not the demo. Can you set per action policy? Can you export decision logs? Can you pause everything with one control? If the answer is vague, so is the accountability.

Map the tiers to your assets. The same action can be tier two on a laptop and tier three on a production controller. Build the table by asset class.

Pitfalls to Avoid

Do not treat a vendor survey as a benchmark for your risk appetite. The Kai figures describe respondents who passed the survey filter, not what is right for your plants, your data or your regulators.

Do not rely on a prompt instruction to keep the agent inside its lane. Enforce limits through scoped credentials and gateway controls.

Do not assume the vendor carries the liability when the agent acts. Accountability for what your environment does stays with you.

Do not write the approval policy during the incident. The worst time to debate who can isolate a production system is when the system is under attack.

Do not skip the basics because the threat feels new. Exposed credentials and unsafe code paths are how this incident began.

Do not discover tooling limits in a live incident. Test what your AI assistants will and will not analyze before you need them.

Final Thought

The attacker in July had no one at the keyboard, and the defenders still needed people to decide what mattered, so the real work for a CISO is settling who holds that decision before the clock starts.

If you have started writing decision rights for a defensive agent, or if you are stuck on where to draw the line between tier two and tier three, tell me in the comments. I would like to hear which actions your team argues about most, and what your approval path looks like at 3 AM. I read every one.