Broken Face and the AI, Privacy, and Security Weekly Update for the Week ending July 21st 2026
EP 301.
In this week's update:
The world's biggest AI model repository just got broken into, and for the first time, nobody was typing the commands.
OpenAI's flagship model has a filing cabinet problem, and the fix looks a lot like "please stop giving it root."
A ransomware gang built malware that doesn't just lock your files - it goes hunting specifically for your AI models, because those are worth more.
Somewhere on corporate networks right now, a calendar invite dated the year 2050 might be quietly talking to a spy.
One missing digit on a license plate turned an innocent man's night into a month in jail - and the AI reading the plate never caught its own mistake.
Britain just spent two years and £1.8 billion building a national digital ID scheme. It's about to be scrapped in a single afternoon.
For years, the Pentagon's cloud support ran through China - and the paperwork Microsoft filed with the government never mentioned the word.
And after a summer of AI agents deleting production databases and quietly phoning home with your data, someone may have finally built the guardrail that actually holds.
Welcome back, everyone. This week's stories all orbit one uncomfortable question: what happens when we hand AI agents the keys - to our infrastructure, our credentials, our identities - before we've finished building the locks? We move from an AI-versus-AI breach at the world's largest model hub, through ransomware built to kill AI models on purpose, into an old-school surveillance scandal, and land, finally, on what might be the first real sign that agentic AI security is starting to grow up.
Let's get into it.
Hugging Face, the open-source platform that hosts more than 45,000 AI models and datasets for over 50,000 organizations, disclosed that its production infrastructure was breached last week by an autonomous AI agent system, from start to finish, with no human at the keyboard on the attacker's side.
The intrusion began with a poisoned dataset that abused two code-execution flaws in Hugging Face's data-processing pipeline: a remote-code dataset loader and a template-injection bug in a dataset configuration.
Once the agent had a foothold on a processing worker, it escalated to node-level access, harvested cloud and cluster credentials, and moved laterally across several internal clusters over the course of a single weekend, executing what Hugging Face described as many thousands of individual actions across a swarm of short-lived, self-migrating sandboxes.
Hugging Face says it found no evidence the attacker tampered with public models, datasets, or Spaces, and that its software supply chain came back "verified clean," though the investigation into whether partner or customer data was touched is still ongoing.
The most striking twist came during the response: when Hugging Face's own security team tried to use a frontier commercial AI model to analyze the attack, the model's safety guardrails refused to help with malware analysis, so the team turned instead to GLM-5.2, a Chinese open-weight model, running it locally without those restrictions to reconstruct the intrusion.
The company has since closed the vulnerable code paths, rebuilt every compromised node, rotated all affected credentials, alerted law enforcement, and is now telling other security teams to keep a vetted AI model on standby, because the next AI-driven attack won't wait for a human to notice.
So what's the upshot for you?
If your organization stores anything of value with a third-party AI platform, treat "autonomous attacker" as a line item in your threat model starting today, not next year. And if your own incident response plan assumes a human is always on the other end of the keyboard, this is the week to rewrite that assumption.
That was an AI agent breaking in. Our next story is an AI agent breaking things on its own - no attacker required.
US: OpenAI Admits Its Newest Model Can Delete Your Files, Calls It an "Honest Mistake"
OpenAI has confirmed what several of its power users already learned the hard way: its GPT-5.6 model family can, under the wrong conditions, delete files it was never asked to touch.
Investor Matt Shumer reported that GPT-5.6-Sol "just accidentally deleted almost all" of his Mac's files, and days later software engineer Bruno Lemos said the same model wiped his entire production database.
Thibault Sottiaux, OpenAI's engineering lead for Codex, explained on social media that these incidents cluster around a specific setup: full access mode enabled, with Codex running outside its sandbox and without auto-review, the safeguard that checks for high-risk actions before they execute.
In that configuration, the model attempts to override the $HOME environment variable to define a temporary directory, and Sottiaux says it "makes an honest mistake" and deletes $HOME instead of the folder it meant to clear.
OpenAI insists these incidents are rare, but the company is taking concrete steps in response: updating the developer messaging around permission modes, steering more users toward safer defaults, and adding harness-level safeguards on top of the existing sandbox and auto-review protections.
A full post-mortem detailing the root cause and additional mitigations is expected soon, but for now the burden of avoiding a wiped hard drive sits with users choosing the right permission mode before they let the model loose.
So what's the upshot for you?
If you're running any coding agent in "full access" or unsandboxed mode, that convenience is exactly the setting most likely to bite you.
Keep sandboxing, and auto-review turned on, and treat any AI agent with filesystem access the way you'd treat a new employee's first day: closely supervised, until it's earned otherwise.
Deleted files are bad. Deliberately destroyed AI models are worse - and that's exactly what our next story's ransomware was built to do.
Global: Ransomware Gang Builds Malware Specifically Designed to Destroy AI Models
The threat actor known as JadePuffer, previously documented as running the first ransomware campaign ever executed end-to-end by a large language model, has resurfaced with a purpose-built weapon called ENCFORGE, a compiled Go ransomware binary engineered specifically to destroy AI and machine-learning infrastructure.
Where JadePuffer's earlier campaign made do with improvised Python scripts and MySQL's own encryption function, ENCFORGE targets roughly 180 file extensions covering the entire modern AI stack: model checkpoints, vector databases, training datasets, and embedding indices in nearly every format in use, including LoRA fine-tune adapters and legacy GGML weights called out by name in the binary's own help text.
Entry came through the same door as before, a missing-authentication flaw in Langflow's code-validation endpoint, <tracked as CVE-2025-3248, that CISA added to its Known Exploited Vulnerabilities catalog back in May of last year.>
Once inside, the AI-driven operator ran through reconnaissance and credential harvesting, discovered a mounted Docker socket, and rebuilt its own delivery mechanism on the fly after its first attempt to fetch the ransomware payload failed.
Researchers at Sysdig say ENCFORGE encrypts using AES-256 in counter mode with an RSA-2048 key exchange, kills any process holding a file lock before encrypting, and self-deletes afterward, and they found no exfiltration capability or leak site in the binary at all, which sets JadePuffer apart from the double-extortion playbook most ransomware crews now follow.
That absence of a data-theft angle only underscores the point: this operation exists purely to destroy, and researchers estimate that rebuilding a single production AI model from scratch, once its weights are gone, can cost between $75,000 and $500,000 in compute and engineering time.
So what's the upshot for you?
If your organization runs Langflow or any similar AI-agent framework, patch CVE-2025-3248 immediately and check whether your model checkpoints and training data sit on the same accessible storage as everything else. Back up your model artifacts the same way you'd back up a database you couldn't afford to lose, because a locked model, unlike a locked spreadsheet, may simply be gone for good.
ENCFORGE went looking for AI models to destroy. Our next piece of malware went looking for something quieter: a place to hide in plain sight.
Researchers at Group-IB have uncovered an espionage implant called HollowGraph that swaps out the usual attacker-controlled command-and-control server for something far less conspicuous: a compromised Microsoft 365 calendar.
Instead of reaching out to external infrastructure for instructions, the malware rummages through calendar events on the hijacked account, picks up encrypted tasking hidden inside appointment details, and smuggles stolen files back out as attachments on calendar events dated decades into the future, some as far out as the year 2050.
Because all of that traffic flows through the legitimate Microsoft Graph API, it blends in with normal Microsoft 365 activity rather than tripping the alarms that watch for unusual outbound connections.
Researchers have linked HollowGraph to the Cavern framework, based on its shared use of Microsoft 365 calendars and Graph APIs as a stealthy communications channel, suggesting the technique may already be in use by more than one operator.
The approach represents a broader shift in how espionage groups think about command-and-control: rather than standing up new infrastructure that defenders can eventually fingerprint and block, they are increasingly repurposing the productivity tools already trusted and whitelisted inside the organizations they target.
A calendar invite thirty years in the future is exactly the kind of anomaly a human analyst might dismiss as a data-entry error rather than recognize as a covert channel.
So what's the upshot for you?
Security teams should start treating Microsoft Graph API activity, including calendar behavior, as a monitored surface rather than an assumed-safe corner of the environment.
An appointment nobody remembers scheduling, especially one dated far in the future or the past, is worth a second look before you dismiss it as clutter.
That story was about hiding in plain sight. Our next one is about a single hidden digit - and how much damage its absence can do.
Security researcher Bruce Schneier has drawn renewed attention to a case that shows exactly how automated license-plate tracking can go wrong when a small human error meets a confident piece of AI software.
The saga began when a New Jersey license plate, reported stolen as 34 03 DTM, was entered into a police report as just 34 DTM, dropping the two middle digits entirely; when that truncated plate was later checked against Flock's automated license-plate-reader network, the system matched it to an unrelated vehicle carrying a similar-looking plate.
The mismatch went undetected because Flock's recognition technology wasn't built to flag or question a five-character match against what should have been a seven-character original, and the resulting "hit" was treated by officers as solid corroborating evidence.
That false match became one thread in a wider pattern Schneier has been tracking: automated license-plate readers are now deployed widely enough, and trusted heavily enough by police departments, that a single data-entry slip upstream can cascade into someone's actual arrest downstream, with no human in the loop positioned to catch the discrepancy before it does damage.
The case echoes an earlier, similarly structured incident out of San Diego, in which a Flock hit placed a man at a crime scene he was five miles away from at the time, based on similarly thin corroboration.
Civil liberties advocates argue these aren't isolated glitches but a structural problem: license-plate-reader networks were built for speed and coverage, not for the kind of fuzzy-matching skepticism a human investigator would apply to an incomplete plate number.
So what's the upshot for you?
If you or someone you know is ever implicated by automated surveillance technology, ask specifically how the match was made and what margin of error the system allows, because "the computer flagged it" is not the same as "the computer was right." For policymakers and police departments, this is a clear argument for requiring a documented human verification step before an automated match becomes the basis for an arrest.
Surveillance without a safety net is one kind of privacy failure. Deciding to simply not build the surveillance system at all is another approach entirely - and that's exactly what just happened in Britain.
UK: Britain's Incoming Prime Minister Scraps the National Digital ID Scheme
Incoming British Prime Minister Andy Burnham will scrap the government's digital ID scheme as one of his first acts in office, according to a spokesperson for the new Labour Party leader, redirecting the resources that had been earmarked for the program toward other priorities, including the cost of living.
The scheme had already been labeled a "fiasco" by a cross-party committee of lawmakers before Burnham's decision, and the UK's Office for Budget Responsibility had estimated its cost at around £1.8 billion between the 2026-27 and 2028-29 financial years.
"All the time and resource that was going to be spent on a national ID scheme will go instead to where it's most needed," Burnham's spokesperson said, framing the cancellation as a matter of practical priorities rather than a principled rejection of digital identity itself.
The reversal marks a significant course change for a program that had absorbed years of planning and public debate, and it leaves the UK without the centralized identity infrastructure that supporters argued could streamline everything from benefits access to age verification.
Privacy advocates who had raised concerns about surveillance potential and function creep are likely to see this as a win, while proponents of digital identity modernization will point to the abandoned investment as a missed opportunity.
Either way, the decision underscores how quickly a flagship technology policy can be shelved when a change in leadership meets a program that never built durable public or political support.
So what's the upshot for you?
If you were tracking UK digital ID as a bellwether for how other governments might approach national identity systems, this is a reminder that political will, not just technical readiness, determines whether these programs survive. For anyone building compliance plans around an assumed digital ID rollout, it's worth revisiting those assumptions now rather than after the next budget cycle.
Britain chose to unplug a system before it was ever switched on. Our next story is about a system that ran quietly for years before anyone in Washington realized where it actually lived.
US: How Microsoft's "Digital Escorts" Quietly Routed Pentagon Tech Support Through China
ProPublica reporter Renee Dudley's investigation revealed that Microsoft had, for years, provided technical support to the U.S. Defense Department using engineers based in China, America's most significant cybersecurity adversary, through an arrangement the company called "digital escorts."
Under that arrangement, a China-based engineer who actually understood how to fix a given problem would write the solution and pass it to a U.S.-based "digital escort," who would simply copy and paste the code, allowing Microsoft to technically satisfy the government's requirement that only U.S. citizens or permanent residents handle sensitive data, while the real technical work happened overseas.
Dudley first found the arrangement through a job ad offering $18 to $28 an hour to Americans willing to serve as digital escorts, then tracked down people who had answered similar ads on LinkedIn, and ultimately confirmed the practice by finding Microsoft's own patents describing the digital-escort system, filed under the names of current and former Microsoft employees.
When Dudley reviewed what Microsoft had actually disclosed to the government about the arrangement, she found no mention of foreign engineers being involved, and no mention of China at all.
The investigation's impact was immediate: ProPublica published the story on a Tuesday, and by that Friday, Microsoft said it had stopped using China-based engineers to support Defense Department cloud systems, while the Pentagon opened its own investigation into whether any of those China-based engineers had compromised national security along the way.
This just goes to show how a workaround designed to satisfy the letter of a security requirement can hollow out its actual purpose, leaving the government believing a rule was being followed when the substance of that rule had been quietly bypassed for years.
So what's the upshot for you?
If your organization relies on subcontractors or "escort" arrangements to meet data-residency or personnel-citizenship requirements, this is the moment to audit whether those arrangements satisfy the rule in practice, not just on paper. Trust in a compliance framework is only as strong as your willingness to verify what's actually happening behind it, and this story is a reminder that nobody checked for years.
That story showed what happens when nobody verifies the arrangement behind the credentials. Our final story is about a company trying to build verification into the credentials themselves, before an AI agent ever touches them.
US: 1Password Lets Claude Log Into Your Accounts Without Ever Seeing Your Passwords
1Password has launched 1Password for Claude, a new integration that lets Claude complete browser-based tasks requiring a login or one-time passcode without the credential ever reaching the model, its memory, or Anthropic's systems.
When a task calls for a saved login, the user approves the request via Touch ID or Face ID, and 1Password injects the credential directly into the target site rather than exposing it to Claude at any point in the process; as 1Password's CTO Nancy Wang put it, "Claude knows it used your login; it does not need the password or one-time code in its context."
Access is scoped to the current task only and expires once that task finishes, and after every autofill, the system scans the page to confirm no secret was accidentally exposed, wiping any filled values if a form submission fails.
Alongside the Claude-specific integration, 1Password introduced Agentic Mode for all of its users, a browser-extension feature that automatically locks down the moment it detects any compatible AI agent taking control of the browser, hiding the extension's interface and limiting the agent to only the credentials explicitly granted for that specific task, with the rest of the vault staying out of reach entirely.
Agentic Mode works even when the Claude integration itself isn't configured, and 1Password says it's designed to extend to other browser-based AI agents as they gain similar capabilities, positioning this less as a one-off partnership and more as a template for how credential security needs to work once agents, not just humans, are the ones logging in.
The launch lands as researchers have separately demonstrated that AI browsers can be tricked via prompt injection into leaking credentials, a timing reminder that this kind of zero-exposure architecture is arriving alongside real, documented risk rather than ahead of it.
And for those of you wondering, apparently BitLocker is working on Agentic integration too; they just got pipped at the post.
So what's the upshot for you?
If you're using AI agents to handle logins or purchases on your behalf, look for this kind of task-scoped, zero-exposure credential model rather than any setup that hands an agent your raw passwords. The bar for trusting an AI agent with your accounts should be the same bar you'd set for a new hire: access to exactly what the job requires, for exactly as long as the job takes, and nothing more.
So in summary...
Hugging Face got “broken” by an autonomous AI agent that acted entirely on its own, and the company had to enlist an AI of its own just to understand what had happened to it. The age of the human hacker typing commands at a keyboard isn't over, but it just got some very fast, very tireless competition.
OpenAI's GPT-5.6 can wipe your files when it's run without a sandbox, and the company is calling it an honest mistake rather than a design flaw. Either way, the fix is the same: don't hand an AI agent more access than the task actually requires.
JadePuffer's ENCFORGE ransomware doesn't want your data; it wants your AI models gone entirely, and it knows exactly which file extensions to hunt for. When the thing being destroyed can cost half a million dollars to rebuild, patching the entry point stops being optional.
HollowGraph turned an ordinary Microsoft 365 calendar into a covert spy channel, hidden inside appointments nobody would think to question. The tools we trust most are exactly the ones attackers are learning to hide inside.
A single missing digit on a license plate cost an innocent man his freedom for a month, and the AI reading that plate never once questioned its own confidence. Automation without a human skepticism check isn't efficiency; it's just a faster way to be wrong.
Britain looked at its own £1.8 billion digital ID scheme and simply walked away from it before it ever went live. Sometimes the most responsible technology decision a government can make is choosing not to build the thing at all.
Microsoft ran Pentagon tech support through China for years behind a paperwork workaround that never mentioned the word, and it took an investigative reporter, not an audit, to surface it. A compliance checkbox is only as honest as the people who never bothered to look behind it.
And 1Password built a credential system where Claude can log you in without ever seeing your password, treating AI agents like a genuinely new class of identity that needs its own guardrails. It's the first real evidence this year that agentic AI security might be catching up to agentic AI capability.
That's your week: an AI catching an AI, a model deleting files it was never asked to touch, ransomware hunting AI models specifically, a spy hiding in your calendar, a plate reader that never doubted itself, a government that chose not to build what it started, a support contract that hid where the work actually happened, and a password manager finally treating your AI agent like the new employee it is.
The throughline is trust, and this week showed us both how badly it can be misplaced and what it looks like when someone builds the guardrail first instead of after the breach. So this week's challenge: find the one system in your life that's currently running on blind trust, and go put a checkpoint on it.
And that brings us to our quote of the week, from Ronald Reagan - 'Trust, but verify.'
Reagan borrowed that line from an old Russian proverb, and he used it to describe how two superpowers could cooperate without either one being naive about the other. It fits this week's stories almost too perfectly. Hugging Face trusted its own infrastructure until an autonomous agent proved it needed verifying. Microsoft's paperwork said one thing while the actual work happened somewhere else entirely, because nobody verified it for years. And 1Password's whole pitch to you this week is that trust and verification don't have to be opposites: you can let an AI agent use your credentials while still verifying, every single time, that it never actually sees them. That's the balance every story this week was really circling. Trust the tools, trust the AI, trust the systems you rely on. Just build in the verification before you need it, not after.
That’s it for this week. Stay safe, stay secure, remain unbroken, and we’ll see you in se7en.
Comments
Post a Comment