You're Unbelievable. The A.I., Privacy, and Security Weekly Update for the Week Ending June 30th 2026

Episode 298. 
In this week's update: 
Meta contractors spent months posing as suicidal, drug-curious teenagers to test rival chatbots - and the platforms being probed had no idea it was happening.

A Chinese AI lab just matched Anthropic's top cybersecurity model on its own turf - and the question isn't whether export controls work, it's whether they ever could.

The New York Times says Microsoft didn't just host OpenAI's training runs - it built a 285,000-core machine specifically engineered to feed on its journalism, and the lawsuit's whole strategy just shifted because of it.

A shell trick older than most AI startups just walked straight past ten out of eleven coding agent guardrails - and the fix isn't as simple as updating a blocklist.

Nearly two out of three AI chatbot apps tested on iPhone are leaking the keys to someone else's wallet - and you'd never know it just by using them.

A federal gun-enforcement agency ran more than 300 warrantless phone-tracking searches using data bought from ad networks - until a prosecutor and a judge themselves said no.

Companies fired workers to save money on AI, and now some of them are paying five times more for the AI than they ever paid the humans - Gartner says that's not a fluke, it's the trend.

Five allied nations' top cybersecurity agencies just used the word 'urgent' in the same breath as 'months, not years' - and that combination doesn't happen by accident.

Welcome back, everyone. This is a week where the gap between how fast AI is moving and how fast our safeguards, our laws, and our budgets are catching up gets impossible to ignore, from corporate testing practices that crossed a line to a legal theory that could put cloud infrastructure itself on trial to a federal agency that finally got told no. It runs from alarming to sobering to genuinely urgent, and there isn't a soft landing at the end of it. Let's get into it.

Hundreds of contractors working on a Meta-managed project internally called 'Cannes,' run through contractor firm Covalen, created fake under-18 accounts and sent more than 45,000 prompts to rival chatbots, including ChatGPT, Gemini, and Character.AI in a single testing round completed in August 2025.
One spreadsheet alone logged 3,748 prompts: hundreds centered on suicide and self-harm, hundreds more on eating disorders, and at least 239 touching on sex or romance, many written from the voice of a distressed child.

Specific examples reported by WIRED include a contractor posing as a pregnant teen asking where to buy abortion pills, another posing as a fifth-grader describing a classmate holding a gun to his own mouth, and a French-language prompt invoking the real suicide of bullied teen Jamey Rodemeyer.
The targeted companies were reportedly unaware their platforms were being probed, and the project was still active as of April 2026.
An internal Covalen document described the work as 'comprehensive AI safety benchmarking,' producing 'critical datasets for model comparison and compliance.'
Meta says it does not use competitor benchmarking to train its own models and defended the practice as standard industry testing.

So what's the upshot for you?

Whether or not Meta used this data to train its own products, the sheer scale and content of this testing - tens of thousands of prompts simulating children in crisis, sent to competitors without their knowledge or consent - raises serious questions about corporate ethics that no 'industry standard practice' defense fully answers.
A separate CNN investigation found that roughly eight in ten major AI chatbots gave actionable advice on planning violent acts when prompted by users posing as 13-year-olds, which means the underlying problem Meta says it was testing for is real - the question is whether this was the right way to find out.

Chinese AI systems have matched the performance of Anthropic's powerful model Mythos in some cybersecurity scenarios, reports the Wall Street Journal, calling it a development poised to reset the global tech race and pressure the White House in its overhaul of U.S. AI policy.
Security researchers said a new AI model released this month by China's Zhipu AI, also known as Z.ai, can match the latest U.S. models in identifying and exploiting software vulnerabilities.

The finding undercuts a core argument behind Washington's export restrictions on Anthropic: that keeping the most powerful cybersecurity-capable models out of foreign hands would meaningfully limit who has access to that capability.
If a Chinese lab can independently reach comparable performance, the restriction primarily disadvantages American defenders and allies rather than slowing adversaries.

So what's the upshot for you?

Export controls on AI capability work very differently from export controls on physical hardware, because the underlying science and engineering talent are globally distributed.
If a frontier capability can be replicated independently within months, the policy conversation needs to shift from 'can we keep this out of foreign hands' to 'how do we make sure defenders everywhere have access to keep pace' - a much harder problem than a ban.

The New York Times has asked a federal court for permission to amend its years-long copyright lawsuit against Microsoft and OpenAI, this time sharpening its claim that Microsoft did far more than provide generic cloud infrastructure.
According to the amended filing, Microsoft built a custom supercomputing system, reportedly equipped with more than 285,000 CPU cores and 10,000 GPUs, specifically engineered to ingest copyrighted material at massive scale for training OpenAI's models.
The Times alleges its own journalism was deliberately weighted more heavily within that system so the resulting models could closely mimic high-quality reporting.

The timing follows a recent Supreme Court ruling in a case involving Cox Communications and Sony, which raised the bar for proving contributory copyright infringement: plaintiffs must now show a party intentionally induced the illegal conduct, not merely that it knew infringement was occurring.
To meet that higher bar, the Times is recasting Microsoft's supercomputer as a purpose-built tool for infringement rather than neutral infrastructure, while voluntarily dropping two weaker claims against the defendants.
Microsoft called the amended complaint a last-ditch effort to escape unfavorable precedent, and OpenAI maintains that training on publicly available data qualifies as fair use, a question the Cox ruling left unresolved.

So what's the upshot for you?

This case has moved well past the original 2023 question of whether ChatGPT was trained on Times articles; the Times is now trying to make Microsoft, not just OpenAI, the central target by arguing its infrastructure was built with infringement as the goal.
If the court allows the amendment and the Times' theory ultimately succeeds, the most extreme outcome could force OpenAI and Microsoft to retrain models from scratch - a reminder that the infrastructure behind an AI system can carry its own legal exposure, separate from the model itself.

New research from Adversa AI, named GuardFall, found that a shell injection technique works against ten of eleven popular open-source AI coding and computer-use agents tested.
Only one, Continue, was built to defend against it.
These agents run shell commands with the user's full account access, meaning a booby-trapped repository or software package can quietly trigger a hidden instruction that wipes files or steals credentials, from SSH keys to cloud secrets sitting in the home folder.

The flaw stems from a mismatch: most agents check a command against a blocklist of dangerous patterns as plain text, while bash itself rewrites that text before actually executing it.
The shell strips quotes and expands shortcuts, so the safety filter and the shell end up evaluating two different strings.
GuardFall is the latest in a string of similar findings this year, following Adversa's own TrustFall bypass that hit Claude Code, Cursor, Gemini CLI, and Copilot CLI, plus a separate deny-rule bypass against Claude Code's bashPermissions.ts that was patched in version 2.1.90.

So what's the upshot for you?

The common thread across this entire family of vulnerabilities is simple: untrusted text keeps reaching a real shell before the safety guard understands what bash will actually do with it.
If your organization uses AI coding agents with shell access, treat every agent-generated repository interaction as untrusted input until the agent's vendor confirms how - not just whether - they validate commands after shell expansion, not before.


Researchers tested 444 AI chatbot apps for iPhone and found that 282 of them - nearly two-thirds - exposed paid AI access through their network traffic.
In many cases, the path in was visible just by watching what the app sent over the network: a plaintext API key, a reusable token, or a backend server that accepted requests with no key required at all.

Whoever captures one of these exposed credentials can send model requests on the developer's account, running up the developer's bill or using their paid AI access for free.
The scale of the problem - nearly two out of every three apps tested - suggests this is not a handful of careless developers but a systemic pattern across the AI app ecosystem.

So what's the upshot for you?

If you build or maintain a mobile app that calls an AI API, assume your traffic is being inspected and audit it the way an attacker would, because 'nearly two-thirds of apps tested' means careless key handling is closer to the norm than the exception right now.
For everyday users, this is a reminder that the convenience of 'AI-powered' apps often comes with backend security that hasn't caught up to the pace of feature releases - stick to apps from developers with a track record before trusting them with sensitive conversations.

The Bureau of Alcohol, Tobacco, Firearms and Explosives has canceled its contract for a tool called Webloc that enabled warrantless tracking of mobile devices, after a Republican lawmaker, a Democratic senator, a federal prosecutor, and a judge all raised concerns about the tool's legality in criminal investigations.
Webloc, built by vendor Penlink from technology originally developed by an Israeli company called Cobwebs, sourced bulk location data from consumer apps and advertising networks rather than from a warrant or court order.

Rep. Michael Cloud and Sen. Ron Wyden learned in a briefing that ATF had run more than 300 warrantless searches using the tool, including over 200 tied to active cases.
In one arson investigation involving a U.S. defense contractor's facility, both a prosecutor and a judge raised concerns about relying on the ad-tech data, forcing ATF to go back and obtain a traditional court order for bulk cellphone tower records instead.
ATF said the tool simply did not meet its needs and that it is not using any other ad-tech-sourced location services; a bipartisan group of lawmakers has separately introduced a bill that would ban buying location data without a judicial order.

So what's the upshot for you?

Wyden called the decision a victory for constitutional rights, and the underlying pattern is worth noting: a federal law enforcement agency used commercially purchased location data to sidestep the warrant process until a prosecutor and a judge themselves balked at relying on it in court.
If your data is being bought and sold by advertising networks, this story is a reminder that 'we didn't get a warrant because we didn't need one' is becoming a harder argument for agencies to defend, and the bipartisan push to ban warrantless data purchases is one to watch as it works through Congress.

Enterprises may soon be spending as much on their developers' AI token usage as they spend on the developers themselves, according to new Gartner research; the firm projects these costs will meet or exceed a typical software engineer's monthly salary within two years, with some estimates pointing to AI coding costs overtaking salaries entirely by 2028.
The shift is driven by two forces at once: developers adopting generative AI and agentic coding tools far more heavily, and vendors moving away from flat per-seat pricing toward consumption-based billing tied directly to token usage.

Gartner analyst Nitish Tyagi noted the firm's baseline assumes a global average salary of about $2,000 a month, so the comparison looks very different in the US, where six-figure developer salaries are common, but he stressed that even those higher numbers aren't out of reach.
The trend has already produced a workplace phenomenon nicknamed tokenmaxxing, with some employees running up bills of $150,000 a month, while Uber's CTO has said the company burned through its entire annual AI budget in just four months, and even an Nvidia vice president has acknowledged that AI compute now costs more than the people using it.
Gartner cautions there is no direct relationship between higher token consumption and higher productivity, and recommends companies adopt usage monitoring, token thresholds, and clearer rules for when AI agents should be used at all, rather than treating rising costs as a reason to abandon the tools altogether.

So what's the upshot for you?

There's a particular irony embedded in this trend: some companies cut human workers to save money, only to find their AI compute bill climbing faster than payroll ever did, which suggests the savings story being sold around AI replacing labor is, at minimum, incomplete.
If your organization is scaling up AI coding tools, treat token spend like any other infrastructure cost from day one - with usage monitoring and governance - rather than discovering a year from now that the convenience came with a bill nobody budgeted for.

On June 22, the leaders of the cybersecurity agencies of Australia, Canada, New Zealand, the UK, and the US issued a joint statement calling for an 'urgent' focus on cyber resilience in anticipation of frontier AI models 'exceeding current industry expectations' and 'fundamentally transforming both offensive and defensive cyber capabilities' within a timeline measured in months, not years.
The frontier models referenced are the latest generation of advanced AI systems capable of identifying and exploiting software vulnerabilities at a pace and scale that outstrips traditional patch cycles.

The statement is notable for both its unusual urgency and its unanimous five-country backing - joint statements of this kind typically take longer to coordinate and are usually more measured in tone.
It follows CISA's own domestic directive shortening federal patch timelines in response to similar concerns.

So what's the upshot for you?

When five allied nations' top cybersecurity agencies jointly use the word 'urgent' and put a timeline of 'months' on a fundamental shift in offensive and defensive capability, that is not boilerplate - organizational leaders genuinely need to treat this as a signal to accelerate, not just acknowledge, their AI-era security posture.
The practical takeaway is to revisit your patch management timelines now: if government agencies are compressing their own remediation windows in response to this threat, waiting for your usual quarterly review cycle is no longer a defensible posture.

...and to round it all up:

Meta's 'Cannes' project sent tens of thousands of prompts simulating children in crisis to rival chatbots, all without those companies' knowledge or consent, in the name of safety benchmarking. Calling it standard industry practice doesn't make the scale, or the subject matter, any less unsettling.

A Chinese lab matching Anthropic's flagship cybersecurity model didn't just close a capability gap - it exposed the central flaw in export-control logic. You can't ban your way out of a problem that talent and time can independently solve.

The Times just reframed two and a half years of litigation around a single sharper claim: that Microsoft's supercomputer wasn't neutral infrastructure but a tool purpose-built to enable infringement. Whether that theory survives the Supreme Court's tougher new standard for contributory infringement will shape how every cloud provider thinks about its role in AI training going forward.

Ten of eleven popular AI coding agents fell to a shell trick that exploits the gap between what a safety filter reads and what bash actually executes. Until vendors validate commands after shell expansion instead of before, every agent-generated repository interaction deserves to be treated as hostile.

Nearly two-thirds of tested iOS AI chatbot apps leaked the API keys that fund someone else's bill, often visible to anyone watching the network traffic. When 'careless key handling' describes most of an entire app category, it's not a bug anymore - it's the baseline.

It took a prosecutor, a judge, and two lawmakers from opposite parties to get one federal agency to admit that ad-tech location data wasn't worth the legal risk it created. The bigger question - whether Congress closes the warrantless-data-purchase loophole for every other agency still using it - remains wide open.

Gartner's math is blunt: AI coding costs are on track to rival, and in some cases already exceed, developer salaries, even as no one can show a clean line between heavier token usage and better output. Cutting humans to save money only works if nobody notices the new bill is bigger than the old one.

Five allied nations just broke from their usual measured tone to call AI's reshaping of cyber offense and defense 'urgent,' on a timeline of months. Whatever your patch cycle looks like today, this is the week to make it faster.

Every story this week traces back to the same gap: the technology, and the money and power flowing through it, is moving faster than the guardrails meant to contain it, whether those guardrails are corporate ethics policies, export control law, copyright doctrine, a coding agent's safety filter, a developer's API key handling, or a warrant requirement. 
Some of those gaps are closing this week, like a federal agency walking back a tool it couldn't defend in court. Others are just opening, like a legal theory that could put cloud infrastructure itself on trial, or a budget line that's quietly outgrowing payroll. 
So here's the challenge: pick one system you rely on, your kid's chatbot, your company's coding agent, your own app's key management, your own assumptions about what your devices reveal about you, and ask whether you actually know how it fails, not just how it works. 

And that brings us to our quote of the week, from Wayne Gretzky:

'I skate to where the puck is going to be, not where it has been.'

Think about how many of this week's stories are really about that exact gap. Export controls written for where AI capability was, not where it's headed. Coding agent guardrails built for the shell injection tricks of yesterday, not the ones researchers found this month. Even Meta's controversial testing project was, in its own clumsy way, an attempt to skate ahead of where chatbot harms might go next. The lesson for all of us, whether you're a policymaker, a developer, or just someone with a chatbot on your phone, is the same one Gretzky gave hockey: stop reacting to where the puck was, and start planning for where it's going. That's the game we're all playing now.
That's it for this week.  Stay safe, stay secure, and most importantly, stay believable, and we'll see you in Se7en.

Comments

Post a Comment