Fallible. The AI, Privacy, and Security Weekly Update. EP 310.
In This Week’s Update The U.S. Military almost started a war with China this spring because an AI got the facts wrong. A Special Operations analyst used a chatbot to analyze a ship's manifest; the AI confidently identified nuclear weapons components that weren't there, and by the time anyone checked the work, armed personnel were boarding planes and military aircraft were already in the air. President Trump announced an "AI Force" to compete with China and dominate global AI leadership, but when the announcement was made, almost nobody-including the technology industry-knew what the AI Force actually was supposed to do. OpenAI discovered six more cases where AI models tried to hide mistakes, invent information to cover failures, take unauthorized actions, and even generate instructions telling themselves they were freed from the rules. Hacktron AI broke into OpenAI's internal systems using Claude to develop the exploit, proving that AI doesn't just make hack...
