Systems that no one will test
Reflecting on a massive Brazilian government data leak he found in 2020, Christian Perone warns that advanced AI agents will soon automate large-scale vulnerability discovery for unpatched public systems.
summary is enough The essay offers a sobering perspective on automated AI vulnerability discovery, though technical readers can grasp the core argument quickly through a summary.
summary
In 2020, the author easily discovered a vulnerability exposing sensitive personal records for over 200 million Brazilians. Comparing that era to modern advancements in reinforcement learning and long-horizon AI agents, he argues that frontier labs or bad actors can now automate large-scale vulnerability hunting. He warns that developing countries and legacy government systems lack the resources or security baselines to withstand automated AI-driven pentesting. Commenters expand on systemic failures in IT administration, pointing out that poor baseline hygiene and insurance models perpetuate these vulnerabilities.
key points
- In 2020, the author found an access flaw exposing records for 200+ million Brazilians, which was quickly fixed after a direct phone call.
- Rapid advances in reinforcement learning and long-horizon AI agents make it trivial to build automated environments and discover similar vulnerabilities at scale.
- Commenters note that poor IT hygiene, lack of air gaps, and a reliance on cybersecurity insurance rather than robust defense perpetuate these infrastructure risks.
who should read it
Software engineers, security professionals, and policy makers interested in AI safety and infrastructure security.
discussion skeptical
consensus
Commenters largely agree that global IT infrastructure suffers from poor baseline hygiene, noting that porous government databases and a reliance on cyber insurance create systemic vulnerabilities.
pushback
A few commenters debate whether advanced AI agents represent a novel security threat or simply accelerate existing administrative negligence in IT management.
Every since the OPM hack of 2015, I've been apparent to me that my former field of IT administration has lost the plot. Nobody knows what a data diode is, or why you would use one. Systems that should clearly be air-gapped aren't.
While it's easy to lay this at the feet of AI getting better at hacking. I see it as an primarily an IT issue. We've collectively ignored the lessons of history, and made do with patch jobs over poorly chosen operating systems instead.
--
We need air gaps, data diodes, and capability based operating systems. Now that I'm retired, when I have the energy, I'm …
It argues that the core crisis is an IT administration failure—specifically the abandonment of air gaps and data diodes—rather than just an emerging AI threat.
Generated by an LLM from the article and thread. Check the sources before relying on it.