Security auditing in the age of (good enough) AI
Trail of Bits used AI agents over six months to build custom analysis tools and formal models, uncovering critical bugs in the Miden zero-knowledge VM.
read it Provides a compelling case study on how AI changes the economics of building custom security tooling and formal verification models from scratch.
summary
Trail of Bits prepared for an audit of the Miden zero-knowledge VM by tasking AI agents with building an LSP server, a decompiler, a static analysis engine, and a Lean formal model from scratch over six months. This exploratory tooling enabled abstract interpretation and 95 machine-checked correctness proofs that would have been economically unviable previously. The process uncovered over 400 minor validation issues, two subtle bugs in core binary arithmetic, and a high-severity flaw that could have allowed forging Falcon signatures. The authors argue that AI has transformed the economics of security research, making expansive custom tooling feasible for complex codebases.
key points
- AI agents built a custom LSP server, decompiler, static analysis engine, and Lean formal model for the Miden zero-knowledge VM over a six-month preparation period.
- The tooling and abstract interpretation passes uncovered a high-severity bug in mod_12289 that could let a malicious prover forge Falcon signatures.
- Agent-driven formal modeling in Lean produced 95 correctness proofs and caught two edge-case logic bugs missed by standard unit tests.
who should read it
Security engineers, auditors, and developers interested in formal verification and practical applications of AI agents for code analysis.
discussion positive
I suspect you two are using the same words for different terms / connotations.
"Good enough" in colloquial speak usually means the minimum required for some particular requirement.
For security, there is usually no exact threshold that differs between insecure and secure. It's a spectrum that involves costs and tradeoffs, which are subjective value judgements.
A SaaS startup in pre-seed mode with no customers will have VASTLY different value judgements than a bank that handles $trillions in assets. Hence they will make very different security choices and "good enough" will mean very …
Clarifies that 'good enough' security depends heavily on an organization's risk profile and economic tradeoffs rather than an absolute threshold.
Generated by an LLM from the article and thread. Check the sources before relying on it.