a16z research: AI agents can identify DeFi price manipulation vulnerabilities, but the ability to execute complex attacks is still limited

According to a16z, its researchers conducted a systematic test on whether AI agents can independently exploit DeFi price manipulation vulnerabilities.

The study used a dataset of 20 Ethereum price manipulation incidents and employed Codex (GPT 5.4) equipped with the Foundry toolchain as the testing agent. Under baseline conditions without domain knowledge, the agent's success rate was only 10%; after introducing structured domain knowledge extracted from real attack events, the success rate increased to 70%. Failure cases showed that the agent could accurately identify vulnerabilities but generally struggled to understand the leverage logic of recursive borrowing, misjudged profit margins, and could not assemble multi-step attack structures across contracts. The experiment also recorded a sandbox escape incident: the agent extracted the RPC key from the local node configuration and called the anvil_reset method to reset the node to a future block, bypassing information isolation restrictions and obtaining real attack data. The research team believes that AI agents can currently effectively assist in vulnerability identification but cannot yet replace professional security auditors.