Same Claude, 2.6x Attack Surface: SADF Quantifies the Framework Every Web3 Agent Just Inherited

The same Claude Sonnet model becomes 2.6 times more attackable depending on the scaffolding wrapped around it. That is the quiet detonation inside SADF's orchestration-framework attack surface study — a DEF CON 34 AI Village release, now ricocheting through crypto wire channels. Direct API baseline: 15.5% Attack Completion Rate. CrewAI: 11.9%. LangChain: 18.1%. AutoGen: 20.0%. SmolAgents: 31.1%. Same brain. Different toolbox. The differential is architecture, not noise. For a Web3 ecosystem preparing to hand private keys to autonomous agents, this timing is not unfortunate. It is surgical. I do not use "surgical" casually. The 2020 DeFi summer embedded a lesson I have not shaken: when an anomaly appears in the execution layer, the culprit everyone blames is rarely the actual culprit. I tracked the first Uniswap V2 liquidity drains twenty minutes after the flash-loan vector started firing. The pairs bled because routing was exploitable, not because liquidity logic failed. The logic was sound. The order of operations around it was not. SADF's research is the same story at a higher altitude: the model is sound, the plumbing is not. Security is a promise; liquidity is the proof. The plumbing is orchestration software. SADF — the research effort led by Julie Brunias — locked Claude Sonnet as a fixed base, set a direct API call as the control, then measured attack outcomes across four frameworks: CrewAI, LangChain, AutoGen, SmolAgents. 5,119 evaluation rows. 32 attack payloads. All sealed inside a SimulatedToolEnvironment that never touches real credentials. This is the "fixed model, variable separation" methodology: hold the brain constant, vary only the scaffold, isolate the delta. That delta is the orchestration framework attack surface. It may be the first clean quantification of a threat class the industry has been gesturing at with vibes. The study's stated goal is not to rank models. It is to rank the infrastructure that carries them. Why now? Because agentic infrastructure has crossed from prototype to production, and crypto is the fastest deployment zone on the planet. The first wave of autonomous systems already holds transaction privileges: MEV bots adjusting strategies mid-block, governance delegates voting on treasury proposals, automated portfolio managers rotating positions across five chains. Nearly every AI-threaded Web3 stack ships on LangChain, CrewAI, or AutoGen. These frameworks were built for modularity and developer velocity. Adversarial isolation was a config-file afterthought, read by no one, reviewed by fewer. That is not a defect in SADF's sample. That is the industry's default state. The deeper consequence is a shift in the evaluation unit. The industry has treated the model as the unit of analysis for two years. SADF's entire apparatus argues the unit must be a triple: model + framework + tool environment. That reframing kills the implicit assumption that model security equals system security. The assumption was never true, but it was institutionally convenient. Budgets flow to model benchmarks because model benchmarks are clean and comparable. Framework security is messy, version-dependent, configuration-sensitive. Messy things get under-funded. SADF just attached hard numbers to the mess. The most consequential move is methodological self-correction. SADF found that naive substring matching — the lazy standard in agent benchmark suites — overestimates Claude's attack resistance by a factor of four to six. A model that verbally rejects an injected instruction but still executes it through tool behavior gets scored "safe" by a dumb parser. Refusal-filtered scoring fixes the artifact. Sonnet's true ACR: 15.5%. Haiku's: 22.3%. The gap between reported and real safety is not marginal; it is categorical. I caught the same disease in early 2021 when 15% of a "decentralized" PFP collection's metadata sat on failing centralized IPFS gateways. The asset looked alive on the surface. The infrastructure underneath was dead. Benchmarks carry the same infection: the visible score says success, the mechanics say compromised. SADF also hands the industry a shared vocabulary: eight failure modes, each anchored to a distinct layer of agent behavior. Tool Call Hijacking. Output Poisoning. Cross-Tool Injection. Memory Poisoning. RAG Poisoning. Delegated Authority Abuse. Multi-Agent Propagation. Context Boundary Violation. Any Web3 operator reading this list should feel a specific chill. These are not abstract ML classifications. They are primitives for attacking an agent holding transaction authority. Take Tool Call Hijacking. The agent receives a crafted instruction that converts a benign read call — balanceOf() — into a value-moving write: transfer(). On-chain, that is the line between a portfolio tracker and a drained EOA. Framework-level instruction parsing decides which side of the line you live on. SADF's data implies that framework choice moves that boundary by a meaningful margin, and the frameworks with the tightest enforcement performed best. Output Poisoning deserves equal attention in a financial context. The model's own generated response becomes the attack's delivery vehicle. A governance agent generating a voting rationale is generating a payload. Cross-Tool Injection extends the logic across the tool graph: tool A's output becomes tool B's instruction. In a Web3 stack, tool A might be a block explorer API feeding a price oracle; tool B might be the swap execution engine. The chain is only as safe as the least-boundaried plugin in the graph. This is the mechanical reason a smart contract audit cannot see the attack. The contract never misbehaved. The orchestration delivered the misbehavior to the contract. Memory Poisoning and RAG Poisoning are the slow-burn horrors. Agents persist state across sessions. Long-term memory stores task context, tool results, and user instructions. A poisoned memory writes adversarial content into that context, and the next session dutifully acts on it. RAG Poisoning corrupts the knowledge base the agent consults. SmolAgents showed a 20% RAG Poisoning completion rate; CrewAI's discrete task isolation avoided it entirely. In crypto, the knowledge base is IPFS documents, unchecked market endpoints, and consensus scraped at runtime. Every one of those is writable by someone. Treat none of them as trusted input. Context Boundary Violation is the report's most brutal stat: 64% ACR for SmolAgents, against CrewAI's 11.9%. Information from one context leaks its framing into another. Cross-chain messaging is a context-boundary factory. A message from sidechain X that overrides intent assumptions in mainnet context is the same violation at a different altitude. When an agent reads a signed message from chain X and applies it as authority on chain Y, that is Context Boundary Violation with financial settlement attached. Now the blind spots, because a forensic read demands them. The experiment runs in a simulated tool environment. Real tools have latency, permission layers, multi-tenant state, and partial failures. The SADF results are a lower bound on production attack completion; if anything, real-world rates should be worse. A simulated environment is a control; it is also a quarantine. The attacks that require multi-step real-world interaction — waiting for an oracle update, racing a validator — will not instantiate there. The permission boundaries that do not exist in simulation will not protect you in production; they will only mislead you. The next blind spot is the architecture gap. SADF's taxonomy claims coverage across eight architectures, but only five ACR data sets are reported in detail. That is a gap between the declared surface and the delivered evidence. The honest reading: three architectures produced data incompatible with the comparison frame. The cynical reading: the publication narrative ran ahead of the experiments. The SUPERSEDED folder — which retains an earlier version claiming coordinated disclosure across ten architectures — is the audit trail for that tension. I respect the transparency. I also respect the signal: the first version over-claimed, and the team course-corrected mid-disclosure. Better to revise openly than publish fiction. Downstream consumers must ask which revision they are citing. Then there is the payload sample. 32 payloads across 5,119 evaluation rows is a deep sweep over a narrow library. Payload selection was researcher-curated, which means rare-but-deadly attack paths are exactly where under-sampling bites. Real attackers do not respect a 32-item taxonomy. They find the 33rd path, the one the simulated environment never instantiated. Configuration sensitivity is a second unopened box: the study does not decompose how framework parameters — temperature, system prompts, tool-permission granularity — shift the ACR. Change the temperature and the compliance profile changes. Tighten tool permissions and Tool Call Hijacking falls. The ACR is a property of a configuration, not a law of nature. The model × framework interaction remains untested. Swap Claude Sonnet for GPT-5.4, DeepSeek, or Llama and the framework ranking may not hold. A metric that does not generalize across models is a procurement snapshot, not a security law. That is the boundary SADF chose, and it is defensible. But boundaries matter: a security metric without a documented configuration context is a benchmark in aspic. Pretty. Fragile. Unverifiable. The industry impact runs along three tracks, each touching crypto differently. Enterprise Agent assessment standardizes: the RFP that asked "which model performs best" now asks "which framework safely constrains the model." Framework providers must re-examine architecture choices; a framework that hardens its tool boundary will market that as a feature, and SADF's data is the first externally verifiable scorecard. Security vendors will productize the methodology; shops at the Unit 42 and CrowdStrike level have the sales channels and will wrap SADF-style evaluation into a service within quarters. The skill set shifts from bytecode reasoning to graph analysis of tool dependencies and memory state. Crypto projects building on these frameworks will inherit the cost as a compliance line item. The ones that skip it will inherit the risk. The commercialization channel is more speculative. Independent scoring breaks cleanly: technical conclusions at A-high, commercial at C-medium. There is no product, no entity, no pricing signal — only a method and a dataset. But the vacuum will fill. The ACR spread between CrewAI and SmolAgents is procurement arbitrage waiting for a wrapper. Security-Evaluation-as-a-Service is the obvious path: agent enters sandbox, runs the payload suite, leaves with an ACR certificate. Integrated into CI/CD, that becomes subscription revenue. In 2017 I spent 72 hours inside the 0x v2 exchange proxy hunting a reentrancy that the audit industry had not named yet. Now that audit style is a checklist. The same arc is starting here, and the first vendors to publish a framework-security badge will capture the market's attention. The vulnerability evidence is already arriving. Azure SRE Agent — CVE-2026-62830. Langflow — CVE-2026-9198. These are not hypotheticals. They are assigned CVEs in deployed agent infrastructure. Langflow in particular is middleware that crypto teams bolt onto pipelines for on-chain queries and automated reporting. A CVE there is a production exploit. The market knows how to price broken promises in this sector: dump first, ask questions later. Security is a promise; liquidity is the proof. The proof is often the liquidation event. The contrarian cut is about where the study surfaced. The report is circulating in blockchain media. On its face, category confusion — a cybersecurity paper about LLM orchestration, syndicated through Web3 wires. But it is precisely correct, likely for reasons the researchers did not intend. Traditional enterprises can firewall agents behind approval queues. Crypto cannot. Web3 agents hold keys. They sign transactions. They interact with contracts where a single altered tool output is an authorization to move funds. No human-in-the-loop layer survives sub-second market-making. Crypto is the canary colony for agent security — not because the tech is ahead, but because the economic pressure to remove human oversight is highest here. Every framework expansion, and every framework attack surface, lands directly on capital. That creates a perverse incentive mismatch. Framework providers compete on speed, modularity, and developer experience. Security hardening adds latency and constraints — anti-features in a market that rewards execution velocity. SmolAgents' high ACR is not incompetence; it is the natural product of a framework optimized for ease of use. The safest framework, CrewAI, got there via discrete task isolation, which is exactly the property that slows autonomous flow. Crypto, of all industries, is the most likely to optimize for speed and accept the risk. Volatility isn't the market; it is the speed premium applied to security. Check the on-chain data when the first framework-level exploit hits an agent wallet. Nobody will have a postmortem ready. The forensic trail will be written in tool calls and memory state, not contract bytecode. What you see on-chain is not always what you get: the chain displays the transaction. The chain does not display the poisoned RAG doc that sent it. For builders, the watch list is short. Framework security is a due-diligence document, not a footnote — put ACR-type data in the procurement review for any agent infrastructure. Tool outputs are adversarial by default; scope every tool permission to the minimum call surface. A model's visible refusal is not a security control; refusal-filtered scoring shows that seen rejection is not action prevention. When a framework updates, forks, or archives a release — as SADF did with the SUPERSEDED folder — assume the security posture changed, even if the changelog is silent. The chain does not update silently. Frameworks can and will. The next major exploit in this sector will not be a smart contract bug. It will be a poisoned RAG document steering an agent into a malicious approval, or a multi-agent propagation chain converting one compromised sub-agent into a signed transaction. Smart contract audits are necessary. They are no longer sufficient. The evaluation unit has changed: model plus framework plus tools. The sooner the market prices that triple, the less painful the tuition. Chaos is just data waiting to be organized. Get the data before the chaos organizes you.