Why Everyone Is Talking About AI Agent Leak Rates Reaching 86% Despite 'Unlearning'

A new benchmark reveals that large language models deployed as autonomous agents leak confidential data in 22% to 86% of queries, even when certified as having forgotten the information under standard AI unlearning protocols.

What happened

• On September 14, 2026, researchers Guangsheng Yu, Yanna Jiang, Qin Wang, Baihe Ma, and Xu Wang released version 2 of their paper (arXiv:2609.12808, first submitted September 11) introducing K-Bench [1].
• The benchmark tests 20 publicly available machine learning unlearning techniques across all six transmission channels of ReAct agents, including chain-of-thought, tool calls, tool observations, and summarization [1].
• When confidential data is stored in prompts or retrieval stores, agents leak confidential details in 22% to 86% of queries, despite standard unlearning frameworks reporting zero leaks [1].
• Out of 20 unlearning techniques evaluated, 0 successfully demonstrated provable removal of secrets residing in model weights (0% success rate) [1].

Why it matters

In artificial intelligence, "unlearning" refers to removing sensitive personal details, passwords, or proprietary corporate data from a model's memory [1]. (This is unrelated to popular culture homonyms, such as Canadian singer-songwriter Matt Brouwer's February 2005 album Unlearning, which reached No. 30 on the Billboard Christian Songs chart in 2006 [2]).

Existing AI unlearning benchmarks, such as TOFU and MUSE, evaluate only the model's final response channel [1]. Under those legacy tests, a model receives unlearning certification if it simply refuses to answer a query [1].

However, in practical agentic deployments where models use ReAct frameworks to browse, call external tools, and execute workflows, secrets bypassed the final answer screen and remained fully visible in tool observations and internal reasoning steps [1]. In structured retrieval environments, even if the final output was scrubbed, confidential text persisted verbatim inside tool observation channels, leaving the overall leakage rate unchanged [1].

The numbers

The evaluation of 20 unlearning techniques under K-Bench demonstrates a severe gap between legacy benchmarks and agent deployments [1]:

• Weight Erasure Success Rate: 0% (0 successful techniques ÷ 20 evaluated techniques × 100 = 0%) [1].
• Selective Forgetting: Only input distortion intervention achieved selective forgetting under the evaluated observers [1].
• Optimal Method: The highest-ranking unlearning method varies depending on the underlying base model, showing no single universal solution [1].

Fact-check

Claim: Existing benchmarks like TOFU and MUSE guarantee that confidential data is safe in agent workflows.

MISLEADING. Existing benchmarks inspected only the final answer channel and certified unlearning upon simple refusal [1]. In practical agent setups, K-Bench showed that data stored in prompts or retrieval stores leaked in 22% to 86% of queries across internal channels [1].

Claim: Zero out of twenty tested machine learning unlearning methods provably purged secrets from model weights.

TRUE. Across all 20 public unlearning techniques tested in the study, none provably eliminated confidential information when stored directly in model weights (a 0% success rate) [1].

Claim: Refusal tuning permanently erases underlying sensitive data from an LLM.

MISLEADING. The research team explicitly noted that refusal tuning methods merely "defend evaluated extraction without verified knowledge elimination," altering external output behavior without confirming that the information was destroyed inside the weights [1].

What's next

The K-Bench paper (v2 posted September 14, 2026 on arXiv cs.AI) establishes the K-Score, which measures weights, prompts, and retrieval stores separately while requiring that agent utility is maintained to certify unlearning [1]. Moving forward, developers deploying ReAct agents must inspect all six transmission channels—including chain-of-thought, tool calls, tool observations, and summarization—as any disclosure across any single channel constitutes a security leak [1].

Sources

1. K-Bench: A Benchmark for LLM Unlearning in Agentic Deployments — arXiv cs.AI 신규 논문
2. Unlearning — Wikipedia (en)