What a speculative-⁠decoding paper says about agent context nobody governs

Note on sourcing: the trigger for this article is a Reddit r/LocalLLaMA thread titled "[Paper] DLite: History-free Speculative Decoding for Agentic AI." The thread itself could not be fetched for this piece (Reddit blocked automated retrieval), and a direct link to the underlying paper could not be located through search. The only material available is the abstract fragment carried in the thread's title and preview text. Nothing below states a number, benchmark result or author claim from the paper itself, because none could be verified against a fetched source. The analysis instead treats the problem the title names — agentic workloads accumulating long, continuously evolving context — as the subject, using only previously verified figures from AANCER's own sourcing record.

What happened

A paper surfaced on r/LocalLLaMA under the title "DLite: History-free Speculative Decoding for Agentic AI," describing work on speeding up LLM inference for agents. The visible abstract fragment states the motivating problem plainly: speculative decoding speeds up inference by having a small draft model propose tokens that a larger target model verifies, but "agentic workloads introduce long and continuously evolving contexts and frequent multi-turn interactions," and under that kind of load the authors observed that existing draft models degrade.

That observation is an engineering one, aimed at inference speed. But it describes, as a side effect, something enterprises running agents should already be tracking for a different reason: agent context in production does not reset. It accumulates, turn after turn, tool call after tool call, and nothing about the architecture of a long-running agent session forces anyone to look back at what is still sitting in that context.

Why it is not an isolated observation

The inference-research framing is new, but the underlying fact about agent context has been visible from the governance side for a while.

Microsoft's own deployment guidance for M365 Copilot warns that an AI layer surfaces everything underneath it that was already reachable, including years of permission debt such as SharePoint folders with broken inheritance (Microsoft, 2025). An agent that holds a long-running, continuously evolving context has the same property one level up: whatever entered that context early in a session, intentionally or not, is still reachable by every later step unless something actively removes it.

Reco's State of Agent Security 2026 (26 August 2026), built from anonymised enterprise telemetry and analysis of 500 public Model Context Protocol servers, found 80% of AI tools running with no oversight at all, and 414 unsanctioned AI tools per 1,000 employees in smaller organisations. Agents built or extended inside that kind of unmonitored estate are exactly the ones most likely to carry long, unreviewed context from one integration into the next.

And the credential pattern described by Orchid Security co-founder Roy Katmor — permissions accumulating past an agent's original job as integrations are added one at a time — is the same shape of problem as context accumulating past what a session originally needed. In both cases nobody designed the growth. It just happened, step by step, and nobody was assigned to check it.

What it means for a regulated enterprise

If an agent's context is the thing an attacker, a careless integration, or an over-eager retrieval step can poison, then the context is also the thing an auditor should be able to ask about. Two obligations converge here.

The EU AI Act's record-keeping requirements assume an organisation can state what a system did and on what basis. A long-running agent whose context has absorbed a hundred turns of tool output, retrieved documents and prior user instructions cannot have "what it did" explained unless something recorded what was in scope at each step, not just the final answer.

NIS2's management-level accountability for cybersecurity risk extends to the same question from the other direction: if context accumulation is a known failure mode in agent architectures — which the speculative-decoding research literature is now describing from the performance side — then not managing it is a risk management gap, not a technical curiosity.

The practical exposure is concrete. An agent carrying forward a customer record, a secret, or a stale instruction from twenty turns earlier can act on it at step 120 without anyone having re-approved that it still belongs there.

What actually addresses it

The fix is not a faster draft model. It is making context a governed object rather than an implicit one.

That means a few concrete mechanisms, independent of any particular inference technique: context that is scoped per task rather than carried indefinitely across a session; a record of what entered an agent's working context and when, so a review has something to read rather than a transcript to re-derive; and a way to revoke or clear context quickly when something in it should not have persisted, without having to tear down the whole agent session to do it.

How AANCER answers

AANCER treats an agent's working context as something that is recorded, not assumed. Every request an agent makes — what was retrieved, which model answered, whether anything crossed the sovereign perimeter — is written to an append-only, tamper-evident audit ledger, so a question about what was in scope at a given step has an answer that does not depend on the agent's own memory of itself.

Scoped access works the same way context should: a Certified Agent's passport names the tools, data and spend it may reach, and that passport is revocable in under 30 seconds, without redeploying anything. That gives a concrete, fast control for the moment something is found in an agent's reach that should not be there, whether that is a credential, a document, or context that should have aged out. None of this is a claim about speculative decoding or inference speed; it is a claim about being able to answer, afterwards, what an agent held and why.

What to check on Monday

Pick one agent that runs long or multi-turn sessions in your environment — a support agent, a coding assistant, an onboarding workflow — and ask whoever owns it three questions: what is the maximum context it can accumulate before anything resets; is there a log of what entered that context and when; and if something needs to be purged from an active session right now, how long would that take and who can do it. If the honest answer to any of the three is "we don't know," that is the gap this paper's framing points at, independent of what the paper itself turns out to claim.

Sources

• https://www.reddit.com/r/LocalLLaMA/comments/1xasomp/paper_dlite_historyfree_speculative_decoding_for/ (fetch failed; title and abstract fragment only, as quoted above)
• Microsoft 365 Copilot oversharing-mitigation guidance, 2025
• Reco, State of Agent Security 2026, 26 August 2026
• Orchid Security, Roy Katmor, co-founder and CEO, on agent identity and permission accumulation, 2026