What practitioners actually say about running agents in production

QCon San Francisco 2026 runs 16 to 20 November at the Hyatt Regency San Francisco, and its programme makes an unusual admission for an engineering conference: the hard problem with AI agents is no longer getting them to work (InfoQ, QCon San Francisco 2026 session listing). Four of the headline sessions, from four different companies, each at a different point in the stack, converge on the same subject: what happens once the agent is already live.

What happened

Weiping Peng, a Distinguished Engineer at Airbnb, is presenting on how the company guardrailed its AI customer support agent, which acts on millions of real customer accounts and needs input sanitisation, classifiers, shadow testing and rapid-response mitigations to run safely (InfoQ, QCon San Francisco 2026 session listing). Liz Fong-Jones, Honeycomb's Technical Fellow, is speaking on making production observability legible to agents through an MCP server already used by more than 40% of Honeycomb's weekly active users, and the token economy and evaluation framework it took to get there (InfoQ, QCon San Francisco 2026 session listing). Brian Yang of OpenAI is describing an operating model that used coding agents from day one to build a product that reached $100 million in annual recurring revenue within six weeks, and the feedback loops and human ownership boundaries that made that speed survivable (InfoQ, QCon San Francisco 2026 session listing). Netflix engineers Joseph Lynch and Ayushi Singh round out the track with a systems talk on safe rollout strategies at the scale of billions of daily requests (InfoQ, QCon San Francisco 2026 session listing).

None of these are AI-safety talks in the abstract sense. They are operations talks, from teams that already shipped the agent and are now living with what it does.

Why this is not an isolated instance

The pattern these four sessions describe, build fast, then discover that the agent's blast radius outran the controls built for it, is the same one surfacing across the industry over the past two years.

Orchid Security's framing of agent identity, reported in an earlier AANCER article, is that agents accumulate authority past their original job the moment a second integration gets bolted on or a scope gets copied from a working service account, and that most organisations cannot state what an agent's credentials actually permit ([[agent-credentials-nobody-reviews]]). Reco's State of Agent Security 2026 found 80% of AI tools running with no oversight at all, and in smaller organisations, 414 unsanctioned AI tools per 1,000 employees (Reco, 26 August 2026; cited in [[ungoverned-prompt]]). The common thread in both is that production agents keep outrunning the review process that was supposed to govern them, whether the gap is in credentials, in tool scope, or in the sheer count of agents nobody catalogued.

Airbnb's guardrail work and Honeycomb's evaluation framework exist precisely because that gap is now visible from inside well-resourced engineering organisations, not only from outside audits.

What it means for a regulated enterprise

A support agent acting on millions of customer accounts, as Airbnb's does, is the kind of system a bank, insurer or public-sector body would need to defend in an incident review, an audit, or a regulatory inquiry. Under the EU AI Act's record-keeping obligations and under NIS2's management-level accountability for cybersecurity risk, the question an auditor asks is not whether the agent was guardrailed in principle but whether the organisation can produce evidence: what the agent was permitted to do, what classifiers or shadow tests caught before anything shipped, and what it actually did on a given account on a given day.

Airbnb needed shadow testing and false-positive management because classifiers are imperfect and the cost of a wrong block or a wrong allow is paid in real customer trust. A regulated enterprise carries the same imperfection with a sharper downside: a wrongly-blocked claim or a wrongly-approved transaction is not just a support ticket, it is a finding.

What actually addresses it

The mechanism all four QCon sessions reach for, under different names, is the same one: separate what an agent is permitted to do from what it actually did, record both, and compare them continuously rather than at review time. Honeycomb's approach makes production state legible to the agent itself through a defined, scoped tool interface rather than open-ended access. Airbnb's shadow testing runs new guardrail logic against live traffic before it is trusted to act. Both are forms of the same discipline: nothing an agent does should exist only as a side effect, it should exist as a recorded event you can query afterwards.

This is the operational core of agent governance, not a policy statement about responsible AI, but an audit trail that exists because the architecture produces it, and a revocation path that works without redeploying the system the agent was talking to.

What to check on Monday

Pick one agent already in production and try to answer four questions without asking the team that built it: what tools and data can it reach, what did it actually touch in the last seven days, is there a record of any blocked or escalated action it attempted, and could you revoke its access this afternoon without taking down anything else. If the answer to any of those requires someone's memory rather than a log, that agent is running the way the pre-guardrail version of Airbnb's support bot once did, before the work described at QCon existed to fix it.