The ungoverned prompt, documented: the incident record every AI policy should cite

We made the general case in [The ungoverned prompt](article-ungoverned-prompt.html): employees adopted AI before their companies did, and the result is a data flow nobody logs and nobody can produce on demand. This piece is the evidence file. Every incident below is documented, dated, and sourced — the record a CISO can put in front of a board, and the one an AI policy should cite. The question it answers is not whether ungoverned prompting is risky in theory. It is what has already gone wrong in practice.

What has actually gone wrong when employees prompt public AI?

This is not a hypothetical risk; it is an incident record.

• Samsung, 2023. Within weeks of permitting ChatGPT use, Samsung identified three separate incidents of engineers pasting sensitive material into it — including semiconductor-related source code submitted for debugging and a recorded internal meeting submitted for minutes. Samsung banned generative-AI tools on company devices in May 2023 (Bloomberg, Forbes, May 2023).
• The Dutch DPA's breach files, 2024. The Autoriteit Persoonsgegevens issued formal guidance after receiving multiple data-breach notifications caused by employees entering personal data into AI chatbots — among them an employee of a medical practice who entered patient medical data, and a telecom employee who entered a file of customer addresses (Autoriteit Persoonsgegevens, 6 August 2024).

The pattern in both: the employees were not malicious. They were productive. The tooling simply offered no governed way to do what they needed, so the data went where the productivity was.

Even when your people do everything right, the provider can be the breach

An ungoverned prompt inherits every failure of the service it lands on.

• March 2023: a Redis client bug at OpenAI let some users see other users' chat titles, and for a nine-hour window about 1.2% of ChatGPT Plus subscribers may have had name, billing address and partial card details visible to others (OpenAI postmortem, 24 March 2023).
• July–August 2025: ChatGPT's "make this chat discoverable" share option ended with over 4,500 shared conversations indexed by Google — some containing names, résumés and confidential work content — before OpenAI removed the feature and began de-indexing (Fast Company; Search Engine Land, August 2025).

Neither incident required an employee to do anything wrong. The data was simply outside the company's perimeter, subject to someone else's bugs and someone else's product decisions.

Doesn't a governed suite like Copilot solve this?

It solves the perimeter problem and uncovers a different one: oversharing. An enterprise assistant answers with everything the signed-in user can technically access — years of "Everyone" links, inherited permissions and forgotten sites included. Concentric AI's analysis puts 16% of business-critical data in the overshared category — on average about 802,000 exposed files per organization — and Microsoft's own deployment guidance now centers on remediating oversharing with Purview and SharePoint Advanced Management before rolling the assistant out (Concentric AI; Microsoft, 2024–2025). The lesson generalizes: an AI assistant is a permission auditor you did not ask for. If retrieval is not permission-aware and logged, the assistant will find what your access model forgot.

What do regulators expect you to do about it?

Two signals matter.

• Entering personal data into a chatbot can itself be a notifiable breach. That is the explicit position behind the Dutch DPA's 2024 guidance — the notifications it cites were filed because an employee prompted an AI with personal data (Autoriteit Persoonsgegevens, 6 August 2024).
• The provider side is under enforcement, not just advisories. Italy's Garante fined OpenAI over €15 million in December 2024, finding among other things that user data had been processed for training without an adequate legal basis (Garante decision, 20 December 2024; OpenAI has said it will appeal).

An organization that cannot reconstruct what its employees sent to an AI service cannot answer the questions either development will eventually put to it.

What does it cost to leave the prompt ungoverned?

• No audit trail: you cannot list what left, so you cannot notify, remediate, or defend.
• Retention and training-use uncertainty: what a public service keeps, and what it learns from, is governed by its terms — which change — not by your policy.
• Access-model exposure: ungoverned assistants surface whatever permissions allow, which is rarely what policy intends.
• Policy theater: banning tools without offering a governed alternative moves usage to personal devices, where visibility is exactly zero. Samsung's ban was rational triage — it is not an end state.

What has "governed" to mean before AI touches company data?

• The assistant, retrieval and inference run inside your perimeter, so a prompt is an internal event, not an export.
• Every prompt and answer lands in an audit trail that compliance can actually read.
• Retrieval is permission-aware per user — the asker sees only what they are cleared to see, with citations.
• Sensitive work routes to local models by default; anything leaving the perimeter is a deliberate, policy-checked exception, not a default.
• Egress is default-deny: nothing talks out until someone decides it may.

Employees will keep prompting — they are right to; the productivity is real. The only durable answer is to give the prompt a governed place to happen.

Sources: Bloomberg / Forbes (Samsung ban, May 2023); OpenAI postmortem "March 20 ChatGPT outage" (24 Mar 2023); Fast Company & Search Engine Land (indexed shared chats, Aug 2025); Autoriteit Persoonsgegevens guidance (6 Aug 2024); Garante decision on OpenAI (20 Dec 2024); Concentric AI data-risk research & Microsoft 365 Copilot deployment guidance (2024–2025). Figures as reported by these sources.