Eight tests you run yourself, on your own installation.

1Sovereignty: protected data never leaves the perimeter

Claim under test. A request that would send data classified sovereign to a cloud model is refused inside the perimeter with 403 SOVEREIGNTY_BLOCK…, before any packet leaves.

Setup

1. Admin → Workspaces → create pov-sovereign, data classification Sovereign.
2. Admin → AI Gateway → make sure at least one cloud tier is configured (any provider) and one local tier.
3. Upload a document with a recognisable secret string (for example POV-SECRET-4711) to that workspace's knowledge.
4. Start capturing egress at the perimeter, filtered to the cloud provider's hosts.

Steps

1. In Chat, in workspace pov-sovereign, select the Cloud Reasoning tier and ask: "What is the secret in the uploaded document?"
2. Repeat over the API to see the exact code:

   curl -s -H "Cookie: $COOKIE" -H 'content-type: application/json' \
     -d '{"workspaceId":"pov-sovereign","modelTier":"cloud-reasoning","dataClassification":"sovereign","message":"What is the secret?"}' \
     $GW/chat/message -o /dev/null -w '%{http_code}\n'

3. Switch to the Local tier and ask the same question.
4. Admin → Governance → Audit log: filter action sovereignty.data_classification.

Observables

• Step 1: the UI refuses with the sovereignty message; no answer is produced.
• Step 2: HTTP 403, body code: SOVEREIGNTY_BLOCK_DATA_CLASSIFICATION.
• Step 3: the local tier answers with the secret and a citation to the document.
• Step 4: one audit row per refusal with the workspace, user and rule sovereign-data-cloud-tier.
• Perimeter capture: zero connections to the cloud provider during steps 1 and 2. This is the observable that matters; the 403 alone only proves the gate exists, the capture proves it sits before egress.

Pass: all five observables. Does not prove: that every code path is gated; that is what the audit row and the capture across your other tests (2, 5) accumulate.

Variant B (air gapped)

Install the air gap package with AANCER_OFFLINE_INSTALL=1 on a host whose firewall denies all outbound traffic. Boot, log in, run test 5. Expected: the platform is fully usable on local models, the licence activates from the offline token, and the firewall log shows no outbound attempts from the platform hosts (there is no call home).

2Agent safety: an unauthorised action is technically denied, not merely discouraged

Claim under test. An agent's tools are bounded by its signed passport (scope, classification ceiling, budget) and by the seat. A tool call outside the envelope is refused by the platform with a PolicyDenied result the model cannot talk its way past, and the refusal is audited.

Setup

1. Agents → Design → create pov-agent with two tools: one read tool (for example Search knowledge) and one action tool (for example Append to dataset, or any connector write you have).
2. Certify it (Agent Administration → Certificates → Issue) with a classification ceiling of internal and scope limited to the read tool only.
3. Create a dataset pov_actions in an internal workspace and a second one in the pov-sovereign workspace from test 1.

Steps

1. As Platform Admin, chat with pov-agent in the internal workspace: "Append a row {note:'hello'} to dataset pov_actions." (The write tool is bound to the agent but not in the passport scope.)
2. Then: "Search the knowledge for POV-SECRET." (in scope).
3. Re-issue the passport with the write tool in scope; repeat step 1.
4. Now chat with the same agent from the pov-sovereign workspace and ask it to search knowledge.
5. As the Knowledge Worker seat (no agent.execute_code), ask any agent that has the Run code tool to "compute 2**64 exactly".
6. Admin → Governance → Audit log: filter agent.tool.call.

Observables

• Step 1: the agent reports it cannot perform the action; the run trace shows the tool call with result PolicyDenied / envelope.out_of_scope; no row was written (check the dataset).
• Step 2: succeeds, citation shown.
• Step 3: the row is written; the trace shows allowed.
• Step 4: PolicyDenied / sovereignty.classification_ceiling; the sovereign workspace is above the passport's internal ceiling; nothing was retrieved.
• Step 5: the code tool is refused for the seat (permission), the agent answers without it.
• Step 6: one audit row per tool call with outcome: denied and the decision rule above; the denied rows carry no tool arguments beyond their names.

Pass: all six. Does not prove: the model's judgement; it proves the platform does not rely on it.

3Revocation: a revoked passport stops within 30 seconds

Claim under test. Revoking an agent's certificate stops further tool execution within 30 s (the platform caches revocation checks for at most 30 s); the check fails closed if the revocation store is unreachable.

Setup

pov-agent from test 2, certified, with a workflow or prompt that makes it call a tool repeatedly (for example "search the knowledge for each of these 40 terms, one call per term").

Steps

1. Start the long running conversation and note the time (T0).
2. While it runs: Agent Administration → Certificates → the agent's cert → Revoke (or POST $GW/admin/agent/certs/<lid>/revoke). Note T1.
3. Watch the run trace; note the timestamp of the last allowed tool call (T2) and the first PolicyDenied / revoked (T3).
4. Start a new conversation with the revoked agent.
5. Optional fail closed check (Docker): docker stop <postgres> for 20 s during a run.

Observables

• T3 − T1 ≤ 30 s; every tool call after T3 is denied; the run ends with an honest error, not a hang.
• Step 4: refused immediately: "Invalid or revoked agent token".
• Step 5: tool calls are denied while the store is unreachable (never allowed by default), and resume when it is back.
• Audit: agent.cert.revoked and the denied tool calls.

Pass: T3 − T1 ≤ 30 s and fail closed behaviour. Does not prove: interruption of a single tool call already in flight (a running connector call completes or times out on its own budget, perCallBudget.maxWallMs).

4Connector depth: a real process end to end, not a logo on a page

Claim under test. The connectors you need support the actual operations, authentication and error handling your process requires.

Setup

Pick one real process with two of your systems (for example "new supplier in SAP → approval → record in ServiceNow", or M365 mail → extraction → SharePoint). Use test credentials on your side.

Steps

1. Automations → Design workflow → build the process with the platform's connectors: trigger, read, an approval step, a write. Use Test step on each step.
2. Run it once end to end with valid data. Approve in Approvals.
3. Run it with invalid data (a mandatory field missing on the write).
4. Run it with the target system's credentials revoked on your side.
5. Rerun step 2 while the target is temporarily unreachable (block it at the firewall for one minute), then unblock.
6. Activity & Results → open each run.

Observables

• Step 2: the write appears in the target system; the run shows every step's input and output; the approval shows who decided and when.
• Step 3: the run fails at the write step with the target's validation message surfaced (not a generic error); nothing partial was written.
• Step 4: the connector fails with an authentication error; the platform did not retry blindly; the credential is flagged in Connectivity.
• Step 5: the step retries and completes after the target returns; the run detail shows the retries.
• Audit: run.completed / run.failed events; connector.* rows name the host, never the payload.

Pass: all five. Does not prove: transaction semantics inside your target system; verify compensation logic in your own process design.

5Knowledge and RAG: citations, permissions, scanned documents

Claim under test. Answers cite the passages they came from; a user only ever sees what their seat may see; scanned PDFs are readable when a vision model is configured.

Setup

1. Two workspaces, pov-hr and pov-ops; upload 10 real documents to each, including at least two scanned PDFs and two documents that contradict each other (an old and a new policy).
2. The Knowledge Worker seat is a member of pov-ops only.

Steps

1. As admin in pov-hr: ask a question answered only by the scanned PDF.
2. Ask a question the two contradicting documents answer differently.
3. As the Knowledge Worker, ask the pov-hr question from pov-ops, and try to open the pov-hr knowledge set directly (/knowledge?workspace=pov-hr).
4. Delete one document; ask its question again.
5. Admin → Governance → Records of processing (Art. 30) → export.

Observables

• Step 1: an answer with a citation to the scanned file and page; the citation opens the passage. (If no vision model is configured the upload says so at once: "This looks like a scanned document. Extracting its text needs a vision model…", honest, rather than a silently empty index.)
• Step 2: the answer surfaces both sources with dates and says they differ, or cites the newer one and names the older; it does not blend them into one confident claim.
• Step 3: no answer from pov-hr content; the direct URL is refused (403 on the API, an honest empty state in the UI).
• Step 4: the answer no longer cites the deleted document; the audit shows knowledge.delete.
• Step 5: the export lists the knowledge sets as processing activities with their data categories.

Pass: all five. Does not prove: retrieval quality on your corpus at scale; run it on a representative 1,000 document set before deciding.

6Extension platform: build one department pack and take it through its life

Claim under test. A department's customisation is a signed pack of data: it installs, activates with parameters, upgrades, rolls back and retires cleanly, and a pack signed by an unknown key is refused.

Setup

Node 20 on a laptop; the tools/aancer-pack CLI from the release bundle; the customer guide chapter Authoring a pack. A workspace pov-dept.

Steps

1. aancer-pack keygen --out dept.key. Write manifest.json with one dataset, one business model, one dashboard with a KPI and a chart that emits a cross filter, one hook on dataset.rows.published, one theme (your colours), one home card. Sign it: aancer-pack sign manifest.json content/ -o dept-1.0.0.ancpack --key dept.key.
2. Platform Settings → Extensions → Import pack before adding your key.
3. Add dept.pub under Trust keys; import again; Configure & activate, binding pov-dept.
4. Import 50 rows into the dataset by CSV; open the dashboard; click a bar.
5. Change the chart title, bump to 1.1.0, sign, import, activate. Then Roll back to 1.0.0.
6. Retire the pack.
7. Modify one byte inside a signed .ancpack (for example edit content/theme.json in the zip) and import it.

Observables

• Step 2: refused, PACK_SIGNER_UNKNOWN; nothing stored.
• Step 3: "Bundle verified", items listed with any contrast warnings; after activation every item is Active, the dashboard appears in the navigation, the theme is applied, the home card shows, the hook shows managed by pack.
• Step 4: the dashboard renders your rows; the click filters the KPI and a chip appears; the hook's deliveries show one delivery for the CSV import (if its handler flow is published with an Event trigger; otherwise the hook shows handler paused, also correct).
• Step 5: the new title after upgrade; the old title after rollback; both versions listed.
• Step 6: dashboard, card, theme and hook are gone; the datasets' rows are retained for audit; stock branding is back.
• Step 7: refused with a hash mismatch error.
• Audit: extension.pack.imported|activated|rolled_back|retired, signature failures as denials.

Pass: all seven. Does not prove: third party pack quality; it proves the platform's boundary.

7Failure and recovery: kill things mid run

Claim under test. Runs, approvals and the audit ledger survive component failure; retries are bounded; the ledger stays intact and detects tampering.

Setup

Docker installation (Helm: use kubectl delete pod equivalently). A workflow with three steps and an approval in the middle (test 4's will do). A hook from test 6.

Steps

1. Start the workflow; while step 1 runs: docker restart <automation-runner>.
2. Start it again; while it waits for approval: docker restart <gateway>; then approve.
3. Import a CSV that fires the hook; immediately docker stop <postgres> for 60 s; start it.
4. Ledger integrity: GET $GW/admin/compliance/audit-chain/verify → note ok:true. Then, as the database superuser, attempt UPDATE audit_events SET action='x' WHERE id=(SELECT id FROM audit_events LIMIT 1); and TRUNCATE audit_events;.
5. If you have direct DB access and want the tamper detection itself: disable the trigger, update one row's payload, re-enable, and call the verify endpoint again.
6. GET $GW/metrics (with the metrics token) and /health.

Observables

• Step 1: the run ends failed with an honest error or completes after the runner returns, never a silent half run; the run detail says which.
• Step 2: the approval is still pending after the restart; approving resumes the run and it completes with the same run id.
• Step 3: the hook delivery is retried after the database returns (30 s, 120 s, 480 s back off) and delivered once; aancer_event_outbox_pending on /metrics rises then returns to 0; no duplicate run.
• Step 4: both statements are refused by the database ("audit_events is append-only"); verify still reports ok:true.
• Step 5: verify reports ok:false, brokenAt:<row>; the chain detects the change.
• Step 6: /health 200; /metrics lists gateway and outbox gauges.

Pass: all six. Does not prove: HA under sustained load; that is a sizing exercise with the Enterprise sidecars.

8Exit: take everything with you

Claim under test. A customer can leave with their data, their automations and their extension IP, and can keep an installed version running without the vendor.

Steps

1. Datasets: Export CSV on each dataset (GET $GW/datasets/<id>/export.csv).
2. Dashboards: Export as pack on each published dashboard, a signed .ancpack containing the dashboard, widgets, models and dataset schemas.
3. Workflows: Designer → Export JSON for each flow (and Import JSON on a second installation to prove the round trip).
4. Compliance: Governance → Export the audit log, the Art. 30 records and the DIFC Reg. 10 report (/compliance-log/export, /compliance/gdpr-art30/export, /compliance/difc-reg10/export).
5. A user's own data: Admin → Users → Access export (Art. 15).
6. Extension packs: keep your signed .ancpack files and your private key; the platform never holds the key.
7. Frozen version: on the air gapped host from test 1B, disconnect it from everything for a week; log in daily.

Observables

• Steps 1 to 5 produce complete files you can open outside AANCER (CSV, JSON, signed zip); the second installation imports the flows and runs them.
• Step 7: the installation keeps working; the licence, being an offline signed token with no call home, does not expire on disconnection (only on its own expiry date, which you can read under Platform Settings → Licence).

Pass: all. Does not prove, and we say so plainly: there is no single "export the whole installation" button today; the exit is the sum of the per object exports above plus a database backup, and the Backup sidecar is an Enterprise component. Source code escrow is a contractual matter, not a product feature.