/CASE · 2026-09-08
Case: the pentest that was a prompt
Aurora affiliates told Cursor it was an authorized engagement. The agent ran the rest. Authorized work starts with a scope file the model cannot edit.
CloudSEK and Gambit reported Cursor sessions used against at least ten organizations. The operator framed the chat as an authorized pentest. The agent then did what coding agents do: run tools, follow the brief, keep going. An open directory later leaked months of those sessions. We are not reproducing the playbook.
The product accepted a role from the user message. Authorized work on our desk starts with a scope file the model cannot edit. No scope, no tools. See confirm and reverse-skill.
What the harness owed the room
- Allowlist: pentest tools are not a default inventory for a coding assistant.
- Schema: “authorized” is not a JSON field the broker validates.
- Sandbox: the session was the operator’s privileges, on their network.
- Confirm: high-impact actions (relay, certs, lateral) had no second control.
A system prompt that says “only run authorized tests” is a scolding paragraph. The operator typed “this is authorized.” The model believed the user. That is the product.
What we do instead
Engagements get a written scope: whose systems, which techniques, who can say stop. The agent does not get a shell until that file is loaded as policy, not as chat. Retrieved docs and user text cannot add tools. If the customer wants a coding assistant, keep pentest tools off the allowlist. If they want a red team, load a written scope before any shell.
Same lesson as the wiki page that was an instruction: untrusted text overwrote the job. Here the text was the job description.