AI Release Gate
47 items · source
The "Do Not Trust" List
The security architecture should explicitly treat all of these as untrusted unless independently verified:
- user prompts
- uploaded files
- webpages
- emails
- documents
- database text
- Storage metadata
- search results
- vector results
- retrieved documents
- model output
- tool output
- agent output
- MCP output
- A2A messages
- generated SQL
- generated code
- generated URLs
- generated shell commands
- generated API parameters
- generated authorization claims
- generated role names
- model-generated policy decisions
Production Release Gate
Do not release an autonomous/high-privilege AI feature until:
- Agent threat model is complete.
- Tool inventory is complete.
- Agent identity model is complete.
- Tool authorization matrix is complete.
- Prompt injection testing is complete.
- Indirect prompt injection testing is complete.
- RAG security testing is complete.
- Memory security testing is complete.
- Cross-tenant testing is complete.
- Tool authorization tests pass.
- Output-validation tests pass.
- SSRF tests pass.
- Code-execution sandbox is verified where applicable.
- Cost/rate controls are verified.
- Human approval controls are verified.
- Audit logging is verified.
- Incident kill switch is verified.
- Production credentials are least privilege.
- Agent cannot disable its own security controls.
- Agent cannot obtain unrestricted production credentials.
- Agent cannot silently expand its tool permissions.
- Agent cannot approve its own privileged actions.
- Adversarial regression suite passes.
- Security reviewer approves the release.
# AI Release Gate ## The "Do Not Trust" List The security architecture should explicitly treat all of these as untrusted unless independently verified: * [ ] user prompts * [ ] uploaded files * [ ] webpages * [ ] emails * [ ] documents * [ ] database text * [ ] Storage metadata * [ ] search results * [ ] vector results * [ ] retrieved documents * [ ] model output * [ ] tool output * [ ] agent output * [ ] MCP output * [ ] A2A messages * [ ] generated SQL * [ ] generated code * [ ] generated URLs * [ ] generated shell commands * [ ] generated API parameters * [ ] generated authorization claims * [ ] generated role names * [ ] model-generated policy decisions ## Production Release Gate Do not release an autonomous/high-privilege AI feature until: * [ ] Agent threat model is complete. * [ ] Tool inventory is complete. * [ ] Agent identity model is complete. * [ ] Tool authorization matrix is complete. * [ ] Prompt injection testing is complete. * [ ] Indirect prompt injection testing is complete. * [ ] RAG security testing is complete. * [ ] Memory security testing is complete. * [ ] Cross-tenant testing is complete. * [ ] Tool authorization tests pass. * [ ] Output-validation tests pass. * [ ] SSRF tests pass. * [ ] Code-execution sandbox is verified where applicable. * [ ] Cost/rate controls are verified. * [ ] Human approval controls are verified. * [ ] Audit logging is verified. * [ ] Incident kill switch is verified. * [ ] Production credentials are least privilege. * [ ] Agent cannot disable its own security controls. * [ ] Agent cannot obtain unrestricted production credentials. * [ ] Agent cannot silently expand its tool permissions. * [ ] Agent cannot approve its own privileged actions. * [ ] Adversarial regression suite passes. * [ ] Security reviewer approves the release. 47 items · https://github.com/FarzamHabibi/pre-production-checklist · CC BY 4.0