Insecure Output Handling Is the Quiet AI Risk

A model response can look polished and still be unsafe for direct execution. The quiet risk appears when generated output is passed into another system without validation.

011. Treat Output as Untrusted

Generated SQL, JSON, HTML, shell commands, workflow payloads, and emails should be validated like any other untrusted input. The source is a model, not a guarantee.

If an output can change data, trigger communication, or affect a customer, deterministic checks should run before the action.

Generated output needs validation before it enters business systems.

022. Validate Structure and Meaning

Schema validation catches malformed fields. Business validation catches fields that are well-formed but wrong: invalid status transitions, excessive amounts, impossible dates, or unauthorized changes.

Both layers matter. A perfect JSON object can still request a bad action.

033. Escape for the Destination

An output displayed in HTML, inserted into SQL, sent to a command line, or passed to a downstream API requires destination-specific handling.

Never assume that a model will consistently escape content correctly. Make the application responsible.

044. Prefer Drafts for High-Risk Content

For sensitive workflows, generate drafts rather than final actions. Let users review emails, data changes, and approvals before the system commits them.

Insecure output handling is preventable when AI is treated as a reasoning component, not an execution authority.