Fabio Marcello Salvadori reports on the fundamental friction between high-level AI policy debates and the actual engineering logic controlling autonomous systems. While industry leaders argue over oversight bodies, the true security boundary of any AI agent comes down to a simple boolean function running inside local application code.
OpenAI started publishing reports on unexpected model behavior.
How Ambiguous Code Creates Vulnerabilities in Agent Actions
An independent stress test found a critical flaw in PIC, an open protocol maintained for checking agent actions before execution. No cryptography failed.
Consider an agent proposing a payment. Unless the protocol adds a rule of its own, this payload is valid, even though the identifiers are not unique and the evidence reference points at two different entries.
Why JSON Schema Fails to Resolve Conflicting Implementations
To apply that policy, the verifier has to work out which entry an identifier refers to. Two reasonable engineers resolve it differently depending on their implementation choices.
Implementation A, written in Python, stops at the first match:
def resolve(provenance, evidence_id):
for entry in provenance:
if entry["id"] == evidence_id:
return entry
return Noneconst index = Object.fromEntries(provenance.map(e => [e.id, e]));
const resolve = (id) => index[id];Given the same payload, Implementation A resolves the evidence to the trusted entry and the payment executes, while Implementation B resolves it to the untrusted entry and the payment is blocked. Both verifiers run the same protocol version, with the same policy and the same trusted keys. The only difference is how each one looks up an identifier.
Security engineers have seen this class of vulnerability before in HTTP parameter pollution, XML signature wrapping, JWT algorithm confusion, and HTTP request smuggling.
Separating Data Integrity from Cryptographic Authority
Another layer of ambiguity involves who gets to declare an input trustworthy. The proposal is produced upstream of the verifier, frequently by the very agent requesting permission to act. Honoring a trust field at face value lets the requester supply its own authority, which is the classic confused deputy problem translated into JSON.
Trust has to be derived inside the enforcement boundary. An attacker can supply a file together with the correct hash of that same file. The integrity check passes because nothing was changed, but nobody with any authority approved the action.
Otherwise, a genuine signature can approve one thing while the action itself is something else entirely.
