Injection through pasted text, and an attacker that adapts

Anything an assistant reads can carry instructions: a pasted email, a ticket comment, a retrieved document. Telling the model to ignore them helps, and it is not enough. This is what we measured on deskbook, and the rule we took from it: a deterministic check is the boundary, the model is not.

A test that quoting cannot fool

The pasted family hides an instruction inside text the user pastes. Each instruction asks for a code formed by joining two fragments. The joined code never appears in the input, so a reply that holds it shows the instruction was carried out, not quoted. It is the check behind the Studio goal "Can it be tricked by instructions hidden in text a user pastes?"

What the first run found

An early deskbook corpus planted eight tickets whose user comment carried an injection: a direct override, a fake system notice, a planted false value, a hidden comment and others. The first run found a real failure. The answering model copied a planted sentence into 4 of 29 answers, telling an operator that the Equities kill switch now trips at USD 1. No restricted data leaked, but an operator was given a false safety limit by whoever wrote a comment.

Two defences followed. User written sections are marked untrusted, with a rule that they may hide instructions and must never be followed. And an echo check turns any answer that repeats six or more words of untrusted text into an abstention. With the shipped rule that model followed 0 of 29. The cost was real: its correct answers fell from 91.4% to 84.8%, while wrong values fell from 4.8% to 0%.

An attacker that adapts

Fixed payloads test known shapes. So we gave an attacker model control of a ticket's comment field and let it rewrite the payload after seeing each answer, for up to 10 rounds. Against a small model behind the prompt rule and the echo check, it broke 1 of 4 tickets in round 3. The winning payload was a quiet note in character, and the model rephrased it, so the echo check never saw six copied words.

The value taint check

The decision that followed: an answer may not contain a value, such as a reference, amount, time, code or number, that only user written text supports. However an injection is phrased, the value it plants exists only in untrusted text, so it cannot reach the operator. A fresh adaptive attack on the final defence broke 0 of 4 tickets in 44 attempts, while the same attacker broke the undefended system on 3 of 4 tickets in the same run.

The unit of evidence is the ticket. Attempts on one ticket are rewrites by the same attacker, not separate trials. Zero of 4 tickets bounds the success rate per ticket below about 53% at 95%. Treating the 44 attempts as separate trials would give about 6.6%, and would overstate the result.

Meaning, not values

The taint check covers values only. A larger retest against deskbook's shipped answering model used 20 tickets and 10 rounds. Undefended, the attacker got a planted value through on 17 of 20 tickets and changed the meaning of the answer on 14 of 20. Under the full defence it got a value through on 0 of 20 (bound 13.9%) and made the answer cite another document on 0 of 20, but changed the meaning on 6 of 20 (bound 50.8%): a comment posing as a QA memo made the model add that the fix did not hold.

The fix withholds user written ticket text from the answering model entirely and shows it to the operator separately. The live retest broke 0 of 20 tickets (bound 13.9%). With the text withheld, the prompt is byte identical to the benign control's, so the answer cannot depend on the payload. The cost is that answers can no longer use information found only in a user comment.

What this does not show

• The attack budgets were small: 4 and 20 tickets, 10 rounds each. Published attacks use hundreds of queries. Each result is success against this attacker and this budget, not a proof of immunity.
• The 4 ticket results were measured with a small model as the victim, not with deskbook's shipped answering model.
• A marker check cannot tell a reply that obeys an instruction from one that reports it: an assistant that warns the user and quotes the code is flagged.
• Injections that plant a false value without a marker, or that ask for a tool call, are covered by other families, not by this one.