Home Gadgets & Devices Researchers Recover Hidden Reasoning Steps from Encrypted AI API Calls

Researchers Recover Hidden Reasoning Steps from Encrypted AI API Calls

0
WinBuzzer


TL;DR

  • Trace Recovery: Alexander Panfilov and seven coauthors report recovering provider-decrypted reasoning text from tested AI APIs, making opaque fields in shared agent logs a data-exposure risk.
  • Public Logs: The team reconstructed 315,320 reasoning blocks from 6,708 public agent logs and found 704 distinct privacy artifacts in genuine non-benchmark sessions.
  • Limited Scope: The experiments did not establish broken encryption, provider-system access, malicious exploitation, or faithful access to a model’s latent computation.
  • Developer Action: Developers should treat opaque reasoning fields as secrets, remove them from shared logs, and rotate any exposed credentials found inside.

A team of eight researchers reports that encrypted reasoning blocks returned by major AI model APIs can sometimes be replayed to a compatible, less guarded model, which then transcribes the provider-decrypted text. The experiments covered combinations of models within Anthropic, OpenAI, and Google APIs. A separate scan of public development logs found secrets and personal information inside opaque blocks, including material absent from the visible conversations.

Alexander Panfilov and seven coauthors published the work as an arXiv version-one preprint. The team combined researchers from Germany’s University of Tübingen, Max Planck Institute, MATS Research, and Snyk. The researchers recovered text carried through particular API workflows; they did not access provider infrastructure or encryption keys themselves. No malicious campaign using the method was established. API clients can carry hidden model context into a later request.

How the Replay Exposes Hidden Text

Reasoning models can generate intermediate scratch text before returning a visible answer. Some APIs keep the full trace hidden while sending the client an opaque block that preserves context for a later call. The client returns that block to the provider, which can decrypt or verify it while rebuilding the model’s context. This raw hidden text differs from the structured summaries that products sometimes show users. For example, Since censoring its o3 reasoning model, OpenAI’s ChatGPT presents summaries rather than raw chain-of-thought. While chatbots usually show only a short summary, its opaque message blocks retain different, more detailed text.

Google’s Gemini API and OpenAI’s stateless call mode both let clients return opaque state, but their behaviors differ. Gemini returns thought signatures that clients send back with the associated content. For OpenAI stateless calls, developers can return opaque reasoning items and encrypted content

Recovering text required more than possession of random ciphertext. The researchers needed an unmodified block that the provider would accept in another request, a model within the same provider that could receive the block, and a less guarded model that could be induced to repeat the decrypted content. They also needed enough output capacity and, in some cases, repeated attempts. The provider performed the decryption for its model; the researchers did not crack this encryption.

Cryptographer Matthew Green had earlier shown that some encrypted blocks could be replayed across accounts and contexts. But his tests did not establish a general way to dump hidden text. Panfilov’s team reports that choosing a compatible, less guarded model and prompting it to transcribe its context turned replay into text recovery across tested combinations at all three providers.

What the Public Logs Contained

The authors first tested whether the decoded output was roughly complete. Across 120 Codeforces programming prompts, they decoded length closely tracked the hidden thinking-token count reported by the API, with one-to-one agreement for most prompts. However, their aggregate comparison cannot support an exact failure rate or a provider-by-provider success claim. Length agreement supports completeness by token count; it does not show that the natural-language trace faithfully represents every internal computation that caused the answer.

The team then examined public agent logs posted to GitHub and Hugging Face that still contained opaque blocks. The 120-prompt test compared decoded length with hidden-token counts; the repository scan counted blocks and privacy artifacts, with the privacy count filtered to genuine sessions.

The obtained artifacts included API keys, passwords, access tokens, and personal email addresses. Sixty-four appeared only in reasoning blocks and nowhere in the visible session. Sanitizing the readable conversation can therefore leave sensitive information inside an opaque block that remains in a published log. The paper separately reports 367 personal-information artifacts and 182 credentials without reconciling that narrower partition with the broader total of 704.

The authors also ran a separate 90-question experiment to ask whether a small fragment of proprietary reasoning changed the visible answers of models whose parameters are available to developers. Kimi K3 sometimes shifted toward the wording of Claude Opus 4.8 or GPT-5.6 Sol, while DeepSeek and Inkling did not show an equivalent pattern. The mixed outcomes cannot proof that Kimi or any other model was trained on stolen traces, however. The researchers instead established direct privacy and integrity risks by recovering private data from public blocks and separately demonstrating that inserted instructions survived replay.

Safety Risks Go Beyond Exposed Secrets

Private data was not the only material hidden from the visible answer. In one worked example, reasoning produced by Claude Opus 4.8 and decoded through Haiku 4.5 contained detailed discussion of vehicle-theft vulnerabilities even though the answer shown to the user was framed as defensive advice. The example shows how recovered text can reveal safety-relevant detail that a user never saw.

The authors also report placing malicious instructions inside an opaque block so that replaying the block carried an instruction absent from the plaintext transcript. The hidden instruction demonstrates that client-carried state can become a prompt-injection channel.

Provider Changes Leave the Current Risk Unclear

After recovering private data and demonstrating that inserted instructions could survive replay, Panfilov’s team alerted Anthropic, OpenAI, and Google in July, according to WIRED. All three providers reportedly adjusted their APIs before the study became public. Anthropic said it had started building short-term mitigations for the replay behavior, while Google and OpenAI declined to comment for that report.

Panfilov said the changes stopped the method from extracting private information, although some reasoning traces could still be recovered. Without an independent post-change reproduction in the available reporting, the durability and model coverage of the API changes remain unknown. Anthropic, OpenAI, and Google disclosed no exact controls, rollout state, affected-model list, or completion of a durable fix in the available reporting. Google still documents thought-signature reuse, and OpenAI still documents encrypted content for stateless calls. 

Developers who export or publish agent logs should therefore handle encrypted reasoning blocks and thought signatures like other secrets. Logs should exclude those fields unless they are required, access should be restricted when they are retained, and credentials found in previously shared traces should be rotated. The study shows that unreadable-looking API state can preserve sensitive text after the visible transcript has been cleaned.



Source link

No Comments

Exit mobile version