Daily AI Paper

Capability evaluations are supposed to answer a deceptively simple question: what can a model actually do, especially if those abilities would be dangerous in the wrong setting? That matters because frontier models are increasingly being deployed in high-stakes domains, and for risks like cyber abuse, fraud, or CBRN-related assistance, you do not want to discover a dangerous capability only after release. The paper’s core warning is that elicitation-based audits have a fundamental blind spot under adversarial conditions: if someone intentionally hides a capability inside the weights, then even a white-box auditor with the full model may not be able to surface it. To understand the landscape, it helps to separate two ideas that often get conflated. Safety is about how a model behaves on average under normal use and normal testing. Security is about worst-case behavior under an active adversary. A lot of capability audits implicitly assume the model is trying to cooperate, or at least is not actively hiding. But there is a known phenomenon called sandbagging, where a model underperforms on evaluations to look weaker than it is. Related work has also built password-locked models and sleeper agents, where a trigger unlocks hidden behavior. Those systems are useful as model organisms for studying elicitation, but they do not fully answer the strongest adversarial question: can a malicious actor embed an arbitrary computation in the weights so that it is effectively impossible to recover or activate without a secret trigger? That is the question this paper tackles. The authors study a weight-based supply-chain attack. Imagine a hostile insider, a compromised model vendor, or some other attacker who can modify the model before deployment. The attacker wants to hide a capability so that the deployed model behaves normally during audit, but executes the hidden function when presented with an attacker-chosen trigger. The defender gets full access to the architecture and weights, and can apply any polynomial-time elicitation method: prompting, fine-tuning, noise, gradient-based trigger search, probing, or other white-box attacks. The point is not that the defender is powerless in an absolute sense, but that under the paper’s cryptographic assumptions, there are constructions for which recovering the trigger or determining the hidden capability is computationally infeasible. The key technical move is to go beyond hiding a fixed string or a simple behavior. Prior cryptographic backdoor work had shown that certain triggers can be hidden or made unelicitable, and there were earlier results for transformer-based backdoors. This paper extends that idea to arbitrary computations. Instead of just embedding a password-locked response, the authors compile an entire neural network into encrypted form and place it inside the host model’s weights. They use a cryptographic primitive called a digital locker, which is essentially a program containing encrypted data that cannot be deciphered without the right password, even if the adversary can inspect the full implementation. Here, the secret trigger is used to decrypt the hidden network’s weights. On each forward pass, the host model tries to decrypt the locker; if the trigger is correct, the hidden weights are recovered and executed, and the model reveals the embedded capability. What is important here is the guarantee. Under the stated cryptographic assumptions, it is computationally hard for a polynomial-time adversary to recover the trigger or even distinguish which of two equal-size capabilities is hidden. That is a stronger statement than just saying a particular attack failed in experiments. It means that, assuming the cryptography holds, no efficient elicitation method should reliably extract the concealed computation. The authors are careful to note what this does not mean. It does not prevent an auditor from learning the same task from outside data. It also does not necessarily make the presence of some strange compiled circuit completely invisible. The guarantee is about the trigger and the hidden capability, not about proving the model is ordinary in every respect. The paper also places this work in the context of neural algorithmic reasoning and model compilation. Modern networks can implement exact algorithms, and prior systems like Tracr and Stravinsky have shown that human-readable programs can be compiled into transformer weights. This paper uses a related philosophy, but instead of compiling a symbolic program into a transformer, it builds explicit neural circuits that can implement the cryptographic machinery. Their framework uses multilayer perceptrons directly as circuit components rather than treating them as lookup tables. That makes the constructions more efficient in depth and parameter count for the cryptographic setting they study, though it comes with a wider residual stream. For the proof of concept, the authors implement the constructions in PyTorch using SwiGLU MLPs and RMSNorm. They hide a small neural network trained to predict whether a set of reagents yields a candidate product. This is a binary chemical reaction feasibility task. It is deliberately constrained so that it is not directly useful for misuse, but it still serves as a proxy for CBRN-relevant capability because reaction prediction can lower barriers to synthesizing controlled substances or precursors. So the task is small, but the motivation is serious. The experimental question is whether standard elicitation methods can surface the hidden capability despite the cryptographic construction. The authors test several approaches. First is supervised fine-tuning, or SFT, where the model is directly trained on the task. They also try weight noising, which perturbs the model parameters to see whether the hidden circuit can be shaken loose. They use gradient-based trigger search, analogous in spirit to methods like Greedy Coordinate Gradient search, to try to discover the special input that activates the backdoor. And they evaluate latent adversarial training, which attempts to make models robust to adversarial perturbations in internal activations. The results are straightforward but important. In their experiments, supervised fine-tuning fails to surface the hidden capability, even when the models are trained directly on the task. Weight noising does not reveal it either. Gradient-based trigger search fails to recover the secret trigger. Latent adversarial training also does not break the construction. In other words, the standard tools that are often used to elicit latent behaviors do not reliably expose the hidden network in these model organisms. The authors are careful not to oversell the practical impact. Their implementation is a proof of concept, not a turnkey attack. It hides only a small binary classifier, and there is substantial overhead. They also note that the presence of a cryptographic circuit might itself be detectable in some settings, so this is not a claim of perfect stealth in every sense. But the main point is that the hidden capability can remain inaccessible to elicitation, even with white-box access, and that this is not just an empirical fluke but follows from the cryptographic design. They also discuss the scale of the risk. The harm depends on both the hidden capability and the authority of the host model. A small hidden circuit could, in principle, augment an already capable model and support misuse while remaining dormant during normal evaluation. The paper does not demonstrate that full scenario, but it argues that the possibility is enough to motivate stronger security research. There are a few limitations and caveats worth emphasizing. First, these are adversarial constructions, not something the authors claim naturally emerges from ordinary training. Second, the proof-of-concept is small and computationally inefficient compared with a real deployment-scale backdoor. Third, the guarantee is conditional on cryptographic assumptions, as is standard in this kind of work. Fourth, the construction targets the inability to elicit or identify the hidden capability, not the broader problem of detecting every form of anomalous model behavior. And finally, the experiments are on a proxy task, not on an actual dangerous chemistry model. Still, the broader message is strong. If a malicious party can modify weights before deployment, then capability audits that rely on elicitation have a worst-case security limit. A model can be built so that the dangerous behavior is not just hidden by training dynamics or poor prompting, but mathematically hard to recover under the usual computational assumptions. The authors release their models to help others test defenses, and that is probably the most constructive takeaway: if we want capability audits to remain meaningful in adversarial settings, we need methods that are robust not just to difficult models, but to models that are deliberately designed to resist being understood.