Physics Steering: Causal Control of Cross-Domain Concepts in a Physics Foundation Model

원문 초록

Foundation models trained on text and images have been shown to develop abstract internal features that align with human concepts and that can be directly manipulated via activation steering to alter model behaviour. Whether scientific foundation models learn similarly abstract and domain-general representations has remained an open question. Inspired by recent work identifying single directions in activation space that control complex behaviours in large language models, we show that Walrus, a large physics foundation model, learns linearly steerable representations of physical phenomena. By computing the difference between activations representing contrasting physical regimes, we identify single directions in activation space that correspond to vorticity, diffusion, and even temporal progression. We find that injecting these concept directions back into the model during inference enables fine-grained causal control: vortices can be induced or removed, diffusion enhanced or suppressed, and simulations sped up or slowed down. Moreover, directions derived from one physical system also steer the behaviour of other physical systems, suggesting that they are domain-general. These results provide early evidence that scientific foundation models can learn general representations of physical principles, and further support for the Linear Representation Hypothesis.