PersonaDrive: Human-Style Retrieval-Augmented VLA Agents for Closed-Loop Driving Simulation

원문 초록

Closed-loop driving simulators mostly fill their environments with traffic agents that behave alike, whether they are rule-based or learned. Recent methods that control driving style learn it from labels inferred from existing driving recordings, from synthetic data generated to reflect each style, or from reward weights estimated by a large language model (LLM). All of these are indirect estimates of a style, not recordings of people who were instructed to drive in it. We introduce PersonaDrive, a vision-language-action (VLA) driving agent that takes its style from retrieved human demonstrations. The demonstrations come from a new dataset in which 15 participants drove CARLA routes on a driving simulator under conservative, neutral, and aggressive instructions. PersonaDrive trains a lightweight retrieval head on features from a frozen image encoder and on control signals over per-style databases, and fine-tunes one VLA backbone over the retrieved demonstrations. Switching style only means searching a different FAISS index, with no retraining. On the Bench2Drive benchmark without style conditioning, PersonaDrive improves the Driving Score by 4.6% over SimLingo and by 2.5% over HiP-AD, a gain that fine-tuning without retrieval does not reproduce. With style conditioning, PersonaDrive outperforms the strongest baseline, Drive My Way, in every style, with a Driving Score 6.7% to 11.6% higher. Its driving also follows the instruction, since average speed rises by 19.8% from the conservative to the aggressive style. Each traffic agent is evaluated individually as a driving policy. The dataset and model weights are available on the project page: https://pervasiveautonomylab.github.io/personadrive-landpage/.