SimGen: Simulator-conditioned Driving Scene Generation
원문 초록
Controllable synthetic data generation can substantially lower the annotation cost of training data. Prior works use diffusion models to generate driving images conditioned on the 3D object layout. However, those models are trained on small-scale datasets like nuScenes, which lack appearance and layout diversity. Moreover, overfitting often happens, where the trained models can only generate images based on the layout data from the validation set of the same dataset. In this work, we introduce a simulator-conditioned scene generation framework called SimGen that can learn to generate diverse driving scenes by mixing data from the simulator and the real world. It uses a novel cascade diffusion pipeline to address challenging sim-to-real gaps and multi-condition conflicts. A driving video dataset DIVA is collected to enhance the generative diversity of SimGen, which contains over 147.5 hours of real-world driving videos from 73 locations worldwide and simulated driving data from the MetaDrive simulator. SimGen achieves superior generation quality and diversity while preserving controllability based on the text prompt and the layout pulled from a simulator. We further demonstrate the improvements brought by SimGen for synthetic data augmentation on the BEV detection and segmentation task and showcase its capability in safety-critical data generation.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Graph-Convolutional-Beta-VAE for Synthetic Abdominal Aorta Aneurysm Generation
$I^{2}$-World: Intra-Inter Tokenization for Efficient Dynamic 4D Scene Forecasting
How Good Are Synthetic Requirements ? Evaluating LLM-Generated Datasets for AI4RE
World Model-Based End-to-End Scene Generation for Accident Anticipation in Autonomous Driving
TigAug: Data Augmentation for Testing Traffic Light Detection in Autonomous Driving Systems
From 2D to 3D Cognition: A Brief Survey of General World Models
토론
이 논문과 연결된 포럼 글이 아직 없습니다.