Characterizing Concurrency Mechanisms for NVIDIA GPUs under Deep Learning Workloads
원문 초록
We investigate the performance of the concurrency mechanisms available on NVIDIA's new Ampere GPU microarchitecture under deep learning training and inference workloads. In contrast to previous studies that treat the GPU as a black box, we examine scheduling at the microarchitectural level. We find that the lack of fine-grained preemption mechanisms, robust task prioritization options, and contention-aware thread block placement policies limits the effectiveness of NVIDIA's concurrency mechanisms. In summary, the sequential nature of deep learning workloads and their fluctuating resource requirements and kernel runtimes make executing such workloads while maintaining consistently high utilization and low, predictable turnaround times difficult on current NVIDIA hardware.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Confucius3-Math: A Lightweight High-Performance Reasoning LLM for Chinese K-12 Mathematics Learning
ConsumerBench: Benchmarking Generative AI Applications on End-User Devices
Vine Copulas as Differentiable Computational Graphs
FlashDMoE: Fast Distributed MoE in a Single Kernel
Diagonal Batching Unlocks Parallelism in Recurrent Memory Transformers for Long Contexts
CF-DETR: Coarse-to-Fine Transformer for Real-Time Object Detection
토론
이 논문과 연결된 포럼 글이 아직 없습니다.