Compass: A Decentralized Scheduler for Latency-Sensitive ML Workflows
원문 초록
We consider ML query processing in distributed systems where GPU-enabled workers coordinate to execute complex queries: a computing style often seen in applications that interact with users in support of image processing and natural language processing. In such systems, coscheduling of GPU memory management and task placement represents a promising opportunity. We propose Compass, a novel framework that unifies these functions to reduce job latency while using resources efficiently, placing tasks where data dependencies will be satisfied, collocating tasks from the same job (when this will not overload the host or its GPU), and efficiently managing GPU memory. Comparison with other state of the art schedulers shows a significant reduction in completion times while requiring the same amount or even fewer resources. In one case, just half the servers were needed for processing the same workload.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
AKG: Automatic Kernel Generation for Neural Processing Units using Polyhedral Transformations
Elastic Deep Learning in Multi-Tenant GPU Clusters
Arctic Inference with Shift Parallelism: Fast and Efficient Open Source Inference System for Enterprise AI
Relative Entropy Pathwise Policy Optimization
DEARLi: Decoupled Enhancement of Recognition and Localization for Semi-supervised Panoptic Segmentation
Scaling Attention to Very Long Sequences in Linear Time with Wavelet-Enhanced Random Spectral Attention (WERSA)
토론
이 논문과 연결된 포럼 글이 아직 없습니다.