Failure Tolerant Training with Persistent Memory Disaggregation over CXL
원문 초록
This paper proposes TRAININGCXL that can efficiently process large-scale recommendation datasets in the pool of disaggregated memory while making training fault tolerant with low overhead. To this end, i) we integrate persistent memory (PMEM) and GPU into a cache-coherent domain as Type-2. Enabling CXL allows PMEM to be directly placed in GPU's memory hierarchy, such that GPU can access PMEM without software intervention. TRAININGCXL introduces computing and checkpointing logic near the CXL controller, thereby training data and managing persistency in an active manner. Considering PMEM's vulnerability, ii) we utilize the unique characteristics of recommendation models and take the checkpointing overhead off the critical path of their training. Lastly, iii) TRAININGCXL employs an advanced checkpointing technique that relaxes the updating sequence of model parameters and embeddings across training batches. The evaluation shows that TRAININGCXL achieves 5.2x training performance improvement and 76% energy savings, compared to the modern PMEM-based recommendation systems.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
LiteTopK: 장문맥 희소 어텐션용 통합 Indexer-TopK 커널을 위한 차원의 저주 활용
LiteTopK: Exploiting the Curse of Dimensionality for a Fused Indexer-TopK Kernel in Long-Context Sparse Attention
만족을 넘어서: 콘텐츠, 리뷰, 웰빙 간의 연관성 학습
Beyond Satisfaction: Learning Associations Between Content, Reviews, and Well-Being
정교한 축출 정책이 실패할 때: LLM 접두 문맥 재사용을 위한 캐시 교체 정책의 재고
When Fancy Eviction Fails: Rethinking Cache Replacement For LLM Prefix Reuse
SeKV: 장문 컨텍스트 대규모 언어 모델 추론을 위한 계층적 시맨틱 메모리 기반 해상도 적응형 KV 캐시
SeKV: Resolution-Adaptive KV Cache with Hierarchical Semantic Memory for Long-Context LLM Inference
HyperVAttention: 비디오 디퓨전을 위한 시공간 클러스터링 기반 효율적인 희소 어텐션
HyperVAttention: Efficient Sparse Attention with Spatio-Temporal Clustering for Video Diffusion
일반 함수 근사 하에서의 최적·효율적 문맥적 조합 세미-밴딧
Optimal and Efficient Contextual Combinatorial Semi-bandits with General Function Approximation
토론
이 논문과 연결된 포럼 글이 아직 없습니다.