Neural Cache: Bit-Serial In-Cache Acceleration of Deep Neural Networks
원문 초록
This paper presents the Neural Cache architecture, which re-purposes cache structures to transform them into massively parallel compute units capable of running inferences for Deep Neural Networks. Techniques to do in-situ arithmetic in SRAM arrays, create efficient data mapping and reducing data movement are proposed. The Neural Cache architecture is capable of fully executing convolutional, fully connected, and pooling layers in-cache. The proposed architecture also supports quantization in-cache. Our experimental results show that the proposed architecture can improve inference latency by 18.3x over state-of-art multi-core CPU (Xeon E5), 7.7x over server class GPU (Titan Xp), for Inception v3 model. Neural Cache improves inference throughput by 12.4x over CPU (2.2x over GPU), while reducing power consumption by 50% over CPU (53% over GPU).
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
MNN-LLM: A Generic Inference Engine for Fast Large Language Model Deployment on Mobile Devices
any4: Learned 4-bit Numeric Representation for LLMs
MathOptAI.jl: Embed trained machine learning predictors into JuMP models
CommVQ: Commutative Vector Quantization for KV Cache Compression
ConsumerBench: Benchmarking Generative AI Applications on End-User Devices
Versatile and Fast Location-Based Private Information Retrieval with Fully Homomorphic Encryption over the Torus
토론
이 논문과 연결된 포럼 글이 아직 없습니다.