Serpent: Scalable and Efficient Image Restoration via Multi-scale Structured State Space Models
원문 초록
The landscape of computational building blocks of efficient image restoration architectures is dominated by a combination of convolutional processing and various attention mechanisms. However, convolutional filters, while efficient, are inherently local and therefore struggle with modeling long-range dependencies in images. In contrast, attention excels at capturing global interactions between arbitrary image regions, but suffers from a quadratic cost in image dimension. In this work, we propose Serpent, an efficient architecture for high-resolution image restoration that combines recent advances in state space models (SSMs) with multi-scale signal processing in its core computational block. SSMs, originally introduced for sequence modeling, can maintain a global receptive field with a favorable linear scaling in input size. We propose a novel hierarchical architecture inspired by traditional signal processing principles, that converts the input image into a collection of sequences and processes them in a multi-scale fashion. Our experimental results demonstrate that Serpent can achieve reconstruction quality on par with state-of-the-art techniques, while requiring orders of magnitude less compute (up to $150$ fold reduction in FLOPS) and a factor of up to $5\times$ less GPU memory while maintaining a compact model size. The efficiency gains achieved by Serpent are especially notable at high image resolutions.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
MODEM: A Morton-Order Degradation Estimation Mechanism for Adverse Weather Image Recovery
MT-PCR: A Hybrid Mamba-Transformer with Spatial Serialization for Hierarchical Point Cloud Registration
Generalizable, real-time neural decoding with hybrid state-space models
FastMamba: A High-Speed and Efficient Mamba Accelerator on FPGA with Accurate Quantization
APE: Selective Fine-tuning with Acceptance Criteria for Language Model Adaptation
Arctic Inference with Shift Parallelism: Fast and Efficient Open Source Inference System for Enterprise AI
토론
이 논문과 연결된 포럼 글이 아직 없습니다.