On Training of Kolmogorov-Arnold Networks
원문 초록
Kolmogorov-Arnold Networks have recently been introduced as a flexible alternative to multi-layer Perceptron architectures. In this paper, we examine the training dynamics of different KAN architectures and compare them with corresponding MLP formulations. We train with a variety of different initialization schemes, optimizers, and learning rates, as well as utilize back propagation free approaches like the HSIC Bottleneck. We find that (when judged by test accuracy) KANs are an effective alternative to MLP architectures on high-dimensional datasets and have somewhat better parameter efficiency, but suffer from more unstable training dynamics. Finally, we provide recommendations for improving training stability of larger KAN models.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Pushing the Performance of Synthetic Speech Detection with Kolmogorov-Arnold Networks and Self-Supervised Learning Models
Automated Modeling Method for Pathloss Model Discovery
"KAN you hear me?" Exploring Kolmogorov-Arnold Networks for Spoken Language Understanding
TK-Mamba: Marrying KAN with Mamba for Text-Driven 3D Medical Image Segmentation
X-KAN: Optimizing Local Kolmogorov-Arnold Networks via Evolutionary Rule-Based Machine Learning
Improving Generalizability of Kolmogorov-Arnold Networks via Error-Correcting Output Codes
토론
이 논문과 연결된 포럼 글이 아직 없습니다.