Rethinking the Function of Neurons in KANs
원문 초록
The neurons of Kolmogorov-Arnold Networks (KANs) perform a simple summation motivated by the Kolmogorov-Arnold representation theorem, which asserts that sum is the only fundamental multivariate function. In this work, we investigate the potential for identifying an alternative multivariate function for KAN neurons that may offer increased practical utility. Our empirical research involves testing various multivariate functions in KAN neurons across a range of benchmark Machine Learning tasks. Our findings indicate that substituting the sum with the average function in KAN neurons results in significant performance enhancements compared to traditional KANs. Our study demonstrates that this minor modification contributes to the stability of training by confining the input to the spline within the effective range of the activation function. Our implementation and experiments are available at: \url{https://github.com/Ghaith81/dropkan}
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Pushing the Performance of Synthetic Speech Detection with Kolmogorov-Arnold Networks and Self-Supervised Learning Models
Automated Modeling Method for Pathloss Model Discovery
"KAN you hear me?" Exploring Kolmogorov-Arnold Networks for Spoken Language Understanding
TK-Mamba: Marrying KAN with Mamba for Text-Driven 3D Medical Image Segmentation
X-KAN: Optimizing Local Kolmogorov-Arnold Networks via Evolutionary Rule-Based Machine Learning
Improving Generalizability of Kolmogorov-Arnold Networks via Error-Correcting Output Codes
토론
이 논문과 연결된 포럼 글이 아직 없습니다.