Is Smoothness the Key to Robustness? A Comparison of Attention and Convolution Models Using a Novel Metric
원문 초록
Robustness is a critical aspect of machine learning models. Existing robustness evaluation approaches often lack theoretical generality or rely heavily on empirical assessments, limiting insights into the structural factors contributing to robustness. Moreover, theoretical robustness analysis is not applicable for direct comparisons between models. To address these challenges, we propose $\textit{TopoLip}$, a metric based on layer-wise analysis that bridges topological data analysis and Lipschitz continuity for robustness evaluation. TopoLip provides a unified framework for both theoretical and empirical robustness comparisons across different architectures or configurations, and it reveals how model parameters influence the robustness of models. Using TopoLip, we demonstrate that attention-based models typically exhibit smoother transformations and greater robustness compared to convolution-based models, as validated through theoretical analysis and adversarial tasks. Our findings establish a connection between architectural design, robustness, and topological properties.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Comparing the Effects of Persistence Barcodes Aggregation and Feature Concatenation on Medical Imaging
Topological Machine Learning for Protein-Nucleic Acid Binding Affinity Changes Upon Mutation
Topology across Scales on Heterogeneous Cell Data
Short-PHD: Detecting Short LLM-generated Text with Topological Data Analysis After Off-topic Content Insertion
A Topological Data Analysis Framework for Quantifying Necrosis in Glioblastomas
Persistent Homology-induced Graph Ensembles for Time Series Regressions
토론
이 논문과 연결된 포럼 글이 아직 없습니다.