Evaluating Loss Landscapes from a Topology Perspective
원문 초록
Characterizing the loss of a neural network with respect to model parameters, i.e., the loss landscape, can provide valuable insights into properties of that model. Various methods for visualizing loss landscapes have been proposed, but less emphasis has been placed on quantifying and extracting actionable and reproducible insights from these complex representations. Inspired by powerful tools from topological data analysis (TDA) for summarizing the structure of high-dimensional data, here we characterize the underlying shape (or topology) of loss landscapes, quantifying the topology to reveal new insights about neural networks. To relate our findings to the machine learning (ML) literature, we compute simple performance metrics (e.g., accuracy, error), and we characterize the local structure of loss landscapes using Hessian-based metrics (e.g., largest eigenvalue, trace, eigenvalue spectral density). Following this approach, we study established models from image pattern recognition (e.g., ResNets) and scientific ML (e.g., physics-informed neural networks), and we show how quantifying the shape of loss landscapes can provide new insights into model performance and learning dynamics.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Comparing the Effects of Persistence Barcodes Aggregation and Feature Concatenation on Medical Imaging
Topological Machine Learning for Protein-Nucleic Acid Binding Affinity Changes Upon Mutation
Topology across Scales on Heterogeneous Cell Data
Short-PHD: Detecting Short LLM-generated Text with Topological Data Analysis After Off-topic Content Insertion
A Topological Data Analysis Framework for Quantifying Necrosis in Glioblastomas
Persistent Homology-induced Graph Ensembles for Time Series Regressions
토론
이 논문과 연결된 포럼 글이 아직 없습니다.