The Landscape of Multi-Layer Linear Neural Network From the Perspective of Algebraic Geometry
원문 초록
The clear understanding of the non-convex landscape of neural network is a complex incomplete problem. This paper studies the landscape of linear (residual) network, the simplified version of the nonlinear network. By treating the gradient equations as polynomial equations, we use algebraic geometry tools to solve it over the complex number field, the attained solution can be decomposed into different irreducible complex geometry objects. Then three hypotheses are proposed, involving how to calculate the loss on each irreducible geometry object, the losses of critical points have a certain range and the relationship between the dimension of each irreducible geometry object and strict saddle condition. Finally, numerical algebraic geometry is applied to verify the rationality of these three hypotheses which further clarify the landscape of linear network and the role of residual connection.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
RoHOI: Robustness Benchmark for Human-Object Interaction Detection
Car Object Counting and Position Estimation via Extension of the CLIP-EBC Framework
NOCTIS: Novel Object Cyclic Threshold based Instance Segmentation
Refine Any Object in Any Scene
DenseWorld-1M: Towards Detailed Dense Grounded Caption in the Real World
Towards Reliable Detection of Empty Space: Conditional Marked Point Processes for Object Detection
토론
이 논문과 연결된 포럼 글이 아직 없습니다.