VectorAdam for Rotation Equivariant Geometry Optimization
원문 초록
The Adam optimization algorithm has proven remarkably effective for optimization problems across machine learning and even traditional tasks in geometry processing. At the same time, the development of equivariant methods, which preserve their output under the action of rotation or some other transformation, has proven to be important for geometry problems across these domains. In this work, we observe that Adam $-$ when treated as a function that maps initial conditions to optimized results $-$ is not rotation equivariant for vector-valued parameters due to per-coordinate moment updates. This leads to significant artifacts and biases in practice. We propose to resolve this deficiency with VectorAdam, a simple modification which makes Adam rotation-equivariant by accounting for the vector structure of optimization variables. We demonstrate this approach on problems in machine learning and traditional geometric optimization, showing that equivariant VectorAdam resolves the artifacts and biases of traditional Adam when applied to vector-valued data, with equivalent or even improved rates of convergence.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
AnalogVNN: A fully modular framework for modeling and optimizing photonic neural networks
Evolutionary bagging for ensemble learning
Neural network accelerator for quantum control
Deep Reinforcement Learning for Multi-Agent Interaction
Disparate Censorship & Undertesting: A Source of Label Bias in Clinical Machine Learning
Unsupervised machine learning framework for discriminating major variants of concern during COVID-19
토론
이 논문과 연결된 포럼 글이 아직 없습니다.