AdaHessian
ADAHESSIAN is a new stochastic optimization algorithm that directly incorporates approximate curvature information from the loss function, and it includes several novel performance-improving features, including a fast Hutchinson based method to approximate the curvature matrix with low computational overhead.
ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learning