In the answer to “what is the Hessian” at this site: https://stackoverflow.com/questions/23297090/how-calculating-hessian-works-for-neural-network-learning the person answering the question concludes by saying “Fun fact, adding the momentum term to your gradient based optimization is (under sufficient conditions) approximating the hessian based optimization (and is far less computationally expensive).” I have tried to understand this point and look … Continue reading "How is momentum an approximation of Hessian based optimization?"
The post How is momentum an approximation of Hessian based optimization? appeared first on 100% Private Proxies - Fast, Anonymous, Quality, Unlimited USA Private Proxy!.
