Document worth reading: “A Survey of Optimization Methods from a Machine Learning Perspective”

Machine learning develops shortly, which has made many theoretical breakthroughs and is broadly utilized in diverse fields. Optimization, as an important half of machine learning, has attracted a lot consideration of researchers. With the exponential improvement of data amount and the rise of model complexity, optimization methods in machine learning face an growing quantity of challenges. Loads of work on fixing optimization points or bettering optimization methods in machine learning has been proposed successively. The systematic retrospect and summary of the optimization methods from the angle of machine learning are of good significance, which could present guidance for every developments of optimization and machine learning evaluation. In this paper, we first describe the optimization points in machine learning. Then, we introduce the principles and progresses of usually used optimization methods. Next, we summarize the needs and developments of optimization methods in some in model machine learning fields. Finally, we uncover and supplies some challenges and open points for the optimization in machine learning. A Survey of Optimization Methods from a Machine Learning Perspective