摘要
We consider a fundamental problem in the field of machine learning—structural risk minimization,which can be represented as the average of a large number of smooth component functions plus a simple and convex(but possibly non-smooth)function.In this paper,we propose a novel proximal variance reducing stochastic method building on the introduced Point-SAGA.Our method achieves two proximal operator calculations by combining the fast Douglas–Rachford splitting and refers to the scheme of the FISTA algorithm in the choice of momentum factors.We show that the objective function value converges to the iteration point at the rate of O(1/k)when each loss function is convex and smooth.In addition,we prove that our method achieves a linear convergence rate for strongly convex and smooth loss functions.Experiments demonstrate the effectiveness of the proposed algorithm,especially when the loss function is ill-conditioned with good acceleration.
作者简介
Jialin Lei received the BEng degree from Henan Institute of Science and Technology,China in 2020.He is currently a master student at Zhejiang Normal University,China.His research interests include machine learning and optimization algorithm.E-mail:jialinlei@zjnu.edu.cn;To whom correspondence should be addressed:Ying Zhang received the PhD degree in operations research and cybernetics from Shanghai University,China in 2009.She is currently an associate professor at Zhejiang Normal University,China.Her research interests include optimal control,global optimization,and machine learning.She has published some papers in international journals,such as Applied Mathematical Modelling,Journal of Computational and Applied Mathematics,and Communications in Nonlinear Science and Numerical Simulation.znuzy@zjnu.cn;Zhao Zhang received the PhD degree in applied mathematics from Xinjiang University,China in 2003.She is currently a distinguished professor at Zhejiang Normal University,China.Her research interests include approximation algorithm,combinatorial optimization,and machine learning.She has published more than 180 academic papers.zhaozhang@zjnu.cn.
