The Stackelberg prediction game(SPG)is a bilevel optimization frame-work for modeling strategic interactions between a learner and a follower.Existing meth-ods for solving this problem with general loss functions are ...The Stackelberg prediction game(SPG)is a bilevel optimization frame-work for modeling strategic interactions between a learner and a follower.Existing meth-ods for solving this problem with general loss functions are computationally expensive and scarce.We propose a novel hyper-gradient type method with a warm-start strategy to address this challenge.Particularly,we first use a Taylor expansion-based approach to obtain a good initial point.Then we apply a hyper-gradient descent method with an ex-plicit approximate hyper-gradient.We establish the convergence results of our algorithm theoretically.Furthermore,when the follower employs the least squares loss function,our method is shown to reach an e-stationary point by solving quadratic subproblems.Numerical experiments show our algorithms are empirically orders of magnitude faster than the state-of-the-art.展开更多
基于结构风险最小的最小二乘支持向量机(least squares support vector machine,LSSVM)为标准支持向量机(SVM)的约简,训练简易,性能良好。其模型精度受超参数影响,常规的网络搜索法很难搜得最佳超参数。在快速留一法的基础上,以全样本...基于结构风险最小的最小二乘支持向量机(least squares support vector machine,LSSVM)为标准支持向量机(SVM)的约简,训练简易,性能良好。其模型精度受超参数影响,常规的网络搜索法很难搜得最佳超参数。在快速留一法的基础上,以全样本留一预测误差平方和最小化为目标,导出基于梯度的最优化算法,用以优选为LSSVM超参数,进而构建G-LSSVM模型。以柠檬酸发酵过程为算例对G-LSSVM进行检验,结果表明G-LSSVM的超参数选取耗时少,模型稳定性良好,且拟合和预报性能都优于标准SVM和神经网络。有望适用于机理不明、高度非线性、小样本的化工过程建模。展开更多
文摘The Stackelberg prediction game(SPG)is a bilevel optimization frame-work for modeling strategic interactions between a learner and a follower.Existing meth-ods for solving this problem with general loss functions are computationally expensive and scarce.We propose a novel hyper-gradient type method with a warm-start strategy to address this challenge.Particularly,we first use a Taylor expansion-based approach to obtain a good initial point.Then we apply a hyper-gradient descent method with an ex-plicit approximate hyper-gradient.We establish the convergence results of our algorithm theoretically.Furthermore,when the follower employs the least squares loss function,our method is shown to reach an e-stationary point by solving quadratic subproblems.Numerical experiments show our algorithms are empirically orders of magnitude faster than the state-of-the-art.
文摘基于结构风险最小的最小二乘支持向量机(least squares support vector machine,LSSVM)为标准支持向量机(SVM)的约简,训练简易,性能良好。其模型精度受超参数影响,常规的网络搜索法很难搜得最佳超参数。在快速留一法的基础上,以全样本留一预测误差平方和最小化为目标,导出基于梯度的最优化算法,用以优选为LSSVM超参数,进而构建G-LSSVM模型。以柠檬酸发酵过程为算例对G-LSSVM进行检验,结果表明G-LSSVM的超参数选取耗时少,模型稳定性良好,且拟合和预报性能都优于标准SVM和神经网络。有望适用于机理不明、高度非线性、小样本的化工过程建模。
