摘要
通过研究基于回报函数学习的学徒学习的发展历史和目前的主要工作,概述了基于回报函数学习的学徒学习方法.分别在回报函数为线性和非线性条件下讨论,并且在线性条件下比较了2类方法——基于逆向增强学习(IRL)和最大化边际规划(MMP)的学徒学习.前者有较为快速的近似算法,但对于演示的最优性作了较强的假设;后者形式上更易于扩展,但计算量大.最后,提出了该领域现在还存在的问题和未来的研究方向,如把学徒学习应用于POMDP环境下,用PBVI等近似算法或者通过PCA等降维方法对数据进行学习特征的提取,从而减少高维度带来的大计算量问题.
This paper focuses on apprenticeship learning, based on reward function learning. Both the historical basis of this field and a broad selection of current work were investigated. In this paper, two kinds of algorithm--apprenticeship learning methods based on inverse reinforcement learning （IRL） and maximum margin planning （MMP） frameworks were discussed under respective assumptions of linear and nonlinear reward functions. Comparison was made under the linear assumption conditions. The former can be implemented with an efficient approximate method but has made a strong supposition of optimal demonstration. The latter has a relatively easy to extend form but may take large amounts of computation. Finally, some suggestions were given for further research in reward function learning in a partially observable Markov decision process （POMDP） environment and in continuous/ high dimensional space, using either an approximate algorithm such as point-based value iteration （PBVI） or a feature abstraction algorithm using dimension reduction methods such as principle component analysis （PCA）. Adopting these may alleviate the curse of dimensionality.
出处
《智能系统学报》
2009年第3期208-212,共5页
CAAI Transactions on Intelligent Systems
基金
国家自然科学基金资助项目(90820306)
浙江省科技厅重大资助项目(006c13096)
关键词
学徒学习
回报函数
逆向增强学习
最大化边际规划
apprenticeship learning
reward function
inverse reinforcement learning
maximum margin planning
作者简介
金卓军，男，1984年生，博士研究生，主要研究方向为机器学习．
通信作者：钱徽．E—mail：qianhui@zju．edu．cn．钱徽，男，1974年生，副教授，人工智能学会智能机器人专业委员会委员，主要研究方向为人工智能、计算机视觉．
陈沈轶，男，1980生，博士研究生，主要研究方向为机器学习．
