Reinforcement Learning and Apprenticeship Learning for Robotic Control
(invited lecture for ALT and DS 2006)
Author: Andrew Ng
Affiliation: Computer Science Department,
Stanford University, Stanford, U.S.A.
Many control problems, such as autonomous helicopter flight,
legged robot locomotion, and autonomous driving are difficult
(i) It is hard to write down, in closed form, a formal
specification of the control task (for example, what is the
cost function for "driving well"?),
(ii) It is difficult to learn
good models of the robot's dynamics, and
(iii) It is expensive to
find closed-loop controllers for high dimensional, highly stochastic
domains. Using apprenticeship learning—in which we learn from a
human demonstration of a task—as a unifying theme, I will present
formal results showing how many control problems can be efficiently
addressed given access to a demonstration. In presenting these
ideas, I will also draw from a number of case studies, including
applications in autonomous helicopter flight, quadruped obstacle
negotiation, snake robot locomotion, and high-speed off-road navigation.
Finally, I will also describe the application of these ideas to
the STAIR (STanford AI Robot) project, which has the long term goal
of integrating methods from all major areas of AI—including spoken
dialog/NLP, manipulation, vision, navigation, and planning—to
build a general-purpose, "intelligent" home/office robotic assistant.
Joint work with Pieter Abbeel, Adam Coates, Ashutosh Saxena,
Jeremy Kolter, Honglak Lee, Yirong Shen, Justin Driemeyer,
Justin Kearns, and Chioma Osondu.
©Copyright 2006 Author