About
Welcome to the planning and learning reading group website! In this reading group we will discuss current papers and developments in the fields of planning and learning in robotics. Topics of interest include but are not limited to:
- (deep) reinforcement learning
- planning under uncertainty
- planning in unknown or partially observable environments
- hierarchical planning
- classical planning
- ...
Who, When and Where
The planning and learning reading group is jointly run by Frank Broz and Jose L. Part. Meetings take place every two weeks on Thursdays from 15:00 to 16:00 in Room EM 1.58 in the Earl Mountbatten Building at Heriot-Watt University.
Meetings are suspended indefinitely.
Next Meeting
Date: 27/02/2020
Time: 15:00
Room: EMB 1.58
Presenter: Frank Broz
Title: Efficient Model Learning from Joint-Action Demonstrations for Human-Robot Collaborative Tasks
Authors: Stefanos Nikolaidis, Ramya Ramakrishnan, Keren Gu, Julie Shah
Abstract: We present a framework for automatically learning human user models from joint-action demonstrations that enables a robot to compute a robust policy for a collaborative task with a human. First, the demonstrated action sequences are clustered into different human types using an unsupervised learning algorithm. A reward function is then learned for each type through the employment of an inverse reinforcement learning algorithm. The learned model is then incorporated into a mixed-observability Markov decision process (MOMDP) formulation, wherein the human type is a partially observable variable. With this framework, we can infer online the human type of a new user that was not included in the training set, and can compute a policy for the robot that will be aligned to the preference of this user. In a human subject experiment (n=30), participants agreed more strongly that the robot anticipated their actions when working with a robot incorporating the proposed framework (p<0.01), compared to manually annotating robot actions. In trials where participants faced difficulty annotating the robot actions to complete the task, the proposed framework significantly improved team efficiency (p<0.01). The robot incorporating the framework was also found to be more responsive to human actions compared to policies computed using a hand-coded reward function by a domain expert (p<0.01). These results indicate that learning human user models from joint-action demonstrations and encoding them in a MOMDP formalism can support effective teaming in human-robot collaborative tasks.
Previous Meetings
Date: 24/06/2019
Presenter: Jose L. Part
Title: Social Cobots: Anticipatory Decision-Making for Collaborative Robots Incorporating Unexpected Human Behaviors
Authors: Orhan Can Görür, Benjamin Rosman, Fikret Sivrikaya, Sahin Albayrak
Abstract: We propose an architecture as a robot's decision-making mechanism to anticipate a human's state of mind, and so plan accordingly...
Date: 20/05/2019
Presenter: Ingo Keller
Title: Anticipatory Bayesian Policy Selection for Online Adaptation of Collaborative Robots to Unknown Human Types
Authors: Orhan Can Görür, Benjamin Rosman, Sahin Albayrak
Abstract: As a key component of collaborative robots (cobots) working with humans, existing decision-making approaches try to model the uncertainty in...
Date: 08/04/2019
Presenter: Martin Ross
Title: Where to Add Actions in Human-in-the-Loop Reinforcement Learning
Authors: Travis Mandel, Yun-En Liu, Emma Brunskill, and Zoran Popović
Abstract: In order for reinforcement learning systems to learn quickly in vast action spaces such as the space of all possible...
Date: 25/03/2019
Presenter: Frank Broz
Title: Planning with Trust for Human-Robot Collaboration
Authors: Min Chen, Stefanos Nikolaidis, Harold Soh, David Hsu and Siddhartha Srinivasa
Abstract: Trust is essential for human-robot collaboration and user adoption of autonomous systems, such as robot assistants. This paper introduces a...
Date: 11/03/2019
Presenter: Jose L. Part
Title: Learning Latent Dynamics for Planning from Pixels
Authors: Danijar Hafner, Timothy Lillicrap, Ian Fischer, Ruben Villegas, David Ha, Honglak Lee and James Davidson
Abstract: Planning has been very successful for control tasks with known environment dynamics. To leverage planning in unknown environments, the agent...
Contact
Frank Broz: f.broz@hw.ac.ukJose L. Part: j.part@hw.ac.uk