Computational Human-Robot Interaction

Instructor: Stefanos Nikolaidis (snikolai at alumni dot cmu dot edu)

Lectures: Tue / Thu 10:00 - 11:50am LVL 13

Office Hours: Tue 13:30-15:30pm SAL 242 (by appointment)

Course Description: In this advanced graduate-level class, you will learn about the theory and algorithms that enable robots to account for people in their decision making in a principled way. The course will contrast decision-theoretic and learning-based paradigms that allow robots to reason in the presence of uncertainty with studies in human-robot interaction. It will then focus on what makes some of these algorithms particularly effective and scalable in real-world human-robot interaction scenarios. By the end of this class, you will be able to describe and compare algorithms for deployed robotic systems interacting with people, design user studies to evaluate these algorithms and communicate your ideas to a peer audience. Evaluation is mainly based on student presentations, a final project and short quizzes based on the assigned reading material.

Syllabus

Learning Objectives: In this course, you will gain knowledge about planning and learning algorithms in human-robot interaction and skills in interpreting and presenting research. By the end of this course you should be able to:

identify and discuss the different components that make up decision-theoretic reasoning (e.g., MDPs, POMDPs) and learning-based techniques (learning from demonstration, reinforcement learning) that support human-robot interaction
explain the computational and practical challenges of applying these techniques in real-world interaction settings and compare them in terms of robustness, scalability and performance
analyze the design and implementation of a user study to evaluate algorithms for HRI
critique a research paper’s methods and analysis
communicate effectively scientific research to a peer audience

Prerequisites: There are no formal prerequisites, but knowledge of probability theory and linear algebra is encouraged.

Grading:

Component	Percentage
Paper Presentations	30%
Final Project	40%
Weekly Quizzes	15%
Participation	10%
Scribing	5%

Assessment of Assignments

Paper Presentations: Each week, students will present assigned papers in class, on a rotating schedule. Presentations will be 17 minutes long. You will be evaluated based on your demonstrated understanding of the technical content, how you relate the paper to the lectures and previous readings, the clarity, structure and timing of the presentation and how well you answer questions.
Final Project: It should be a substantial piece of work and is expected to take between 60-80 hours over 8 weeks. In the beginning of the class, we will discuss your background and research interests, and we will work together to find potential projects that relate to the course material and that match your interests. The project can be a research project or an extensive in-depth, publication quality literature review(~40-50 papers, grouped based on different categorization criteria, identification of gaps in the literature). You will write a project proposal, work on the project upon approval and present it at the end of the term. You are strongly encouraged to work on something related to your research and think of the project as preliminary work for a paper at a selective, peer-reviewed conference. You will be evaluated based on proposal quality, demonstrated mastery of content, novelty of contribution or categorization of previous work and identification of gaps in the case of a survey, and quality of the final presentation.
Quizzes: There will be a short (10 minute) quiz on the material from the lecture one week before and the readings for the day. The quiz will frequently connect different concepts from multiple readings and/or the material. This quiz is intended to ensure that you are keeping pace with the material and are prepared for the day’s presentations, and is not meant to be onerous. I will drop your two lowest quiz grades when calculating your final grade in the course.
Participation: Students will get the most out of this class if they are active and engaged. This includes asking questions and participating in discussions. There will be explicit time for questions and discussion after each student presentation. I expect students to participate during all lecture sessions.
Scribing: You will take turns compiling the lecture notes. The notes do not need to be very detailed, but should include the main concepts of the lecture.

Important Dates:

October 23rd: Project Proposal Submission. You are encouraged to meet with me before the submission to discuss project ideas.

Project Proposal:

Research Project.
- Identify the problem.
- Summarize previous work
- Describe key insight
- Hypothesize on the results: what do you expect to see? It is perfectly OK to be wrong :)

Research survey.
- Scope the survey: what are you going to include and what will you ignore?
- Identify categorization criteria
- Start the survey with 10 papers.

Schedule:

Day	Date	Topic	Reading	Notes
Tue	Aug 21st	What is Computational HRI?	Computational Human-Robot Interaction, Thomaz, Hoffman and Cakmak. (Optional) Human modeling for human–robot collaboration, Hiatt et al. (Optional) The Grand Challenges in Socially Assistive Robotics, Tapus et al. (Optional) Social robots that interact with people, Breazal et al. (Optional)	Scribing: template.tex questionnaire slides notes
Thu	Aug 23rd	Probability and Bayesian inference	Russell & Norvig (2009). Artificial Intelligence: a Modern Approach (3rd ed.). Prentice Hall. Chapters 13, 14 and 15. Real-Time American Sign Language Recognition from Video Using Hidden Markov Models, Starner and Pentland.	code notes
Tue	Aug 28th	Bayesian inference (cont'd) and decision making under uncertainty	Russell & Norvig (2009). Artificial Intelligence: a Modern Approach (3rd ed.). Prentice Hall. Chapter 16.	notes
Thu	Aug 30th	Markov decision processes and applications in HRI	Russell & Norvig (2009). Artificial Intelligence: a Modern Approach (3rd ed.). Prentice Hall. Chapter 17.1- 17.3.	code notes
Tue	Sept 4th	Action selection for collaboration (student presentations)	Cost-Based Anticipatory Action Selection for Human-Robot Fluency, Hoffman and Breazal. (Sayan) Anticipating human actions for collaboration in the presence of task and sensor uncertainty, Hawkins et al. (Xin) Joint action: bodies and minds moving together, Sebanz et al.(Shihan)
Thu	Sept 6th	Experimental Design	Which statistical test to choose (optional) Toward a science of robotics: Goals and standards for experimental research, Takayama. (optional)	Sample consent form notes
Tue	Sept 11th	Training of human teams and shared mental models (student presentations)	The Impact of Cross-Training on Team Effectiveness, Marks et al. (Setareh) The Influence of Shared Mental Models on Team Process and Performance, Mathieu et al. (Jessica) Planning, Shared Mental Models and Coordinated Performance: An Empirical Link is Established, Stout et al. (Shixian)
Thu	Sept 13th	Action coordination in human-robot teams (student presentations)	Human-Robot Cross-Training: Computational Formulation, Modeling and Evaluation of a Human Team Training Strategy, Nikolaidis and Shah (Lauren) Improved Human-Robot Team Performance Using Chaski, A Human-Inspired Plan Execution System, Shah et al. (Audrow) Adaptive Coordination Strategies for Human-Robot Handovers, Huang et al. (Kiran)
Tue	Sept 18th	Intent inference (student presentations)	Goal Inference as Inverse Planning, Baker et al. (Yash) 'Obsessed with goals': Functions and mechanisms of teleological interpretation of actions in humans, Csibra and Gergely. (Chris) Planning-based Prediction for Pedestrians, Ziebart et al. (Nitu)
Thu	Sept 20th	Expressiveness in robot motion (student presentations)	Expressing thought: improving robot readability with animation principles, Takayama et al. (Naghmeh) The Illusion of Robotic Life, Ribeiro and Paiva. (Emily) Perception of Affect Elicited by Robot Motion, Saerbeck and Bartneck. (Leili)
Tue	Sept 25th	Generation of expressive motion (student presentations)	Generating Legible Robot Motion, Dragan and Srinivasa. (David) Enhancing Interaction Through Exaggerated Motion Synthesis, Gielniak and Thomaz. (Arshdeep) Communication of Intent in Assistive Free Flyers, Szafir1 et al. (Isabel)
Thu	Sept 27th	Planning with partial observability	Planning and acting in partially observable stochastic domains, Kaelbling et al. "A brief introduction to Partially Observable Markov Decision Processes" (optional)	notes
Tue	Oct 2nd	Planning with partially observable human states (student presentations)	Intention-aware motion planning, Bandyopadhyay et al. (Tricia) Belief Space Planning for Sidekicks in Cooperative Games, Macindowe et al. (Shixian) Automated handwashing assistance for persons with dementia using video and a partially observable Markov decision process, Hoey et al. (Xin)
Thu	Oct 4th	Planning with human state dynamics (student presentations)	Formalizing Human-Robot Mutual Adaptation: A Bounded Memory Model, Nikolaidis et al. (Shihan) Planning for Autonomous Cars that Leverage Effects on Human Actions, Sadigh et al. (Setareh) Game-theoretic modeling of human adaptation in human-robot collaboration, Nikolaidis et al. (Jessica)
Tue	Oct 9th	Planning in shared autonomy domains (student presentations)	Shared Autonomy via Hindsight Optimization, Javdani et al. (Tom) Autonomy Infused Teleoperation with Application to BCI Manipulation, Muelling et al. (Sayan) Comparing shared control approaches for alternative interfaces: A wheelchair simulator experiment, Ezeh1 et al. (Lauren)
Thu	Oct 11th	Learning techniques for HRI	Russell & Norvig (2009). Artificial Intelligence: a Modern Approach (3rd ed.). Prentice Hall. Chapter 20. The Expectation Maximization Algorithm, A short tutorial, Borman. (optional) A survey of robot learning from demonstration, Argall et al.(optional)	notes
Tue	Oct 16th	Learning from demonstration in HRI (student presentations)	Trajectories and Keyframes for Kinesthetic Teaching: A Human-Robot Interaction Perspective, Akgun et al. (Yash) Confidence-based policy learning from demonstration using gaussian mixture models, Chernova and Veloso. (Kiran) Learning to Tutor from Expert Demonstrators via Apprenticeship Scheduling, Gombolay et al. (Audrow)
Thu	Oct 18th	Active learning in HRI (student presentations)	Designing robot learners that ask good questions, Cakmak and Thomaz (Chris). Active Preference-Based Learning of Reward Functions, Sadigh et al (Nitu). Discovering task constraints through observation and active learning, Hayes and Scassellati (Naghmeh).
Tue	Oct 23rd	Reinforcement learning		Project proposal submission notes
Thu	Oct 25th	Reinforcement learning with human feedback (student presentations)	Reinforcement Learning with Human Teachers, Thomaz and Breazeal. (Leili) Combining Manual Feedback with Subsequent MDP Reward Signals for Reinforcement Learning, Knox and Stone. (Emily) Interactive Learning from Policy-Dependent Human Feedback, MacGlashan et al. (David)
Tue	Oct 30th	Integrating learning and planning in HRI (student presentations)	Efficient model learning for dialog management, Doshi and Roy. (Arshdeep) Efficient Model Learning from Joint-Action Demonstrations for Human-Robot Collaborative Tasks, Nikolaidis et al. (Isabel) Planning with Trust for Human-Robot Collaboration, Chen et al. (Tricia)
Thu	Nov 1st	Optimal teaching (student presentations)	Machine Teaching for Inverse Reinforcement Learning: Algorithms and Applications, sections 1-6, Brown and Niekum (Tom) Algorithmic and human teaching of sequential decision tasks, Cakmak and Lopez. (Xin) How do humans teach: On curriculum learning and teaching dimension, sections 1-3, Khan et al. (Kiran)
Tue	Nov 6th	Pedagogical reasoning (student presentations)	Cooperative inverse reinforcement learning, Hadfield et al. (David) Pragmatic-Pedagogic Value Alignment, Fisac et al. (Arshdeep) Showing versus doing: Teaching by demonstration, Ho et al. (Shihan)
Thu	Nov 8th	Communication and signaling (student presentations)	Effects of Robot Sound on Auditory Localization in Human-Robot Collaboration, Cha et al. (Sayan) ConTaCT: Deciding to Communicate during Time-Critical Collaborative Tasks in Unknown, Deterministic Domains, Unhelkar and Shah (Yash) Implicit Communication in a Joint Action, Knepper et al. (Emily)
Tue	Nov 13th	Socially assistive robotics (guest lecture: Maja Mataric)
Thu	Nov 15th	Natural language dialog systems (guest lecture: David Traum)
Tue	Nov 20th	Projects discussion
Thu	Nov 22th	no class
Tue	Nov 27th	Project presentations
Thu	Nov 29th	Project presentations
Fri	Dec 7th			Final report due

Expectations: You can expect me to come to class on time, clearly communicate expectations for the presentations structure, format and clarity, give you feedback on a timely manner, adjust lecture material based on performance on presentations and quizzes and be available to meet regularly to discuss the progress of your project. I can expect you to come to class on time, be attentive and engaged in class, take notes and ask questions when something is not clear, spend an adequate amount of time on the readings each week (at least 3 hours), spend 60-80 hours on your final project.

Additional Policies: Please see the syllabus for the statement on academic conduct and student support systems. Unless you are assigned to compile lecture notes, please refrain from using laptops or other electronic devices during class.

Related Courses: You are encouraged to expand your readings from related courses, for example: