Computational Human-Robot Interaction

Instructor: Stefanos Nikolaidis (nikolaid at usc dot edu)

Lectures: Mon / Wed 08:00 - 09:50 (VHE 206)

TA: Ya-Chuan (Sophie) Hsu

Office Hours: RTH 401 ( by appointment)

Course Description: In this advanced graduate-level class, you will learn about the theory and algorithms that enable robots to account for people in their decision making in a principled way. The course will contrast decision-theoretic and learning-based paradigms that allow robots to reason in the presence of uncertainty with studies in human-robot interaction. It will then focus on what makes some of these algorithms particularly effective and scalable in real-world human-robot interaction scenarios. By the end of this class, you will be able to describe and compare algorithms for deployed robotic systems interacting with people, design user studies to evaluate these algorithms and communicate your ideas to a peer audience. Evaluation is mainly based on student presentations, a final project and short quizzes based on the assigned reading material.

Learning Objectives: In this course, you will gain knowledge about planning and learning algorithms in human-robot interaction and skills in interpreting and presenting research. By the end of this course you should be able to:

identify and discuss the different components that make up decision-theoretic reasoning (e.g., MDPs, POMDPs) and learning-based techniques (learning from demonstration, reinforcement learning) that support human-robot interaction
explain the computational and practical challenges of applying these techniques in real-world interaction settings and compare them in terms of robustness, scalability and performance
analyze the design and implementation of a user study to evaluate algorithms for HRI
critique a research paper’s methods and analysis
communicate effectively scientific research to a peer audience

Prerequisites: There are no formal prerequisites, but knowledge of probability theory and linear algebra is encouraged.

Grading:

Component	Percentage
Paper Presentations	30%
Final Project	40%
Weekly Quizzes	10%
Participation	10%
Homework	10%

Assessment of Assignments

Paper Presentations: Each week, students will present assigned papers in class, on a rotating schedule. There will be two presentation types: (1) Main: this will be 15 minutes long and the goal will be to provide an overview of the paper. (2) Con: in some papers we will have an additional 5 minute presentation where the goal will be to discuss technical limitations of the proposed method and ideas on how to overcome them. You will be evaluated based on your demonstrated understanding of the technical content, how you relate the paper to the lectures and previous readings, the clarity, structure and timing of the presentation and how well you answer questions.
Final Project: It should be a substantial piece of work and is expected to take between 60-80 hours over 8 weeks. The project can be a research project or an extensive in-depth, publication quality literature review(~40-50 papers, grouped based on different categorization criteria, identification of gaps in the literature). You will write a project proposal, work on the project upon approval and present it at the end of the term. You are strongly encouraged to work on something related to your research and think of the project as preliminary work for a paper at a selective, peer-reviewed conference. You will be evaluated based on proposal quality, demonstrated mastery of content, novelty of contribution or categorization of previous work and identification of gaps in the case of a survey, and quality of the final presentation.
Quizzes: There will be a short (10 minute) quiz on the material from the lecture one week before and the readings for the day. The quiz will frequently connect different concepts from multiple readings and/or the material. This quiz is intended to ensure that you are keeping pace with the material and are prepared for the day’s presentations, and is not meant to be onerous. I will drop your four lowest quiz grades when calculating your final grade in the course. There will be no make-up quizzes.
Participation: Students will get the most out of this class if they are active and engaged. This includes asking questions and participating in discussions. There will be explicit time for questions and discussion after each student presentation.
Homework: We will have 2-3 homework assignments in the class based on the topics covered in the lectures and papers. You can work on the assignments on your own or in teams of two or three students.

Important Dates:

March 1st: Project Proposal Submission.

Project Proposal:

Research Project.
- Identify the problem.
- Summarize previous work
- Describe key insight
- Hypothesize on the results: what do you expect to see? It is perfectly OK to be wrong :)

Research survey.
- Scope the survey: what are you going to include and what will you ignore?
- Identify categorization criteria
- Start the survey with 10 papers.

Schedule:

Day	Date	Topic	Reading	Notes
Mon	Jan 8th	What is Computational HRI?	Computational Human-Robot Interaction, Thomaz, Hoffman and Cakmak. (Optional) Inferring Human Intent and Predicting Human Action in Human–Robot Collaboration , Hoffman et al. (Optional) The Grand Challenges in Socially Assistive Robotics, Tapus et al. (Optional)	Slides
Wed	Jan 10th	Probability and Bayesian inference	Russell & Norvig (2009). Artificial Intelligence: a Modern Approach (3rd ed.). Prentice Hall. Chapters 13, 14 and 15. Real-Time American Sign Language Recognition from Video Using Hidden Markov Models, Starner and Pentland.	code notes_A notes_B Slides
Mon	Jan 15th	Martin Luther King’s Birthday (no class)
Wed	Jan 17th	Bayesian Inference (cont'd)
Mon	Jan 22th	Bayesian Inference (cont'd)
Wed	Jan 24th	Decision making under uncertainty	Russell & Norvig (2009). Artificial Intelligence: a Modern Approach (3rd ed.). Prentice Hall. Chapter 16.	notes
Mon	Jan 29th	Experimental Design	Which statistical test to choose (optional) Toward a science of robotics: Goals and standards for experimental research, Takayama. (optional)	Sample consent form notes Slides
Wed	Jan 31st	Action selection for collaboration (student presentations)	Cost-Based Anticipatory Action Selection for Human-Robot Fluency, Hoffman and Breazal. (Main: Patrick , Con: Pavel) Joint action: bodies and minds moving together, Sebanz et al.(Main: Ziwei)
Mon	Feb 5th	Action selection for collaboration II (student presentations)	Coordination With Humans Via Strategy Matching., Zhao et al.. (Main: Jade , Con: Saeed) Probabilistic Human Intent Recognition for Shared Autonomy in Assistive Robotics, Jain and Argall (Main: Zhehui , Con: Yigit)
Wed	Feb 7th	Training of human teams and shared mental models (student presentations)	The Impact of Cross-Training on Team Effectiveness, Marks et al. (Main: Shipeng) Planning, Shared Mental Models and Coordinated Performance: An Empirical Link is Established, Stout et al. (Main:Nima)
Mon	Feb 12th	Action coordination in human-robot teams (student presentations)	Human-Robot Cross-Training: Computational Formulation, Modeling and Evaluation of a Human Team Training Strategy, Nikolaidis and Shah (Main: Bosheng, Con:Lina) Adaptive Coordination Strategies for Human-Robot Handovers, Huang et al. (Main:Xiye, Con:Shihan)
Wed	Feb 14th	Guest Lecture: Erdem Biyik
Mon	Feb 19th	President's Day (no class)
Wed	Feb 21st	Intent Inference (Student Presentations)	Goal Inference as Inverse Planning, Baker et al. (Main: Shuqin) Planning-based Prediction for Pedestrians, Ziebart et al (Main: Pavel , Con:Patrick)
Mon	Feb 26th	Expressiveness in robot motion (student presentations)	Expressing thought: improving robot readability with animation principles, Takayama et al. (Main:Saeed, Con:Ziwei) The Illusion of Robotic Life, Ribeiro and Paiva. (Main: Yigit, Con:Jade)
Wed	Feb 28th	Generation of expressive motion (student presentations)	Generating Legible Robot Motion, Dragan and Srinivasa. (Main:Lina, Con:Zhehui) Enhancing Interaction Through Exaggerated Motion Synthesis, Gielniak and Thomaz. (Main:Shihan, Con:Shipeng)
Mon	Mar 4th	Planning with partial observability	Planning and acting in partially observable stochastic domains, Kaelbling et al. "A brief introduction to Partially Observable Markov Decision Processes" (optional) POMDPs for Dummies	Project Proposal Due notes Slides
Wed	Mar 6th	Planning with partially observable human states (student presentations)	Intention-aware motion planning, Bandyopadhyay et al. (Main: Patrick , Con: Nima ) Belief Space Planning for Sidekicks in Cooperative Games, Macindowe et al (Main: Pavel, Con: Bosheng)

Mon	Mar 11th	Spring Recess (no class)
Wed	Mar 13th	Spring Recess (no class)
Mon	Mar 18th	Planning with human state dynamics (student presentations)	Towards Modeling and Influencing the Dynamics of Human Learning, Tian et al. (Main: Ziwei , Con: Xiye) Learning Latent Representations to Influence Multi-Agent Interaction, Xie et al. (Main: Saeed , Con: Shuqin)
Wed	Mar 20th	Planning in shared autonomy domains (student presentations)	Shared Autonomy via Hindsight Optimization, Javdani et al. (Main: Zhehui, Con: Shuqin) Autonomy Infused Teleoperation with Application to BCI Manipulation, Muelling et al. (Main: Yigit , Con: Shihan )
Mon	Mar 25th	Integrating Learning and Planning in HRI	Human-Robot Team Coordination with Dynamic and Latent Human Task Proficiencies, Tian et al. (Main: Shipeng, Con: Xiye) Planning with Trust for Human-Robot Collaboration, Xie et al. (Main: Nima , Con: Lina)
Wed	Mar 27th	Language for HRI (student presentations)	Generative Expressive Robot Behaviors Using Large Language Models, Mahadevan et al. (Main: Boshen, Con: Patrick) Sketching Robot Programs On the Fly, Porfirio et al. (Main: Lina , Con: Pavel)
Mon	Apr 1st	Guest Lecture (TBD)
Mon	Apr 3rd	Guest Lecture (TBD)
Mon	Apr 8th	Quality Diversity (student presentations)	Illuminating search spaces by mapping elites, Mouret and Clune (Main: ) Confronting the Challenge of Quality Diversity, Pugh et al.(Main: )
Wed	Apr 10th	Authoring Human-Robot Interactions	A Quality Diversity Approach to Automatically Generating Human-Robot Interaction Scenarios in Shared Autonomy, Fontaine and Nikolaidis (Main: , Con: ) RCare World: A Human-centric Simulation World for Caregiving Robots, Ye et al (Main: ,Con: )
Mon	Apr 15th
Wed	Apr 17th
Mon	Apr 22nd	Project Presentations
Wed	Apr 24th	Project Presentations
Wed	May 1st			Final report due

Expectations: You can expect me to come to class on time, clearly communicate expectations for the presentations structure, format and clarity, give you feedback on a timely manner, adjust lecture material based on performance on presentations and quizzes and be available to meet regularly to discuss the progress of your project. I can expect you to spend an adequate amount of time on the readings each week (at least 3 hours), spend 60-80 hours on your final project.

Additional Policies: Please see the syllabus for the statement on academic conduct and student support systems. Unless you are assigned to compile lecture notes, please refrain from using laptops or other electronic devices during class.

Related Courses: You are encouraged to expand your readings from related courses, for example: