CS309: Robot Learning from Demonstration and Interaction

Course Info

Semester: Spring 2019
Time: 12:00 - 13:30 Tue / Thu
Location: WEL 2.128
Website: /archive/fri_rl_spring19/

  Instructor   Teaching Assistant
  Rudolf Lioutikov   Chia-Wen Cheng
Email: [javascript protected email address]   [javascript protected email address]
Office Hours: Fr 11:15-12:15 (GDC 3.430)
or by appointment
  Th 17:00-18:00
GDC 1.302 Tutorial Lobby

  Cameron Wolfe Christian Sweet Kamil Ali Vikram Idury
Email: [javascript protected email address] [javascript protected email address] [javascript protected email address] [javascript protected email address]
Office Hours: Fr 12:00-13:30
GDC 1.302 Tutorial Lobby
Mo 12:00-13:30
GDC 1.302 Tutorial Lobby
Tu 15:30-17:00
GDC 1.302 Tutorial Lobby
We 14:00-15:30
GDC 1.302 Tutorial Lobby

Piazza: http://piazza.com/utexas/spring2019/50670

Course Description

Currently, most robots are programmed meticulously by robotics experts in a controlled laboratory setting—a model that is obviously not scalable to large-scale deployment. Robot learning from demonstration has emerged as an alternative paradigm for teaching robots new tasks by simply showing them what to do, rather than by writing code. Thus, learning from demonstration research tries to answer the question: “How can robots learn to interpret and generalize human demonstrations?” Solving this core research problem will enable the next generation of personal robots to revolutionize the home and workplace in coming years.

This research stream will place students at the cutting-edge of robot learning from demonstration research, working with robots to perform complex manipulation tasks, such as autonomously building IKEA furniture. Students will be given instruction in three core areas of robotics: manipulation, perception, and human-robot interaction. Additionally, students will learn and practice programming skills via hands-on mini-projects in each of these areas. After these key competencies have been acquired, students will devise and implement research projects in the area of learning from demonstration.

Organization Class sessions will be held in WEL 2.128 on Tuesdays and Thursdays from 12:00 to 13:30 if not announced otherwise. Attendance is mandatory. Students are expected to email the instructor in advance to inform of any potential absences.

Readings: There is no textbook for this course. There will be a series of 4 readings assignments corresponding to different sections of the 2009 learning from demonstration survey paper by Argall et al.

Prerequisites: None, except some intrinsic motivation and a healthy thirst for knowledge.

Language: Course programming assignments will be in Python 3. We do not assume that students have previous experience with the language, but we do expect you to learn the basics very rapidly. Homework 1 is designed to teach you the basics of Python 3 if you are not already familiar.


There are four types of assignments that you will be expected to complete:

1. Reading assignments: These will cover different section of the survey paper.
All reading assignments should be completed before the class period for which they are assigned.

2. Homework exercises: Throughout the semester, problem sets will be assigned and graded. The goal of these problems is to get you comfortable with the material and prepared for the final.

3. Tutorials: Every last lecture of the month will be held in the FRI lab in GDC 5.710A and GDC 5.816. This lectures will require hand on work with the two collaborative robots. You will demonstrate and solve given tasks with the robot but also define and present your own tasks to your peers.

4. Data Collection: You will be required to demonstrate certain tasks several times to the robots. This will help you to develop a feeling for the capabilitites of the robots as well as improve your robot interaction and teaching skills. After a mandatory minimal number of demonstrations you can earn a bonus on your grade by providing additional demonstrations.

Planned Lecture Schedule

(Subject to change due to pace of class, see website for updates)
Day Topic Readings Assignments
  Install Conda & Jupyter Notebook  
Tue 01/22 Motivation, Organization  
Thu 01/24 Python: general     Python: solutions  

Tue 01/29 Python: continued  
Thu 01/31 Tutorial: Introduction to the Sawyers and the initial task    

Tue 02/05 Robotics: basics, position, orientation, transformation  
Thu 02/07 SCARA Robotics: fk, ik, singularities, dynamics  

Tue 02/12 Teaching Types: Kinesthetic, Tele-Operation, Observation
Designing (Collaborative) Tasks: goals, requirements, challenges
1. Introduction
2. Design choices
Homework 1:
Thu 02/14 Trajectory Representation  

Tue 02/19 Homework in Lecture  
Thu 02/21 Classification: FLD, Perceptron, SVM  

Tue 02/26 Classification: FLD, Perceptron, SVM  
Thu 02/28 Tutorial: Students present solutions for the first task 3. Gathering examples:
How the dataset is built

Tue 03/05 Regression: OLS, Ridge   Homework 2:
Thu 03/07 Regression: OLS, Ridge  

Tue 03/12 Homework in Lecture  
Thu 03/14 Clustering: KDE, KNN, Mean-Shift, K-means  

Tue 03/26 Clustering: KDE, KNN, Mean-Shift, K-means  
Thu 03/28 Tutorial: Students introduce final task  

Tue 04/02 Evaluation: Model Selection, Cross Validation   Homework 3:
Reinforcement Learning
Thu 04/04 Reinforcement Learning: MDP, Features  

Tue 04/09 Reinforcement Learning: Bellman Optimality, Exploration vs Exploitation  
Thu 04/11 Homework in Lecture 4. Deriving a policy:
The source of the
state to action mapping

Tue 04/16 Imitation Learning: Movement Primitives  
Thu 04/18 Imitation Learning: Movement Primitives  

Tue 04/23 Imitation Learning: Behavioral Cloning   Homework 4:
Imitation Learning,
Thu 04/25 Homework in Lecture  

Tue 04/30 Tutorial: Students present solutions for final task  
Thu 05/02 ROS: general, messages, pub-sub, rosbag  

Tue 05/07 ROS: services, tf2   Final Project
Thu 05/09 Inverse Reinforcement Learning 5. Limitations of the
demonstration dataset
6. Future directions
7. Conclusion

Tue 05/21
(subject to change)
Final Project Presentations  
The "Homework in Lecture" lectures will give you the opportunity to ask question about the current homework assignment and discuss it with your peers. Furthermore, these lectures will also be used as buffer lectures in case we fall behind schedule.


Homework assignments are due by the time the corresponding lecture begins.
For example, Homework 1 will be published Tue 02/12 12:00 and is due Thu 02/28 12:00.

Overall grades will be determined from:

  • Homework Exercises: 40%
  • Tutorials: 20%
  • Data Collection: 20%
  • Final Project: 20%

Grades will be assigned using both plus and minus grades.

Academic honesty policy

You are encouraged to discuss the readings and concepts with classmates, but all written work must be your own. Programming assignments must be your own, except for 2-person teams when teams are authorized. You may NOT look online for existing implementations of algorithms related to the programming assignments, even as a reference. Your code will be analyzed by automatic tools that detect plagiarism to ensure that it is original.

Students caught cheating will automatically fail the course and will be reported to the university. If in doubt about the ethics of any particular action, look at the departmental guidelines and/or ask—ignorance of the rules will not shield you from potential consequences.

Notice about students with disabilities

The University of Texas at Austin provides upon request appropriate academic accommodations for qualified students with disabilities. For more information, contact the Division of Diversity and Community Engagement — Services for Students with Disabilities at 512-471-6529; 512-471-4641 TTY.

Notice about missed work due to religious holy days

A student who misses an examination, work assignment, or other project due to the observance of a religious holy day will be given an opportunity to complete the work missed within a reasonable time after the absence, provided that he or she has properly notified the instructor. It is the policy of the University of Texas at Austin that the student must notify the instructor at least fourteen days prior to the classes scheduled on dates he or she will be absent to observe a religious holy day. For religious holy days that fall within the first two weeks of the semester, the notice should be given on the first day of the semester. The student will not be penalized for these excused absences, but the instructor may appropriately respond if the student fails to complete satisfactorily the missed assignment or examination within a reasonable time after the excused absence.