Online Lecture Modifications

The lecture will be taught in an online-format for the remainder of the semester.

Lectures will be held in a zoom meeting during the usual lecture times Tue/Thu 15:30 - 17:00 CST.

The lectures will be recorded and made available to you.

Class recordings are reserved only for the use of members of this class (students, TAs, and the instructor) and only for educational purposes. Recordings should not be shared outside the class in any form. Violation of this restriction could lead to Student Misconduct proceedings.

More changes and adaptation regarding data collection and simulation environments will be discussed and announced in the first meeting (Tue. 03/31/2020).

Robot Learning from Demonstration and Interaction (CS309)

  • semester
    spring 2020
  • time
    tue 15:30-17:00
    thu 15:30-17:00
  • lecture room
    PAR 208
  • lab
    GDC 5.710A
  • instructor
    Rudolf Lioutikov
  • e-mail

Course Description

Currently, most robots are programmed meticulously by robotics experts in a controlled laboratory setting—a model that is obviously not scalable to large-scale deployment. Robot learning from demonstration has emerged as an alternative paradigm for teaching robots new tasks by simply showing them what to do, rather than by writing code. Thus, learning from demonstration research tries to answer the question: “How can robots learn to interpret and generalize human demonstrations?” Solving this core research problem will enable the next generation of personal robots to revolutionize the home and workplace in coming years.

This research stream will place students at the cutting-edge of robot learning from demonstration research, working with robots to perform complex manipulation tasks, such as autonomously building IKEA furniture. Students will be given instruction in three core areas of robotics: manipulation, perception, and human-robot interaction. Additionally, students will learn and practice programming skills via hands-on mini-projects in each of these areas. After these key competencies have been acquired, students will devise and implement research projects in the area of learning from demonstration.


Class sessions will be held in PAR 208 on Tuesdays and Thursdays from 15:30 to 17:00 if not announced otherwise. Attendance is mandatory. Students are expected to email the instructor in advance to inform of any potential absences.


None, except some intrinsic motivation and a healthy thirst for knowledge.

Programming Language

Course programming assignments will be in Python 3. We do not assume that students have previous experience with the language, but the students are expected to learn the basics very rapidly.


There are six types of assignments that the students are expected to complete.

Homework Exercises

Throughout the semester, problem sets will be assigned and graded. The goal of these problems is to get the students comfortable with the material and prepared for the final project.

Lecture Notes

Students will be randomly selected to present a summary of the last two lectures at the next lecture.


Tutorials will be held in the FRI lab in GDC 5.710A. This lectures will require hand on work with the two collaborative robots. The students will demonstrate and solve given tasks with the robot but also define and present your own tasks to your peers.

Data Collection

The students will be required to demonstrate certain tasks several times to the robots. This will help them to develop an understanding of the capabilities of the robots as well as improve your robot interaction and teaching skills.

Final Project

In the final project the students apply learned methods on a robot learning task performed on a collaborative robot platform. The students report their acquired knowledge in a short research paper as well as a short project presentation.

Reading Assignments

In addition, the students are expected to read through literature related to the course and the final project. In particular, there will be 4 nongraded reading assignments corresponding to different sections of learning from demonstration survey by Argall et al.


The overall grade will consist of five of the coursework assignments. Grades will be assigned using both plus and minus grades.

Lecture Schedule

The schedule is subject to change due to pace of class, see website for updates.

Inverse Reinforcement Learning
Day Topic Readings Assignments
Tue 01/21 Motivation, Organization   Homework 0:
Thu 01/23 Linear Algebra Refresher  
Tue 01/28 Regression: OLS, Ridge Regression  
Thu 01/30 Regression: Lasso, Elastic Nets  
Tue 02/04 Regression: Features, Polynomial Regression    
Tue 02/06 Robotics: basics, position, orientation    
Tue 02/011 Robotics: homogeneous transformation, ROS basics    
Thu 02/13 Tutorial: Introduction to the Sawyers and the initial task   Homework 1:
Tue 02/18 Robotics: Inverse Kinematics, Singularities, Dynamics  
Thu 02/20 Teaching Types: Kinesthetic, Tele-Operation, Observation
Designing (Collaborative) Tasks: goals, requirements, challenges
1. Introduction
2. Design choices
Tue 02/25 Trajectory Representation  
Thu 02/27 Classification: FLD, Perceptron  
Tue 03/03 Classification: SVM   Homework 2:
Model Evaluation
Thu 03/05 Tutorial: Students present solutions for the first task 3. Gathering examples:
How the dataset is built
Thu 03/10 Clustering: KDE, KNN  
Tue 03/12 Clustering: Mean-Shift, K-means  
Tue 03/24 Evaluation: Model Selection, Cross Validation  
Thu 03/26 Reinforcement Learning: MDP, Features  
Tue 03/31 Introduction: Pytorch   Homework 3:
Reinforcement Learning,
Imitation Learning
Thu 04/02 Tutorial: Students introduce final task  
Tue 04/07 Reinforcement Learning: Bellman Optimality, Exploration vs Exploitation  
Thu 04/09 Imitation Learning: Behavioral Cloning 4. Deriving a policy:
The source of the
state to action mapping
Tue 04/14 Imitation Learning: Inverse Reinforcement Learning  
Thu 04/16 Imitation Learning: Inverse Reinforcement Learning  
Tue 04/21 Intermediate Presentation  
Thu 04/23 Project Meetings   Final Project
Tue 04/28 Project Meetings  
Thu 04/30 Project Meetings  
Tue 05/05 Project Meetings  
Thu 05/07 Project Meetings 5. Limitations of the
demonstration dataset
6. Future directions
7. Conclusion
Tue 05/14
(subject to change)
Final Project Presentations    

Office Hours

Several office hours are offered by various mentors. The office hours are held in the basement pc lab/pool in GDC (1.302 / 1.308 / 1.310). Always arrive at the beginning of the office. The mentors are instructed to end the office hours if there are no students in the first 20 minutes. If you run late to an office hour reach out to the respective mentor via e-mail.

Monday Tuesday Wednesday Thursday Friday

Academic Honesty Policy

You are encouraged to discuss the readings and concepts with classmates, but all written work must be your own. Programming assignments must be your own, except for 2-person teams when teams are authorized. You may NOT look online for existing implementations of algorithms related to the programming assignments, even as a reference. Your code will be analyzed by automatic tools that detect plagiarism to ensure that it is original.
Students caught cheating will automatically fail the course and will be reported to the university. If in doubt about the ethics of any particular action, look at the departmental guidelines and/or ask — ignorance of the rules will not shield you from potential consequences.

Notice About Students With Disabilities

The University of Texas at Austin provides upon request appropriate academic accommodations for qualified students with disabilities. For more information, contact the Division of Diversity and Community Engagement — Services for Students with Disabilities at 512-471-6529; 512-471-4641 TTY.

Notice About Missed Work Due To Religious Holy Days

A student who misses an examination, work assignment, or other project due to the observance of a religious holy day will be given an opportunity to complete the work missed within a reasonable time after the absence, provided that he or she has properly notified the instructor. It is the policy of the University of Texas at Austin that the student must notify the instructor at least fourteen days prior to the classes scheduled on dates he or she will be absent to observe a religious holy day. For religious holy days that fall within the first two weeks of the semester, the notice should be given on the first day of the semester. The student will not be penalized for these excused absences, but the instructor may appropriately respond if the student fails to complete satisfactorily the missed assignment or examination within a reasonable time after the excused absence.