Durham University
Programme and Module Handbook

Undergraduate Programme and Module Handbook 2022-2023 (archived)

Module COMP3667: REINFORCEMENT LEARNING

Department: Computer Science

COMP3667: REINFORCEMENT LEARNING

Type Open Level 3 Credits 10 Availability Available in 2022/23 Module Cap None. Location Durham

Prerequisites

  • (COMP2261 Artificial Intelligence AND COMP2271 Data Science)

Corequisites

  • COMP3547 Deep Learning

Excluded Combination of Modules

  • None

Aims

  • To understand computational models of learning in dynamic environments.
  • Learning how to plan and design agents with intelligent behaviour, taking actions to control the environment and maximise cumulative future rewards.

Content

  • Introduction to reinforcement learning.
  • Markov decision processes and planning by dynamic programming.
  • Model free prediction and control.
  • Value-based and policy-based reinforcement learning.
  • Scaling up reinforcement learning approaches with deep learning.
  • Integrating learning and planning, and balancing exploration/exploitation.

Learning Outcomes

Subject-specific Knowledge:
  • On completion of the module, students will be able to demonstrate:
  • an understanding of the key features of reinforcement learning and differences with non-interactive learning.
  • an understanding of state-of-the-art reinforcement learning algorithms.
  • an understanding of the issues faced in scaling reinforcement learning approaches using deep learning.
Subject-specific Skills:
  • On completion of the module, students will be able to demonstrate:
  • an ability to use modern libraries to design, train, validate and test deep reinforcement learning models.
  • an ability to find RL based solutions with respect to the task or environment.
  • an ability to design bespoke RL algorithms based on the problem and the environment, such as whether in continuous or discrete action spaces.
  • an ability to solve complex learning and planning problems in dynamic environments.
Key Skills:
  • On completion of the module, students will be able to demonstrate:
  • the scientific approach to the design, training, validation, and testing of reinforcement techniques in a broad range of applications.
  • an ability to design new environments with OpenAI gym, and design tailored agents that learn to control the environments.
  • an ability to identify the problem area and subsequently design and implement state-of-the-art reinforcement learning approaches.

Modes of Teaching, Learning and Assessment and how these contribute to the learning outcomes of the module

  • Lectures enable the students to learn new material relevant to reinforcement learning, as well as its applications.
  • Practicals enable students to acquire the necessary coding skills, learn about the relevant libraries and packages and receive feedback on their work.
  • Summative assessments assess the knowledge of relevant libraries and application of methods and techniques.

Teaching Methods and Learning Hours

Activity Number Frequency Duration Total/Hours
lectures 10 1 per week 1 hour 10
practicals 10 1 per week 1 hour 10
preparation and reading 80
total 100

Summative Assessment

Component: Coursework Component Weighting: 100%
Element Length / duration Element Weighting Resit Opportunity
Summative Assignment 100% No

Formative Assessment:

Example formative exercises are given during the course. The first few lab practicals are dedicated to formative assignments aimed at familiarising students with state-of-the-art packages and libraries used in reinforcement learning. Feedback will be provided to the students on the summative assignments and lecture materials during the practicals. Additional revision lectures may be arranged in the module's lecture slots in the 3rd term.


Attendance at all activities marked with this symbol will be monitored. Students who fail to attend these activities, or to complete the summative or formative assessment specified above, will be subject to the procedures defined in the University's General Regulation V, and may be required to leave the University