Information Theory

Undergraduate | 2026

Course page banner
area/catalogue icon
Area/Catalogue
COMP 4007
Course ID icon
Course ID
203300
Level of study
Level of study
Undergraduate
Unit value icon
Unit value
6
Course level icon
Course level
4
Study abroad and student exchange icon
Inbound study abroad and exchange
Inbound study abroad and exchange
The fee you pay will depend on the number and type of courses you study.
Yes
University-wide elective icon
University-wide elective course
Yes
Single course enrollment
Single course enrolment
Yes
alt
Note:
Course data is interim and subject to change

Course overview

Information theory is one of the seminal ideas of the 20th century. It enabled modern telecommunications and computing, and now is frequently used in modern Machine Learning. Information theory lies at the heart of how we communicate, store and preserve what we know. In some sense it is the most fundamental theory, as without symbols (representations of information) we couldn't even do mathematics. In this course, learners will develop knowledge of Information Theory and skills in using it to efficiently and reliably code signals. This course will help to unify concepts from statistical mechanics, statistical estimation theory and entropy, broadening and deepening knowledge gained from earlier units.

  • Concepts Of Information And Uncertainty
  • Coding And Communication
  • Data Science And Other Applications

Course learning outcomes

  • Rationalise different definitions and uses of entropy
  • Derive key measures (e.g., mutual information) from known distributions
  • Propose mechanisms for coding and compression of data in different contexts
  • Implement core theoretical findings in applied settings including cyber security, network coding, game theory, cryptography and statistical estimation
  • Quantify uncertainty both mathematically and programmatically

Prerequisite(s)

N/A

Corequisite(s)

N/A

Antirequisite(s)

N/A