Course overview
This course will introduce the basic tools and concepts in information theory, and practical coding schemes for data compression and transmission over noisy channels. The first part of this course is about information theory. The course aims at answering two fundamental questions in data communications: What is the ultimate data compression? What is the ultimate transmission rate of a communication channel? To answer these questions, the basic tools and concepts in information theory are introduced. Practical coding schemes for data compression and transmission over noisy channels are introduced in the second part of this course. Review of Mathematical Background and Basic Concepts in Probability Theory. Introduction of Information Measures and their Properties: Entropy; Mutual Information; Conditional Entropy; Conditional Mutual Information; Kullback-Leibler Divergence; Log-sum inequality; Data Processing Inequality; Fano's inequality. Asymptotic Equipartition Property (AEP): Law of Large Numbers; Types and Typicality. Lossless source coding theorem and algorithms. Noisy Channel Models and Shannon's Channel Coding Theorem. Fundamentals of Linear Codes: Introduction to Error Detection and Correction; Types of Codes; Minimum Distance. Block Coding Principles: Generator Matrix Description, Systematic Codes, Detection and Correction Bounds. Convolutional Codes (Encoding): encoders; generator polynomials; constraint length; state diagrams; tree and trellis diagrams; distance measures. Convolutional Codes (Decoding): Viterbi algorithm; Soft-Decision Decoding; MAP Decoding. LDPC Codes: Code Density, Tanner Graphs, Belief Propagation. Special topics: Network Coding and Digital Fountain.
Course learning outcomes
- Examine mathematically the notion of information contained in data and the fundamental limits to compress data for efficient storage.
- Analyse and compare the roles of redundancy in combating the effect of noise in various communications media.
- Develop the connections between information-theoretic schemes and algebraic/convolutional/graph-based schemes for communicating data over discrete memoryless channels..
- Examine and derive the operations involved in different encoders and decoders, and be able to compare the advantages and disadvantages of various coding paradigms.
- Use problem-solving and computing skills to analyse and derive efficient means of representation, storage and transmission of information in communications networks.
Fee calculator
To display course fees, please select your status and program below:
We’re updating this Fee Calculator. It currently shows fees for programs only. Please check the relevant program for full fee details.
Study Abroad student tuition fees are available here.
Only some Postgraduate Coursework programs are available as Commonwealth Supported. Please check your program for specific fee information.
The Student Contribution amount displayed below is for students commencing a new program from 2021 onwards. If you are continuing in a program you commenced prior to 1 January 2021, or are commencing an Honours degree relating to an undergraduate degree you commenced prior to 1 January 2021, you may be charged a different Student Contribution amount from the amount displayed below. Please check the Student Contribution bands for continuing students here. If you are an international student, or a domestic student studying in a full fee paying place, and are continuing study that you commenced in 2025 or earlier, your fees will be available here before enrolments open for 2026.