Theory of Information
3 ECTS
Aim:These lectures deal with statistical information processing and theory. The course first introduces estimation theory, with the fundamendal Cramer-Rao bound associated with Fisher information. Then we deal with classical information theory: ayesia entropy, Kullback-Leibler divergence and present the source and channel coding theorems, with applications in compression and data transmission. The lectures are illustrated with labs in estimation concepts and on an implementation of a compression algorithm.
Content:
- Estimation theory (maximum likelihood, Bayesian approaches)
- Entropies and divergences
- Source coding (compression)
- Channel coding