Information theory concerns the relationship between data, models, and reality. Its striking generality allows information theory to show up in a myriad of places and lend a powerfully unifying perspective. As such, it serves as a critical bridge to bring physicists to the issues of today. It is, first and foremost, the language of deep learning and therefore a route to using and understanding the capacities of modern AI — undoubtedly a timely matter.
It is also the language of model forming and model evaluation, be it a human’s imperfect model of reality influenced by social media or a trillion-parameter language model containing hidden biases, or a policy-guiding model of climate change. This tutorial will showcase a range of ways to bring ideas from information theory to the understanding of models and will serve as a rich foray into the topic to excite all levels of background.