What is Entropy?

TL;DR

Entropy in Information Theory There are many definitions of information in various literatures but all of them have the same property of distinction from a message: information is the substantive content independent of transmission processes.

Abstract

This short book is an elementary course on entropy, leading up to a calculation of the entropy of hydrogen gas at standard temperature and pressure. Topics covered include information, Shannon entropy and Gibbs entropy, the principle of maximum entropy, the Boltzmann distribution, temperature and coolness, the relation between entropy, expected energy and temperature, the equipartition theorem, the partition function, the relation between expected energy, free energy and entropy, the entropy of a classical harmonic oscillator, the entropy of a classical particle in a box, and the entropy of a classical ideal gas.