Musing upon Information Theory

 

Professor Te Sun Han, NICT

Basic logics underlying typical theorems in informtion theory are scrutinized from the viewpoint of information-spectra. Very general sources, which may be neither stationary nor ergodic, are introduced to establish conceptually simple formulars holding for the generalized AEP theorem, the optimal source coding theorem along with the reliability function, the fundamental limits of random number generation, the folklore theorem about source coding, the invariance theorem of information-spectra in the process of optimal source coding, the type II error rate function for hypothesis testing, etc. To do so, it plays the key role to define the limit superior in probability and the limit inferior in probability, which look like reasonable extentions of the well-established concept of "limit in probability." It turns out also that source coding may be regarded as being a mere special case of hypothesis testing in the generalized sense.