Premium
An Introduction to Coding Theory and the Two‐Part Minimum Description Length Principle
Author(s) -
Lee Thomas C. M.
Publication year - 2001
Publication title -
international statistical review
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.051
H-Index - 54
eISSN - 1751-5823
pISSN - 0306-7734
DOI - 10.1111/j.1751-5823.2001.tb00455.x
Subject(s) - minimum description length , coding (social sciences) , mathematics , computer science , coding theory , algorithm , statistical theory , statistics
Summary This article provides a tutorial introduction to the so‐called two‐part minimum description length (MDL) principle proposed by Rissanen. This two‐part MDL principle is a powerful methodology for solving many statistical model selection problems. However, it seems that this powerful methodology is only adopted by a small number of statisticians to tackle a small number of problems. One plausible reason for this is that the coding theory results required by the MDL principle are somewhat new to most statisticians, and that there are not many readily accessible articles introducing these results appearing in the statistical literature. The first part of this article is devoted to a discussion of such coding theory results. Then, in the second part of the article, the two‐part MDL principle is introduced and explained. In doing so, only those coding theory results that are presented in the first part of the article are used. Finally, the applicability of the two‐part MDL principle is demonstrated by applying it to tackle four different statistical problems.