Source coding theorems on generalized entropy and its application in data compression

Abstract

Classical source coding theorems, grounded in Shannon entropy, provide the foundational limits of data compression. However, in various real-world systems - such as those exhibiting long-range dependencies, non-Markovian structures, or multifractal distributions-Shannon entropy may not fully capture the complexity of the source. This paper investigates source coding theorems under generalized entropy measures, particularly Tsallis and & Racute;enyi entropies, and extends classical results to these broader frameworks. We establish necessary and sufficient conditions for source coding under generalized entropy formulations, derive bounds on compression rates, and demonstrate their relevance through examples involving memory-laden and heavy-tailed sources. Furthermore, we explore practical implications in data compression schemes, including entropy coding and universal coding, and validate our theoretical findings via numerical simulations.