Average entropy and tail heaviness of bell-shaped distributions: graphs and calculations

This paper revisits the concept of average entropy, a functional measure of uncertainty introduced to address limitations in traditional entropy measures, particularly for continuous distributions. We investigate its utility as a consistent and interpretable measure of tail heaviness in bell-shaped probability distributions. Unlike kurtosis, which is undefined for many heavy-tailed distributions, average entropy is well-defined, location- and scale-invariant, and can be empirically estimated from data. We demonstrate that higher average entropy is associated with heavier distribution tails. This property is then used to define three canonical regions of the distribution: the peak, the shoulder, and the tail. The boundaries of these regions are identified in a consistent and unified manner across different distributions. These regions allow for objective comparisons of tail weight. The approach is applied to several well-known symmetric distributions, including the normal, student’s t, exponential power, logistic, and Cramér distributions. Additionally, we demonstrate a practical estimation method using histogram-based approximations. The findings establish average entropy as a robust measure complementing kurtosis for characterizing tail behavior in symmetric distributions.

Department

NSMTU

Publisher

Informa UK Limited

Sponsor

none

Copyright

Attribution-NonCommercial-NoDerivatives 4.0 International

cbn