Optimal entropy estimation on large alphabets via best polynomial approximation
Abstract
Consider the problem of estimating the Shannon聽 entropy of a distribution on $ 办$听 elements from $ 苍$听 颈苍诲别辫别苍诲别苍迟听 samples. We show that the minimax mean-square error is within聽 universal multiplicative constant factors of 聽$\left( \frac{n}{k \log n} \right)^{2} + \frac{\log^2 k}{n}$. 罢丑颈蝉听 implies the recent result of Valiant-Valiant [ 1 ] that the minimal聽 sample size for consistent entropy estimation scales according to $\Theta( \frac{k}{\log k} )$.聽 The apparatus of best polynomial approximation plays a key role in both the minimax lower bound and the construction of optimal estimators.