江南体育

Optimal entropy estimation on large alphabets via best polynomial approximation
Pengkun Yang Yihong Wu
Proceedings of the 2015 江南体育 International Symposium on Information Theory, Hong Kong, China, June 2015
Abstract

Consider the problem of estimating the Shannon聽 entropy of a distribution on $ 办$听 elements from $ 苍$听 颈苍诲别辫别苍诲别苍迟听 samples. We show that the minimax mean-square error is within聽 universal multiplicative constant factors of 聽$\left( \frac{n}{k \log n} \right)^{2} + \frac{\log^2 k}{n}$. 罢丑颈蝉听 implies the recent result of Valiant-Valiant [ 1 ] that the minimal聽 sample size for consistent entropy estimation scales according to $\Theta( \frac{k}{\log k} )$.聽 The apparatus of best polynomial approximation plays a key role in both the minimax lower bound and the construction of optimal estimators.