On the Estimation of alpha-Divergences (2011)

Barnabas Poczos, Jeff Schneider


We propose new nonparametric, consistent Renyi-alpha and
Tsallis-alpha divergence estimators for continuous
distributions. Given two independent and identically distributed
samples, a ``naive'' approach would be to simply estimate the
underlying densities and plug the estimated densities into the
corresponding formulas. Our proposed estimators,
in contrast, avoid density estimation completely, estimating the
divergences directly using only simple k-nearest-neighbor
statistics. We are nonetheless able to prove that the estimators are
consistent under certain conditions. We also describe how to apply
these estimators to mutual information and demonstrate their
efficiency via numerical experiments.

Full text

Download (application/pdf, 828.4 kB)

Approximate BibTeX Entry

    Howpublished = {AISTATS 2011},
    Year = {2011},
    Booktitle = {AISTATS 2011},
    Author = { Barnabas Poczos, Jeff Schneider },
    Title = {On the Estimation of alpha-Divergences}

Copyright 2010, Carnegie Mellon University, Auton Lab. All Rights Reserved.