D-CIS Publication Database

Publication

Type of publication:Inproceedings
Entered by:JOSM
TitleHybrid Variational/Gibbs Collapsed Inference in Topic Models
Bibtex cite ID
Booktitle Proceedings of the 24th Conference on Uncertainty in Artificial Intelligence (UAI 2008)
Year published 2008
Month July
Pages 587-594
Location 9-12 July 2008, Helsinki, Finland
Keywords hybrid variational/gibbs,collapsed inference,topic models
Abstract
Variational Bayesian inference and (collapsed) Gibbs sampling are the two important classes of inference algorithms for Bayesian networks. Both have their advantages and disadvantages: collapsed Gibbs sampling is unbiased but is also inefficient for large count values and requires averaging over many samples to reduce variance. On the other hand, variational Bayesian inference is efficient and accurate for large count values but suffers from bias for small counts. We propose a hybrid algorithm that combines the best of both worlds: it samples very small counts and applies variational updates to large counts. This hybridization is shown to significantly improve testset perplexity relative to variational inference at no computational cost.
Authors
Welling, Max
Teh, Yee Whye
Kappen, Hilbert J.
Topics
=SEE CLASSIFICATION DIFFERENCE FROM OTHERS=
BibTeXBibTeX
RISRIS
Attachments
 
Total mark: 5