Question

I've been experimenting with LDA topic modelling using Gensim. I couldn't seem to find any topic model evaluation facility in Gensim, which could report on the perplexity of a topic model on held-out evaluation texts thus facilitates subsequent fine tuning of LDA parameters (e.g. number of topics). It would be greatly appreciated if anyone could shed some light on how I can perform topic model evaluation in Gensim. This question has also been posted on metaoptimize.

Was it helpful?

Solution

Found the answer on the gensim mailing list.

In short, the bound() method of LdaModel computes a lower bound on perplexity, based on a held-out corpus.

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top