org.apache.spark.mllib.clustering
Class OnlineLDAOptimizer

Object
  org.apache.spark.mllib.clustering.OnlineLDAOptimizer

All Implemented Interfaces:

LDAOptimizer

public final class OnlineLDAOptimizer
extends Object
implements LDAOptimizer

:: DeveloperApi ::

An online optimizer for LDA. The Optimizer implements the Online variational Bayes LDA algorithm, which processes a subset of the corpus on each iteration, and updates the term-topic distribution adaptively.

Original Online LDA paper: Hoffman, Blei and Bach, "Online Learning for Latent Dirichlet Allocation." NIPS, 2010.

Constructor Summary

OnlineLDAOptimizer()

Method Summary

double	getKappa() Learning rate: exponential decay rate
double	getMiniBatchFraction() Mini-batch fraction, which sets the fraction of document sampled and used in each iteration
double	getTau0() A (positive) learning parameter that downweights early iterations.
OnlineLDAOptimizer	setKappa(double kappa) Learning rate: exponential decay rate---should be between (0.5, 1.0] to guarantee asymptotic convergence.
OnlineLDAOptimizer	setMiniBatchFraction(double miniBatchFraction) Mini-batch fraction in (0, 1], which sets the fraction of document sampled and used in each iteration.
OnlineLDAOptimizer	setTau0(double tau0) A (positive) learning parameter that downweights early iterations.

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

OnlineLDAOptimizer

public OnlineLDAOptimizer()

Method Detail

getTau0

public double getTau0()

A (positive) learning parameter that downweights early iterations. Larger values make early iterations count less.

Returns:

(undocumented)

setTau0

public OnlineLDAOptimizer setTau0(double tau0)

A (positive) learning parameter that downweights early iterations. Larger values make early iterations count less. Default: 1024, following the original Online LDA paper.

Parameters:

tau0 - (undocumented)

Returns:

(undocumented)

getKappa

public double getKappa()

Learning rate: exponential decay rate

Returns:

(undocumented)

setKappa

public OnlineLDAOptimizer setKappa(double kappa)

Learning rate: exponential decay rate---should be between (0.5, 1.0] to guarantee asymptotic convergence. Default: 0.51, based on the original Online LDA paper.

Parameters:

kappa - (undocumented)

Returns:

(undocumented)

getMiniBatchFraction

public double getMiniBatchFraction()

Mini-batch fraction, which sets the fraction of document sampled and used in each iteration

Returns:

(undocumented)

setMiniBatchFraction

public OnlineLDAOptimizer setMiniBatchFraction(double miniBatchFraction)

Mini-batch fraction in (0, 1], which sets the fraction of document sampled and used in each iteration.

Note that this should be adjusted in synch with LDA.setMaxIterations() so the entire corpus is used. Specifically, set both so that maxIterations * miniBatchFraction >= 1.

Default: 0.05, i.e., 5% of total documents.

Parameters:

miniBatchFraction - (undocumented)

Returns:

(undocumented)