org.apache.spark.mllib.clustering

LDAModel

abstract class LDAModel extends Saveable

:: Experimental ::

Latent Dirichlet Allocation (LDA) model.

This abstraction permits for different underlying representations, including local and distributed data structures.

Annotations
@Experimental() @Since( "1.3.0" )
Linear Supertypes
Saveable, AnyRef, Any
Known Subclasses
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. LDAModel
  2. Saveable
  3. AnyRef
  4. Any
  1. Hide All
  2. Show all
Learn more about member selection
Visibility
  1. Public
  2. All

Abstract Value Members

  1. abstract def describeTopics(maxTermsPerTopic: Int): Array[(Array[Int], Array[Double])]

    Return the topics described by weighted terms.

    Return the topics described by weighted terms.

    maxTermsPerTopic

    Maximum number of terms to collect for each topic.

    returns

    Array over topics. Each topic is represented as a pair of matching arrays: (term indices, term weights in topic). Each topic's terms are sorted in order of decreasing weight.

    Annotations
    @Since( "1.3.0" )
  2. abstract def docConcentration: Vector

    Concentration parameter (commonly named "alpha") for the prior placed on documents' distributions over topics ("theta").

    Concentration parameter (commonly named "alpha") for the prior placed on documents' distributions over topics ("theta").

    This is the parameter to a Dirichlet distribution.

    Annotations
    @Since( "1.5.0" )
  3. abstract def formatVersion: String

    Current version of model save/load format.

    Current version of model save/load format.

    Attributes
    protected
    Definition Classes
    Saveable
  4. abstract def gammaShape: Double

    Shape parameter for random initialization of variational parameter gamma.

    Shape parameter for random initialization of variational parameter gamma. Used for variational inference for perplexity and other test-time computations.

    Attributes
    protected
  5. abstract def k: Int

    Number of topics

    Number of topics

    Annotations
    @Since( "1.3.0" )
  6. abstract def save(sc: SparkContext, path: String): Unit

    Save this model to the given path.

    Save this model to the given path.

    This saves:

    • human-readable (JSON) model metadata to path/metadata/
    • Parquet formatted data to path/data/

    The model may be loaded using Loader.load.

    sc

    Spark context used to save model data.

    path

    Path specifying the directory in which to save this model. If the directory already exists, this method throws an exception.

    Definition Classes
    Saveable
    Annotations
    @Since( "1.3.0" )
  7. abstract def topicConcentration: Double

    Concentration parameter (commonly named "beta" or "eta") for the prior placed on topics' distributions over terms.

    Concentration parameter (commonly named "beta" or "eta") for the prior placed on topics' distributions over terms.

    This is the parameter to a symmetric Dirichlet distribution.

    Note: The topics' distributions over terms are called "beta" in the original LDA paper by Blei et al., but are called "phi" in many later papers such as Asuncion et al., 2009.

    Annotations
    @Since( "1.5.0" )
  8. abstract def topicsMatrix: Matrix

    Inferred topics, where each topic is represented by a distribution over terms.

    Inferred topics, where each topic is represented by a distribution over terms. This is a matrix of size vocabSize x k, where each column is a topic. No guarantees are given about the ordering of the topics.

    Annotations
    @Since( "1.3.0" )
  9. abstract def vocabSize: Int

    Vocabulary size (number of terms or terms in the vocabulary)

    Vocabulary size (number of terms or terms in the vocabulary)

    Annotations
    @Since( "1.3.0" )

Concrete Value Members

  1. final def !=(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  2. final def !=(arg0: Any): Boolean

    Definition Classes
    Any
  3. final def ##(): Int

    Definition Classes
    AnyRef → Any
  4. final def ==(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  5. final def ==(arg0: Any): Boolean

    Definition Classes
    Any
  6. final def asInstanceOf[T0]: T0

    Definition Classes
    Any
  7. def clone(): AnyRef

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  8. def describeTopics(): Array[(Array[Int], Array[Double])]

    Return the topics described by weighted terms.

    Return the topics described by weighted terms.

    WARNING: If vocabSize and k are large, this can return a large object!

    returns

    Array over topics. Each topic is represented as a pair of matching arrays: (term indices, term weights in topic). Each topic's terms are sorted in order of decreasing weight.

    Annotations
    @Since( "1.3.0" )
  9. final def eq(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  10. def equals(arg0: Any): Boolean

    Definition Classes
    AnyRef → Any
  11. def finalize(): Unit

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  12. final def getClass(): Class[_]

    Definition Classes
    AnyRef → Any
  13. def hashCode(): Int

    Definition Classes
    AnyRef → Any
  14. final def isInstanceOf[T0]: Boolean

    Definition Classes
    Any
  15. final def ne(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  16. final def notify(): Unit

    Definition Classes
    AnyRef
  17. final def notifyAll(): Unit

    Definition Classes
    AnyRef
  18. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes
    AnyRef
  19. def toString(): String

    Definition Classes
    AnyRef → Any
  20. final def wait(): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  21. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  22. final def wait(arg0: Long): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from Saveable

Inherited from AnyRef

Inherited from Any

Ungrouped