Package org.apache.spark.mllib.classification

Class LogisticRegressionWithLBFGS

Object
org.apache.spark.mllib.regression.GeneralizedLinearAlgorithm<LogisticRegressionModel>
org.apache.spark.mllib.classification.LogisticRegressionWithLBFGS
All Implemented Interfaces:
Serializable, org.apache.spark.internal.Logging
public class LogisticRegressionWithLBFGS extends GeneralizedLinearAlgorithm<LogisticRegressionModel> implements Serializable
Train a classification model for Multinomial/Binary Logistic Regression using Limited-memory BFGS. Standard feature scaling and L2 regularization are used by default.

Earlier implementations of LogisticRegressionWithLBFGS applies a regularization penalty to all elements including the intercept. If this is called with one of standard updaters (L1Updater, or SquaredL2Updater) this is translated into a call to ml.LogisticRegression, otherwise this will use the existing mllib GeneralizedLinearAlgorithm trainer, resulting in a regularization penalty to the intercept.

See Also:
Note:
Labels used in Logistic Regression should be {0, 1, ..., k - 1} for k classes multi-label classification problem.

  • Nested Class Summary

    Nested classes/interfaces inherited from interface org.apache.spark.internal.Logging

    org.apache.spark.internal.Logging.LogStringContext, org.apache.spark.internal.Logging.SparkShellLoggingFilter

  • Constructor Summary

    Constructors
    Constructor
    Description
    LogisticRegressionWithLBFGS()
     

  • Method Summary

    Modifier and Type
    Method
    Description
    LBFGS
    optimizer()
    The optimizer to solve the problem.
    LogisticRegressionModel
    run(RDD<LabeledPoint> input)
    Run Logistic Regression with the configured parameters on an input RDD of LabeledPoint entries.
    LogisticRegressionModel
    run(RDD<LabeledPoint> input, Vector initialWeights)
    Run Logistic Regression with the configured parameters on an input RDD of LabeledPoint entries starting from the initial weights provided.
    LogisticRegressionWithLBFGS
    setNumClasses(int numClasses)
    Set the number of possible outcomes for k classes classification problem in Multinomial Logistic Regression.

    Methods inherited from class org.apache.spark.mllib.regression.GeneralizedLinearAlgorithm

    getNumFeatures, isAddIntercept, setIntercept, setValidateData

    Methods inherited from class java.lang.Object

    equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    Methods inherited from interface org.apache.spark.internal.Logging

    initializeForcefully, initializeLogIfNecessary, initializeLogIfNecessary, initializeLogIfNecessary$default$2, isTraceEnabled, log, logBasedOnLevel, logDebug, logDebug, logDebug, logDebug, logError, logError, logError, logError, logInfo, logInfo, logInfo, logInfo, logName, LogStringContext, logTrace, logTrace, logTrace, logTrace, logWarning, logWarning, logWarning, logWarning, MDC, org$apache$spark$internal$Logging$$log_, org$apache$spark$internal$Logging$$log__$eq, withLogContext

  • Constructor Details

    • LogisticRegressionWithLBFGS

      public LogisticRegressionWithLBFGS()

  • Method Details

    • optimizer

      public LBFGS optimizer()
      Description copied from class: GeneralizedLinearAlgorithm
      The optimizer to solve the problem.

      Specified by:
      optimizer in class GeneralizedLinearAlgorithm<LogisticRegressionModel>
      Returns:
      (undocumented)

    • run

      public LogisticRegressionModel run(RDD<LabeledPoint> input)
      Run Logistic Regression with the configured parameters on an input RDD of LabeledPoint entries.

      If a known updater is used calls the ml implementation, to avoid applying a regularization penalty to the intercept, otherwise defaults to the mllib implementation. If more than two classes or feature scaling is disabled, always uses mllib implementation. If using ml implementation, uses ml code to generate initial weights.

      Overrides:
      run in class GeneralizedLinearAlgorithm<LogisticRegressionModel>
      Parameters:
      input - (undocumented)
      Returns:
      (undocumented)

    • run

      public LogisticRegressionModel run(RDD<LabeledPoint> input, Vector initialWeights)
      Run Logistic Regression with the configured parameters on an input RDD of LabeledPoint entries starting from the initial weights provided.

      If a known updater is used calls the ml implementation, to avoid applying a regularization penalty to the intercept, otherwise defaults to the mllib implementation. If more than two classes or feature scaling is disabled, always uses mllib implementation. Uses user provided weights.

      In the ml LogisticRegression implementation, the number of corrections used in the LBFGS update can not be configured. So optimizer.setNumCorrections() will have no effect if we fall into that route.

      Overrides:
      run in class GeneralizedLinearAlgorithm<LogisticRegressionModel>
      Parameters:
      input - (undocumented)
      initialWeights - (undocumented)
      Returns:
      (undocumented)

    • setNumClasses

      public LogisticRegressionWithLBFGS setNumClasses(int numClasses)
      Set the number of possible outcomes for k classes classification problem in Multinomial Logistic Regression. By default, it is binary logistic regression so k will be set to 2.
      Parameters:
      numClasses - (undocumented)
      Returns:
      (undocumented)