Package org.apache.spark.ml.classification

Class LogisticRegression

Object
org.apache.spark.ml.PipelineStage
org.apache.spark.ml.Estimator<M>
org.apache.spark.ml.Predictor<FeaturesType,E,M>
org.apache.spark.ml.classification.Classifier<FeaturesType,E,M>
org.apache.spark.ml.classification.ProbabilisticClassifier<Vector,LogisticRegression,LogisticRegressionModel>
org.apache.spark.ml.classification.LogisticRegression
All Implemented Interfaces:
Serializable, org.apache.spark.internal.Logging, org.apache.spark.ml.classification.ClassifierParams, org.apache.spark.ml.classification.LogisticRegressionParams, org.apache.spark.ml.classification.ProbabilisticClassifierParams, Params, HasAggregationDepth, HasElasticNetParam, HasFeaturesCol, HasFitIntercept, HasLabelCol, HasMaxBlockSizeInMB, HasMaxIter, HasPredictionCol, HasProbabilityCol, HasRawPredictionCol, HasRegParam, HasStandardization, HasThreshold, HasThresholds, HasTol, HasWeightCol, org.apache.spark.ml.PredictorParams, DefaultParamsWritable, Identifiable, MLWritable
public class LogisticRegression extends ProbabilisticClassifier<Vector,LogisticRegression,LogisticRegressionModel> implements org.apache.spark.ml.classification.LogisticRegressionParams, DefaultParamsWritable, org.apache.spark.internal.Logging
Logistic regression. Supports: - Multinomial logistic (softmax) regression. - Binomial logistic regression.

This class supports fitting traditional logistic regression model by LBFGS/OWLQN and bound (box) constrained logistic regression model by LBFGSB.

Since 3.1.0, it supports stacking instances into blocks and using GEMV/GEMM for better performance. The block size will be 1.0 MB, if param maxBlockSizeInMB is set 0.0 by default.

See Also:

  • Constructor Details

    • LogisticRegression

      public LogisticRegression(String uid)

    • LogisticRegression

      public LogisticRegression()

  • Method Details

    • load

      public static LogisticRegression load(String path)

    • read

      public static MLReader<T> read()

    • family

      public final Param<String> family()
      Specified by:
      family in interface org.apache.spark.ml.classification.LogisticRegressionParams

    • lowerBoundsOnCoefficients

      public Param<Matrix> lowerBoundsOnCoefficients()
      Specified by:
      lowerBoundsOnCoefficients in interface org.apache.spark.ml.classification.LogisticRegressionParams

    • upperBoundsOnCoefficients

      public Param<Matrix> upperBoundsOnCoefficients()
      Specified by:
      upperBoundsOnCoefficients in interface org.apache.spark.ml.classification.LogisticRegressionParams

    • lowerBoundsOnIntercepts

      public Param<Vector> lowerBoundsOnIntercepts()
      Specified by:
      lowerBoundsOnIntercepts in interface org.apache.spark.ml.classification.LogisticRegressionParams

    • upperBoundsOnIntercepts

      public Param<Vector> upperBoundsOnIntercepts()
      Specified by:
      upperBoundsOnIntercepts in interface org.apache.spark.ml.classification.LogisticRegressionParams

    • maxBlockSizeInMB

      public final DoubleParam maxBlockSizeInMB()
      Description copied from interface: HasMaxBlockSizeInMB
      Param for Maximum memory in MB for stacking input data into blocks. Data is stacked within partitions. If more than remaining data size in a partition then it is adjusted to the data size. Default 0.0 represents choosing optimal value, depends on specific algorithm. Must be &gt;= 0..
      Specified by:
      maxBlockSizeInMB in interface HasMaxBlockSizeInMB
      Returns:
      (undocumented)

    • aggregationDepth

      public final IntParam aggregationDepth()
      Description copied from interface: HasAggregationDepth
      Param for suggested depth for treeAggregate (&gt;= 2).
      Specified by:
      aggregationDepth in interface HasAggregationDepth
      Returns:
      (undocumented)

    • threshold

      public DoubleParam threshold()
      Description copied from interface: HasThreshold
      Param for threshold in binary classification prediction, in range [0, 1].
      Specified by:
      threshold in interface HasThreshold
      Returns:
      (undocumented)

    • weightCol

      public final Param<String> weightCol()
      Description copied from interface: HasWeightCol
      Param for weight column name. If this is not set or empty, we treat all instance weights as 1.0.
      Specified by:
      weightCol in interface HasWeightCol
      Returns:
      (undocumented)

    • standardization

      public final BooleanParam standardization()
      Description copied from interface: HasStandardization
      Param for whether to standardize the training features before fitting the model.
      Specified by:
      standardization in interface HasStandardization
      Returns:
      (undocumented)

    • tol

      public final DoubleParam tol()
      Description copied from interface: HasTol
      Param for the convergence tolerance for iterative algorithms (&gt;= 0).
      Specified by:
      tol in interface HasTol
      Returns:
      (undocumented)

    • fitIntercept

      public final BooleanParam fitIntercept()
      Description copied from interface: HasFitIntercept
      Param for whether to fit an intercept term.
      Specified by:
      fitIntercept in interface HasFitIntercept
      Returns:
      (undocumented)

    • maxIter

      public final IntParam maxIter()
      Description copied from interface: HasMaxIter
      Param for maximum number of iterations (&gt;= 0).
      Specified by:
      maxIter in interface HasMaxIter
      Returns:
      (undocumented)

    • elasticNetParam

      public final DoubleParam elasticNetParam()
      Description copied from interface: HasElasticNetParam
      Param for the ElasticNet mixing parameter, in range [0, 1]. For alpha = 0, the penalty is an L2 penalty. For alpha = 1, it is an L1 penalty.
      Specified by:
      elasticNetParam in interface HasElasticNetParam
      Returns:
      (undocumented)

    • regParam

      public final DoubleParam regParam()
      Description copied from interface: HasRegParam
      Param for regularization parameter (&gt;= 0).
      Specified by:
      regParam in interface HasRegParam
      Returns:
      (undocumented)

    • uid

      public String uid()
      Description copied from interface: Identifiable
      An immutable unique ID for the object and its derivatives.
      Specified by:
      uid in interface Identifiable
      Returns:
      (undocumented)

    • setRegParam

      public LogisticRegression setRegParam(double value)
      Set the regularization parameter. Default is 0.0.

      Parameters:
      value - (undocumented)
      Returns:
      (undocumented)

    • setElasticNetParam

      public LogisticRegression setElasticNetParam(double value)
      Set the ElasticNet mixing parameter. For alpha = 0, the penalty is an L2 penalty. For alpha = 1, it is an L1 penalty. For alpha in (0,1), the penalty is a combination of L1 and L2. Default is 0.0 which is an L2 penalty.

      Note: Fitting under bound constrained optimization only supports L2 regularization, so throws exception if this param is non-zero value.

      Parameters:
      value - (undocumented)
      Returns:
      (undocumented)

    • setMaxIter

      public LogisticRegression setMaxIter(int value)
      Set the maximum number of iterations. Default is 100.

      Parameters:
      value - (undocumented)
      Returns:
      (undocumented)

    • setTol

      public LogisticRegression setTol(double value)
      Set the convergence tolerance of iterations. Smaller value will lead to higher accuracy at the cost of more iterations. Default is 1E-6.

      Parameters:
      value - (undocumented)
      Returns:
      (undocumented)

    • setFitIntercept

      public LogisticRegression setFitIntercept(boolean value)
      Whether to fit an intercept term. Default is true.

      Parameters:
      value - (undocumented)
      Returns:
      (undocumented)

    • setFamily

      public LogisticRegression setFamily(String value)
      Sets the value of param family(). Default is "auto".

      Parameters:
      value - (undocumented)
      Returns:
      (undocumented)

    • setStandardization

      public LogisticRegression setStandardization(boolean value)
      Whether to standardize the training features before fitting the model. The coefficients of models will be always returned on the original scale, so it will be transparent for users. Note that with/without standardization, the models should be always converged to the same solution when no regularization is applied. In R's GLMNET package, the default behavior is true as well. Default is true.

      Parameters:
      value - (undocumented)
      Returns:
      (undocumented)

    • setThreshold

      public LogisticRegression setThreshold(double value)
      Specified by:
      setThreshold in interface org.apache.spark.ml.classification.LogisticRegressionParams

    • getThreshold

      public double getThreshold()
      Specified by:
      getThreshold in interface HasThreshold
      Specified by:
      getThreshold in interface org.apache.spark.ml.classification.LogisticRegressionParams

    • setWeightCol

      public LogisticRegression setWeightCol(String value)
      Sets the value of param weightCol(). If this is not set or empty, we treat all instance weights as 1.0. Default is not set, so all instances have weight one.

      Parameters:
      value - (undocumented)
      Returns:
      (undocumented)

    • setThresholds

      public LogisticRegression setThresholds(double[] value)
      Specified by:
      setThresholds in interface org.apache.spark.ml.classification.LogisticRegressionParams
      Overrides:
      setThresholds in class ProbabilisticClassifier<Vector,LogisticRegression,LogisticRegressionModel>

    • getThresholds

      public double[] getThresholds()
      Specified by:
      getThresholds in interface HasThresholds
      Specified by:
      getThresholds in interface org.apache.spark.ml.classification.LogisticRegressionParams

    • setAggregationDepth

      public LogisticRegression setAggregationDepth(int value)
      Suggested depth for treeAggregate (greater than or equal to 2). If the dimensions of features or the number of partitions are large, this param could be adjusted to a larger size. Default is 2.

      Parameters:
      value - (undocumented)
      Returns:
      (undocumented)

    • setLowerBoundsOnCoefficients

      public LogisticRegression setLowerBoundsOnCoefficients(Matrix value)
      Set the lower bounds on coefficients if fitting under bound constrained optimization.

      Parameters:
      value - (undocumented)
      Returns:
      (undocumented)

    • setUpperBoundsOnCoefficients

      public LogisticRegression setUpperBoundsOnCoefficients(Matrix value)
      Set the upper bounds on coefficients if fitting under bound constrained optimization.

      Parameters:
      value - (undocumented)
      Returns:
      (undocumented)

    • setLowerBoundsOnIntercepts

      public LogisticRegression setLowerBoundsOnIntercepts(Vector value)
      Set the lower bounds on intercepts if fitting under bound constrained optimization.

      Parameters:
      value - (undocumented)
      Returns:
      (undocumented)

    • setUpperBoundsOnIntercepts

      public LogisticRegression setUpperBoundsOnIntercepts(Vector value)
      Set the upper bounds on intercepts if fitting under bound constrained optimization.

      Parameters:
      value - (undocumented)
      Returns:
      (undocumented)

    • setMaxBlockSizeInMB

      public LogisticRegression setMaxBlockSizeInMB(double value)
      Sets the value of param maxBlockSizeInMB(). Default is 0.0, then 1.0 MB will be chosen.

      Parameters:
      value - (undocumented)
      Returns:
      (undocumented)

    • setInitialModel

      public LogisticRegression setInitialModel(LogisticRegressionModel model)

    • copy

      public LogisticRegression copy(ParamMap extra)
      Description copied from interface: Params
      Creates a copy of this instance with the same UID and some extra params. Subclasses should implement this method and set the return type properly. See defaultCopy().
      Specified by:
      copy in interface Params
      Specified by:
      copy in class Predictor<Vector,LogisticRegression,LogisticRegressionModel>
      Parameters:
      extra - (undocumented)
      Returns:
      (undocumented)