Package org.apache.spark.ml.regression

Class GeneralizedLinearRegression

Object
org.apache.spark.ml.PipelineStage
org.apache.spark.ml.Estimator<M>
org.apache.spark.ml.Predictor<FeaturesType,Learner,M>
org.apache.spark.ml.regression.Regressor<Vector,GeneralizedLinearRegression,GeneralizedLinearRegressionModel>
org.apache.spark.ml.regression.GeneralizedLinearRegression
All Implemented Interfaces:
Serializable, org.apache.spark.internal.Logging, Params, HasAggregationDepth, HasFeaturesCol, HasFitIntercept, HasLabelCol, HasMaxIter, HasPredictionCol, HasRegParam, HasSolver, HasTol, HasWeightCol, org.apache.spark.ml.PredictorParams, org.apache.spark.ml.regression.GeneralizedLinearRegressionBase, DefaultParamsWritable, Identifiable, MLWritable
public class GeneralizedLinearRegression extends Regressor<Vector,GeneralizedLinearRegression,GeneralizedLinearRegressionModel> implements org.apache.spark.ml.regression.GeneralizedLinearRegressionBase, DefaultParamsWritable, org.apache.spark.internal.Logging
Fit a Generalized Linear Model (see Generalized linear model (Wikipedia)) specified by giving a symbolic description of the linear predictor (link function) and a description of the error distribution (family). It supports "gaussian", "binomial", "poisson", "gamma" and "tweedie" as family. Valid link functions for each family is listed below. The first link function of each family is the default one. - "gaussian" : "identity", "log", "inverse" - "binomial" : "logit", "probit", "cloglog" - "poisson" : "log", "identity", "sqrt" - "gamma" : "inverse", "identity", "log" - "tweedie" : power link function specified through "linkPower". The default link power in the tweedie family is 1 - variancePower.
See Also:

  • Constructor Details

    • GeneralizedLinearRegression

      public GeneralizedLinearRegression(String uid)

    • GeneralizedLinearRegression

      public GeneralizedLinearRegression()

  • Method Details

    • load

      public static GeneralizedLinearRegression load(String path)

    • read

      public static MLReader<T> read()

    • family

      public final Param<String> family()
      Specified by:
      family in interface org.apache.spark.ml.regression.GeneralizedLinearRegressionBase

    • variancePower

      public final DoubleParam variancePower()
      Specified by:
      variancePower in interface org.apache.spark.ml.regression.GeneralizedLinearRegressionBase

    • link

      public final Param<String> link()
      Specified by:
      link in interface org.apache.spark.ml.regression.GeneralizedLinearRegressionBase

    • linkPower

      public final DoubleParam linkPower()
      Specified by:
      linkPower in interface org.apache.spark.ml.regression.GeneralizedLinearRegressionBase

    • linkPredictionCol

      public final Param<String> linkPredictionCol()
      Specified by:
      linkPredictionCol in interface org.apache.spark.ml.regression.GeneralizedLinearRegressionBase

    • offsetCol

      public final Param<String> offsetCol()
      Specified by:
      offsetCol in interface org.apache.spark.ml.regression.GeneralizedLinearRegressionBase

    • solver

      public final Param<String> solver()
      Description copied from interface: HasSolver
      Param for the solver algorithm for optimization.
      Specified by:
      solver in interface org.apache.spark.ml.regression.GeneralizedLinearRegressionBase
      Specified by:
      solver in interface HasSolver
      Returns:
      (undocumented)

    • aggregationDepth

      public final IntParam aggregationDepth()
      Description copied from interface: HasAggregationDepth
      Param for suggested depth for treeAggregate (&gt;= 2).
      Specified by:
      aggregationDepth in interface HasAggregationDepth
      Returns:
      (undocumented)

    • weightCol

      public final Param<String> weightCol()
      Description copied from interface: HasWeightCol
      Param for weight column name. If this is not set or empty, we treat all instance weights as 1.0.
      Specified by:
      weightCol in interface HasWeightCol
      Returns:
      (undocumented)

    • regParam

      public final DoubleParam regParam()
      Description copied from interface: HasRegParam
      Param for regularization parameter (&gt;= 0).
      Specified by:
      regParam in interface HasRegParam
      Returns:
      (undocumented)

    • tol

      public final DoubleParam tol()
      Description copied from interface: HasTol
      Param for the convergence tolerance for iterative algorithms (&gt;= 0).
      Specified by:
      tol in interface HasTol
      Returns:
      (undocumented)

    • maxIter

      public final IntParam maxIter()
      Description copied from interface: HasMaxIter
      Param for maximum number of iterations (&gt;= 0).
      Specified by:
      maxIter in interface HasMaxIter
      Returns:
      (undocumented)

    • fitIntercept

      public final BooleanParam fitIntercept()
      Description copied from interface: HasFitIntercept
      Param for whether to fit an intercept term.
      Specified by:
      fitIntercept in interface HasFitIntercept
      Returns:
      (undocumented)

    • uid

      public String uid()
      Description copied from interface: Identifiable
      An immutable unique ID for the object and its derivatives.
      Specified by:
      uid in interface Identifiable
      Returns:
      (undocumented)

    • setFamily

      public GeneralizedLinearRegression setFamily(String value)
      Sets the value of param family(). Default is "gaussian".

      Parameters:
      value - (undocumented)
      Returns:
      (undocumented)

    • setVariancePower

      public GeneralizedLinearRegression setVariancePower(double value)
      Sets the value of param variancePower(). Used only when family is "tweedie". Default is 0.0, which corresponds to the "gaussian" family.

      Parameters:
      value - (undocumented)
      Returns:
      (undocumented)

    • setLinkPower

      public GeneralizedLinearRegression setLinkPower(double value)
      Sets the value of param linkPower(). Used only when family is "tweedie".

      Parameters:
      value - (undocumented)
      Returns:
      (undocumented)

    • setLink

      public GeneralizedLinearRegression setLink(String value)
      Sets the value of param link(). Used only when family is not "tweedie".

      Parameters:
      value - (undocumented)
      Returns:
      (undocumented)

    • setFitIntercept

      public GeneralizedLinearRegression setFitIntercept(boolean value)
      Sets if we should fit the intercept. Default is true.

      Parameters:
      value - (undocumented)
      Returns:
      (undocumented)

    • setMaxIter

      public GeneralizedLinearRegression setMaxIter(int value)
      Sets the maximum number of iterations (applicable for solver "irls"). Default is 25.

      Parameters:
      value - (undocumented)
      Returns:
      (undocumented)

    • setTol

      public GeneralizedLinearRegression setTol(double value)
      Sets the convergence tolerance of iterations. Smaller value will lead to higher accuracy with the cost of more iterations. Default is 1E-6.

      Parameters:
      value - (undocumented)
      Returns:
      (undocumented)

    • setRegParam

      public GeneralizedLinearRegression setRegParam(double value)
      Sets the regularization parameter for L2 regularization. The regularization term is
      $$ 0.5 * regParam * L2norm(coefficients)^2 $$
      Default is 0.0.

      Parameters:
      value - (undocumented)
      Returns:
      (undocumented)

    • setWeightCol

      public GeneralizedLinearRegression setWeightCol(String value)
      Sets the value of param weightCol(). If this is not set or empty, we treat all instance weights as 1.0. Default is not set, so all instances have weight one. In the Binomial family, weights correspond to number of trials and should be integer. Non-integer weights are rounded to integer in AIC calculation.

      Parameters:
      value - (undocumented)
      Returns:
      (undocumented)

    • setOffsetCol

      public GeneralizedLinearRegression setOffsetCol(String value)
      Sets the value of param offsetCol(). If this is not set or empty, we treat all instance offsets as 0.0. Default is not set, so all instances have offset 0.0.

      Parameters:
      value - (undocumented)
      Returns:
      (undocumented)

    • setSolver

      public GeneralizedLinearRegression setSolver(String value)
      Sets the solver algorithm used for optimization. Currently only supports "irls" which is also the default solver.

      Parameters:
      value - (undocumented)
      Returns:
      (undocumented)

    • setLinkPredictionCol

      public GeneralizedLinearRegression setLinkPredictionCol(String value)
      Sets the link prediction (linear predictor) column name.

      Parameters:
      value - (undocumented)
      Returns:
      (undocumented)

    • setAggregationDepth

      public GeneralizedLinearRegression setAggregationDepth(int value)

    • copy

      public GeneralizedLinearRegression copy(ParamMap extra)
      Description copied from interface: Params
      Creates a copy of this instance with the same UID and some extra params. Subclasses should implement this method and set the return type properly. See defaultCopy().
      Specified by:
      copy in interface Params
      Specified by:
      copy in class Predictor<Vector,GeneralizedLinearRegression,GeneralizedLinearRegressionModel>
      Parameters:
      extra - (undocumented)
      Returns:
      (undocumented)

    • estimateModelSize

      public long estimateModelSize(Dataset<?> dataset)