Package org.apache.spark.ml.regression

Class GBTRegressor

Object
org.apache.spark.ml.PipelineStage
org.apache.spark.ml.Estimator<M>
org.apache.spark.ml.Predictor<FeaturesType,Learner,M>
org.apache.spark.ml.regression.Regressor<Vector,GBTRegressor,GBTRegressionModel>
org.apache.spark.ml.regression.GBTRegressor
All Implemented Interfaces:
Serializable, org.apache.spark.internal.Logging, Params, HasCheckpointInterval, HasFeaturesCol, HasLabelCol, HasMaxIter, HasPredictionCol, HasSeed, HasStepSize, HasValidationIndicatorCol, HasWeightCol, org.apache.spark.ml.PredictorParams, org.apache.spark.ml.tree.DecisionTreeParams, org.apache.spark.ml.tree.GBTParams, org.apache.spark.ml.tree.GBTRegressorParams, org.apache.spark.ml.tree.HasVarianceImpurity, org.apache.spark.ml.tree.TreeEnsembleParams, org.apache.spark.ml.tree.TreeEnsembleRegressorParams, org.apache.spark.ml.tree.TreeRegressorParams, DefaultParamsWritable, Identifiable, MLWritable
public class GBTRegressor extends Regressor<Vector,GBTRegressor,GBTRegressionModel> implements org.apache.spark.ml.tree.GBTRegressorParams, DefaultParamsWritable, org.apache.spark.internal.Logging
Gradient-Boosted Trees (GBTs) learning algorithm for regression. It supports both continuous and categorical features.

The implementation is based upon: J.H. Friedman. "Stochastic Gradient Boosting." 1999.

Notes on Gradient Boosting vs. TreeBoost: - This implementation is for Stochastic Gradient Boosting, not for TreeBoost. - Both algorithms learn tree ensembles by minimizing loss functions. - TreeBoost (Friedman, 1999) additionally modifies the outputs at tree leaf nodes based on the loss function, whereas the original gradient boosting method does not. - When the loss is SquaredError, these methods give the same result, but they could differ for other loss functions. - We expect to implement TreeBoost in the future: [https://issues.apache.org/jira/browse/SPARK-4240]

See Also:

  • Constructor Details

    • GBTRegressor

      public GBTRegressor(String uid)

    • GBTRegressor

      public GBTRegressor()

  • Method Details

    • supportedLossTypes

      public static final String[] supportedLossTypes()
      Accessor for supported loss settings: squared (L2), absolute (L1)

    • load

      public static GBTRegressor load(String path)

    • read

      public static MLReader<T> read()

    • lossType

      public Param<String> lossType()
      Specified by:
      lossType in interface org.apache.spark.ml.tree.GBTRegressorParams

    • impurity

      public final Param<String> impurity()
      Specified by:
      impurity in interface org.apache.spark.ml.tree.HasVarianceImpurity

    • validationTol

      public final DoubleParam validationTol()
      Specified by:
      validationTol in interface org.apache.spark.ml.tree.GBTParams

    • stepSize

      public final DoubleParam stepSize()
      Description copied from interface: HasStepSize
      Param for Step size to be used for each iteration of optimization (&gt; 0).
      Specified by:
      stepSize in interface org.apache.spark.ml.tree.GBTParams
      Specified by:
      stepSize in interface HasStepSize
      Returns:
      (undocumented)

    • validationIndicatorCol

      public final Param<String> validationIndicatorCol()
      Description copied from interface: HasValidationIndicatorCol
      Param for name of the column that indicates whether each row is for training or for validation. False indicates training; true indicates validation..
      Specified by:
      validationIndicatorCol in interface HasValidationIndicatorCol
      Returns:
      (undocumented)

    • maxIter

      public final IntParam maxIter()
      Description copied from interface: HasMaxIter
      Param for maximum number of iterations (&gt;= 0).
      Specified by:
      maxIter in interface HasMaxIter
      Returns:
      (undocumented)

    • subsamplingRate

      public final DoubleParam subsamplingRate()
      Specified by:
      subsamplingRate in interface org.apache.spark.ml.tree.TreeEnsembleParams

    • featureSubsetStrategy

      public final Param<String> featureSubsetStrategy()
      Specified by:
      featureSubsetStrategy in interface org.apache.spark.ml.tree.TreeEnsembleParams

    • leafCol

      public final Param<String> leafCol()
      Specified by:
      leafCol in interface org.apache.spark.ml.tree.DecisionTreeParams

    • maxDepth

      public final IntParam maxDepth()
      Specified by:
      maxDepth in interface org.apache.spark.ml.tree.DecisionTreeParams

    • maxBins

      public final IntParam maxBins()
      Specified by:
      maxBins in interface org.apache.spark.ml.tree.DecisionTreeParams

    • minInstancesPerNode

      public final IntParam minInstancesPerNode()
      Specified by:
      minInstancesPerNode in interface org.apache.spark.ml.tree.DecisionTreeParams

    • minWeightFractionPerNode

      public final DoubleParam minWeightFractionPerNode()
      Specified by:
      minWeightFractionPerNode in interface org.apache.spark.ml.tree.DecisionTreeParams

    • minInfoGain

      public final DoubleParam minInfoGain()
      Specified by:
      minInfoGain in interface org.apache.spark.ml.tree.DecisionTreeParams

    • maxMemoryInMB

      public final IntParam maxMemoryInMB()
      Specified by:
      maxMemoryInMB in interface org.apache.spark.ml.tree.DecisionTreeParams

    • cacheNodeIds

      public final BooleanParam cacheNodeIds()
      Specified by:
      cacheNodeIds in interface org.apache.spark.ml.tree.DecisionTreeParams

    • weightCol

      public final Param<String> weightCol()
      Description copied from interface: HasWeightCol
      Param for weight column name. If this is not set or empty, we treat all instance weights as 1.0.
      Specified by:
      weightCol in interface HasWeightCol
      Returns:
      (undocumented)

    • seed

      public final LongParam seed()
      Description copied from interface: HasSeed
      Param for random seed.
      Specified by:
      seed in interface HasSeed
      Returns:
      (undocumented)

    • checkpointInterval

      public final IntParam checkpointInterval()
      Description copied from interface: HasCheckpointInterval
      Param for set checkpoint interval (&gt;= 1) or disable checkpoint (-1). E.g. 10 means that the cache will get checkpointed every 10 iterations. Note: this setting will be ignored if the checkpoint directory is not set in the SparkContext.
      Specified by:
      checkpointInterval in interface HasCheckpointInterval
      Returns:
      (undocumented)

    • uid

      public String uid()
      Description copied from interface: Identifiable
      An immutable unique ID for the object and its derivatives.
      Specified by:
      uid in interface Identifiable
      Returns:
      (undocumented)

    • setMaxDepth

      public GBTRegressor setMaxDepth(int value)

    • setMaxBins

      public GBTRegressor setMaxBins(int value)

    • setMinInstancesPerNode

      public GBTRegressor setMinInstancesPerNode(int value)

    • setMinWeightFractionPerNode

      public GBTRegressor setMinWeightFractionPerNode(double value)

    • setMinInfoGain

      public GBTRegressor setMinInfoGain(double value)

    • setMaxMemoryInMB

      public GBTRegressor setMaxMemoryInMB(int value)

    • setCacheNodeIds

      public GBTRegressor setCacheNodeIds(boolean value)

    • setCheckpointInterval

      public GBTRegressor setCheckpointInterval(int value)
      Specifies how often to checkpoint the cached node IDs. E.g. 10 means that the cache will get checkpointed every 10 iterations. This is only used if cacheNodeIds is true and if the checkpoint directory is set in SparkContext. Must be at least 1. (default = 10)
      Parameters:
      value - (undocumented)
      Returns:
      (undocumented)

    • setImpurity

      public GBTRegressor setImpurity(String value)
      The impurity setting is ignored for GBT models. Individual trees are built using impurity "Variance."

      Parameters:
      value - (undocumented)
      Returns:
      (undocumented)

    • setSubsamplingRate

      public GBTRegressor setSubsamplingRate(double value)

    • setSeed

      public GBTRegressor setSeed(long value)

    • setMaxIter

      public GBTRegressor setMaxIter(int value)

    • setStepSize

      public GBTRegressor setStepSize(double value)

    • setLossType

      public GBTRegressor setLossType(String value)

    • setFeatureSubsetStrategy

      public GBTRegressor setFeatureSubsetStrategy(String value)

    • setValidationIndicatorCol

      public GBTRegressor setValidationIndicatorCol(String value)

    • setWeightCol

      public GBTRegressor setWeightCol(String value)
      Sets the value of param weightCol(). If this is not set or empty, we treat all instance weights as 1.0. By default the weightCol is not set, so all instances have weight 1.0.

      Parameters:
      value - (undocumented)
      Returns:
      (undocumented)

    • copy

      public GBTRegressor copy(ParamMap extra)
      Description copied from interface: Params
      Creates a copy of this instance with the same UID and some extra params. Subclasses should implement this method and set the return type properly. See defaultCopy().
      Specified by:
      copy in interface Params
      Specified by:
      copy in class Predictor<Vector,GBTRegressor,GBTRegressionModel>
      Parameters:
      extra - (undocumented)
      Returns:
      (undocumented)