Package org.apache.spark.ml.classification

Class GBTClassifier

Object
org.apache.spark.ml.PipelineStage
org.apache.spark.ml.Estimator<M>
org.apache.spark.ml.Predictor<FeaturesType,E,M>
org.apache.spark.ml.classification.Classifier<FeaturesType,E,M>
org.apache.spark.ml.classification.ProbabilisticClassifier<Vector,GBTClassifier,GBTClassificationModel>
org.apache.spark.ml.classification.GBTClassifier
All Implemented Interfaces:
Serializable, org.apache.spark.internal.Logging, org.apache.spark.ml.classification.ClassifierParams, org.apache.spark.ml.classification.ProbabilisticClassifierParams, Params, HasCheckpointInterval, HasFeaturesCol, HasLabelCol, HasMaxIter, HasPredictionCol, HasProbabilityCol, HasRawPredictionCol, HasSeed, HasStepSize, HasThresholds, HasValidationIndicatorCol, HasWeightCol, org.apache.spark.ml.PredictorParams, org.apache.spark.ml.tree.DecisionTreeParams, org.apache.spark.ml.tree.GBTClassifierParams, org.apache.spark.ml.tree.GBTParams, org.apache.spark.ml.tree.HasVarianceImpurity, org.apache.spark.ml.tree.TreeEnsembleClassifierParams, org.apache.spark.ml.tree.TreeEnsembleParams, DefaultParamsWritable, Identifiable, MLWritable
public class GBTClassifier extends ProbabilisticClassifier<Vector,GBTClassifier,GBTClassificationModel> implements org.apache.spark.ml.tree.GBTClassifierParams, DefaultParamsWritable, org.apache.spark.internal.Logging
Gradient-Boosted Trees (GBTs) (http://en.wikipedia.org/wiki/Gradient_boosting) learning algorithm for classification. It supports binary labels, as well as both continuous and categorical features.

The implementation is based upon: J.H. Friedman. "Stochastic Gradient Boosting." 1999.

Notes on Gradient Boosting vs. TreeBoost: - This implementation is for Stochastic Gradient Boosting, not for TreeBoost. - Both algorithms learn tree ensembles by minimizing loss functions. - TreeBoost (Friedman, 1999) additionally modifies the outputs at tree leaf nodes based on the loss function, whereas the original gradient boosting method does not. - We expect to implement TreeBoost in the future: [https://issues.apache.org/jira/browse/SPARK-4240]

See Also:
Note:
Multiclass labels are not currently supported.

  • Constructor Details

    • GBTClassifier

      public GBTClassifier(String uid)

    • GBTClassifier

      public GBTClassifier()

  • Method Details

    • supportedLossTypes

      public static final String[] supportedLossTypes()
      Accessor for supported loss settings: logistic

    • load

      public static GBTClassifier load(String path)

    • read

      public static MLReader<T> read()

    • lossType

      public Param<String> lossType()
      Specified by:
      lossType in interface org.apache.spark.ml.tree.GBTClassifierParams

    • impurity

      public final Param<String> impurity()
      Specified by:
      impurity in interface org.apache.spark.ml.tree.HasVarianceImpurity

    • validationTol

      public final DoubleParam validationTol()
      Specified by:
      validationTol in interface org.apache.spark.ml.tree.GBTParams

    • stepSize

      public final DoubleParam stepSize()
      Description copied from interface: HasStepSize
      Param for Step size to be used for each iteration of optimization (&gt; 0).
      Specified by:
      stepSize in interface org.apache.spark.ml.tree.GBTParams
      Specified by:
      stepSize in interface HasStepSize
      Returns:
      (undocumented)

    • validationIndicatorCol

      public final Param<String> validationIndicatorCol()
      Description copied from interface: HasValidationIndicatorCol
      Param for name of the column that indicates whether each row is for training or for validation. False indicates training; true indicates validation..
      Specified by:
      validationIndicatorCol in interface HasValidationIndicatorCol
      Returns:
      (undocumented)

    • maxIter

      public final IntParam maxIter()
      Description copied from interface: HasMaxIter
      Param for maximum number of iterations (&gt;= 0).
      Specified by:
      maxIter in interface HasMaxIter
      Returns:
      (undocumented)

    • subsamplingRate

      public final DoubleParam subsamplingRate()
      Specified by:
      subsamplingRate in interface org.apache.spark.ml.tree.TreeEnsembleParams

    • featureSubsetStrategy

      public final Param<String> featureSubsetStrategy()
      Specified by:
      featureSubsetStrategy in interface org.apache.spark.ml.tree.TreeEnsembleParams

    • leafCol

      public final Param<String> leafCol()
      Specified by:
      leafCol in interface org.apache.spark.ml.tree.DecisionTreeParams

    • maxDepth

      public final IntParam maxDepth()
      Specified by:
      maxDepth in interface org.apache.spark.ml.tree.DecisionTreeParams

    • maxBins

      public final IntParam maxBins()
      Specified by:
      maxBins in interface org.apache.spark.ml.tree.DecisionTreeParams

    • minInstancesPerNode

      public final IntParam minInstancesPerNode()
      Specified by:
      minInstancesPerNode in interface org.apache.spark.ml.tree.DecisionTreeParams

    • minWeightFractionPerNode

      public final DoubleParam minWeightFractionPerNode()
      Specified by:
      minWeightFractionPerNode in interface org.apache.spark.ml.tree.DecisionTreeParams

    • minInfoGain

      public final DoubleParam minInfoGain()
      Specified by:
      minInfoGain in interface org.apache.spark.ml.tree.DecisionTreeParams

    • maxMemoryInMB

      public final IntParam maxMemoryInMB()
      Specified by:
      maxMemoryInMB in interface org.apache.spark.ml.tree.DecisionTreeParams

    • cacheNodeIds

      public final BooleanParam cacheNodeIds()
      Specified by:
      cacheNodeIds in interface org.apache.spark.ml.tree.DecisionTreeParams

    • weightCol

      public final Param<String> weightCol()
      Description copied from interface: HasWeightCol
      Param for weight column name. If this is not set or empty, we treat all instance weights as 1.0.
      Specified by:
      weightCol in interface HasWeightCol
      Returns:
      (undocumented)

    • seed

      public final LongParam seed()
      Description copied from interface: HasSeed
      Param for random seed.
      Specified by:
      seed in interface HasSeed
      Returns:
      (undocumented)

    • checkpointInterval

      public final IntParam checkpointInterval()
      Description copied from interface: HasCheckpointInterval
      Param for set checkpoint interval (&gt;= 1) or disable checkpoint (-1). E.g. 10 means that the cache will get checkpointed every 10 iterations. Note: this setting will be ignored if the checkpoint directory is not set in the SparkContext.
      Specified by:
      checkpointInterval in interface HasCheckpointInterval
      Returns:
      (undocumented)

    • uid

      public String uid()
      Description copied from interface: Identifiable
      An immutable unique ID for the object and its derivatives.
      Specified by:
      uid in interface Identifiable
      Returns:
      (undocumented)

    • setMaxDepth

      public GBTClassifier setMaxDepth(int value)

    • setMaxBins

      public GBTClassifier setMaxBins(int value)

    • setMinInstancesPerNode

      public GBTClassifier setMinInstancesPerNode(int value)

    • setMinWeightFractionPerNode

      public GBTClassifier setMinWeightFractionPerNode(double value)

    • setMinInfoGain

      public GBTClassifier setMinInfoGain(double value)

    • setMaxMemoryInMB

      public GBTClassifier setMaxMemoryInMB(int value)

    • setCacheNodeIds

      public GBTClassifier setCacheNodeIds(boolean value)

    • setCheckpointInterval

      public GBTClassifier setCheckpointInterval(int value)
      Specifies how often to checkpoint the cached node IDs. E.g. 10 means that the cache will get checkpointed every 10 iterations. This is only used if cacheNodeIds is true and if the checkpoint directory is set in SparkContext. Must be at least 1. (default = 10)
      Parameters:
      value - (undocumented)
      Returns:
      (undocumented)

    • setImpurity

      public GBTClassifier setImpurity(String value)
      The impurity setting is ignored for GBT models. Individual trees are built using impurity "Variance."

      Parameters:
      value - (undocumented)
      Returns:
      (undocumented)

    • setSubsamplingRate

      public GBTClassifier setSubsamplingRate(double value)

    • setSeed

      public GBTClassifier setSeed(long value)

    • setMaxIter

      public GBTClassifier setMaxIter(int value)

    • setStepSize

      public GBTClassifier setStepSize(double value)

    • setFeatureSubsetStrategy

      public GBTClassifier setFeatureSubsetStrategy(String value)

    • setLossType

      public GBTClassifier setLossType(String value)

    • setValidationIndicatorCol

      public GBTClassifier setValidationIndicatorCol(String value)

    • setWeightCol

      public GBTClassifier setWeightCol(String value)
      Sets the value of param weightCol(). If this is not set or empty, we treat all instance weights as 1.0. By default the weightCol is not set, so all instances have weight 1.0.

      Parameters:
      value - (undocumented)
      Returns:
      (undocumented)

    • copy

      public GBTClassifier copy(ParamMap extra)
      Description copied from interface: Params
      Creates a copy of this instance with the same UID and some extra params. Subclasses should implement this method and set the return type properly. See defaultCopy().
      Specified by:
      copy in interface Params
      Specified by:
      copy in class Predictor<Vector,GBTClassifier,GBTClassificationModel>
      Parameters:
      extra - (undocumented)
      Returns:
      (undocumented)