Package org.apache.spark.ml.classification

Class GBTClassificationModel

Object
org.apache.spark.ml.PipelineStage
org.apache.spark.ml.Transformer
org.apache.spark.ml.Model<M>
org.apache.spark.ml.PredictionModel<FeaturesType,M>
org.apache.spark.ml.classification.ClassificationModel<FeaturesType,M>
org.apache.spark.ml.classification.ProbabilisticClassificationModel<Vector,GBTClassificationModel>
org.apache.spark.ml.classification.GBTClassificationModel
All Implemented Interfaces:
Serializable, org.apache.spark.internal.Logging, org.apache.spark.ml.classification.ClassifierParams, org.apache.spark.ml.classification.ProbabilisticClassifierParams, Params, HasCheckpointInterval, HasFeaturesCol, HasLabelCol, HasMaxIter, HasPredictionCol, HasProbabilityCol, HasRawPredictionCol, HasSeed, HasStepSize, HasThresholds, HasValidationIndicatorCol, HasWeightCol, org.apache.spark.ml.PredictorParams, org.apache.spark.ml.tree.DecisionTreeParams, org.apache.spark.ml.tree.GBTClassifierParams, org.apache.spark.ml.tree.GBTParams, org.apache.spark.ml.tree.HasVarianceImpurity, org.apache.spark.ml.tree.TreeEnsembleClassifierParams, org.apache.spark.ml.tree.TreeEnsembleModel<DecisionTreeRegressionModel>, org.apache.spark.ml.tree.TreeEnsembleParams, Identifiable, MLWritable
public class GBTClassificationModel extends ProbabilisticClassificationModel<Vector,GBTClassificationModel> implements org.apache.spark.ml.tree.GBTClassifierParams, org.apache.spark.ml.tree.TreeEnsembleModel<DecisionTreeRegressionModel>, MLWritable, Serializable
Gradient-Boosted Trees (GBTs) (http://en.wikipedia.org/wiki/Gradient_boosting) model for classification. It supports binary labels, as well as both continuous and categorical features.

param: _trees Decision trees in the ensemble. param: _treeWeights Weights for the decision trees in the ensemble.

See Also:
Note:
Multiclass labels are not currently supported.

  • Constructor Details

    • GBTClassificationModel

      public GBTClassificationModel(String uid, DecisionTreeRegressionModel[] _trees, double[] _treeWeights)
      Construct a GBTClassificationModel

      Parameters:
      _trees - Decision trees in the ensemble.
      _treeWeights - Weights for the decision trees in the ensemble.
      uid - (undocumented)

  • Method Details

    • read

      public static MLReader<GBTClassificationModel> read()

    • load

      public static GBTClassificationModel load(String path)

    • totalNumNodes

      public int totalNumNodes()
      Specified by:
      totalNumNodes in interface org.apache.spark.ml.tree.TreeEnsembleModel<DecisionTreeRegressionModel>

    • lossType

      public Param<String> lossType()
      Specified by:
      lossType in interface org.apache.spark.ml.tree.GBTClassifierParams

    • impurity

      public final Param<String> impurity()
      Specified by:
      impurity in interface org.apache.spark.ml.tree.HasVarianceImpurity

    • validationTol

      public final DoubleParam validationTol()
      Specified by:
      validationTol in interface org.apache.spark.ml.tree.GBTParams

    • stepSize

      public final DoubleParam stepSize()
      Description copied from interface: HasStepSize
      Param for Step size to be used for each iteration of optimization (&gt; 0).
      Specified by:
      stepSize in interface org.apache.spark.ml.tree.GBTParams
      Specified by:
      stepSize in interface HasStepSize
      Returns:
      (undocumented)

    • validationIndicatorCol

      public final Param<String> validationIndicatorCol()
      Description copied from interface: HasValidationIndicatorCol
      Param for name of the column that indicates whether each row is for training or for validation. False indicates training; true indicates validation..
      Specified by:
      validationIndicatorCol in interface HasValidationIndicatorCol
      Returns:
      (undocumented)

    • maxIter

      public final IntParam maxIter()
      Description copied from interface: HasMaxIter
      Param for maximum number of iterations (&gt;= 0).
      Specified by:
      maxIter in interface HasMaxIter
      Returns:
      (undocumented)

    • subsamplingRate

      public final DoubleParam subsamplingRate()
      Specified by:
      subsamplingRate in interface org.apache.spark.ml.tree.TreeEnsembleParams

    • featureSubsetStrategy

      public final Param<String> featureSubsetStrategy()
      Specified by:
      featureSubsetStrategy in interface org.apache.spark.ml.tree.TreeEnsembleParams

    • leafCol

      public final Param<String> leafCol()
      Specified by:
      leafCol in interface org.apache.spark.ml.tree.DecisionTreeParams

    • maxDepth

      public final IntParam maxDepth()
      Specified by:
      maxDepth in interface org.apache.spark.ml.tree.DecisionTreeParams

    • maxBins

      public final IntParam maxBins()
      Specified by:
      maxBins in interface org.apache.spark.ml.tree.DecisionTreeParams

    • minInstancesPerNode

      public final IntParam minInstancesPerNode()
      Specified by:
      minInstancesPerNode in interface org.apache.spark.ml.tree.DecisionTreeParams

    • minWeightFractionPerNode

      public final DoubleParam minWeightFractionPerNode()
      Specified by:
      minWeightFractionPerNode in interface org.apache.spark.ml.tree.DecisionTreeParams

    • minInfoGain

      public final DoubleParam minInfoGain()
      Specified by:
      minInfoGain in interface org.apache.spark.ml.tree.DecisionTreeParams

    • maxMemoryInMB

      public final IntParam maxMemoryInMB()
      Specified by:
      maxMemoryInMB in interface org.apache.spark.ml.tree.DecisionTreeParams

    • cacheNodeIds

      public final BooleanParam cacheNodeIds()
      Specified by:
      cacheNodeIds in interface org.apache.spark.ml.tree.DecisionTreeParams

    • weightCol

      public final Param<String> weightCol()
      Description copied from interface: HasWeightCol
      Param for weight column name. If this is not set or empty, we treat all instance weights as 1.0.
      Specified by:
      weightCol in interface HasWeightCol
      Returns:
      (undocumented)

    • seed

      public final LongParam seed()
      Description copied from interface: HasSeed
      Param for random seed.
      Specified by:
      seed in interface HasSeed
      Returns:
      (undocumented)

    • checkpointInterval

      public final IntParam checkpointInterval()
      Description copied from interface: HasCheckpointInterval
      Param for set checkpoint interval (&gt;= 1) or disable checkpoint (-1). E.g. 10 means that the cache will get checkpointed every 10 iterations. Note: this setting will be ignored if the checkpoint directory is not set in the SparkContext.
      Specified by:
      checkpointInterval in interface HasCheckpointInterval
      Returns:
      (undocumented)

    • uid

      public String uid()
      Description copied from interface: Identifiable
      An immutable unique ID for the object and its derivatives.
      Specified by:
      uid in interface Identifiable
      Returns:
      (undocumented)

    • numFeatures

      public int numFeatures()
      Description copied from class: PredictionModel
      Returns the number of features the model was trained on. If unknown, returns -1
      Overrides:
      numFeatures in class PredictionModel<Vector,GBTClassificationModel>

    • numClasses

      public int numClasses()
      Description copied from class: ClassificationModel
      Number of classes (values which the label can take).
      Specified by:
      numClasses in class ClassificationModel<Vector,GBTClassificationModel>

    • trees

      public DecisionTreeRegressionModel[] trees()
      Specified by:
      trees in interface org.apache.spark.ml.tree.TreeEnsembleModel<DecisionTreeRegressionModel>

    • getNumTrees

      public int getNumTrees()
      Number of trees in ensemble
      Returns:
      (undocumented)

    • treeWeights

      public double[] treeWeights()
      Specified by:
      treeWeights in interface org.apache.spark.ml.tree.TreeEnsembleModel<DecisionTreeRegressionModel>

    • transformSchema

      public StructType transformSchema(StructType schema)
      Description copied from class: PipelineStage
      Check transform validity and derive the output schema from the input schema.

      We check validity for interactions between parameters during transformSchema and raise an exception if any parameter value is invalid. Parameter value checks which do not depend on other parameters are handled by Param.validate().

      Typical implementation should first conduct verification on schema change and parameter validity, including complex parameter interaction checks.

      Overrides:
      transformSchema in class ProbabilisticClassificationModel<Vector,GBTClassificationModel>
      Parameters:
      schema - (undocumented)
      Returns:
      (undocumented)

    • transform

      public Dataset<Row> transform(Dataset<?> dataset)
      Description copied from class: ProbabilisticClassificationModel
      Transforms dataset by reading from PredictionModel.featuresCol(), and appending new columns as specified by parameters: - predicted labels as PredictionModel.predictionCol() of type Double - raw predictions (confidences) as ClassificationModel.rawPredictionCol() of type Vector - probability of each class as ProbabilisticClassificationModel.probabilityCol() of type Vector.

      Overrides:
      transform in class ProbabilisticClassificationModel<Vector,GBTClassificationModel>
      Parameters:
      dataset - input dataset
      Returns:
      transformed dataset

    • predict

      public double predict(Vector features)
      Description copied from class: ClassificationModel
      Predict label for the given features. This method is used to implement transform() and output PredictionModel.predictionCol().

      This default implementation for classification predicts the index of the maximum value from predictRaw().

      Overrides:
      predict in class ClassificationModel<Vector,GBTClassificationModel>
      Parameters:
      features - (undocumented)
      Returns:
      (undocumented)

    • predictRaw

      public Vector predictRaw(Vector features)
      Description copied from class: ClassificationModel
      Raw prediction for each possible label. The meaning of a "raw" prediction may vary between algorithms, but it intuitively gives a measure of confidence in each possible label (where larger = more confident). This internal method is used to implement transform() and output ClassificationModel.rawPredictionCol().

      Specified by:
      predictRaw in class ClassificationModel<Vector,GBTClassificationModel>
      Parameters:
      features - (undocumented)
      Returns:
      vector where element i is the raw prediction for label i. This raw prediction may be any real number, where a larger value indicates greater confidence for that label.

    • copy

      public GBTClassificationModel copy(ParamMap extra)
      Description copied from interface: Params
      Creates a copy of this instance with the same UID and some extra params. Subclasses should implement this method and set the return type properly. See defaultCopy().
      Specified by:
      copy in interface Params
      Specified by:
      copy in class Model<GBTClassificationModel>
      Parameters:
      extra - (undocumented)
      Returns:
      (undocumented)

    • toString

      public String toString()
      Specified by:
      toString in interface Identifiable
      Specified by:
      toString in interface org.apache.spark.ml.tree.TreeEnsembleModel<DecisionTreeRegressionModel>
      Overrides:
      toString in class Object

    • featureImportances

      public Vector featureImportances()

    • evaluateEachIteration

      public double[] evaluateEachIteration(Dataset<?> dataset)
      Method to compute error or loss for every iteration of gradient boosting.

      Parameters:
      dataset - Dataset for validation.
      Returns:
      (undocumented)

    • write

      public MLWriter write()
      Description copied from interface: MLWritable
      Returns an MLWriter instance for this ML instance.
      Specified by:
      write in interface MLWritable
      Returns:
      (undocumented)