Package org.apache.spark.ml.regression
Class RandomForestRegressor
Object
org.apache.spark.ml.PipelineStage
org.apache.spark.ml.Estimator<M>
org.apache.spark.ml.Predictor<FeaturesType,Learner,M>
org.apache.spark.ml.regression.Regressor<Vector,RandomForestRegressor,RandomForestRegressionModel>
org.apache.spark.ml.regression.RandomForestRegressor
- All Implemented Interfaces:
Serializable
,org.apache.spark.internal.Logging
,Params
,HasCheckpointInterval
,HasFeaturesCol
,HasLabelCol
,HasPredictionCol
,HasSeed
,HasWeightCol
,PredictorParams
,DecisionTreeParams
,HasVarianceImpurity
,RandomForestParams
,RandomForestRegressorParams
,TreeEnsembleParams
,TreeEnsembleRegressorParams
,TreeRegressorParams
,DefaultParamsWritable
,Identifiable
,MLWritable
,scala.Serializable
public class RandomForestRegressor
extends Regressor<Vector,RandomForestRegressor,RandomForestRegressionModel>
implements RandomForestRegressorParams, DefaultParamsWritable
Random Forest
learning algorithm for regression.
It supports both continuous and categorical features.
- See Also:
-
Nested Class Summary
Nested classes/interfaces inherited from interface org.apache.spark.internal.Logging
org.apache.spark.internal.Logging.SparkShellLoggingFilter
-
Constructor Summary
-
Method Summary
Modifier and TypeMethodDescriptionfinal BooleanParam
Whether bootstrap samples are used when building trees.final BooleanParam
If false, the algorithm will pass trees to executors to match instances with nodes.final IntParam
Param for set checkpoint interval (>= 1) or disable checkpoint (-1).Creates a copy of this instance with the same UID and some extra params.The number of features to consider for splits at each tree node.impurity()
Criterion used for information gain calculation (case-insensitive).leafCol()
Leaf indices column name.static RandomForestRegressor
final IntParam
maxBins()
Maximum number of bins used for discretizing continuous features and for choosing how to split on features at each node.final IntParam
maxDepth()
Maximum depth of the tree (nonnegative).final IntParam
Maximum memory in MB allocated to histogram aggregation.final DoubleParam
Minimum information gain for a split to be considered at a tree node.final IntParam
Minimum number of instances each child must have after split.final DoubleParam
Minimum fraction of the weighted sample count that each child must have after split.final IntParam
numTrees()
Number of trees to train (at least 1).static MLReader<T>
read()
final LongParam
seed()
Param for random seed.setBootstrap
(boolean value) setCacheNodeIds
(boolean value) setCheckpointInterval
(int value) Specifies how often to checkpoint the cached node IDs.setFeatureSubsetStrategy
(String value) setImpurity
(String value) setMaxBins
(int value) setMaxDepth
(int value) setMaxMemoryInMB
(int value) setMinInfoGain
(double value) setMinInstancesPerNode
(int value) setMinWeightFractionPerNode
(double value) setNumTrees
(int value) setSeed
(long value) setSubsamplingRate
(double value) setWeightCol
(String value) Sets the value of paramweightCol()
.final DoubleParam
Fraction of the training data used for learning each decision tree, in range (0, 1].static final String[]
Accessor for supported featureSubsetStrategy settings: auto, all, onethird, sqrt, log2static final String[]
Accessor for supported impurity settings: varianceuid()
An immutable unique ID for the object and its derivatives.Param for weight column name.Methods inherited from class org.apache.spark.ml.Predictor
featuresCol, fit, labelCol, predictionCol, setFeaturesCol, setLabelCol, setPredictionCol, transformSchema
Methods inherited from class org.apache.spark.ml.PipelineStage
params
Methods inherited from class java.lang.Object
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
Methods inherited from interface org.apache.spark.ml.tree.DecisionTreeParams
getCacheNodeIds, getLeafCol, getMaxBins, getMaxDepth, getMaxMemoryInMB, getMinInfoGain, getMinInstancesPerNode, getMinWeightFractionPerNode, getOldStrategy, setLeafCol
Methods inherited from interface org.apache.spark.ml.util.DefaultParamsWritable
write
Methods inherited from interface org.apache.spark.ml.param.shared.HasCheckpointInterval
getCheckpointInterval
Methods inherited from interface org.apache.spark.ml.param.shared.HasFeaturesCol
featuresCol, getFeaturesCol
Methods inherited from interface org.apache.spark.ml.param.shared.HasLabelCol
getLabelCol, labelCol
Methods inherited from interface org.apache.spark.ml.param.shared.HasPredictionCol
getPredictionCol, predictionCol
Methods inherited from interface org.apache.spark.ml.tree.HasVarianceImpurity
getImpurity, getOldImpurity
Methods inherited from interface org.apache.spark.ml.param.shared.HasWeightCol
getWeightCol
Methods inherited from interface org.apache.spark.ml.util.Identifiable
toString
Methods inherited from interface org.apache.spark.internal.Logging
initializeForcefully, initializeLogIfNecessary, initializeLogIfNecessary, initializeLogIfNecessary$default$2, isTraceEnabled, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logName, logTrace, logTrace, logWarning, logWarning, org$apache$spark$internal$Logging$$log_, org$apache$spark$internal$Logging$$log__$eq
Methods inherited from interface org.apache.spark.ml.util.MLWritable
save
Methods inherited from interface org.apache.spark.ml.param.Params
clear, copyValues, defaultCopy, defaultParamMap, explainParam, explainParams, extractParamMap, extractParamMap, get, getDefault, getOrDefault, getParam, hasDefault, hasParam, isDefined, isSet, onParamChange, paramMap, params, set, set, set, setDefault, setDefault, shouldOwn
Methods inherited from interface org.apache.spark.ml.tree.RandomForestParams
getBootstrap, getNumTrees
Methods inherited from interface org.apache.spark.ml.tree.TreeEnsembleParams
getFeatureSubsetStrategy, getOldStrategy, getSubsamplingRate
Methods inherited from interface org.apache.spark.ml.tree.TreeEnsembleRegressorParams
validateAndTransformSchema
-
Constructor Details
-
RandomForestRegressor
-
RandomForestRegressor
public RandomForestRegressor()
-
-
Method Details
-
supportedImpurities
Accessor for supported impurity settings: variance -
supportedFeatureSubsetStrategies
Accessor for supported featureSubsetStrategy settings: auto, all, onethird, sqrt, log2 -
load
-
read
-
impurity
Description copied from interface:HasVarianceImpurity
Criterion used for information gain calculation (case-insensitive). This impurity type is used in DecisionTreeRegressor, RandomForestRegressor, GBTRegressor and GBTClassifier (since GBTClassificationModel is internally composed of DecisionTreeRegressionModels). Supported: "variance". (default = variance)- Specified by:
impurity
in interfaceHasVarianceImpurity
- Returns:
- (undocumented)
-
numTrees
Description copied from interface:RandomForestParams
Number of trees to train (at least 1). If 1, then no bootstrapping is used. If greater than 1, then bootstrapping is done. TODO: Change to always do bootstrapping (simpler). SPARK-7130 (default = 20)Note: The reason that we cannot add this to both GBT and RF (i.e. in TreeEnsembleParams) is the param
maxIter
controls how many trees a GBT has. The semantics in the algorithms are a bit different.- Specified by:
numTrees
in interfaceRandomForestParams
- Returns:
- (undocumented)
-
bootstrap
Description copied from interface:RandomForestParams
Whether bootstrap samples are used when building trees.- Specified by:
bootstrap
in interfaceRandomForestParams
- Returns:
- (undocumented)
-
subsamplingRate
Description copied from interface:TreeEnsembleParams
Fraction of the training data used for learning each decision tree, in range (0, 1]. (default = 1.0)- Specified by:
subsamplingRate
in interfaceTreeEnsembleParams
- Returns:
- (undocumented)
-
featureSubsetStrategy
Description copied from interface:TreeEnsembleParams
The number of features to consider for splits at each tree node. Supported options: - "auto": Choose automatically for task: If numTrees == 1, set to "all." If numTrees greater than 1 (forest), set to "sqrt" for classification and to "onethird" for regression. - "all": use all features - "onethird": use 1/3 of the features - "sqrt": use sqrt(number of features) - "log2": use log2(number of features) - "n": when n is in the range (0, 1.0], use n * number of features. When n is in the range (1, number of features), use n features. (default = "auto")These various settings are based on the following references: - log2: tested in Breiman (2001) - sqrt: recommended by Breiman manual for random forests - The defaults of sqrt (classification) and onethird (regression) match the R randomForest package.
- Specified by:
featureSubsetStrategy
in interfaceTreeEnsembleParams
- Returns:
- (undocumented)
- See Also:
-
leafCol
Description copied from interface:DecisionTreeParams
Leaf indices column name. Predicted leaf index of each instance in each tree by preorder. (default = "")- Specified by:
leafCol
in interfaceDecisionTreeParams
- Returns:
- (undocumented)
-
maxDepth
Description copied from interface:DecisionTreeParams
Maximum depth of the tree (nonnegative). E.g., depth 0 means 1 leaf node; depth 1 means 1 internal node + 2 leaf nodes. (default = 5)- Specified by:
maxDepth
in interfaceDecisionTreeParams
- Returns:
- (undocumented)
-
maxBins
Description copied from interface:DecisionTreeParams
Maximum number of bins used for discretizing continuous features and for choosing how to split on features at each node. More bins give higher granularity. Must be at least 2 and at least number of categories in any categorical feature. (default = 32)- Specified by:
maxBins
in interfaceDecisionTreeParams
- Returns:
- (undocumented)
-
minInstancesPerNode
Description copied from interface:DecisionTreeParams
Minimum number of instances each child must have after split. If a split causes the left or right child to have fewer than minInstancesPerNode, the split will be discarded as invalid. Must be at least 1. (default = 1)- Specified by:
minInstancesPerNode
in interfaceDecisionTreeParams
- Returns:
- (undocumented)
-
minWeightFractionPerNode
Description copied from interface:DecisionTreeParams
Minimum fraction of the weighted sample count that each child must have after split. If a split causes the fraction of the total weight in the left or right child to be less than minWeightFractionPerNode, the split will be discarded as invalid. Should be in the interval [0.0, 0.5). (default = 0.0)- Specified by:
minWeightFractionPerNode
in interfaceDecisionTreeParams
- Returns:
- (undocumented)
-
minInfoGain
Description copied from interface:DecisionTreeParams
Minimum information gain for a split to be considered at a tree node. Should be at least 0.0. (default = 0.0)- Specified by:
minInfoGain
in interfaceDecisionTreeParams
- Returns:
- (undocumented)
-
maxMemoryInMB
Description copied from interface:DecisionTreeParams
Maximum memory in MB allocated to histogram aggregation. If too small, then 1 node will be split per iteration, and its aggregates may exceed this size. (default = 256 MB)- Specified by:
maxMemoryInMB
in interfaceDecisionTreeParams
- Returns:
- (undocumented)
-
cacheNodeIds
Description copied from interface:DecisionTreeParams
If false, the algorithm will pass trees to executors to match instances with nodes. If true, the algorithm will cache node IDs for each instance. Caching can speed up training of deeper trees. Users can set how often should the cache be checkpointed or disable it by setting checkpointInterval. (default = false)- Specified by:
cacheNodeIds
in interfaceDecisionTreeParams
- Returns:
- (undocumented)
-
weightCol
Description copied from interface:HasWeightCol
Param for weight column name. If this is not set or empty, we treat all instance weights as 1.0.- Specified by:
weightCol
in interfaceHasWeightCol
- Returns:
- (undocumented)
-
seed
Description copied from interface:HasSeed
Param for random seed. -
checkpointInterval
Description copied from interface:HasCheckpointInterval
Param for set checkpoint interval (>= 1) or disable checkpoint (-1). E.g. 10 means that the cache will get checkpointed every 10 iterations. Note: this setting will be ignored if the checkpoint directory is not set in the SparkContext.- Specified by:
checkpointInterval
in interfaceHasCheckpointInterval
- Returns:
- (undocumented)
-
uid
Description copied from interface:Identifiable
An immutable unique ID for the object and its derivatives.- Specified by:
uid
in interfaceIdentifiable
- Returns:
- (undocumented)
-
setMaxDepth
-
setMaxBins
-
setMinInstancesPerNode
-
setMinWeightFractionPerNode
-
setMinInfoGain
-
setMaxMemoryInMB
-
setCacheNodeIds
-
setCheckpointInterval
Specifies how often to checkpoint the cached node IDs. E.g. 10 means that the cache will get checkpointed every 10 iterations. This is only used if cacheNodeIds is true and if the checkpoint directory is set inSparkContext
. Must be at least 1. (default = 10)- Parameters:
value
- (undocumented)- Returns:
- (undocumented)
-
setImpurity
-
setSubsamplingRate
-
setSeed
-
setNumTrees
-
setBootstrap
-
setFeatureSubsetStrategy
-
setWeightCol
Sets the value of paramweightCol()
. If this is not set or empty, we treat all instance weights as 1.0. By default the weightCol is not set, so all instances have weight 1.0.- Parameters:
value
- (undocumented)- Returns:
- (undocumented)
-
copy
Description copied from interface:Params
Creates a copy of this instance with the same UID and some extra params. Subclasses should implement this method and set the return type properly. SeedefaultCopy()
.- Specified by:
copy
in interfaceParams
- Specified by:
copy
in classPredictor<Vector,
RandomForestRegressor, RandomForestRegressionModel> - Parameters:
extra
- (undocumented)- Returns:
- (undocumented)
-