Package org.apache.spark.ml.regression

Class GeneralizedLinearRegressionModel

Object

org.apache.spark.ml.PipelineStage

org.apache.spark.ml.Transformer

org.apache.spark.ml.Model<M>

org.apache.spark.ml.PredictionModel<FeaturesType,M>

org.apache.spark.ml.regression.RegressionModel<Vector,GeneralizedLinearRegressionModel>

org.apache.spark.ml.regression.GeneralizedLinearRegressionModel

All Implemented Interfaces:

Serializable, org.apache.spark.internal.Logging, Params, HasAggregationDepth, HasFeaturesCol, HasFitIntercept, HasLabelCol, HasMaxIter, HasPredictionCol, HasRegParam, HasSolver, HasTol, HasWeightCol, PredictorParams, GeneralizedLinearRegressionBase, HasTrainingSummary<GeneralizedLinearRegressionTrainingSummary>, Identifiable, MLWritable

public class GeneralizedLinearRegressionModel
extends RegressionModel<Vector,GeneralizedLinearRegressionModel>
implements GeneralizedLinearRegressionBase, MLWritable, HasTrainingSummary<GeneralizedLinearRegressionTrainingSummary>

Model produced by GeneralizedLinearRegression.

See Also:

• Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.spark.internal.Logging

org.apache.spark.internal.Logging.LogStringContext, org.apache.spark.internal.Logging.SparkShellLoggingFilter

Method Summary

Modifier and Type	Method	Description
final IntParam	aggregationDepth()	Param for suggested depth for treeAggregate (>= 2).
Vector	coefficients()
GeneralizedLinearRegressionModel	copy(ParamMap extra)	Creates a copy of this instance with the same UID and some extra params.
GeneralizedLinearRegressionSummary	evaluate(Dataset<?> dataset)	Evaluate the model on the given dataset, returning a summary of the results.
final Param<String>	family()	Param for the name of family which is a description of the error distribution to be used in the model.
final BooleanParam	fitIntercept()	Param for whether to fit an intercept term.
double	intercept()
final Param<String>	link()	Param for the name of link function which provides the relationship between the linear predictor and the mean of the distribution function.
final DoubleParam	linkPower()	Param for the index in the power link function.
final Param<String>	linkPredictionCol()	Param for link prediction (linear predictor) column name.
static GeneralizedLinearRegressionModel	load(String path)
final IntParam	maxIter()	Param for maximum number of iterations (&gt;= 0).
int	numFeatures()	Returns the number of features the model was trained on.
final Param<String>	offsetCol()	Param for offset column name.
double	predict(Vector features)	Predict label for the given features.
static MLReader<GeneralizedLinearRegressionModel>	read()
final DoubleParam	regParam()	Param for regularization parameter (&gt;= 0).
GeneralizedLinearRegressionModel	setLinkPredictionCol(String value)	Sets the link prediction (linear predictor) column name.
final Param<String>	solver()	The solver algorithm for optimization.
GeneralizedLinearRegressionTrainingSummary	summary()	Gets R-like summary of model on training set.
final DoubleParam	tol()	Param for the convergence tolerance for iterative algorithms (&gt;= 0).
String	toString()
Dataset<Row>	transform(Dataset<?> dataset)	Transforms dataset by reading from PredictionModel.featuresCol(), calling predict, and storing the predictions as a new column PredictionModel.predictionCol().
String	uid()	An immutable unique ID for the object and its derivatives.
final DoubleParam	variancePower()	Param for the power in the variance function of the Tweedie distribution which provides the relationship between the variance and mean of the distribution.
final Param<String>	weightCol()	Param for weight column name.
MLWriter	write()	Returns a MLWriter instance for this ML instance.

Methods inherited from class org.apache.spark.ml.PredictionModel

featuresCol, labelCol, predictionCol, setFeaturesCol, setPredictionCol, transformSchema

Methods inherited from class org.apache.spark.ml.Model

hasParent, parent, setParent

Methods inherited from class org.apache.spark.ml.Transformer

transform, transform, transform

Methods inherited from class org.apache.spark.ml.PipelineStage

params

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.spark.ml.regression.GeneralizedLinearRegressionBase

getFamily, getLink, getLinkPower, getLinkPredictionCol, getOffsetCol, getVariancePower, hasLinkPredictionCol, hasOffsetCol, hasWeightCol, validateAndTransformSchema

Methods inherited from interface org.apache.spark.ml.param.shared.HasAggregationDepth

getAggregationDepth

Methods inherited from interface org.apache.spark.ml.param.shared.HasFeaturesCol

featuresCol, getFeaturesCol

Methods inherited from interface org.apache.spark.ml.param.shared.HasFitIntercept

getFitIntercept

Methods inherited from interface org.apache.spark.ml.param.shared.HasLabelCol

getLabelCol, labelCol

Methods inherited from interface org.apache.spark.ml.param.shared.HasMaxIter

getMaxIter

Methods inherited from interface org.apache.spark.ml.param.shared.HasPredictionCol

getPredictionCol, predictionCol

Methods inherited from interface org.apache.spark.ml.param.shared.HasRegParam

getRegParam

Methods inherited from interface org.apache.spark.ml.param.shared.HasSolver

getSolver

Methods inherited from interface org.apache.spark.ml.param.shared.HasTol

getTol

Methods inherited from interface org.apache.spark.ml.util.HasTrainingSummary

hasSummary, setSummary

Methods inherited from interface org.apache.spark.ml.param.shared.HasWeightCol

getWeightCol

Methods inherited from interface org.apache.spark.internal.Logging

initializeForcefully, initializeLogIfNecessary, initializeLogIfNecessary, initializeLogIfNecessary$default$2, isTraceEnabled, log, logDebug, logDebug, logDebug, logDebug, logError, logError, logError, logError, logInfo, logInfo, logInfo, logInfo, logName, LogStringContext, logTrace, logTrace, logTrace, logTrace, logWarning, logWarning, logWarning, logWarning, org$apache$spark$internal$Logging$$log_, org$apache$spark$internal$Logging$$log__$eq, withLogContext

Methods inherited from interface org.apache.spark.ml.util.MLWritable

save

Methods inherited from interface org.apache.spark.ml.param.Params

clear, copyValues, defaultCopy, defaultParamMap, explainParam, explainParams, extractParamMap, extractParamMap, get, getDefault, getOrDefault, getParam, hasDefault, hasParam, isDefined, isSet, onParamChange, paramMap, params, set, set, set, setDefault, setDefault, shouldOwn

Method Details

read

public static MLReader<GeneralizedLinearRegressionModel> read()

load

public static GeneralizedLinearRegressionModel load(String path)

family

public final Param<String> family()

Description copied from interface: GeneralizedLinearRegressionBase

Param for the name of family which is a description of the error distribution to be used in the model. Supported options: "gaussian", "binomial", "poisson", "gamma" and "tweedie". Default is "gaussian".

Specified by:

family in interface GeneralizedLinearRegressionBase

Returns:

(undocumented)

variancePower

public final DoubleParam variancePower()

Description copied from interface: GeneralizedLinearRegressionBase

Param for the power in the variance function of the Tweedie distribution which provides the relationship between the variance and mean of the distribution. Only applicable to the Tweedie family. (see Tweedie Distribution (Wikipedia)) Supported values: 0 and [1, Inf). Note that variance power 0, 1, or 2 corresponds to the Gaussian, Poisson or Gamma family, respectively.

Specified by:

variancePower in interface GeneralizedLinearRegressionBase

Returns:

(undocumented)

link

public final Param<String> link()

Description copied from interface: GeneralizedLinearRegressionBase

Param for the name of link function which provides the relationship between the linear predictor and the mean of the distribution function. Supported options: "identity", "log", "inverse", "logit", "probit", "cloglog" and "sqrt". This is used only when family is not "tweedie". The link function for the "tweedie" family must be specified through GeneralizedLinearRegressionBase.linkPower().

Specified by:

link in interface GeneralizedLinearRegressionBase

Returns:

(undocumented)

linkPower

public final DoubleParam linkPower()

Description copied from interface: GeneralizedLinearRegressionBase

Param for the index in the power link function. Only applicable to the Tweedie family. Note that link power 0, 1, -1 or 0.5 corresponds to the Log, Identity, Inverse or Sqrt link, respectively. When not set, this value defaults to 1 - GeneralizedLinearRegressionBase.variancePower(), which matches the R "statmod" package.

Specified by:

linkPower in interface GeneralizedLinearRegressionBase

Returns:

(undocumented)

linkPredictionCol

public final Param<String> linkPredictionCol()

Description copied from interface: GeneralizedLinearRegressionBase

Param for link prediction (linear predictor) column name. Default is not set, which means we do not output link prediction.

Specified by:

linkPredictionCol in interface GeneralizedLinearRegressionBase

Returns:

(undocumented)

offsetCol

public final Param<String> offsetCol()

Description copied from interface: GeneralizedLinearRegressionBase

Param for offset column name. If this is not set or empty, we treat all instance offsets as 0.0. The feature specified as offset has a constant coefficient of 1.0.

Specified by:

offsetCol in interface GeneralizedLinearRegressionBase

Returns:

(undocumented)

solver

public final Param<String> solver()

Description copied from interface: GeneralizedLinearRegressionBase

The solver algorithm for optimization. Supported options: "irls" (iteratively reweighted least squares). Default: "irls"

Specified by:

solver in interface GeneralizedLinearRegressionBase

Specified by:

solver in interface HasSolver

Returns:

(undocumented)

aggregationDepth

public final IntParam aggregationDepth()

Description copied from interface: HasAggregationDepth

Param for suggested depth for treeAggregate (&gt;= 2).

Specified by:

aggregationDepth in interface HasAggregationDepth

Returns:

(undocumented)

weightCol

public final Param<String> weightCol()

Description copied from interface: HasWeightCol

Param for weight column name. If this is not set or empty, we treat all instance weights as 1.0.

Specified by:

weightCol in interface HasWeightCol

Returns:

(undocumented)

regParam

public final DoubleParam regParam()

Description copied from interface: HasRegParam

Param for regularization parameter (&gt;= 0).

Specified by:

regParam in interface HasRegParam

Returns:

(undocumented)

tol

public final DoubleParam tol()

Description copied from interface: HasTol

Param for the convergence tolerance for iterative algorithms (&gt;= 0).

Specified by:

tol in interface HasTol

Returns:

(undocumented)

maxIter

public final IntParam maxIter()

Description copied from interface: HasMaxIter

Param for maximum number of iterations (&gt;= 0).

Specified by:

maxIter in interface HasMaxIter

Returns:

(undocumented)

fitIntercept

public final BooleanParam fitIntercept()

Description copied from interface: HasFitIntercept

Param for whether to fit an intercept term.

Specified by:

fitIntercept in interface HasFitIntercept

Returns:

(undocumented)

uid

public String uid()

Description copied from interface: Identifiable

An immutable unique ID for the object and its derivatives.

Specified by:

uid in interface Identifiable

Returns:

(undocumented)

coefficients

public Vector coefficients()

intercept

public double intercept()

setLinkPredictionCol

public GeneralizedLinearRegressionModel setLinkPredictionCol(String value)

Sets the link prediction (linear predictor) column name.

Parameters:

value - (undocumented)

Returns:

(undocumented)

predict

public double predict(Vector features)

Description copied from class: PredictionModel

Predict label for the given features. This method is used to implement transform() and output PredictionModel.predictionCol().

Specified by:

predict in class PredictionModel<Vector,GeneralizedLinearRegressionModel>

Parameters:

features - (undocumented)

Returns:

(undocumented)

transform

public Dataset<Row> transform(Dataset<?> dataset)

Description copied from class: PredictionModel

Transforms dataset by reading from PredictionModel.featuresCol(), calling predict, and storing the predictions as a new column PredictionModel.predictionCol().

Overrides:

transform in class PredictionModel<Vector,GeneralizedLinearRegressionModel>

Parameters:

dataset - input dataset

Returns:

transformed dataset with PredictionModel.predictionCol() of type Double

summary

public GeneralizedLinearRegressionTrainingSummary summary()

Gets R-like summary of model on training set. An exception is thrown if there is no summary available.

Specified by:

summary in interface HasTrainingSummary<GeneralizedLinearRegressionTrainingSummary>

Returns:

(undocumented)

evaluate

public GeneralizedLinearRegressionSummary evaluate(Dataset<?> dataset)

Evaluate the model on the given dataset, returning a summary of the results.

Parameters:

dataset - (undocumented)

Returns:

(undocumented)

copy

public GeneralizedLinearRegressionModel copy(ParamMap extra)

Description copied from interface: Params

Creates a copy of this instance with the same UID and some extra params. Subclasses should implement this method and set the return type properly. See defaultCopy().

Specified by:

copy in interface Params

Specified by:

copy in class Model<GeneralizedLinearRegressionModel>

Parameters:

extra - (undocumented)

Returns:

(undocumented)

write

public MLWriter write()

Returns a MLWriter instance for this ML instance.

For GeneralizedLinearRegressionModel, this does NOT currently save the training summary(). An option to save summary() may be added in the future.

Specified by:

write in interface MLWritable

Returns:

(undocumented)

numFeatures

public int numFeatures()

Description copied from class: PredictionModel

Returns the number of features the model was trained on. If unknown, returns -1

Overrides:

numFeatures in class PredictionModel<Vector,GeneralizedLinearRegressionModel>

toString

public String toString()

Specified by:

toString in interface Identifiable

Overrides:

toString in class Object