Package org.apache.spark.ml.regression

Class LinearRegressionModel

Object

org.apache.spark.ml.PipelineStage

org.apache.spark.ml.Transformer

org.apache.spark.ml.Model<M>

org.apache.spark.ml.PredictionModel<FeaturesType,M>

org.apache.spark.ml.regression.RegressionModel<Vector,LinearRegressionModel>

org.apache.spark.ml.regression.LinearRegressionModel

All Implemented Interfaces:

Serializable, org.apache.spark.internal.Logging, Params, HasAggregationDepth, HasElasticNetParam, HasFeaturesCol, HasFitIntercept, HasLabelCol, HasLoss, HasMaxBlockSizeInMB, HasMaxIter, HasPredictionCol, HasRegParam, HasSolver, HasStandardization, HasTol, HasWeightCol, PredictorParams, LinearRegressionParams, GeneralMLWritable, HasTrainingSummary<LinearRegressionTrainingSummary>, Identifiable, MLWritable

public class LinearRegressionModel
extends RegressionModel<Vector,LinearRegressionModel>
implements LinearRegressionParams, GeneralMLWritable, HasTrainingSummary<LinearRegressionTrainingSummary>

Model produced by LinearRegression.

See Also:

• Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.spark.internal.Logging

org.apache.spark.internal.Logging.LogStringContext, org.apache.spark.internal.Logging.SparkShellLoggingFilter

Method Summary

Modifier and Type	Method	Description
final IntParam	aggregationDepth()	Param for suggested depth for treeAggregate (>= 2).
Vector	coefficients()
LinearRegressionModel	copy(ParamMap extra)	Creates a copy of this instance with the same UID and some extra params.
final DoubleParam	elasticNetParam()	Param for the ElasticNet mixing parameter, in range [0, 1].
final DoubleParam	epsilon()	The shape parameter to control the amount of robustness.
LinearRegressionSummary	evaluate(Dataset<?> dataset)	Evaluates the model on a test dataset.
final BooleanParam	fitIntercept()	Param for whether to fit an intercept term.
double	intercept()
static LinearRegressionModel	load(String path)
final Param<String>	loss()	The loss function to be optimized.
final DoubleParam	maxBlockSizeInMB()	Param for Maximum memory in MB for stacking input data into blocks.
final IntParam	maxIter()	Param for maximum number of iterations (&gt;= 0).
int	numFeatures()	Returns the number of features the model was trained on.
double	predict(Vector features)	Predict label for the given features.
static MLReader<LinearRegressionModel>	read()
final DoubleParam	regParam()	Param for regularization parameter (&gt;= 0).
double	scale()
final Param<String>	solver()	The solver algorithm for optimization.
final BooleanParam	standardization()	Param for whether to standardize the training features before fitting the model.
LinearRegressionTrainingSummary	summary()	Gets summary (e.g.
final DoubleParam	tol()	Param for the convergence tolerance for iterative algorithms (&gt;= 0).
String	toString()
String	uid()	An immutable unique ID for the object and its derivatives.
final Param<String>	weightCol()	Param for weight column name.
GeneralMLWriter	write()	Returns a GeneralMLWriter instance for this ML instance.

Methods inherited from class org.apache.spark.ml.PredictionModel

featuresCol, labelCol, predictionCol, setFeaturesCol, setPredictionCol, transform, transformSchema

Methods inherited from class org.apache.spark.ml.Model

hasParent, parent, setParent

Methods inherited from class org.apache.spark.ml.Transformer

transform, transform, transform

Methods inherited from class org.apache.spark.ml.PipelineStage

params

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.spark.ml.param.shared.HasAggregationDepth

getAggregationDepth

Methods inherited from interface org.apache.spark.ml.param.shared.HasElasticNetParam

getElasticNetParam

Methods inherited from interface org.apache.spark.ml.param.shared.HasFeaturesCol

featuresCol, getFeaturesCol

Methods inherited from interface org.apache.spark.ml.param.shared.HasFitIntercept

getFitIntercept

Methods inherited from interface org.apache.spark.ml.param.shared.HasLabelCol

getLabelCol, labelCol

Methods inherited from interface org.apache.spark.ml.param.shared.HasLoss

getLoss

Methods inherited from interface org.apache.spark.ml.param.shared.HasMaxBlockSizeInMB

getMaxBlockSizeInMB

Methods inherited from interface org.apache.spark.ml.param.shared.HasMaxIter

getMaxIter

Methods inherited from interface org.apache.spark.ml.param.shared.HasPredictionCol

getPredictionCol, predictionCol

Methods inherited from interface org.apache.spark.ml.param.shared.HasRegParam

getRegParam

Methods inherited from interface org.apache.spark.ml.param.shared.HasSolver

getSolver

Methods inherited from interface org.apache.spark.ml.param.shared.HasStandardization

getStandardization

Methods inherited from interface org.apache.spark.ml.param.shared.HasTol

getTol

Methods inherited from interface org.apache.spark.ml.util.HasTrainingSummary

hasSummary, setSummary

Methods inherited from interface org.apache.spark.ml.param.shared.HasWeightCol

getWeightCol

Methods inherited from interface org.apache.spark.ml.regression.LinearRegressionParams

getEpsilon, validateAndTransformSchema

Methods inherited from interface org.apache.spark.internal.Logging

initializeForcefully, initializeLogIfNecessary, initializeLogIfNecessary, initializeLogIfNecessary$default$2, isTraceEnabled, log, logBasedOnLevel, logDebug, logDebug, logDebug, logDebug, logError, logError, logError, logError, logInfo, logInfo, logInfo, logInfo, logName, LogStringContext, logTrace, logTrace, logTrace, logTrace, logWarning, logWarning, logWarning, logWarning, MDC, org$apache$spark$internal$Logging$$log_, org$apache$spark$internal$Logging$$log__$eq, withLogContext

Methods inherited from interface org.apache.spark.ml.util.MLWritable

save

Methods inherited from interface org.apache.spark.ml.param.Params

clear, copyValues, defaultCopy, defaultParamMap, estimateMatadataSize, explainParam, explainParams, extractParamMap, extractParamMap, get, getDefault, getOrDefault, getParam, hasDefault, hasParam, isDefined, isSet, onParamChange, paramMap, params, set, set, set, setDefault, setDefault, shouldOwn

Method Details

read

public static MLReader<LinearRegressionModel> read()

load

public static LinearRegressionModel load(String path)

solver

public final Param<String> solver()

Description copied from interface: LinearRegressionParams

The solver algorithm for optimization. Supported options: "l-bfgs", "normal" and "auto". Default: "auto"

Specified by:

solver in interface HasSolver

Specified by:

solver in interface LinearRegressionParams

Returns:

(undocumented)

loss

public final Param<String> loss()

Description copied from interface: LinearRegressionParams

The loss function to be optimized. Supported options: "squaredError" and "huber". Default: "squaredError"

Specified by:

loss in interface HasLoss

Specified by:

loss in interface LinearRegressionParams

Returns:

(undocumented)

epsilon

public final DoubleParam epsilon()

Description copied from interface: LinearRegressionParams

The shape parameter to control the amount of robustness. Must be &gt; 1.0. At larger values of epsilon, the huber criterion becomes more similar to least squares regression; for small values of epsilon, the criterion is more similar to L1 regression. Default is 1.35 to get as much robustness as possible while retaining 95% statistical efficiency for normally distributed data. It matches sklearn HuberRegressor and is "M" from A robust hybrid of lasso and ridge regression. Only valid when "loss" is "huber".

Specified by:

epsilon in interface LinearRegressionParams

Returns:

(undocumented)

maxBlockSizeInMB

public final DoubleParam maxBlockSizeInMB()

Description copied from interface: HasMaxBlockSizeInMB

Param for Maximum memory in MB for stacking input data into blocks. Data is stacked within partitions. If more than remaining data size in a partition then it is adjusted to the data size. Default 0.0 represents choosing optimal value, depends on specific algorithm. Must be &gt;= 0..

Specified by:

maxBlockSizeInMB in interface HasMaxBlockSizeInMB

Returns:

(undocumented)

aggregationDepth

public final IntParam aggregationDepth()

Description copied from interface: HasAggregationDepth

Param for suggested depth for treeAggregate (&gt;= 2).

Specified by:

aggregationDepth in interface HasAggregationDepth

Returns:

(undocumented)

weightCol

public final Param<String> weightCol()

Description copied from interface: HasWeightCol

Param for weight column name. If this is not set or empty, we treat all instance weights as 1.0.

Specified by:

weightCol in interface HasWeightCol

Returns:

(undocumented)

standardization

public final BooleanParam standardization()

Description copied from interface: HasStandardization

Param for whether to standardize the training features before fitting the model.

Specified by:

standardization in interface HasStandardization

Returns:

(undocumented)

fitIntercept

public final BooleanParam fitIntercept()

Description copied from interface: HasFitIntercept

Param for whether to fit an intercept term.

Specified by:

fitIntercept in interface HasFitIntercept

Returns:

(undocumented)

tol

public final DoubleParam tol()

Description copied from interface: HasTol

Param for the convergence tolerance for iterative algorithms (&gt;= 0).

Specified by:

tol in interface HasTol

Returns:

(undocumented)

maxIter

public final IntParam maxIter()

Description copied from interface: HasMaxIter

Param for maximum number of iterations (&gt;= 0).

Specified by:

maxIter in interface HasMaxIter

Returns:

(undocumented)

elasticNetParam

public final DoubleParam elasticNetParam()

Description copied from interface: HasElasticNetParam

Param for the ElasticNet mixing parameter, in range [0, 1]. For alpha = 0, the penalty is an L2 penalty. For alpha = 1, it is an L1 penalty.

Specified by:

elasticNetParam in interface HasElasticNetParam

Returns:

(undocumented)

regParam

public final DoubleParam regParam()

Description copied from interface: HasRegParam

Param for regularization parameter (&gt;= 0).

Specified by:

regParam in interface HasRegParam

Returns:

(undocumented)

uid

public String uid()

Description copied from interface: Identifiable

An immutable unique ID for the object and its derivatives.

Specified by:

uid in interface Identifiable

Returns:

(undocumented)

coefficients

public Vector coefficients()

intercept

public double intercept()

scale

public double scale()

numFeatures

public int numFeatures()

Description copied from class: PredictionModel

Returns the number of features the model was trained on. If unknown, returns -1

Overrides:

numFeatures in class PredictionModel<Vector,LinearRegressionModel>

summary

public LinearRegressionTrainingSummary summary()

Gets summary (e.g. residuals, mse, r-squared ) of model on training set. An exception is thrown if hasSummary is false.

Specified by:

summary in interface HasTrainingSummary<LinearRegressionTrainingSummary>

Returns:

(undocumented)

evaluate

public LinearRegressionSummary evaluate(Dataset<?> dataset)

Evaluates the model on a test dataset.

Parameters:

dataset - Test dataset to evaluate model on.

Returns:

(undocumented)

predict

public double predict(Vector features)

Description copied from class: PredictionModel

Predict label for the given features. This method is used to implement transform() and output PredictionModel.predictionCol().

Specified by:

predict in class PredictionModel<Vector,LinearRegressionModel>

Parameters:

features - (undocumented)

Returns:

(undocumented)

copy

public LinearRegressionModel copy(ParamMap extra)

Description copied from interface: Params

Creates a copy of this instance with the same UID and some extra params. Subclasses should implement this method and set the return type properly. See defaultCopy().

Specified by:

copy in interface Params

Specified by:

copy in class Model<LinearRegressionModel>

Parameters:

extra - (undocumented)

Returns:

(undocumented)

write

public GeneralMLWriter write()

Returns a GeneralMLWriter instance for this ML instance.

For LinearRegressionModel, this does NOT currently save the training summary(). An option to save summary() may be added in the future.

This also does not save the Model.parent() currently.

Specified by:

write in interface GeneralMLWritable

Specified by:

write in interface MLWritable

Returns:

(undocumented)

toString

public String toString()

Specified by:

toString in interface Identifiable

Overrides:

toString in class Object