LinearRegressionWithSGD

Related Docs: object LinearRegressionWithSGD | package regression

class LinearRegressionWithSGD extends GeneralizedLinearAlgorithm[LinearRegressionModel] with Serializable

Train a linear regression model with no regularization using Stochastic Gradient Descent. This solves the least squares regression formulation f(weights) = 1/n ||A weights-y||2 (which is the mean squared error). Here the data matrix has n rows, and the input RDD holds the set of rows of A, each with its corresponding right hand side label y. See also the documentation for the precise formulation.

Annotations
@Since( "0.8.0" )
Source
LinearRegression.scala
Linear Supertypes
GeneralizedLinearAlgorithm[LinearRegressionModel], Serializable, Serializable, Logging, AnyRef, Any
Ordering
1. Alphabetic
2. By Inheritance
Inherited
1. LinearRegressionWithSGD
2. GeneralizedLinearAlgorithm
3. Serializable
4. Serializable
5. Logging
6. AnyRef
7. Any
1. Hide All
2. Show All
Visibility
1. Public
2. All

Instance Constructors

1. new LinearRegressionWithSGD()

Construct a LinearRegression object with default parameters: {stepSize: 1.0, numIterations: 100, miniBatchFraction: 1.0}.

Construct a LinearRegression object with default parameters: {stepSize: 1.0, numIterations: 100, miniBatchFraction: 1.0}.

Annotations
@Since( "0.8.0" ) @deprecated
Deprecated

(Since version 2.0.0) Use ml.regression.LinearRegression or LBFGS

Value Members

1. final def !=(arg0: Any): Boolean

Definition Classes
AnyRef → Any
2. final def ##(): Int

Definition Classes
AnyRef → Any
3. final def ==(arg0: Any): Boolean

Definition Classes
AnyRef → Any

Whether to add intercept (default: false).

Whether to add intercept (default: false).

Attributes
protected
Definition Classes
GeneralizedLinearAlgorithm
5. final def asInstanceOf[T0]: T0

Definition Classes
Any
6. def clone(): AnyRef

Attributes
protected[java.lang]
Definition Classes
AnyRef
Annotations
@throws( ... )
7. def createModel(weights: Vector, intercept: Double): LinearRegressionModel

Create a model given the weights and intercept

Create a model given the weights and intercept

Attributes
protected[org.apache.spark.mllib]
Definition Classes
LinearRegressionWithSGDGeneralizedLinearAlgorithm
8. final def eq(arg0: AnyRef): Boolean

Definition Classes
AnyRef
9. def equals(arg0: Any): Boolean

Definition Classes
AnyRef → Any
10. def finalize(): Unit

Attributes
protected[java.lang]
Definition Classes
AnyRef
Annotations
@throws( classOf[java.lang.Throwable] )
11. def generateInitialWeights(input: RDD[LabeledPoint]): Vector

Generate the initial weights when the user does not supply them

Generate the initial weights when the user does not supply them

Attributes
protected
Definition Classes
GeneralizedLinearAlgorithm
12. final def getClass(): Class[_]

Definition Classes
AnyRef → Any
13. def getNumFeatures: Int

The dimension of training features.

The dimension of training features.

Definition Classes
GeneralizedLinearAlgorithm
Annotations
@Since( "1.4.0" )
14. def hashCode(): Int

Definition Classes
AnyRef → Any
15. def initializeLogIfNecessary(isInterpreter: Boolean, silent: Boolean = false): Boolean

Attributes
protected
Definition Classes
Logging
16. def initializeLogIfNecessary(isInterpreter: Boolean): Unit

Attributes
protected
Definition Classes
Logging

Get if the algorithm uses addIntercept

Get if the algorithm uses addIntercept

Definition Classes
GeneralizedLinearAlgorithm
Annotations
@Since( "1.4.0" )
18. final def isInstanceOf[T0]: Boolean

Definition Classes
Any
19. def isTraceEnabled(): Boolean

Attributes
protected
Definition Classes
Logging
20. def log: Logger

Attributes
protected
Definition Classes
Logging
21. def logDebug(msg: ⇒ String, throwable: Throwable): Unit

Attributes
protected
Definition Classes
Logging
22. def logDebug(msg: ⇒ String): Unit

Attributes
protected
Definition Classes
Logging
23. def logError(msg: ⇒ String, throwable: Throwable): Unit

Attributes
protected
Definition Classes
Logging
24. def logError(msg: ⇒ String): Unit

Attributes
protected
Definition Classes
Logging
25. def logInfo(msg: ⇒ String, throwable: Throwable): Unit

Attributes
protected
Definition Classes
Logging
26. def logInfo(msg: ⇒ String): Unit

Attributes
protected
Definition Classes
Logging
27. def logName: String

Attributes
protected
Definition Classes
Logging
28. def logTrace(msg: ⇒ String, throwable: Throwable): Unit

Attributes
protected
Definition Classes
Logging
29. def logTrace(msg: ⇒ String): Unit

Attributes
protected
Definition Classes
Logging
30. def logWarning(msg: ⇒ String, throwable: Throwable): Unit

Attributes
protected
Definition Classes
Logging
31. def logWarning(msg: ⇒ String): Unit

Attributes
protected
Definition Classes
Logging
32. final def ne(arg0: AnyRef): Boolean

Definition Classes
AnyRef
33. final def notify(): Unit

Definition Classes
AnyRef
34. final def notifyAll(): Unit

Definition Classes
AnyRef
35. var numFeatures: Int

The dimension of training features.

The dimension of training features.

Attributes
protected
Definition Classes
GeneralizedLinearAlgorithm
36. var numOfLinearPredictor: Int

In `GeneralizedLinearModel`, only single linear predictor is allowed for both weights and intercept.

In `GeneralizedLinearModel`, only single linear predictor is allowed for both weights and intercept. However, for multinomial logistic regression, with K possible outcomes, we are training K-1 independent binary logistic regression models which requires K-1 sets of linear predictor.

As a result, the workaround here is if more than two sets of linear predictors are needed, we construct bigger `weights` vector which can hold both weights and intercepts. If the intercepts are added, the dimension of `weights` will be (numOfLinearPredictor) * (numFeatures + 1) . If the intercepts are not added, the dimension of `weights` will be (numOfLinearPredictor) * numFeatures.

Thus, the intercepts will be encapsulated into weights, and we leave the value of intercept in GeneralizedLinearModel as zero.

Attributes
protected
Definition Classes
GeneralizedLinearAlgorithm

The optimizer to solve the problem.

The optimizer to solve the problem.

Definition Classes
LinearRegressionWithSGDGeneralizedLinearAlgorithm
Annotations
@Since( "0.8.0" )
38. def run(input: RDD[LabeledPoint], initialWeights: Vector): LinearRegressionModel

Run the algorithm with the configured parameters on an input RDD of LabeledPoint entries starting from the initial weights provided.

Run the algorithm with the configured parameters on an input RDD of LabeledPoint entries starting from the initial weights provided.

Definition Classes
GeneralizedLinearAlgorithm
Annotations
@Since( "1.0.0" )
39. def run(input: RDD[LabeledPoint]): LinearRegressionModel

Run the algorithm with the configured parameters on an input RDD of LabeledPoint entries.

Run the algorithm with the configured parameters on an input RDD of LabeledPoint entries.

Definition Classes
GeneralizedLinearAlgorithm
Annotations
@Since( "0.8.0" )

Set if the algorithm should add an intercept.

Set if the algorithm should add an intercept. Default false. We set the default to false because adding the intercept will cause memory allocation.

Definition Classes
GeneralizedLinearAlgorithm
Annotations
@Since( "0.8.0" )
41. def setValidateData(validateData: Boolean): LinearRegressionWithSGD.this.type

Set if the algorithm should validate data before training.

Set if the algorithm should validate data before training. Default true.

Definition Classes
GeneralizedLinearAlgorithm
Annotations
@Since( "0.8.0" )
42. final def synchronized[T0](arg0: ⇒ T0): T0

Definition Classes
AnyRef
43. def toString(): String

Definition Classes
AnyRef → Any
44. var validateData: Boolean

Attributes
protected
Definition Classes
GeneralizedLinearAlgorithm
45. val validators: Seq[(RDD[LabeledPoint]) ⇒ Boolean]

Attributes
protected
Definition Classes
GeneralizedLinearAlgorithm
46. final def wait(): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
47. final def wait(arg0: Long, arg1: Int): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
48. final def wait(arg0: Long): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )