GeneralizedLinearAlgorithm

abstract class GeneralizedLinearAlgorithm[M <: GeneralizedLinearModel] extends Logging with Serializable

GeneralizedLinearAlgorithm implements methods to train a Generalized Linear Model (GLM). This class should be extended with an Optimizer to create a new GLM.

Annotations: @Since( "0.8.0" )
Source: GeneralizedLinearAlgorithm.scala

Linear Supertypes

Serializable, Serializable, Logging, AnyRef, Any

Known Subclasses

LogisticRegressionWithLBFGS, LogisticRegressionWithSGD, SVMWithSGD, LassoWithSGD, LinearRegressionWithSGD, RidgeRegressionWithSGD

Ordering

Alphabetic
By Inheritance

Inherited

GeneralizedLinearAlgorithm
Serializable
Serializable
Logging
AnyRef
Any

Hide All
Show All

Visibility

Public
All

Instance Constructors

new GeneralizedLinearAlgorithm()

Abstract Value Members

abstract def createModel(weights: Vector, intercept: Double): M
Create a model given the weights and intercept
Create a model given the weights and intercept

Attributes
protected
abstract def optimizer: Optimizer
The optimizer to solve the problem.
The optimizer to solve the problem.

Annotations
@Since( "0.8.0" )

Concrete Value Members

final def !=(arg0: Any): Boolean

Definition Classes
AnyRef → Any
final def ##(): Int

Definition Classes
AnyRef → Any
final def ==(arg0: Any): Boolean

Definition Classes
AnyRef → Any
var addIntercept: Boolean
Whether to add intercept (default: false).
Whether to add intercept (default: false).

Attributes
protected
final def asInstanceOf[T0]: T0

Definition Classes
Any
def clone(): AnyRef

Attributes
protected[lang]
Definition Classes
AnyRef
Annotations
@throws( ... ) @native()
final def eq(arg0: AnyRef): Boolean

Definition Classes
AnyRef
def equals(arg0: Any): Boolean

Definition Classes
AnyRef → Any
def finalize(): Unit

Attributes
protected[lang]
Definition Classes
AnyRef
Annotations
@throws( classOf[java.lang.Throwable] )
def generateInitialWeights(input: RDD[LabeledPoint]): Vector
Generate the initial weights when the user does not supply them
Generate the initial weights when the user does not supply them

Attributes
protected
final def getClass(): Class[_]

Definition Classes
AnyRef → Any
Annotations
@native()
def getNumFeatures: Int
The dimension of training features.
The dimension of training features.

Annotations
@Since( "1.4.0" )
def hashCode(): Int

Definition Classes
AnyRef → Any
Annotations
@native()
def initializeLogIfNecessary(isInterpreter: Boolean, silent: Boolean): Boolean

Attributes
protected
Definition Classes
Logging
def initializeLogIfNecessary(isInterpreter: Boolean): Unit

Attributes
protected
Definition Classes
Logging
def isAddIntercept: Boolean
Get if the algorithm uses addIntercept
Get if the algorithm uses addIntercept

Annotations
@Since( "1.4.0" )
final def isInstanceOf[T0]: Boolean

Definition Classes
Any
def isTraceEnabled(): Boolean

Attributes
protected
Definition Classes
Logging
def log: Logger

Attributes
protected
Definition Classes
Logging
def logDebug(msg: ⇒ String, throwable: Throwable): Unit

Attributes
protected
Definition Classes
Logging
def logDebug(msg: ⇒ String): Unit

Attributes
protected
Definition Classes
Logging
def logError(msg: ⇒ String, throwable: Throwable): Unit

Attributes
protected
Definition Classes
Logging
def logError(msg: ⇒ String): Unit

Attributes
protected
Definition Classes
Logging
def logInfo(msg: ⇒ String, throwable: Throwable): Unit

Attributes
protected
Definition Classes
Logging
def logInfo(msg: ⇒ String): Unit

Attributes
protected
Definition Classes
Logging
def logName: String

Attributes
protected
Definition Classes
Logging
def logTrace(msg: ⇒ String, throwable: Throwable): Unit

Attributes
protected
Definition Classes
Logging
def logTrace(msg: ⇒ String): Unit

Attributes
protected
Definition Classes
Logging
def logWarning(msg: ⇒ String, throwable: Throwable): Unit

Attributes
protected
Definition Classes
Logging
def logWarning(msg: ⇒ String): Unit

Attributes
protected
Definition Classes
Logging
final def ne(arg0: AnyRef): Boolean

Definition Classes
AnyRef
final def notify(): Unit

Definition Classes
AnyRef
Annotations
@native()
final def notifyAll(): Unit

Definition Classes
AnyRef
Annotations
@native()
var numFeatures: Int
The dimension of training features.
The dimension of training features.

Attributes
protected
var numOfLinearPredictor: Int
In GeneralizedLinearModel, only single linear predictor is allowed for both weights and intercept.
In GeneralizedLinearModel, only single linear predictor is allowed for both weights and intercept. However, for multinomial logistic regression, with K possible outcomes, we are training K-1 independent binary logistic regression models which requires K-1 sets of linear predictor.
As a result, the workaround here is if more than two sets of linear predictors are needed, we construct bigger weights vector which can hold both weights and intercepts. If the intercepts are added, the dimension of weights will be (numOfLinearPredictor) * (numFeatures + 1) . If the intercepts are not added, the dimension of weights will be (numOfLinearPredictor) * numFeatures.
Thus, the intercepts will be encapsulated into weights, and we leave the value of intercept in GeneralizedLinearModel as zero.

Attributes
protected
def run(input: RDD[LabeledPoint], initialWeights: Vector): M
Run the algorithm with the configured parameters on an input RDD of LabeledPoint entries starting from the initial weights provided.
Run the algorithm with the configured parameters on an input RDD of LabeledPoint entries starting from the initial weights provided.

Annotations
@Since( "1.0.0" )
def run(input: RDD[LabeledPoint]): M
Run the algorithm with the configured parameters on an input RDD of LabeledPoint entries.
Run the algorithm with the configured parameters on an input RDD of LabeledPoint entries.

Annotations
@Since( "0.8.0" )
def setIntercept(addIntercept: Boolean): GeneralizedLinearAlgorithm.this.type
Set if the algorithm should add an intercept.
Set if the algorithm should add an intercept. Default false. We set the default to false because adding the intercept will cause memory allocation.

Annotations
@Since( "0.8.0" )
def setValidateData(validateData: Boolean): GeneralizedLinearAlgorithm.this.type
Set if the algorithm should validate data before training.
Set if the algorithm should validate data before training. Default true.

Annotations
@Since( "0.8.0" )
final def synchronized[T0](arg0: ⇒ T0): T0

Definition Classes
AnyRef
def toString(): String

Definition Classes
AnyRef → Any
var validateData: Boolean

Attributes
protected
val validators: Seq[(RDD[LabeledPoint]) ⇒ Boolean]

Attributes
protected
final def wait(): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long, arg1: Int): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long): Unit

Definition Classes
AnyRef
Annotations
@throws( ... ) @native()

Inherited from Logging

Value Members

def initializeLogIfNecessary(isInterpreter: Boolean, silent: Boolean): Boolean

Attributes
protected
Definition Classes
Logging
def initializeLogIfNecessary(isInterpreter: Boolean): Unit

Attributes
protected
Definition Classes
Logging
def isTraceEnabled(): Boolean

Attributes
protected
Definition Classes
Logging
def log: Logger

Attributes
protected
Definition Classes
Logging
def logDebug(msg: ⇒ String, throwable: Throwable): Unit

Attributes
protected
Definition Classes
Logging
def logDebug(msg: ⇒ String): Unit

Attributes
protected
Definition Classes
Logging
def logError(msg: ⇒ String, throwable: Throwable): Unit

Attributes
protected
Definition Classes
Logging
def logError(msg: ⇒ String): Unit

Attributes
protected
Definition Classes
Logging
def logInfo(msg: ⇒ String, throwable: Throwable): Unit

Attributes
protected
Definition Classes
Logging
def logInfo(msg: ⇒ String): Unit

Attributes
protected
Definition Classes
Logging
def logName: String

Attributes
protected
Definition Classes
Logging
def logTrace(msg: ⇒ String, throwable: Throwable): Unit

Attributes
protected
Definition Classes
Logging
def logTrace(msg: ⇒ String): Unit

Attributes
protected
Definition Classes
Logging
def logWarning(msg: ⇒ String, throwable: Throwable): Unit

Attributes
protected
Definition Classes
Logging
def logWarning(msg: ⇒ String): Unit

Attributes
protected
Definition Classes
Logging

Inherited from AnyRef

Value Members

final def !=(arg0: Any): Boolean

Definition Classes
AnyRef → Any
final def ##(): Int

Definition Classes
AnyRef → Any
final def ==(arg0: Any): Boolean

Definition Classes
AnyRef → Any
def clone(): AnyRef

Attributes
protected[lang]
Definition Classes
AnyRef
Annotations
@throws( ... ) @native()
final def eq(arg0: AnyRef): Boolean

Definition Classes
AnyRef
def equals(arg0: Any): Boolean

Definition Classes
AnyRef → Any
def finalize(): Unit

Attributes
protected[lang]
Definition Classes
AnyRef
Annotations
@throws( classOf[java.lang.Throwable] )
final def getClass(): Class[_]

Definition Classes
AnyRef → Any
Annotations
@native()
def hashCode(): Int

Definition Classes
AnyRef → Any
Annotations
@native()
final def ne(arg0: AnyRef): Boolean

Definition Classes
AnyRef
final def notify(): Unit

Definition Classes
AnyRef
Annotations
@native()
final def notifyAll(): Unit

Definition Classes
AnyRef
Annotations
@native()
final def synchronized[T0](arg0: ⇒ T0): T0

Definition Classes
AnyRef
def toString(): String

Definition Classes
AnyRef → Any
final def wait(): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long, arg1: Int): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long): Unit

Definition Classes
AnyRef
Annotations
@throws( ... ) @native()

Inherited from Any

Value Members

final def asInstanceOf[T0]: T0

Definition Classes
Any
final def isInstanceOf[T0]: Boolean

Definition Classes
Any

Ungrouped

abstract def createModel(weights: Vector, intercept: Double): M
Create a model given the weights and intercept
Create a model given the weights and intercept

Attributes
protected
abstract def optimizer: Optimizer
The optimizer to solve the problem.
The optimizer to solve the problem.

Annotations
@Since( "0.8.0" )
final def !=(arg0: Any): Boolean

Definition Classes
AnyRef → Any
final def ##(): Int

Definition Classes
AnyRef → Any
final def ==(arg0: Any): Boolean

Definition Classes
AnyRef → Any
var addIntercept: Boolean
Whether to add intercept (default: false).
Whether to add intercept (default: false).

Attributes
protected
final def asInstanceOf[T0]: T0

Definition Classes
Any
def clone(): AnyRef

Attributes
protected[lang]
Definition Classes
AnyRef
Annotations
@throws( ... ) @native()
final def eq(arg0: AnyRef): Boolean

Definition Classes
AnyRef
def equals(arg0: Any): Boolean

Definition Classes
AnyRef → Any
def finalize(): Unit

Attributes
protected[lang]
Definition Classes
AnyRef
Annotations
@throws( classOf[java.lang.Throwable] )
def generateInitialWeights(input: RDD[LabeledPoint]): Vector
Generate the initial weights when the user does not supply them
Generate the initial weights when the user does not supply them

Attributes
protected
final def getClass(): Class[_]

Definition Classes
AnyRef → Any
Annotations
@native()
def getNumFeatures: Int
The dimension of training features.
The dimension of training features.

Annotations
@Since( "1.4.0" )
def hashCode(): Int

Definition Classes
AnyRef → Any
Annotations
@native()
def initializeLogIfNecessary(isInterpreter: Boolean, silent: Boolean): Boolean

Attributes
protected
Definition Classes
Logging
def initializeLogIfNecessary(isInterpreter: Boolean): Unit

Attributes
protected
Definition Classes
Logging
def isAddIntercept: Boolean
Get if the algorithm uses addIntercept
Get if the algorithm uses addIntercept

Annotations
@Since( "1.4.0" )
final def isInstanceOf[T0]: Boolean

Definition Classes
Any
def isTraceEnabled(): Boolean

Attributes
protected
Definition Classes
Logging
def log: Logger

Attributes
protected
Definition Classes
Logging
def logDebug(msg: ⇒ String, throwable: Throwable): Unit

Attributes
protected
Definition Classes
Logging
def logDebug(msg: ⇒ String): Unit

Attributes
protected
Definition Classes
Logging
def logError(msg: ⇒ String, throwable: Throwable): Unit

Attributes
protected
Definition Classes
Logging
def logError(msg: ⇒ String): Unit

Attributes
protected
Definition Classes
Logging
def logInfo(msg: ⇒ String, throwable: Throwable): Unit

Attributes
protected
Definition Classes
Logging
def logInfo(msg: ⇒ String): Unit

Attributes
protected
Definition Classes
Logging
def logName: String

Attributes
protected
Definition Classes
Logging
def logTrace(msg: ⇒ String, throwable: Throwable): Unit

Attributes
protected
Definition Classes
Logging
def logTrace(msg: ⇒ String): Unit

Attributes
protected
Definition Classes
Logging
def logWarning(msg: ⇒ String, throwable: Throwable): Unit

Attributes
protected
Definition Classes
Logging
def logWarning(msg: ⇒ String): Unit

Attributes
protected
Definition Classes
Logging
final def ne(arg0: AnyRef): Boolean

Definition Classes
AnyRef
final def notify(): Unit

Definition Classes
AnyRef
Annotations
@native()
final def notifyAll(): Unit

Definition Classes
AnyRef
Annotations
@native()
var numFeatures: Int
The dimension of training features.
The dimension of training features.

Attributes
protected
var numOfLinearPredictor: Int
In GeneralizedLinearModel, only single linear predictor is allowed for both weights and intercept.
In GeneralizedLinearModel, only single linear predictor is allowed for both weights and intercept. However, for multinomial logistic regression, with K possible outcomes, we are training K-1 independent binary logistic regression models which requires K-1 sets of linear predictor.
As a result, the workaround here is if more than two sets of linear predictors are needed, we construct bigger weights vector which can hold both weights and intercepts. If the intercepts are added, the dimension of weights will be (numOfLinearPredictor) * (numFeatures + 1) . If the intercepts are not added, the dimension of weights will be (numOfLinearPredictor) * numFeatures.
Thus, the intercepts will be encapsulated into weights, and we leave the value of intercept in GeneralizedLinearModel as zero.

Attributes
protected
def run(input: RDD[LabeledPoint], initialWeights: Vector): M
Run the algorithm with the configured parameters on an input RDD of LabeledPoint entries starting from the initial weights provided.
Run the algorithm with the configured parameters on an input RDD of LabeledPoint entries starting from the initial weights provided.

Annotations
@Since( "1.0.0" )
def run(input: RDD[LabeledPoint]): M
Run the algorithm with the configured parameters on an input RDD of LabeledPoint entries.
Run the algorithm with the configured parameters on an input RDD of LabeledPoint entries.

Annotations
@Since( "0.8.0" )
def setIntercept(addIntercept: Boolean): GeneralizedLinearAlgorithm.this.type
Set if the algorithm should add an intercept.
Set if the algorithm should add an intercept. Default false. We set the default to false because adding the intercept will cause memory allocation.

Annotations
@Since( "0.8.0" )
def setValidateData(validateData: Boolean): GeneralizedLinearAlgorithm.this.type
Set if the algorithm should validate data before training.
Set if the algorithm should validate data before training. Default true.

Annotations
@Since( "0.8.0" )
final def synchronized[T0](arg0: ⇒ T0): T0

Definition Classes
AnyRef
def toString(): String

Definition Classes
AnyRef → Any
var validateData: Boolean

Attributes
protected
val validators: Seq[(RDD[LabeledPoint]) ⇒ Boolean]

Attributes
protected
final def wait(): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long, arg1: Int): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long): Unit

Definition Classes
AnyRef
Annotations
@throws( ... ) @native()

Packages

GeneralizedLinearAlgorithm 

abstract class GeneralizedLinearAlgorithm[M <: GeneralizedLinearModel] extends Logging with Serializable

Instance Constructors

Abstract Value Members

Concrete Value Members

Inherited from Logging

Value Members

Inherited from AnyRef

Value Members

Inherited from Any

Value Members

Ungrouped

GeneralizedLinearAlgorithm