public class ALS extends Object implements scala.Serializable, Logging
ALS attempts to estimate the ratings matrix R
as the product of two lowerrank matrices,
X
and Y
, i.e. X * Yt = R
. Typically these approximations are called 'factor' matrices.
The general approach is iterative. During each iteration, one of the factor matrices is held
constant, while the other is solved for using least squares. The newlysolved factor matrix is
then held constant while solving for the other factor matrix.
This is a blocked implementation of the ALS factorization algorithm that groups the two sets of factors (referred to as "users" and "products") into blocks and reduces communication by only sending one copy of each user vector to each product block on each iteration, and only for the product blocks that need that user's feature vector. This is achieved by precomputing some information about the ratings matrix to determine the "outlinks" of each user (which blocks of products it will contribute to) and "inlink" information for each product (which of the feature vectors it receives from each user block it will depend on). This allows us to send only an array of feature vectors between each user block and product block, and have the product block find the users' ratings and update the products based on these messages.
For implicit preference data, the algorithm used is based on
"Collaborative Filtering for Implicit Feedback Datasets", available at
http://dx.doi.org/10.1109/ICDM.2008.22
, adapted for the blocked approach used here.
Essentially instead of finding the lowrank approximations to the rating matrix R
,
this finds the approximations for a preference matrix P
where the elements of P
are 1 if
r > 0 and 0 if r = 0. The ratings then act as 'confidence' values related to strength of
indicated user
preferences rather than explicit ratings given to items.
Modifier and Type  Class and Description 

static class 
ALS.BlockStats
:: DeveloperApi ::
Statistics of a block in ALS computation.

static class 
ALS.BlockStats$ 
Constructor and Description 

ALS()
Constructs an ALS instance with default parameters: {numBlocks: 1, rank: 10, iterations: 10,
lambda: 0.01, implicitPrefs: false, alpha: 1.0}.

Modifier and Type  Method and Description 

static ALS.BlockStats[] 
analyzeBlocks(RDD<Rating> ratings,
int numUserBlocks,
int numProductBlocks)
:: DeveloperApi ::
Given an RDD of ratings, number of user blocks, and number of product blocks, computes the
statistics of each block in ALS computation.

MatrixFactorizationModel 
run(JavaRDD<Rating> ratings)
Javafriendly version of
ALS.run . 
MatrixFactorizationModel 
run(RDD<Rating> ratings)
Run ALS with the configured parameters on an input RDD of (user, product, rating) triples.

ALS 
setAlpha(double alpha)
:: Experimental ::
Sets the constant used in computing confidence in implicit ALS.

ALS 
setBlocks(int numBlocks)
Set the number of blocks for both user blocks and product blocks to parallelize the computation
into; pass 1 for an autoconfigured number of blocks.

ALS 
setImplicitPrefs(boolean implicitPrefs)
Sets whether to use implicit preference.

ALS 
setIntermediateRDDStorageLevel(StorageLevel storageLevel)
:: DeveloperApi ::
Sets storage level for intermediate RDDs (user/product in/out links).

ALS 
setIterations(int iterations)
Set the number of iterations to run.

ALS 
setLambda(double lambda)
Set the regularization parameter, lambda.

ALS 
setNonnegative(boolean b)
Set whether the leastsquares problems solved at each iteration should have
nonnegativity constraints.

ALS 
setProductBlocks(int numProductBlocks)
Set the number of product blocks to parallelize the computation.

ALS 
setRank(int rank)
Set the rank of the feature matrices computed (number of features).

ALS 
setSeed(long seed)
Sets a random seed to have deterministic results.

ALS 
setUserBlocks(int numUserBlocks)
Set the number of user blocks to parallelize the computation.

double[] 
solveLeastSquares(org.jblas.DoubleMatrix ata,
org.jblas.DoubleMatrix atb,
NNLS.Workspace ws)
Given A^T A and A^T b, find the x minimising Ax  b_2, possibly subject
to nonnegativity constraints if
nonnegative is true. 
static MatrixFactorizationModel 
train(RDD<Rating> ratings,
int rank,
int iterations)
Train a matrix factorization model given an RDD of ratings given by users to some products,
in the form of (userID, productID, rating) pairs.

static MatrixFactorizationModel 
train(RDD<Rating> ratings,
int rank,
int iterations,
double lambda)
Train a matrix factorization model given an RDD of ratings given by users to some products,
in the form of (userID, productID, rating) pairs.

static MatrixFactorizationModel 
train(RDD<Rating> ratings,
int rank,
int iterations,
double lambda,
int blocks)
Train a matrix factorization model given an RDD of ratings given by users to some products,
in the form of (userID, productID, rating) pairs.

static MatrixFactorizationModel 
train(RDD<Rating> ratings,
int rank,
int iterations,
double lambda,
int blocks,
long seed)
Train a matrix factorization model given an RDD of ratings given by users to some products,
in the form of (userID, productID, rating) pairs.

static MatrixFactorizationModel 
trainImplicit(RDD<Rating> ratings,
int rank,
int iterations)
Train a matrix factorization model given an RDD of 'implicit preferences' ratings given by
users to some products, in the form of (userID, productID, rating) pairs.

static MatrixFactorizationModel 
trainImplicit(RDD<Rating> ratings,
int rank,
int iterations,
double lambda,
double alpha)
Train a matrix factorization model given an RDD of 'implicit preferences' given by users to
some products, in the form of (userID, productID, preference) pairs.

static MatrixFactorizationModel 
trainImplicit(RDD<Rating> ratings,
int rank,
int iterations,
double lambda,
int blocks,
double alpha)
Train a matrix factorization model given an RDD of 'implicit preferences' given by users
to some products, in the form of (userID, productID, preference) pairs.

static MatrixFactorizationModel 
trainImplicit(RDD<Rating> ratings,
int rank,
int iterations,
double lambda,
int blocks,
double alpha,
long seed)
Train a matrix factorization model given an RDD of 'implicit preferences' given by users
to some products, in the form of (userID, productID, preference) pairs.

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
initializeIfNecessary, initializeLogging, isTraceEnabled, log_, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logName, logTrace, logTrace, logWarning, logWarning
public ALS()
public static MatrixFactorizationModel train(RDD<Rating> ratings, int rank, int iterations, double lambda, int blocks, long seed)
blocks
.
ratings
 RDD of (userID, productID, rating) pairsrank
 number of features to useiterations
 number of iterations of ALS (recommended: 1020)lambda
 regularization factor (recommended: 0.01)blocks
 level of parallelism to split computation intoseed
 random seedpublic static MatrixFactorizationModel train(RDD<Rating> ratings, int rank, int iterations, double lambda, int blocks)
blocks
.
ratings
 RDD of (userID, productID, rating) pairsrank
 number of features to useiterations
 number of iterations of ALS (recommended: 1020)lambda
 regularization factor (recommended: 0.01)blocks
 level of parallelism to split computation intopublic static MatrixFactorizationModel train(RDD<Rating> ratings, int rank, int iterations, double lambda)
ratings
.
ratings
 RDD of (userID, productID, rating) pairsrank
 number of features to useiterations
 number of iterations of ALS (recommended: 1020)lambda
 regularization factor (recommended: 0.01)public static MatrixFactorizationModel train(RDD<Rating> ratings, int rank, int iterations)
ratings
.
ratings
 RDD of (userID, productID, rating) pairsrank
 number of features to useiterations
 number of iterations of ALS (recommended: 1020)public static MatrixFactorizationModel trainImplicit(RDD<Rating> ratings, int rank, int iterations, double lambda, int blocks, double alpha, long seed)
blocks
.
ratings
 RDD of (userID, productID, rating) pairsrank
 number of features to useiterations
 number of iterations of ALS (recommended: 1020)lambda
 regularization factor (recommended: 0.01)blocks
 level of parallelism to split computation intoalpha
 confidence parameterseed
 random seedpublic static MatrixFactorizationModel trainImplicit(RDD<Rating> ratings, int rank, int iterations, double lambda, int blocks, double alpha)
blocks
.
ratings
 RDD of (userID, productID, rating) pairsrank
 number of features to useiterations
 number of iterations of ALS (recommended: 1020)lambda
 regularization factor (recommended: 0.01)blocks
 level of parallelism to split computation intoalpha
 confidence parameterpublic static MatrixFactorizationModel trainImplicit(RDD<Rating> ratings, int rank, int iterations, double lambda, double alpha)
ratings
.
ratings
 RDD of (userID, productID, rating) pairsrank
 number of features to useiterations
 number of iterations of ALS (recommended: 1020)lambda
 regularization factor (recommended: 0.01)alpha
 confidence parameterpublic static MatrixFactorizationModel trainImplicit(RDD<Rating> ratings, int rank, int iterations)
ratings
.
Model parameters alpha
and lambda
are set to reasonable default values
ratings
 RDD of (userID, productID, rating) pairsrank
 number of features to useiterations
 number of iterations of ALS (recommended: 1020)public static ALS.BlockStats[] analyzeBlocks(RDD<Rating> ratings, int numUserBlocks, int numProductBlocks)
ratings
 an RDD of ratingsnumUserBlocks
 number of user blocksnumProductBlocks
 number of product blockspublic ALS setBlocks(int numBlocks)
public ALS setUserBlocks(int numUserBlocks)
public ALS setProductBlocks(int numProductBlocks)
public ALS setRank(int rank)
public ALS setIterations(int iterations)
public ALS setLambda(double lambda)
public ALS setImplicitPrefs(boolean implicitPrefs)
public ALS setAlpha(double alpha)
public ALS setSeed(long seed)
public ALS setNonnegative(boolean b)
public ALS setIntermediateRDDStorageLevel(StorageLevel storageLevel)
MEMORY_AND_DISK
. Users can change it to a serialized storage, e.g., MEMORY_AND_DISK_SER
and
set spark.rdd.compress
to true
to reduce the space requirement, at the cost of speed.public MatrixFactorizationModel run(RDD<Rating> ratings)
public MatrixFactorizationModel run(JavaRDD<Rating> ratings)
ALS.run
.public double[] solveLeastSquares(org.jblas.DoubleMatrix ata, org.jblas.DoubleMatrix atb, NNLS.Workspace ws)
nonnegative
is true.