Param for the alpha parameter in the implicit preference formulation.
Param for the alpha parameter in the implicit preference formulation.
Check whether the given schema contains an input column.
Check whether the given schema contains an input column.
Parameter name for the input column.
SQL DataType of the input column.
param for checkpoint interval
param for checkpoint interval
Returns the documentation of all params.
Returns the documentation of all params.
Fits a single model to the input data with provided parameter map.
Fits multiple models to the input data with multiple sets of parameters.
Fits multiple models to the input data with multiple sets of parameters. The default implementation uses a for loop on each parameter map. Subclasses could overwrite this to optimize multi-model training.
input dataset
An array of parameter maps. These values override any specified in this Estimator's embedded ParamMap.
fitted models, matching the input parameter maps
Fits a single model to the input data with optional parameters.
Fits a single model to the input data with optional parameters.
input dataset
Optional list of param pairs. These values override any specified in this Estimator's embedded ParamMap.
fitted model
Gets the value of a parameter in the embedded param map.
Gets the value of a parameter in the embedded param map.
Param to decide whether to use implicit preference.
Param to decide whether to use implicit preference.
Checks whether a param is explicitly set.
Checks whether a param is explicitly set.
Param for the column name for item ids.
Param for the column name for item ids.
param for max number of iterations
param for max number of iterations
Param for whether to apply nonnegativity constraints.
Param for whether to apply nonnegativity constraints.
Param for number of item blocks.
Param for number of item blocks.
Param for number of user blocks.
Param for number of user blocks.
Internal param map.
Internal param map.
Returns all params.
Returns all params.
param for prediction column name
param for prediction column name
Param for rank of the matrix factorization.
Param for rank of the matrix factorization.
Param for the column name for ratings.
Param for the column name for ratings.
param for regularization parameter
param for regularization parameter
Sets a parameter in the embedded param map.
Sets a parameter in the embedded param map.
Sets both numUserBlocks and numItemBlocks to the specific value.
:: DeveloperApi ::
:: DeveloperApi ::
Derives the output schema from the input schema and parameters. The schema describes the columns and types of the data.
Input schema to this stage
Parameters passed to this stage
Output schema from this stage
Derives the output schema from the input schema and parameters, optionally with logging.
Derives the output schema from the input schema and parameters, optionally with logging.
Param for the column name for user ids.
Param for the column name for user ids.
Validates parameter values stored internally.
Validates parameter values stored internally. Raise an exception if any parameter value is invalid.
Validates parameter values stored internally plus the input parameter map.
Validates parameter values stored internally plus the input parameter map. Raises an exception if any parameter is invalid.
Validates and transforms the input schema.
Validates and transforms the input schema.
input schema
extra params
output schema
A list of (hyper-)parameter keys this algorithm can take. Users can set and get the parameter values through setters and getters, respectively.
Alternating Least Squares (ALS) matrix factorization.
ALS attempts to estimate the ratings matrix
R
as the product of two lower-rank matrices,X
andY
, i.e.X * Yt = R
. Typically these approximations are called 'factor' matrices. The general approach is iterative. During each iteration, one of the factor matrices is held constant, while the other is solved for using least squares. The newly-solved factor matrix is then held constant while solving for the other factor matrix.This is a blocked implementation of the ALS factorization algorithm that groups the two sets of factors (referred to as "users" and "products") into blocks and reduces communication by only sending one copy of each user vector to each product block on each iteration, and only for the product blocks that need that user's feature vector. This is achieved by pre-computing some information about the ratings matrix to determine the "out-links" of each user (which blocks of products it will contribute to) and "in-link" information for each product (which of the feature vectors it receives from each user block it will depend on). This allows us to send only an array of feature vectors between each user block and product block, and have the product block find the users' ratings and update the products based on these messages.
For implicit preference data, the algorithm used is based on "Collaborative Filtering for Implicit Feedback Datasets", available at http://dx.doi.org/10.1109/ICDM.2008.22, adapted for the blocked approach used here.
Essentially instead of finding the low-rank approximations to the rating matrix
R
, this finds the approximations for a preference matrixP
where the elements ofP
are 1 if r > 0 and 0 if r <= 0. The ratings then act as 'confidence' values related to strength of indicated user preferences rather than explicit ratings given to items.