Run Limited-memory BFGS (L-BFGS) in parallel.
Run Limited-memory BFGS (L-BFGS) in parallel. Averaging the subgradients over different partitions is performed using one standard spark map-reduce in each iteration.
- Input data for L-BFGS. RDD of the set of data examples, each of the form (label, [feature values]).
- Gradient object (used to compute the gradient of the loss function of one single data example)
- Updater function to actually perform a gradient step in a given direction.
- The number of corrections used in the L-BFGS update.
- The convergence tolerance of iterations for L-BFGS which is must be nonnegative. Lower values are less tolerant and therefore generally cause more iterations to be run.
- Maximal number of iterations that L-BFGS can be run.
- Regularization parameter
A tuple containing two elements. The first element is a column matrix containing weights for every feature, and the second element is an array containing the loss computed for every iteration.
:: DeveloperApi :: Top-level method to run L-BFGS.