Usable in Java, Scala, Python, and SparkR.
MLlib fits into Spark's APIs and interoperates with NumPy in Python (starting in Spark 0.9). You can use any Hadoop data source (e.g. HDFS, HBase, or local files), making it easy to plug into Hadoop workflows.
High-quality algorithms, 100x faster than MapReduce.
Spark excels at iterative computation, enabling MLlib to run fast. At the same time, we care about algorithmic performance: MLlib contains high-quality algorithms that leverage iteration, and can yield better results than the one-pass approximations sometimes used on MapReduce.
MLlib contains the following algorithms and utilities:
Refer to the MLlib guide for usage examples.
MLlib is developed as part of the Apache Spark project. It thus gets tested and updated with each Spark release.
If you have questions about the library, ask on the Spark mailing lists.
MLlib is still a young project and welcomes contributions. If you'd like to submit an algorithm to MLlib, read how to contribute to Spark and send us a patch!