Packages
Package
Description
Core Spark classes in Scala.
Spark Java programming APIs.
Set of interfaces to represent functions in Spark's Java API.
Spark's broadcast variables, used to broadcast immutable datasets to all nodes.
ALPHA COMPONENT
GraphX is a graph processing framework built on top of Spark.
Various analytics functions for graphs.
Collections of utilities used by graphx.
IO codecs used for compression.
Library for launching Spark applications programmatically.
DataFrame-based machine learning APIs to let users quickly assemble and configure practical
machine learning pipelines.
ML attributes
Feature transformers
The `ml.feature` package provides common feature transformers that help convert raw data or
features into more suitable forms for model fitting.
RDD-based machine learning APIs (in maintenance mode).
Provides implementation's of various RDDs.
Spark's DAG scheduler.
Pluggable serializers for RDD and shuffle data.
Allows the execution of relational queries, including those expressed in SQL using Spark.
Java APIs for spark streaming.
Various implementations of DStreams.
Spark utilities.
Utilities for random number generation.