Packages

package java

Spark Java programming APIs.

Source
package.scala
Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. java
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. Protected

Package Members

  1. package function

    Set of interfaces to represent functions in Spark's Java API.

    Set of interfaces to represent functions in Spark's Java API. Users create implementations of these interfaces to pass functions to various Java API methods for Spark. Please visit Spark's Java programming guide for more details.

Type Members

  1. class JavaDoubleRDD extends AbstractJavaRDDLike[Double, JavaDoubleRDD]
  2. trait JavaFutureAction[T] extends Future[T]
  3. class JavaHadoopRDD[K, V] extends JavaPairRDD[K, V]
    Annotations
    @DeveloperApi()
  4. class JavaNewHadoopRDD[K, V] extends JavaPairRDD[K, V]
    Annotations
    @DeveloperApi()
  5. class JavaPairRDD[K, V] extends AbstractJavaRDDLike[(K, V), JavaPairRDD[K, V]]
  6. class JavaRDD[T] extends AbstractJavaRDDLike[T, JavaRDD[T]]
  7. trait JavaRDDLike[T, This <: JavaRDDLike[T, This]] extends Serializable

    Defines operations common to several Java RDD implementations.

    Defines operations common to several Java RDD implementations.

    Note

    This trait is not intended to be implemented by user code.

  8. class JavaSparkContext extends Closeable

    A Java-friendly version of org.apache.spark.SparkContext that returns org.apache.spark.api.java.JavaRDDs and works with Java collections instead of Scala ones.

    A Java-friendly version of org.apache.spark.SparkContext that returns org.apache.spark.api.java.JavaRDDs and works with Java collections instead of Scala ones.

    Note

    Only one SparkContext should be active per JVM. You must stop() the active SparkContext before creating a new one.

  9. class JavaSparkStatusTracker extends AnyRef

    Low-level status reporting APIs for monitoring job and stage progress.

    Low-level status reporting APIs for monitoring job and stage progress.

    These APIs intentionally provide very weak consistency semantics; consumers of these APIs should be prepared to handle empty / missing information. For example, a job's stage ids may be known but the status API may not have any information about the details of those stages, so getStageInfo could potentially return null for a valid stage id.

    To limit memory usage, these APIs only provide information on recent jobs / stages. These APIs will provide information for the last spark.ui.retainedStages stages and spark.ui.retainedJobs jobs.

    Note

    This class's constructor should be considered private and may be subject to change.

  10. final class Optional[T] extends Serializable

    Like java.util.Optional in Java 8, scala.Option in Scala, and com.google.common.base.Optional in Google Guava, this class represents a value of a given type that may or may not exist.

    Like java.util.Optional in Java 8, scala.Option in Scala, and com.google.common.base.Optional in Google Guava, this class represents a value of a given type that may or may not exist. It is used in methods that wish to optionally return a value, in preference to returning null.

    In fact, the class here is a reimplementation of the essential API of both java.util.Optional and com.google.common.base.Optional. From java.util.Optional, it implements:

    • #empty()
    • #of(Object)
    • #ofNullable(Object)
    • #get()
    • #orElse(Object)
    • #isPresent()

    From com.google.common.base.Optional it implements:

    • #absent()
    • #of(Object)
    • #fromNullable(Object)
    • #get()
    • #or(Object)
    • #orNull()
    • #isPresent()

    java.util.Optional itself was not used because at the time, the project did not require Java 8. Using com.google.common.base.Optional has in the past caused serious library version conflicts with Guava that can't be resolved by shading. Hence this work-alike clone.

  11. class StorageLevels extends AnyRef

    Expose some commonly useful storage level constants.

Value Members

  1. object JavaDoubleRDD extends Serializable
  2. object JavaPairRDD extends Serializable
  3. object JavaRDD extends Serializable
  4. object JavaSparkContext

Inherited from AnyRef

Inherited from Any

Ungrouped