org.apache

spark

package spark

Visibility
  1. Public
  2. All

Type Members

  1. class Accumulable[R, T] extends Serializable

    A datatype that can be accumulated, i.

  2. trait AccumulableParam[R, T] extends Serializable

    Helper object defining how to accumulate values of a particular type.

  3. class Accumulator[T] extends Accumulable[T, T]

    A simpler value of Accumulable where the result type being accumulated is the same as the types of elements being merged.

  4. trait AccumulatorParam[T] extends AccumulableParam[T, T]

    A simpler version of AccumulableParam where the only datatype you can add in is the same type as the accumulated value.

  5. case class Aggregator[K, V, C](createCombiner: (V) ⇒ C, mergeValue: (C, V) ⇒ C, mergeCombiners: (C, C) ⇒ C) extends Product with Serializable

    A set of functions used to aggregate data.

  6. abstract class Dependency[T] extends Serializable

    Base class for dependencies.

  7. class HashPartitioner extends Partitioner

    A Partitioner that implements hash-based partitioning using Java's Object.hashCode.

  8. trait Logging extends AnyRef

    Utility trait for classes that want to log data.

  9. abstract class NarrowDependency[T] extends Dependency[T]

    Base class for dependencies where each partition of the parent RDD is used by at most one partition of the child RDD.

  10. class OneToOneDependency[T] extends NarrowDependency[T]

    Represents a one-to-one dependency between partitions of the parent and child RDDs.

  11. trait Partition extends Serializable

    A partition of an RDD.

  12. abstract class Partitioner extends Serializable

    An object that defines how the elements in a key-value pair RDD are partitioned by key.

  13. class RangeDependency[T] extends NarrowDependency[T]

    Represents a one-to-one dependency between ranges of partitions in the parent and child RDDs.

  14. class RangePartitioner[K, V] extends Partitioner

    A Partitioner that partitions sortable records by range into roughly equal ranges.

  15. class SerializableWritable[T <: Writable] extends Serializable

  16. class ShuffleDependency[K, V] extends Dependency[Product2[K, V]]

    Represents a dependency on the output of a shuffle stage.

  17. class SparkContext extends Logging

    Main entry point for Spark functionality.

  18. class SparkEnv extends AnyRef

    Holds all the runtime environment objects for a running Spark instance (either master or worker), including the serializer, Akka actor system, block manager, map output tracker, etc.

  19. class SparkException extends Exception

  20. class SparkFiles extends AnyRef

  21. class TaskContext extends Serializable

Value Members

  1. object Partitioner extends Serializable

  2. object SparkContext extends AnyRef

    The SparkContext object contains a number of implicit conversions and parameters for use with various Spark features.

  3. object SparkEnv extends Logging

  4. object SparkFiles extends

  5. package api

  6. package broadcast

  7. package deploy

  8. package executor

  9. package io

  10. package metrics

  11. package network

  12. package partial

  13. package rdd

  14. package scheduler

  15. package serializer

  16. package storage

  17. package util