Package

org.apache.spark

storage

Permalink

package storage

Visibility
  1. Public
  2. All

Type Members

  1. sealed abstract class BlockId extends AnyRef

    Permalink

    :: DeveloperApi :: Identifies a particular Block of data, usually associated with a single file.

    :: DeveloperApi :: Identifies a particular Block of data, usually associated with a single file. A Block can be uniquely identified by its filename, but each type of Block has a different set of keys which produce its unique name.

    If your BlockId should be serializable, be sure to add it to the BlockId.apply() method.

    Annotations
    @DeveloperApi()
  2. class BlockManagerId extends Externalizable

    Permalink

    :: DeveloperApi :: This class represent an unique identifier for a BlockManager.

    :: DeveloperApi :: This class represent an unique identifier for a BlockManager.

    The first 2 constructors of this class is made private to ensure that BlockManagerId objects can be created only using the apply method in the companion object. This allows de-duplication of ID objects. Also, constructor parameters are private to ensure that parameters cannot be modified from outside this class.

    Annotations
    @DeveloperApi()
  3. class BlockNotFoundException extends Exception

    Permalink
  4. case class BlockStatus(storageLevel: StorageLevel, memSize: Long, diskSize: Long) extends Product with Serializable

    Permalink
    Annotations
    @DeveloperApi()
  5. case class BlockUpdatedInfo(blockManagerId: BlockManagerId, blockId: BlockId, storageLevel: StorageLevel, memSize: Long, diskSize: Long) extends Product with Serializable

    Permalink

    :: DeveloperApi :: Stores information about a block status in a block manager.

    :: DeveloperApi :: Stores information about a block status in a block manager.

    Annotations
    @DeveloperApi()
  6. case class BroadcastBlockId(broadcastId: Long, field: String = "") extends BlockId with Product with Serializable

    Permalink
    Annotations
    @DeveloperApi()
  7. case class RDDBlockId(rddId: Int, splitIndex: Int) extends BlockId with Product with Serializable

    Permalink
    Annotations
    @DeveloperApi()
  8. class RDDInfo extends Ordered[RDDInfo]

    Permalink
    Annotations
    @DeveloperApi()
  9. case class ShuffleBlockId(shuffleId: Int, mapId: Int, reduceId: Int) extends BlockId with Product with Serializable

    Permalink
    Annotations
    @DeveloperApi()
  10. case class ShuffleDataBlockId(shuffleId: Int, mapId: Int, reduceId: Int) extends BlockId with Product with Serializable

    Permalink
    Annotations
    @DeveloperApi()
  11. case class ShuffleIndexBlockId(shuffleId: Int, mapId: Int, reduceId: Int) extends BlockId with Product with Serializable

    Permalink
    Annotations
    @DeveloperApi()
  12. class StorageLevel extends Externalizable

    Permalink

    :: DeveloperApi :: Flags for controlling the storage of an RDD.

    :: DeveloperApi :: Flags for controlling the storage of an RDD. Each StorageLevel records whether to use memory, or ExternalBlockStore, whether to drop the RDD to disk if it falls out of memory or ExternalBlockStore, whether to keep the data in memory in a serialized format, and whether to replicate the RDD partitions on multiple nodes.

    The org.apache.spark.storage.StorageLevel$ singleton object contains some static constants for commonly useful storage levels. To create your own storage level object, use the factory method of the singleton object (StorageLevel(...)).

    Annotations
    @DeveloperApi()
  13. class StorageStatus extends AnyRef

    Permalink

    :: DeveloperApi :: Storage information for each BlockManager.

    :: DeveloperApi :: Storage information for each BlockManager.

    This class assumes BlockId and BlockStatus are immutable, such that the consumers of this class cannot mutate the source of the information. Accesses are not thread-safe.

    Annotations
    @DeveloperApi()
  14. class StorageStatusListener extends SparkListener

    Permalink

    :: DeveloperApi :: A SparkListener that maintains executor storage status.

    :: DeveloperApi :: A SparkListener that maintains executor storage status.

    This class is thread-safe (unlike JobProgressListener)

    Annotations
    @DeveloperApi()
  15. case class StreamBlockId(streamId: Int, uniqueId: Long) extends BlockId with Product with Serializable

    Permalink
    Annotations
    @DeveloperApi()
  16. case class TaskResultBlockId(taskId: Long) extends BlockId with Product with Serializable

    Permalink
    Annotations
    @DeveloperApi()
  17. final class TimeTrackingOutputStream extends OutputStream

    Permalink

Value Members

  1. object BlockId

    Permalink
    Annotations
    @DeveloperApi()
  2. object BlockStatus extends Serializable

    Permalink
    Annotations
    @DeveloperApi()
  3. object StorageLevel extends Serializable

    Permalink

    Various org.apache.spark.storage.StorageLevel defined and utility functions for creating new storage levels.

Ungrouped