Class

org.apache.spark.storage

StorageStatus

Related Doc: package storage

Permalink

class StorageStatus extends AnyRef

:: DeveloperApi :: Storage information for each BlockManager.

This class assumes BlockId and BlockStatus are immutable, such that the consumers of this class cannot mutate the source of the information. Accesses are not thread-safe.

Annotations
@DeveloperApi()
Source
StorageUtils.scala
Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. StorageStatus
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new StorageStatus(bmid: BlockManagerId, maxMem: Long, initialBlocks: Map[BlockId, BlockStatus])

    Permalink

    Create a storage status with an initial set of blocks, leaving the source unmodified.

  2. new StorageStatus(blockManagerId: BlockManagerId, maxMem: Long)

    Permalink

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any
  5. val blockManagerId: BlockManagerId

    Permalink
  6. def blocks: Map[BlockId, BlockStatus]

    Permalink

    Return the blocks stored in this block manager.

    Return the blocks stored in this block manager.

    Note

    This is somewhat expensive, as it involves cloning the underlying maps and then concatenating them together. Much faster alternatives exist for common operations such as contains, get, and size.

  7. def cacheSize: Long

    Permalink

    Return the memory used by caching RDDs

  8. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  9. def containsBlock(blockId: BlockId): Boolean

    Permalink

    Return whether the given block is stored in this block manager in O(1) time.

    Return whether the given block is stored in this block manager in O(1) time.

    Note

    This is much faster than this.blocks.contains, which is O(blocks) time.

  10. def diskUsed: Long

    Permalink

    Return the disk space used by this block manager.

  11. def diskUsedByRdd(rddId: Int): Long

    Permalink

    Return the disk space used by the given RDD in this block manager in O(1) time.

  12. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  13. def equals(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  14. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  15. def getBlock(blockId: BlockId): Option[BlockStatus]

    Permalink

    Return the given block stored in this block manager in O(1) time.

    Return the given block stored in this block manager in O(1) time.

    Note

    This is much faster than this.blocks.get, which is O(blocks) time.

  16. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any
  17. def hashCode(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  18. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any
  19. val maxMem: Long

    Permalink
  20. def memRemaining: Long

    Permalink

    Return the memory remaining in this block manager.

  21. def memUsed: Long

    Permalink

    Return the memory used by this block manager.

  22. def memUsedByRdd(rddId: Int): Long

    Permalink

    Return the memory used by the given RDD in this block manager in O(1) time.

  23. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  24. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef
  25. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef
  26. def numBlocks: Int

    Permalink

    Return the number of blocks stored in this block manager in O(RDDs) time.

    Return the number of blocks stored in this block manager in O(RDDs) time.

    Note

    This is much faster than this.blocks.size, which is O(blocks) time.

  27. def numRddBlocks: Int

    Permalink

    Return the number of RDD blocks stored in this block manager in O(RDDs) time.

    Return the number of RDD blocks stored in this block manager in O(RDDs) time.

    Note

    This is much faster than this.rddBlocks.size, which is O(RDD blocks) time.

  28. def numRddBlocksById(rddId: Int): Int

    Permalink

    Return the number of blocks that belong to the given RDD in O(1) time.

    Return the number of blocks that belong to the given RDD in O(1) time.

    Note

    This is much faster than this.rddBlocksById(rddId).size, which is O(blocks in this RDD) time.

  29. def rddBlocks: Map[BlockId, BlockStatus]

    Permalink

    Return the RDD blocks stored in this block manager.

    Return the RDD blocks stored in this block manager.

    Note

    This is somewhat expensive, as it involves cloning the underlying maps and then concatenating them together. Much faster alternatives exist for common operations such as getting the memory, disk, and off-heap memory sizes occupied by this RDD.

  30. def rddBlocksById(rddId: Int): Map[BlockId, BlockStatus]

    Permalink

    Return the blocks that belong to the given RDD stored in this block manager.

  31. def rddStorageLevel(rddId: Int): Option[StorageLevel]

    Permalink

    Return the storage level, if any, used by the given RDD in this block manager.

  32. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef
  33. def toString(): String

    Permalink
    Definition Classes
    AnyRef → Any
  34. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  35. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  36. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from AnyRef

Inherited from Any

Ungrouped