Package org.apache.spark.storage
package org.apache.spark.storage
-
ClassDescriptionAbstracts away how blocks are stored and provides different ways to read the underlying block data.:: DeveloperApi :: Identifies a particular Block of data, usually associated with a single file.:: DeveloperApi :: This class represent a unique identifier for a BlockManager.The response message of
GetLocationsAndStatusrequest.Driver to Executor message to get a heap histogram.Driver to Executor message to trigger a thread dump.::DeveloperApi:: BlockReplicationPrioritization provides logic for prioritizing a sequence of peers for replicating blocks.:: DeveloperApi :: Stores information about a block status in a block manager.Helper class that ensures a ManagedBuffer is released upon InputStream.close() and also detects stream corruption if streamCompressedOrEncrypted is trueA TopologyMapper that assumes all nodes are in the same rackA simple file based topology mapper.Identifies a block of log data.LogBlockIdGenerator is responsible for generating unique LogBlockIds for log blocks.Base class representing a log line.Helper class forShuffleBlockFetcherIteratorthat encapsulates all the push-based functionality to fetch push-merged block meta and shuffle chunks.A listener to be called at the completion of the ShuffleBlockFetcherIterator param: data the ShuffleBlockFetcherIterator to process:: DeveloperApi :: Flags for controlling the storage of an RDD.A mapper class easy to obtain storage levels based on their names.Helper methods for storage-related objects.Intercepts write calls and tracks total time spent writing in order to update shuffle write metrics.::DeveloperApi:: TopologyMapper provides topology information for a given host param: conf SparkConf to get required properties, if needed