org.apache.spark.storage

Interface BlockReplicationPolicy

All Known Implementing Classes:

RandomBlockReplicationPolicy

public interface BlockReplicationPolicy

::DeveloperApi:: BlockReplicationPrioritization provides logic for prioritizing a sequence of peers for replicating blocks. BlockManager will replicate to each peer returned in order until the desired replication order is reached. If a replication fails, prioritize() will be called again to get a fresh prioritization.

Method Summary

Methods

Modifier and Type	Method and Description
scala.collection.immutable.List<BlockManagerId>	prioritize(BlockManagerId blockManagerId, scala.collection.Seq<BlockManagerId> peers, scala.collection.mutable.HashSet<BlockManagerId> peersReplicatedTo, BlockId blockId, int numReplicas) Method to prioritize a bunch of candidate peers of a block

Method Detail

prioritize

scala.collection.immutable.List<BlockManagerId> prioritize(BlockManagerId blockManagerId,
                                                         scala.collection.Seq<BlockManagerId> peers,
                                                         scala.collection.mutable.HashSet<BlockManagerId> peersReplicatedTo,
                                                         BlockId blockId,
                                                         int numReplicas)

Method to prioritize a bunch of candidate peers of a block

Parameters:

blockManagerId - Id of the current BlockManager for self identification

peers - A list of peers of a BlockManager

peersReplicatedTo - Set of peers already replicated to

blockId - BlockId of the block being replicated. This can be used as a source of randomness if needed.

numReplicas - Number of peers we need to replicate to

Returns:

A prioritized list of peers. Lower the index of a peer, higher its priority. This returns a list of size at most numPeersToReplicateTo.