Package org.apache.spark.storage

Interface BlockReplicationPolicy

All Known Implementing Classes:
BasicBlockReplicationPolicy, RandomBlockReplicationPolicy
public interface BlockReplicationPolicy
::DeveloperApi:: BlockReplicationPrioritization provides logic for prioritizing a sequence of peers for replicating blocks. BlockManager will replicate to each peer returned in order until the desired replication order is reached. If a replication fails, prioritize() will be called again to get a fresh prioritization.

  • Method Summary

    Modifier and Type
    Method
    Description
    scala.collection.immutable.List<BlockManagerId>
    prioritize(BlockManagerId blockManagerId, scala.collection.Seq<BlockManagerId> peers, scala.collection.mutable.HashSet<BlockManagerId> peersReplicatedTo, BlockId blockId, int numReplicas)
    Method to prioritize a bunch of candidate peers of a block

  • Method Details

    • prioritize

      scala.collection.immutable.List<BlockManagerId> prioritize(BlockManagerId blockManagerId, scala.collection.Seq<BlockManagerId> peers, scala.collection.mutable.HashSet<BlockManagerId> peersReplicatedTo, BlockId blockId, int numReplicas)
      Method to prioritize a bunch of candidate peers of a block

      Parameters:
      blockManagerId - Id of the current BlockManager for self identification
      peers - A list of peers of a BlockManager
      peersReplicatedTo - Set of peers already replicated to
      blockId - BlockId of the block being replicated. This can be used as a source of randomness if needed.
      numReplicas - Number of peers we need to replicate to
      Returns:
      A prioritized list of peers. Lower the index of a peer, higher its priority. This returns a list of size at most numPeersToReplicateTo.