org.apache.spark.broadcast
Class TorrentBroadcastFactory

Object
  org.apache.spark.broadcast.TorrentBroadcastFactory

All Implemented Interfaces:

BroadcastFactory

public class TorrentBroadcastFactory
extends Object
implements BroadcastFactory

A Broadcast implementation that uses a BitTorrent-like protocol to do a distributed transfer of the broadcasted data to the executors. Refer to TorrentBroadcast for more details.

Constructor Summary

TorrentBroadcastFactory()

Method Summary

void	initialize(boolean isDriver, SparkConf conf, org.apache.spark.SecurityManager securityMgr)
<T> Broadcast<T>	newBroadcast(T value_, boolean isLocal, long id, scala.reflect.ClassTag<T> evidence$1) Creates a new broadcast variable.
void	stop()
void	unbroadcast(long id, boolean removeFromDriver, boolean blocking) Remove all persisted state associated with the torrent broadcast with the given ID.

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TorrentBroadcastFactory

public TorrentBroadcastFactory()

Method Detail

initialize

public void initialize(boolean isDriver,
                       SparkConf conf,
                       org.apache.spark.SecurityManager securityMgr)

Specified by:

initialize in interface BroadcastFactory

newBroadcast

public <T> Broadcast<T> newBroadcast(T value_,
                                     boolean isLocal,
                                     long id,
                                     scala.reflect.ClassTag<T> evidence$1)

Description copied from interface: BroadcastFactory

Creates a new broadcast variable.

Specified by:

newBroadcast in interface BroadcastFactory

Parameters:

value_ - value to broadcast

isLocal - whether we are in local mode (single JVM process)

id - unique id representing this broadcast variable

evidence$1 - (undocumented)

Returns:

(undocumented)

stop

public void stop()

Specified by:

stop in interface BroadcastFactory

unbroadcast

public void unbroadcast(long id,
                        boolean removeFromDriver,
                        boolean blocking)

Remove all persisted state associated with the torrent broadcast with the given ID.

Specified by:

unbroadcast in interface BroadcastFactory

Parameters:

removeFromDriver - Whether to remove state from the driver.

blocking - Whether to block until unbroadcasted

id - (undocumented)