org.apache.spark.network.netty

NettyBlockTransferService

class NettyBlockTransferService extends BlockTransferService

A BlockTransferService that uses Netty to fetch a set of blocks at at time.

Linear Supertypes
BlockTransferService, Logging, ShuffleClient, Closeable, AutoCloseable, AnyRef, Any
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. NettyBlockTransferService
  2. BlockTransferService
  3. Logging
  4. ShuffleClient
  5. Closeable
  6. AutoCloseable
  7. AnyRef
  8. Any
  1. Hide All
  2. Show all
Learn more about member selection
Visibility
  1. Public
  2. All

Instance Constructors

  1. new NettyBlockTransferService(conf: SparkConf, securityManager: SecurityManager, numCores: Int)

Value Members

  1. final def !=(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  2. final def !=(arg0: Any): Boolean

    Definition Classes
    Any
  3. final def ##(): Int

    Definition Classes
    AnyRef → Any
  4. final def ==(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  5. final def ==(arg0: Any): Boolean

    Definition Classes
    Any
  6. final def asInstanceOf[T0]: T0

    Definition Classes
    Any
  7. def clone(): AnyRef

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  8. def close(): Unit

    Tear down the transfer service.

    Tear down the transfer service.

    Definition Classes
    NettyBlockTransferService → BlockTransferService → Closeable → AutoCloseable
  9. final def eq(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  10. def equals(arg0: Any): Boolean

    Definition Classes
    AnyRef → Any
  11. def fetchBlockSync(host: String, port: Int, execId: String, blockId: String): ManagedBuffer

    A special case of fetchBlocks, as it fetches only one block and is blocking.

    A special case of fetchBlocks, as it fetches only one block and is blocking.

    It is also only available after init is invoked.

    Definition Classes
    BlockTransferService
  12. def fetchBlocks(host: String, port: Int, execId: String, blockIds: Array[String], listener: BlockFetchingListener): Unit

    Fetch a sequence of blocks from a remote node asynchronously, available only after init is invoked.

    Fetch a sequence of blocks from a remote node asynchronously, available only after init is invoked.

    Note that this API takes a sequence so the implementation can batch requests, and does not return a future so the underlying implementation can invoke onBlockFetchSuccess as soon as the data of a block is fetched, rather than waiting for all blocks to be fetched.

    Definition Classes
    NettyBlockTransferService → BlockTransferService → ShuffleClient
  13. def finalize(): Unit

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  14. final def getClass(): Class[_]

    Definition Classes
    AnyRef → Any
  15. def hashCode(): Int

    Definition Classes
    AnyRef → Any
  16. def hostName: String

    Host name the service is listening on, available only after init is invoked.

    Host name the service is listening on, available only after init is invoked.

    Definition Classes
    NettyBlockTransferService → BlockTransferService
  17. def init(blockDataManager: BlockDataManager): Unit

    Initialize the transfer service by giving it the BlockDataManager that can be used to fetch local blocks or put local blocks.

    Initialize the transfer service by giving it the BlockDataManager that can be used to fetch local blocks or put local blocks.

    Definition Classes
    NettyBlockTransferService → BlockTransferService
  18. def init(appId: String): Unit

    Definition Classes
    ShuffleClient
  19. final def isInstanceOf[T0]: Boolean

    Definition Classes
    Any
  20. def isTraceEnabled(): Boolean

    Attributes
    protected
    Definition Classes
    Logging
  21. def log: Logger

    Attributes
    protected
    Definition Classes
    Logging
  22. def logDebug(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging
  23. def logDebug(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging
  24. def logError(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging
  25. def logError(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging
  26. def logInfo(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging
  27. def logInfo(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging
  28. def logName: String

    Attributes
    protected
    Definition Classes
    Logging
  29. def logTrace(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging
  30. def logTrace(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging
  31. def logWarning(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging
  32. def logWarning(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging
  33. final def ne(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  34. final def notify(): Unit

    Definition Classes
    AnyRef
  35. final def notifyAll(): Unit

    Definition Classes
    AnyRef
  36. def port: Int

    Port number the service is listening on, available only after init is invoked.

    Port number the service is listening on, available only after init is invoked.

    Definition Classes
    NettyBlockTransferService → BlockTransferService
  37. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes
    AnyRef
  38. def toString(): String

    Definition Classes
    AnyRef → Any
  39. def uploadBlock(hostname: String, port: Int, execId: String, blockId: BlockId, blockData: ManagedBuffer, level: StorageLevel): Future[Unit]

    Upload a single block to a remote node, available only after init is invoked.

    Upload a single block to a remote node, available only after init is invoked.

    Definition Classes
    NettyBlockTransferService → BlockTransferService
  40. def uploadBlockSync(hostname: String, port: Int, execId: String, blockId: BlockId, blockData: ManagedBuffer, level: StorageLevel): Unit

    Upload a single block to a remote node, available only after init is invoked.

    Upload a single block to a remote node, available only after init is invoked.

    This method is similar to uploadBlock, except this one blocks the thread until the upload finishes.

    Definition Classes
    BlockTransferService
  41. final def wait(): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  42. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  43. final def wait(arg0: Long): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from BlockTransferService

Inherited from Logging

Inherited from ShuffleClient

Inherited from Closeable

Inherited from AutoCloseable

Inherited from AnyRef

Inherited from Any

Ungrouped