Package org.apache.spark.api.plugin

Interface PluginContext

@DeveloperApi public interface PluginContext
:: DeveloperApi :: Context information and operations for plugins loaded by Spark.

An instance of this class is provided to plugins in their initialization method. It is safe for plugins to keep a reference to the instance for later use (for example, to send messages to the plugin's driver component).

Context instances are plugin-specific, so metrics and messages are tied each plugin. It is not possible for a plugin to directly interact with other plugins.

Since:
3.0.0

  • Method Summary

    Modifier and Type
    Method
    Description
    Object
    ask(Object message)
    Send an RPC to the plugin's driver-side component.
    SparkConf
    conf()
    Configuration of the Spark application.
    String
    executorID()
    Executor ID of the process.
    String
    hostname()
    The host name which is being used by the Spark process for communication.
    com.codahale.metrics.MetricRegistry
    metricRegistry()
    Registry where to register metrics published by the plugin associated with this context.
    Map<String,ResourceInformation>
    resources()
    The custom resources (GPUs, FPGAs, etc) allocated to driver or executor.
    void
    send(Object message)
    Send a message to the plugin's driver-side component.

  • Method Details

    • metricRegistry

      com.codahale.metrics.MetricRegistry metricRegistry()
      Registry where to register metrics published by the plugin associated with this context.

    • conf

      SparkConf conf()
      Configuration of the Spark application.

    • executorID

      String executorID()
      Executor ID of the process. On the driver, this will identify the driver.

    • hostname

      String hostname()
      The host name which is being used by the Spark process for communication.

    • resources

      Map<String,ResourceInformation> resources()
      The custom resources (GPUs, FPGAs, etc) allocated to driver or executor.

    • send

      void send(Object message) throws IOException
      Send a message to the plugin's driver-side component.

      This method sends a message to the driver-side component of the plugin, without expecting a reply. It returns as soon as the message is enqueued for sending.

      The message must be serializable.

      Parameters:
      message - Message to be sent.
      Throws:
      IOException

    • ask

      Object ask(Object message) throws Exception
      Send an RPC to the plugin's driver-side component.

      This method sends a message to the driver-side component of the plugin, and blocks until a reply arrives, or the configured RPC ask timeout (spark.rpc.askTimeout) elapses.

      If the driver replies with an error, an exception with the corresponding error will be thrown.

      The message must be serializable.

      Parameters:
      message - Message to be sent.
      Returns:
      The reply from the driver-side component.
      Throws:
      Exception