trait ExecutorPlugin extends AnyRef
:: DeveloperApi ::
Executor component of a SparkPlugin.
- Annotations
- @DeveloperApi()
- Source
- ExecutorPlugin.java
- Since
3.0.0
- Alphabetic
- By Inheritance
- ExecutorPlugin
- AnyRef
- Any
- Hide All
- Show All
- Public
- Protected
Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def ##: Int
- Definition Classes
- AnyRef → Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- def clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.CloneNotSupportedException]) @IntrinsicCandidate() @native()
- final def eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def equals(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef → Any
- final def getClass(): Class[_ <: AnyRef]
- Definition Classes
- AnyRef → Any
- Annotations
- @IntrinsicCandidate() @native()
- def hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @IntrinsicCandidate() @native()
- def init(ctx: PluginContext, extraConf: Map[String, String]): Unit
Initialize the executor plugin.
Initialize the executor plugin.
When a Spark plugin provides an executor plugin, this method will be called during the initialization of the executor process. It will block executor initialization until it returns.
Executor plugins that publish metrics should register all metrics with the context's registry (
PluginContext#metricRegistry()) when this method is called. Metrics registered afterwards are not guaranteed to show up.- ctx
Context information for the executor where the plugin is running.
- extraConf
Extra configuration provided by the driver component during its initialization.
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- final def ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- final def notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @IntrinsicCandidate() @native()
- final def notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @IntrinsicCandidate() @native()
- def onTaskFailed(failureReason: TaskFailedReason): Unit
Perform an action after tasks completes with exceptions.
Perform an action after tasks completes with exceptions.
Same warnings of
onTaskStartapply here.- failureReason
the exception thrown from the failed task.
- Since
3.1.0
- def onTaskStart(): Unit
Perform any action before the task is run.
Perform any action before the task is run.
This method is invoked from the same thread the task will be executed. Task-specific information can be accessed via
org.apache.spark.TaskContext#get.Plugin authors should avoid expensive operations here, as this method will be called on every task, and doing something expensive can significantly slow down a job. It is not recommended for a user to call a remote service, for example.
Exceptions thrown from this method do not propagate - they're caught, logged, and suppressed. Therefore exceptions when executing this method won't make the job fail.
- Since
3.1.0
- def onTaskSucceeded(): Unit
Perform an action after tasks completes without exceptions.
Perform an action after tasks completes without exceptions.
As
onTaskStartexceptions are suppressed, this method will still be invoked even if the corresponding#onTaskStartcall for this task failed.Same warnings of
onTaskStartapply here.- Since
3.1.0
- def shutdown(): Unit
Clean up and terminate this plugin.
Clean up and terminate this plugin.
This method is called during the executor shutdown phase, and blocks executor shutdown.
- final def synchronized[T0](arg0: => T0): T0
- Definition Classes
- AnyRef
- def toString(): String
- Definition Classes
- AnyRef → Any
- final def wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException]) @native()
- final def wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
Deprecated Value Members
- def finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.Throwable]) @Deprecated
- Deprecated
(Since version 9)