| Interface | Description |
|---|---|
| DAGSchedulerEvent |
Types of events that can be handled by the DAGScheduler.
|
| ExternalClusterManager |
A cluster manager interface to plugin external scheduler.
|
| JobListener |
Interface used to listen for job completion or failure events after submitting a job to the
DAGScheduler.
|
| JobResult |
:: DeveloperApi ::
A result of a job in the DAGScheduler.
|
| MapStatus |
Result returned by a ShuffleMapTask to a scheduler.
|
| OutputCommitCoordinationMessage | |
| Schedulable |
An interface for schedulable entities.
|
| SchedulableBuilder |
An interface to build Schedulable tree
buildPools: build the tree nodes(pools)
addTaskSetManager: build the leaf nodes(TaskSetManagers)
|
| SchedulerBackend |
A backend interface for scheduling systems that allows plugging in different ones under
TaskSchedulerImpl.
|
| SchedulingAlgorithm |
An interface for sort algorithm
FIFO: FIFO algorithm between TaskSetManagers
FS: FS algorithm between Pools, and FIFO or FS within Pools
|
| ShuffleOutputStatus |
A common trait between
MapStatus and MergeStatus. |
| SparkListenerBus |
A
SparkListenerEvent bus that relays SparkListenerEvents to its listeners |
| SparkListenerEvent | |
| SparkListenerInterface |
Interface for listening to events from the Spark scheduler.
|
| TaskLocation |
A location where a task should run.
|
| TaskResult<T> | |
| TaskScheduler |
Low-level task scheduler interface, currently implemented exclusively by
TaskSchedulerImpl. |
| Class | Description |
|---|---|
| AccumulableInfo |
:: DeveloperApi ::
Information about an
AccumulatorV2 modified during a task or stage. |
| AllJobsCancelled | |
| AskPermissionToCommitOutput | |
| AsyncEventQueue |
An asynchronous queue for events.
|
| ExcludedExecutor | |
| ExecutorKilled | |
| ExecutorLossMessage | |
| InputFormatInfo |
:: DeveloperApi ::
Parses and holds information about inputFormat (and files) specified as a parameter.
|
| JobSucceeded | |
| LossReasonPending |
A loss reason that means we don't yet know why the executor exited.
|
| MiscellaneousProcessDetails |
:: DeveloperApi ::
Stores information about an Miscellaneous Process to pass from the scheduler to SparkListeners.
|
| ResubmitFailedStages | |
| RuntimePercentage | |
| SchedulingMode |
"FAIR" and "FIFO" determines which policy is used
to order tasks amongst a Schedulable's sub-queues
"NONE" is used when the a Schedulable has no sub-queues.
|
| SparkListener |
:: DeveloperApi ::
A default implementation for
SparkListenerInterface that has no-op implementations for
all callbacks. |
| SparkListenerApplicationEnd | |
| SparkListenerApplicationStart | |
| SparkListenerBlockManagerAdded | |
| SparkListenerBlockManagerRemoved | |
| SparkListenerBlockUpdated | |
| SparkListenerEnvironmentUpdate | |
| SparkListenerExecutorAdded | |
| SparkListenerExecutorBlacklisted | Deprecated
use SparkListenerExecutorExcluded instead.
|
| SparkListenerExecutorBlacklistedForStage | Deprecated
use SparkListenerExecutorExcludedForStage instead.
|
| SparkListenerExecutorExcluded | |
| SparkListenerExecutorExcludedForStage | |
| SparkListenerExecutorMetricsUpdate |
Periodic updates from executors.
|
| SparkListenerExecutorRemoved | |
| SparkListenerExecutorUnblacklisted | Deprecated
use SparkListenerExecutorUnexcluded instead.
|
| SparkListenerExecutorUnexcluded | |
| SparkListenerJobEnd | |
| SparkListenerJobStart | |
| SparkListenerLogStart |
An internal class that describes the metadata of an event log.
|
| SparkListenerMiscellaneousProcessAdded | |
| SparkListenerNodeBlacklisted | Deprecated
use SparkListenerNodeExcluded instead.
|
| SparkListenerNodeBlacklistedForStage | Deprecated
use SparkListenerNodeExcludedForStage instead.
|
| SparkListenerNodeExcluded | |
| SparkListenerNodeExcludedForStage | |
| SparkListenerNodeUnblacklisted | Deprecated
use SparkListenerNodeUnexcluded instead.
|
| SparkListenerNodeUnexcluded | |
| SparkListenerResourceProfileAdded | |
| SparkListenerSpeculativeTaskSubmitted | |
| SparkListenerStageCompleted | |
| SparkListenerStageExecutorMetrics |
Peak metric values for the executor for the stage, written to the history log at stage
completion.
|
| SparkListenerStageSubmitted | |
| SparkListenerTaskEnd | |
| SparkListenerTaskGettingResult | |
| SparkListenerTaskStart | |
| SparkListenerUnpersistRDD | |
| SparkListenerUnschedulableTaskSetAdded | |
| SparkListenerUnschedulableTaskSetRemoved | |
| SplitInfo | |
| StageInfo |
:: DeveloperApi ::
Stores information about a stage to pass from the scheduler to SparkListeners.
|
| StatsReportListener |
:: DeveloperApi ::
Simple SparkListener that logs a few summary statistics when each stage completes.
|
| StopCoordinator | |
| TaskInfo |
:: DeveloperApi ::
Information about a running task attempt inside a TaskSet.
|
| TaskLocality |