class SparkStatusTracker extends AnyRef
Low-level status reporting APIs for monitoring job and stage progress.
These APIs intentionally provide very weak consistency semantics; consumers of these APIs should
be prepared to handle empty / missing information. For example, a job's stage ids may be known
but the status API may not have any information about the details of those stages, so
getStageInfo
could potentially return None
for a valid stage id.
To limit memory usage, these APIs only provide information on recent jobs / stages. These APIs
will provide information for the last spark.ui.retainedStages
stages and
spark.ui.retainedJobs
jobs.
NOTE: this class's constructor should be considered private and may be subject to change.
- Source
- SparkStatusTracker.scala
- Alphabetic
- By Inheritance
- SparkStatusTracker
- AnyRef
- Any
- Hide All
- Show All
- Public
- All
Value Members
-
final
def
!=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
##(): Int
- Definition Classes
- AnyRef → Any
-
final
def
==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
asInstanceOf[T0]: T0
- Definition Classes
- Any
-
def
clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( ... ) @native()
-
final
def
eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
def
equals(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
def
finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( classOf[java.lang.Throwable] )
-
def
getActiveJobIds(): Array[Int]
Returns an array containing the ids of all active jobs.
Returns an array containing the ids of all active jobs.
This method does not guarantee the order of the elements in its result.
-
def
getActiveStageIds(): Array[Int]
Returns an array containing the ids of all active stages.
Returns an array containing the ids of all active stages.
This method does not guarantee the order of the elements in its result.
-
final
def
getClass(): Class[_]
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
-
def
getExecutorInfos: Array[SparkExecutorInfo]
Returns information of all known executors, including host, port, cacheSize, numRunningTasks and memory metrics.
Returns information of all known executors, including host, port, cacheSize, numRunningTasks and memory metrics. Note this include information for both the driver and executors.
-
def
getJobIdsForGroup(jobGroup: String): Array[Int]
Return a list of all known jobs in a particular job group.
Return a list of all known jobs in a particular job group. If
jobGroup
isnull
, then returns all known jobs that are not associated with a job group.The returned list may contain running, failed, and completed jobs, and may vary across invocations of this method. This method does not guarantee the order of the elements in its result.
-
def
getJobInfo(jobId: Int): Option[SparkJobInfo]
Returns job information, or
None
if the job info could not be found or was garbage collected. -
def
getStageInfo(stageId: Int): Option[SparkStageInfo]
Returns stage information, or
None
if the stage info could not be found or was garbage collected. -
def
hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
-
final
def
isInstanceOf[T0]: Boolean
- Definition Classes
- Any
-
final
def
ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
final
def
notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
-
final
def
notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
-
final
def
synchronized[T0](arg0: ⇒ T0): T0
- Definition Classes
- AnyRef
-
def
toString(): String
- Definition Classes
- AnyRef → Any
-
final
def
wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... ) @native()