Package org.apache.spark.scheduler

Class StageInfo

Object

org.apache.spark.scheduler.StageInfo

public class StageInfo
extends Object

:: DeveloperApi :: Stores information about a stage to pass from the scheduler to SparkListeners.

Constructor Summary

Constructors

Constructor

Description

StageInfo(int stageId, int attemptId, String name, int numTasks, scala.collection.Seq<RDDInfo> rddInfos, scala.collection.Seq<Object> parentIds, String details, org.apache.spark.executor.TaskMetrics taskMetrics, scala.collection.Seq<scala.collection.Seq<TaskLocation>> taskLocalityPreferences, scala.Option<Object> shuffleDepId, int resourceProfileId, boolean isShufflePushEnabled, int shuffleMergerCount)

Method Summary

Modifier and Type	Method	Description
scala.collection.mutable.HashMap<Object,AccumulableInfo>	accumulables()	Terminal values of accumulables updated during this stage, including all the user-defined accumulators.
int	attemptNumber()
scala.Option<Object>	completionTime()	Time when the stage completed or when the stage was cancelled.
String	details()
scala.Option<String>	failureReason()	If the stage failed, the reason why.
static StageInfo	fromStage(org.apache.spark.scheduler.Stage stage, int attemptId, scala.Option<Object> numTasks, org.apache.spark.executor.TaskMetrics taskMetrics, scala.collection.Seq<scala.collection.Seq<TaskLocation>> taskLocalityPreferences, int resourceProfileId)	Construct a StageInfo from a Stage.
String	name()
int	numTasks()
scala.collection.Seq<Object>	parentIds()
scala.collection.Seq<RDDInfo>	rddInfos()
int	resourceProfileId()
void	stageFailed(String reason)
int	stageId()
scala.Option<Object>	submissionTime()	When this stage was submitted from the DAGScheduler to a TaskScheduler.
org.apache.spark.executor.TaskMetrics	taskMetrics()

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

StageInfo

public StageInfo(int stageId,
 int attemptId,
 String name,
 int numTasks,
 scala.collection.Seq<RDDInfo> rddInfos,
 scala.collection.Seq<Object> parentIds,
 String details,
 org.apache.spark.executor.TaskMetrics taskMetrics,
 scala.collection.Seq<scala.collection.Seq<TaskLocation>> taskLocalityPreferences,
 scala.Option<Object> shuffleDepId,
 int resourceProfileId,
 boolean isShufflePushEnabled,
 int shuffleMergerCount)

Method Details

fromStage

public static StageInfo fromStage(org.apache.spark.scheduler.Stage stage,
 int attemptId,
 scala.Option<Object> numTasks,
 org.apache.spark.executor.TaskMetrics taskMetrics,
 scala.collection.Seq<scala.collection.Seq<TaskLocation>> taskLocalityPreferences,
 int resourceProfileId)

Construct a StageInfo from a Stage.

Each Stage is associated with one or many RDDs, with the boundary of a Stage marked by shuffle dependencies. Therefore, all ancestor RDDs related to this Stage's RDD through a sequence of narrow dependencies should also be associated with this Stage.

Parameters:

stage - (undocumented)

attemptId - (undocumented)

numTasks - (undocumented)

taskMetrics - (undocumented)

taskLocalityPreferences - (undocumented)

resourceProfileId - (undocumented)

Returns:

(undocumented)

stageId

public int stageId()

name

public String name()

numTasks

public int numTasks()

rddInfos

public scala.collection.Seq<RDDInfo> rddInfos()

parentIds

public scala.collection.Seq<Object> parentIds()

details

public String details()

taskMetrics

public org.apache.spark.executor.TaskMetrics taskMetrics()

resourceProfileId

public int resourceProfileId()

submissionTime

public scala.Option<Object> submissionTime()

When this stage was submitted from the DAGScheduler to a TaskScheduler.

completionTime

public scala.Option<Object> completionTime()

Time when the stage completed or when the stage was cancelled.

failureReason

public scala.Option<String> failureReason()

If the stage failed, the reason why.

accumulables

public scala.collection.mutable.HashMap<Object,AccumulableInfo> accumulables()

Terminal values of accumulables updated during this stage, including all the user-defined accumulators.

Returns:

(undocumented)

stageFailed

public void stageFailed(String reason)

attemptNumber

public int attemptNumber()