Class StageInfo


public class StageInfo extends Object
:: DeveloperApi :: Stores information about a stage to pass from the scheduler to SparkListeners.
  • Constructor Details

    • StageInfo

      public StageInfo(int stageId, int attemptId, String name, int numTasks, scala.collection.Seq<RDDInfo> rddInfos, scala.collection.Seq<Object> parentIds, String details, org.apache.spark.executor.TaskMetrics taskMetrics, scala.collection.Seq<scala.collection.Seq<TaskLocation>> taskLocalityPreferences, scala.Option<Object> shuffleDepId, int resourceProfileId, boolean isShufflePushEnabled, int shuffleMergerCount)
  • Method Details

    • fromStage

      public static StageInfo fromStage(org.apache.spark.scheduler.Stage stage, int attemptId, scala.Option<Object> numTasks, org.apache.spark.executor.TaskMetrics taskMetrics, scala.collection.Seq<scala.collection.Seq<TaskLocation>> taskLocalityPreferences, int resourceProfileId)
      Construct a StageInfo from a Stage.

      Each Stage is associated with one or many RDDs, with the boundary of a Stage marked by shuffle dependencies. Therefore, all ancestor RDDs related to this Stage's RDD through a sequence of narrow dependencies should also be associated with this Stage.

      stage - (undocumented)
      attemptId - (undocumented)
      numTasks - (undocumented)
      taskMetrics - (undocumented)
      taskLocalityPreferences - (undocumented)
      resourceProfileId - (undocumented)
    • stageId

      public int stageId()
    • name

      public String name()
    • numTasks

      public int numTasks()
    • rddInfos

      public scala.collection.Seq<RDDInfo> rddInfos()
    • parentIds

      public scala.collection.Seq<Object> parentIds()
    • details

      public String details()
    • taskMetrics

      public org.apache.spark.executor.TaskMetrics taskMetrics()
    • resourceProfileId

      public int resourceProfileId()
    • submissionTime

      public scala.Option<Object> submissionTime()
      When this stage was submitted from the DAGScheduler to a TaskScheduler.
    • completionTime

      public scala.Option<Object> completionTime()
      Time when the stage completed or when the stage was cancelled.
    • failureReason

      public scala.Option<String> failureReason()
      If the stage failed, the reason why.
    • accumulables

      public scala.collection.mutable.HashMap<Object,AccumulableInfo> accumulables()
      Terminal values of accumulables updated during this stage, including all the user-defined accumulators.
    • stageFailed

      public void stageFailed(String reason)
    • attemptNumber

      public int attemptNumber()