Package org.apache.spark.scheduler

Class StageInfo

Object
org.apache.spark.scheduler.StageInfo
public class StageInfo extends Object
:: DeveloperApi :: Stores information about a stage to pass from the scheduler to SparkListeners.

  • Constructor Summary

    Constructors
    Constructor
    Description
    StageInfo(int stageId, int attemptId, String name, int numTasks, scala.collection.immutable.Seq<RDDInfo> rddInfos, scala.collection.immutable.Seq<Object> parentIds, String details, org.apache.spark.executor.TaskMetrics taskMetrics, scala.collection.immutable.Seq<scala.collection.immutable.Seq<TaskLocation>> taskLocalityPreferences, scala.Option<Object> shuffleDepId, int resourceProfileId, boolean isShufflePushEnabled, int shuffleMergerCount)
     

  • Method Summary

    Modifier and Type
    Method
    Description
    scala.collection.mutable.HashMap<Object,AccumulableInfo>
    accumulables()
    Terminal values of accumulables updated during this stage, including all the user-defined accumulators.
    int
    attemptNumber()
     
    scala.Option<Object>
    completionTime()
    Time when the stage completed or when the stage was cancelled.
    String
    details()
     
    scala.Option<String>
    failureReason()
    If the stage failed, the reason why.
    static StageInfo
    fromStage(org.apache.spark.scheduler.Stage stage, int attemptId, scala.Option<Object> numTasks, org.apache.spark.executor.TaskMetrics taskMetrics, scala.collection.immutable.Seq<scala.collection.immutable.Seq<TaskLocation>> taskLocalityPreferences, int resourceProfileId)
    Construct a StageInfo from a Stage.
    String
    name()
     
    int
    numTasks()
     
    scala.collection.immutable.Seq<Object>
    parentIds()
     
    scala.collection.immutable.Seq<RDDInfo>
    rddInfos()
     
    int
    resourceProfileId()
     
    void
    stageFailed(String reason)
     
    int
    stageId()
     
    scala.Option<Object>
    submissionTime()
    When this stage was submitted from the DAGScheduler to a TaskScheduler.
    org.apache.spark.executor.TaskMetrics
    taskMetrics()
     

    Methods inherited from class java.lang.Object

    equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Constructor Details

    • StageInfo

      public StageInfo(int stageId, int attemptId, String name, int numTasks, scala.collection.immutable.Seq<RDDInfo> rddInfos, scala.collection.immutable.Seq<Object> parentIds, String details, org.apache.spark.executor.TaskMetrics taskMetrics, scala.collection.immutable.Seq<scala.collection.immutable.Seq<TaskLocation>> taskLocalityPreferences, scala.Option<Object> shuffleDepId, int resourceProfileId, boolean isShufflePushEnabled, int shuffleMergerCount)

  • Method Details

    • fromStage

      public static StageInfo fromStage(org.apache.spark.scheduler.Stage stage, int attemptId, scala.Option<Object> numTasks, org.apache.spark.executor.TaskMetrics taskMetrics, scala.collection.immutable.Seq<scala.collection.immutable.Seq<TaskLocation>> taskLocalityPreferences, int resourceProfileId)
      Construct a StageInfo from a Stage.

      Each Stage is associated with one or many RDDs, with the boundary of a Stage marked by shuffle dependencies. Therefore, all ancestor RDDs related to this Stage's RDD through a sequence of narrow dependencies should also be associated with this Stage.

      Parameters:
      stage - (undocumented)
      attemptId - (undocumented)
      numTasks - (undocumented)
      taskMetrics - (undocumented)
      taskLocalityPreferences - (undocumented)
      resourceProfileId - (undocumented)
      Returns:
      (undocumented)

    • stageId

      public int stageId()

    • name

      public String name()

    • numTasks

      public int numTasks()

    • rddInfos

      public scala.collection.immutable.Seq<RDDInfo> rddInfos()

    • parentIds

      public scala.collection.immutable.Seq<Object> parentIds()

    • details

      public String details()

    • taskMetrics

      public org.apache.spark.executor.TaskMetrics taskMetrics()

    • resourceProfileId

      public int resourceProfileId()

    • submissionTime

      public scala.Option<Object> submissionTime()
      When this stage was submitted from the DAGScheduler to a TaskScheduler.

    • completionTime

      public scala.Option<Object> completionTime()
      Time when the stage completed or when the stage was cancelled.

    • failureReason

      public scala.Option<String> failureReason()
      If the stage failed, the reason why.

    • accumulables

      public scala.collection.mutable.HashMap<Object,AccumulableInfo> accumulables()
      Terminal values of accumulables updated during this stage, including all the user-defined accumulators.
      Returns:
      (undocumented)

    • stageFailed

      public void stageFailed(String reason)

    • attemptNumber

      public int attemptNumber()