Packages

o

org.apache.spark.mapred

SparkHadoopMapRedUtil

object SparkHadoopMapRedUtil extends Logging

Source
SparkHadoopMapRedUtil.scala
Linear Supertypes
Logging, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. SparkHadoopMapRedUtil
  2. Logging
  3. AnyRef
  4. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Value Members

  1. def commitTask(committer: OutputCommitter, mrTaskContext: TaskAttemptContext, jobId: Int, splitId: Int): Unit

    Commits a task output.

    Commits a task output. Before committing the task output, we need to know whether some other task attempt might be racing to commit the same output partition. Therefore, coordinate with the driver in order to determine whether this attempt can commit (please see SPARK-4879 for details).

    Output commit coordinator is only used when spark.hadoop.outputCommitCoordination.enabled is set to true (which is the default).