Package org.apache.spark.rdd

Class InputFileBlockHolder

Object
org.apache.spark.rdd.InputFileBlockHolder
public class InputFileBlockHolder extends Object
This holds file names of the current Spark task. This is used in HadoopRDD, FileScanRDD, NewHadoopRDD and InputFileName function in Spark SQL.

  • Constructor Summary

    Constructors
    Constructor
    Description
    InputFileBlockHolder()
     

  • Method Summary

    Modifier and Type
    Method
    Description
    static org.apache.spark.unsafe.types.UTF8String
    getInputFilePath()
    Returns the holding file name or empty string if it is unknown.
    static long
    getLength()
    Returns the length of the block being read, or -1 if it is unknown.
    static long
    getStartOffset()
    Returns the starting offset of the block currently being read, or -1 if it is unknown.
    static void
    initialize()
    Initializes thread local by explicitly getting the value.
    static void
    set(String filePath, long startOffset, long length)
    Sets the thread-local input block.
    static void
    unset()
    Clears the input file block to default value.

    Methods inherited from class java.lang.Object

    equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Constructor Details

    • InputFileBlockHolder

      public InputFileBlockHolder()

  • Method Details

    • getInputFilePath

      public static org.apache.spark.unsafe.types.UTF8String getInputFilePath()
      Returns the holding file name or empty string if it is unknown.
      Returns:
      (undocumented)

    • getStartOffset

      public static long getStartOffset()
      Returns the starting offset of the block currently being read, or -1 if it is unknown.
      Returns:
      (undocumented)

    • getLength

      public static long getLength()
      Returns the length of the block being read, or -1 if it is unknown.
      Returns:
      (undocumented)

    • set

      public static void set(String filePath, long startOffset, long length)
      Sets the thread-local input block.

      Callers of this method must ensure a task completion listener has been registered to unset() the thread local in the task thread.

      Parameters:
      filePath - (undocumented)
      startOffset - (undocumented)
      length - (undocumented)

    • unset

      public static void unset()
      Clears the input file block to default value.

    • initialize

      public static void initialize()
      Initializes thread local by explicitly getting the value. It triggers ThreadLocal's initialValue in the parent thread.