Package org.apache.spark.util

Interface SparkFileUtils

All Superinterfaces:
org.apache.spark.internal.Logging
public interface SparkFileUtils extends org.apache.spark.internal.Logging

  • Nested Class Summary

    Nested classes/interfaces inherited from interface org.apache.spark.internal.Logging

    org.apache.spark.internal.Logging.LogStringContext, org.apache.spark.internal.Logging.SparkShellLoggingFilter

  • Method Summary

    Modifier and Type
    Method
    Description
    void
    cleanDirectory(File dir)
    Delete recursively while keeping the given directory itself.
    boolean
    contentEquals(File file1, File file2)
    Return true if the content of the files are equal or they both don't exist
    void
    copyDirectory(File src, File dir)
    Copy src to the target directory simply.
    void
    copyFile(File src, File dst)
     
    void
    copyFileToDirectory(File file, File dir)
    Copy file to the target directory simply.
    void
    copyURLToFile(URL url, File file)
     
    boolean
    createDirectory(File dir)
    Create a directory given the abstract pathname
    File
    createDirectory(String root, String namePrefix)
    Create a directory inside the given parent directory.
    void
    createParentDirs(File file)
     
    File
    createTempDir()
    Create a temporary directory inside the java.io.tmpdir prefixed with spark.
    File
    createTempDir(String root, String namePrefix)
    Create a temporary directory inside the given parent directory.
    void
    deleteQuietly(File file)
    Delete a file or directory and its contents recursively without throwing exceptions.
    void
    deleteRecursively(File file)
    Delete a file or directory and its contents recursively.
    void
    forceDeleteOnExit(File file)
    Registers the file or directory for deletion when the JVM exists.
    File
    getFile(File parent, scala.collection.immutable.Seq<String> names)
     
    File
    getFile(scala.collection.immutable.Seq<String> names)
     
    Set<File>
    listFiles(File f)
    Lists regular files recursively.
    Set<Path>
    listPaths(File f)
    Lists regular paths recursively.
    void
    moveDirectory(File src, File dst)
    Move src to dst simply.
    void
    moveFile(File src, File dst)
    Move src to dst simply.
    File[]
    recursiveList(File f)
    Lists files recursively.
    URI
    resolveURI(String path)
    Return a well-formed URI for the file described by a user input string.
    long
    sizeOf(File f)
    Size of files recursively.
    void
    touch(File file)
     

    Methods inherited from interface org.apache.spark.internal.Logging

    initializeForcefully, initializeLogIfNecessary, initializeLogIfNecessary, initializeLogIfNecessary$default$2, isTraceEnabled, log, logBasedOnLevel, logDebug, logDebug, logDebug, logDebug, logError, logError, logError, logError, logInfo, logInfo, logInfo, logInfo, logName, LogStringContext, logTrace, logTrace, logTrace, logTrace, logWarning, logWarning, logWarning, logWarning, MDC, org$apache$spark$internal$Logging$$log_, org$apache$spark$internal$Logging$$log__$eq, withLogContext

  • Method Details

    • resolveURI

      URI resolveURI(String path)
      Return a well-formed URI for the file described by a user input string.

      If the supplied path does not contain a scheme, or is a relative path, it will be converted into an absolute path with a file:// scheme.

      Parameters:
      path - (undocumented)
      Returns:
      (undocumented)

    • sizeOf

      long sizeOf(File f)
      Size of files recursively.
      Parameters:
      f - (undocumented)
      Returns:
      (undocumented)

    • recursiveList

      File[] recursiveList(File f)
      Lists files recursively.
      Parameters:
      f - (undocumented)
      Returns:
      (undocumented)

    • listFiles

      Set<File> listFiles(File f)
      Lists regular files recursively.
      Parameters:
      f - (undocumented)
      Returns:
      (undocumented)

    • listPaths

      Set<Path> listPaths(File f)
      Lists regular paths recursively.
      Parameters:
      f - (undocumented)
      Returns:
      (undocumented)

    • createDirectory

      boolean createDirectory(File dir)
      Create a directory given the abstract pathname
      Parameters:
      dir - (undocumented)
      Returns:
      true, if the directory is successfully created; otherwise, return false.

    • createDirectory

      File createDirectory(String root, String namePrefix)
      Create a directory inside the given parent directory. The directory is guaranteed to be newly created, and is not marked for automatic deletion.
      Parameters:
      root - (undocumented)
      namePrefix - (undocumented)
      Returns:
      (undocumented)

    • createTempDir

      File createTempDir()
      Create a temporary directory inside the java.io.tmpdir prefixed with spark. The directory will be automatically deleted when the VM shuts down.
      Returns:
      (undocumented)

    • createTempDir

      File createTempDir(String root, String namePrefix)
      Create a temporary directory inside the given parent directory. The directory will be automatically deleted when the VM shuts down.
      Parameters:
      root - (undocumented)
      namePrefix - (undocumented)
      Returns:
      (undocumented)

    • createParentDirs

      void createParentDirs(File file)

    • cleanDirectory

      void cleanDirectory(File dir)
      Delete recursively while keeping the given directory itself.

    • deleteRecursively

      void deleteRecursively(File file)
      Delete a file or directory and its contents recursively. Don't follow directories if they are symlinks. Throws an exception if deletion is unsuccessful.
      Parameters:
      file - (undocumented)

    • deleteQuietly

      void deleteQuietly(File file)
      Delete a file or directory and its contents recursively without throwing exceptions.

    • forceDeleteOnExit

      void forceDeleteOnExit(File file)
      Registers the file or directory for deletion when the JVM exists.

    • getFile

      File getFile(scala.collection.immutable.Seq<String> names)

    • getFile

      File getFile(File parent, scala.collection.immutable.Seq<String> names)

    • moveFile

      void moveFile(File src, File dst)
      Move src to dst simply. File attribute times are not copied.

    • moveDirectory

      void moveDirectory(File src, File dst)
      Move src to dst simply. File attribute times are not copied.

    • copyDirectory

      void copyDirectory(File src, File dir)
      Copy src to the target directory simply. File attribute times are not copied.

    • copyFileToDirectory

      void copyFileToDirectory(File file, File dir)
      Copy file to the target directory simply. File attribute times are not copied.

    • copyFile

      void copyFile(File src, File dst)

    • copyURLToFile

      void copyURLToFile(URL url, File file)

    • contentEquals

      boolean contentEquals(File file1, File file2)
      Return true if the content of the files are equal or they both don't exist

    • touch

      void touch(File file)