Package org.apache.spark

Interface ReadOnlySparkConf

All Known Implementing Classes:
SparkConf
public interface ReadOnlySparkConf

  • Method Summary

    Modifier and Type
    Method
    Description
    <T> T
    catchIllegalValue(String key, scala.Function0<T> getValue)
    Wrapper method for get() methods which require some specific value format.
    boolean
    contains(String key)
    Does the configuration contain a given parameter?
    boolean
    contains(org.apache.spark.internal.config.ConfigEntry<?> entry)
    Does the configuration have the typed config entry?
    String
    get(String key)
    Get a parameter; throws a NoSuchElementException if it's not set
    String
    get(String key, String defaultValue)
    Get a parameter, falling back to a default if not set
    <T> T
    get(org.apache.spark.internal.config.ConfigEntry<T> entry)
    Retrieves the value of a pre-defined configuration entry.
    scala.Tuple2<String,String>[]
    getAll()
    Get all parameters as a list of pairs
    boolean
    getBoolean(String key, boolean defaultValue)
    Get a parameter as a boolean, falling back to a default if not set
    double
    getDouble(String key, double defaultValue)
    Get a parameter as a double, falling back to a default if not ste
    String
    getenv(String name)
    By using this instead of System.getenv(), environment variables can be mocked in unit tests.
    int
    getInt(String key, int defaultValue)
    Get a parameter as an integer, falling back to a default if not set
    long
    getLong(String key, long defaultValue)
    Get a parameter as a long, falling back to a default if not set
    scala.Option<String>
    getOption(String key)
    Get a parameter as an Option
    long
    getSizeAsBytes(String key)
    Get a size parameter as bytes; throws a NoSuchElementException if it's not set.
    long
    getSizeAsBytes(String key, long defaultValue)
    Get a size parameter as bytes, falling back to a default if not set.
    long
    getSizeAsBytes(String key, String defaultValue)
    Get a size parameter as bytes, falling back to a default if not set.
    long
    getSizeAsGb(String key)
    Get a size parameter as Gibibytes; throws a NoSuchElementException if it's not set.
    long
    getSizeAsGb(String key, String defaultValue)
    Get a size parameter as Gibibytes, falling back to a default if not set.
    long
    getSizeAsKb(String key)
    Get a size parameter as Kibibytes; throws a NoSuchElementException if it's not set.
    long
    getSizeAsKb(String key, String defaultValue)
    Get a size parameter as Kibibytes, falling back to a default if not set.
    long
    getSizeAsMb(String key)
    Get a size parameter as Mebibytes; throws a NoSuchElementException if it's not set.
    long
    getSizeAsMb(String key, String defaultValue)
    Get a size parameter as Mebibytes, falling back to a default if not set.
    long
    getTimeAsMs(String key)
    Get a time parameter as milliseconds; throws a NoSuchElementException if it's not set.
    long
    getTimeAsMs(String key, String defaultValue)
    Get a time parameter as milliseconds, falling back to a default if not set.
    long
    getTimeAsSeconds(String key)
    Get a time parameter as seconds; throws a NoSuchElementException if it's not set.
    long
    getTimeAsSeconds(String key, String defaultValue)
    Get a time parameter as seconds, falling back to a default if not set.

  • Method Details

    • catchIllegalValue

      <T> T catchIllegalValue(String key, scala.Function0<T> getValue)
      Wrapper method for get() methods which require some specific value format. This catches any NumberFormatException or IllegalArgumentException and re-raises it with the incorrectly configured key in the exception message.
      Parameters:
      key - (undocumented)
      getValue - (undocumented)
      Returns:
      (undocumented)

    • contains

      boolean contains(String key)
      Does the configuration contain a given parameter?

    • contains

      boolean contains(org.apache.spark.internal.config.ConfigEntry<?> entry)
      Does the configuration have the typed config entry?

    • get

      String get(String key)
      Get a parameter; throws a NoSuchElementException if it's not set

    • get

      String get(String key, String defaultValue)
      Get a parameter, falling back to a default if not set

    • get

      <T> T get(org.apache.spark.internal.config.ConfigEntry<T> entry)
      Retrieves the value of a pre-defined configuration entry.

      - This is an internal Spark API. - The return type if defined by the configuration entry. - This will throw an exception is the config is not optional and the value is not set.

      Parameters:
      entry - (undocumented)
      Returns:
      (undocumented)

    • getAll

      scala.Tuple2<String,String>[] getAll()
      Get all parameters as a list of pairs

    • getBoolean

      boolean getBoolean(String key, boolean defaultValue)
      Get a parameter as a boolean, falling back to a default if not set

      Parameters:
      key - (undocumented)
      defaultValue - (undocumented)
      Returns:
      (undocumented)
      Throws:
      IllegalArgumentException - If the value cannot be interpreted as a boolean

    • getDouble

      double getDouble(String key, double defaultValue)
      Get a parameter as a double, falling back to a default if not ste

      Parameters:
      key - (undocumented)
      defaultValue - (undocumented)
      Returns:
      (undocumented)
      Throws:
      NumberFormatException - If the value cannot be interpreted as a double

    • getInt

      int getInt(String key, int defaultValue)
      Get a parameter as an integer, falling back to a default if not set

      Parameters:
      key - (undocumented)
      defaultValue - (undocumented)
      Returns:
      (undocumented)
      Throws:
      NumberFormatException - If the value cannot be interpreted as an integer

    • getLong

      long getLong(String key, long defaultValue)
      Get a parameter as a long, falling back to a default if not set

      Parameters:
      key - (undocumented)
      defaultValue - (undocumented)
      Returns:
      (undocumented)
      Throws:
      NumberFormatException - If the value cannot be interpreted as a long

    • getOption

      scala.Option<String> getOption(String key)
      Get a parameter as an Option

    • getSizeAsBytes

      long getSizeAsBytes(String key)
      Get a size parameter as bytes; throws a NoSuchElementException if it's not set. If no suffix is provided then bytes are assumed.

      Parameters:
      key - (undocumented)
      Returns:
      (undocumented)
      Throws:
      NoSuchElementException - If the size parameter is not set
      NumberFormatException - If the value cannot be interpreted as bytes

    • getSizeAsBytes

      long getSizeAsBytes(String key, String defaultValue)
      Get a size parameter as bytes, falling back to a default if not set. If no suffix is provided then bytes are assumed.

      Parameters:
      key - (undocumented)
      defaultValue - (undocumented)
      Returns:
      (undocumented)
      Throws:
      NumberFormatException - If the value cannot be interpreted as bytes

    • getSizeAsBytes

      long getSizeAsBytes(String key, long defaultValue)
      Get a size parameter as bytes, falling back to a default if not set.

      Parameters:
      key - (undocumented)
      defaultValue - (undocumented)
      Returns:
      (undocumented)
      Throws:
      NumberFormatException - If the value cannot be interpreted as bytes

    • getSizeAsGb

      long getSizeAsGb(String key)
      Get a size parameter as Gibibytes; throws a NoSuchElementException if it's not set. If no suffix is provided then Gibibytes are assumed.

      Parameters:
      key - (undocumented)
      Returns:
      (undocumented)
      Throws:
      NoSuchElementException - If the size parameter is not set
      NumberFormatException - If the value cannot be interpreted as Gibibytes

    • getSizeAsGb

      long getSizeAsGb(String key, String defaultValue)
      Get a size parameter as Gibibytes, falling back to a default if not set. If no suffix is provided then Gibibytes are assumed.

      Parameters:
      key - (undocumented)
      defaultValue - (undocumented)
      Returns:
      (undocumented)
      Throws:
      NumberFormatException - If the value cannot be interpreted as Gibibytes

    • getSizeAsKb

      long getSizeAsKb(String key)
      Get a size parameter as Kibibytes; throws a NoSuchElementException if it's not set. If no suffix is provided then Kibibytes are assumed.

      Parameters:
      key - (undocumented)
      Returns:
      (undocumented)
      Throws:
      NoSuchElementException - If the size parameter is not set
      NumberFormatException - If the value cannot be interpreted as Kibibytes

    • getSizeAsKb

      long getSizeAsKb(String key, String defaultValue)
      Get a size parameter as Kibibytes, falling back to a default if not set. If no suffix is provided then Kibibytes are assumed.

      Parameters:
      key - (undocumented)
      defaultValue - (undocumented)
      Returns:
      (undocumented)
      Throws:
      NumberFormatException - If the value cannot be interpreted as Kibibytes

    • getSizeAsMb

      long getSizeAsMb(String key)
      Get a size parameter as Mebibytes; throws a NoSuchElementException if it's not set. If no suffix is provided then Mebibytes are assumed.

      Parameters:
      key - (undocumented)
      Returns:
      (undocumented)
      Throws:
      NoSuchElementException - If the size parameter is not set
      NumberFormatException - If the value cannot be interpreted as Mebibytes

    • getSizeAsMb

      long getSizeAsMb(String key, String defaultValue)
      Get a size parameter as Mebibytes, falling back to a default if not set. If no suffix is provided then Mebibytes are assumed.

      Parameters:
      key - (undocumented)
      defaultValue - (undocumented)
      Returns:
      (undocumented)
      Throws:
      NumberFormatException - If the value cannot be interpreted as Mebibytes

    • getTimeAsMs

      long getTimeAsMs(String key)
      Get a time parameter as milliseconds; throws a NoSuchElementException if it's not set. If no suffix is provided then milliseconds are assumed.

      Parameters:
      key - (undocumented)
      Returns:
      (undocumented)
      Throws:
      NoSuchElementException - If the time parameter is not set
      NumberFormatException - If the value cannot be interpreted as milliseconds

    • getTimeAsMs

      long getTimeAsMs(String key, String defaultValue)
      Get a time parameter as milliseconds, falling back to a default if not set. If no suffix is provided then milliseconds are assumed.

      Parameters:
      key - (undocumented)
      defaultValue - (undocumented)
      Returns:
      (undocumented)
      Throws:
      NumberFormatException - If the value cannot be interpreted as milliseconds

    • getTimeAsSeconds

      long getTimeAsSeconds(String key)
      Get a time parameter as seconds; throws a NoSuchElementException if it's not set. If no suffix is provided then seconds are assumed.

      Parameters:
      key - (undocumented)
      Returns:
      (undocumented)
      Throws:
      NoSuchElementException - If the time parameter is not set
      NumberFormatException - If the value cannot be interpreted as seconds

    • getTimeAsSeconds

      long getTimeAsSeconds(String key, String defaultValue)
      Get a time parameter as seconds, falling back to a default if not set. If no suffix is provided then seconds are assumed.

      Parameters:
      key - (undocumented)
      defaultValue - (undocumented)
      Returns:
      (undocumented)
      Throws:
      NumberFormatException - If the value cannot be interpreted as seconds

    • getenv

      String getenv(String name)
      By using this instead of System.getenv(), environment variables can be mocked in unit tests.
      Parameters:
      name - (undocumented)
      Returns:
      (undocumented)