Package org.apache.spark.util

Class MavenUtils

Object
org.apache.spark.util.MavenUtils
public class MavenUtils extends Object
Provides utility functions to be used inside SparkSubmit.

  • Constructor Details

    • MavenUtils

      public MavenUtils()

  • Method Details

    • JAR_IVY_SETTING_PATH_KEY

      public static String JAR_IVY_SETTING_PATH_KEY()

    • IVY_DEFAULT_EXCLUDES

      public static scala.collection.immutable.Seq<String> IVY_DEFAULT_EXCLUDES()

    • extractMavenCoordinates

      public static scala.collection.immutable.Seq<org.apache.spark.util.MavenUtils.MavenCoordinate> extractMavenCoordinates(String coordinates)
      Extracts maven coordinates from a comma-delimited string. Coordinates should be provided in the format groupId:artifactId:version or groupId/artifactId:version.

      Parameters:
      coordinates - Comma-delimited string of maven coordinates
      Returns:
      Sequence of Maven coordinates

    • buildIvySettings

      public static org.apache.ivy.core.settings.IvySettings buildIvySettings(scala.Option<String> remoteRepos, scala.Option<String> ivyPath, boolean useLocalM2AsCache, PrintStream printStream)
      Build Ivy Settings using options with default resolvers

      Parameters:
      remoteRepos - Comma-delimited string of remote repositories other than maven central
      ivyPath - The path to the local ivy repository
      useLocalM2AsCache - Whether or not use local-m2 repo as cache
      printStream - (undocumented)
      Returns:
      An IvySettings object

    • loadIvySettings

      public static org.apache.ivy.core.settings.IvySettings loadIvySettings(String settingsFile, scala.Option<String> remoteRepos, scala.Option<String> ivyPath, PrintStream printStream)
      Load Ivy settings from a given filename, using supplied resolvers

      Parameters:
      settingsFile - Path to Ivy settings file
      remoteRepos - Comma-delimited string of remote repositories other than maven central
      ivyPath - The path to the local ivy repository
      printStream - (undocumented)
      Returns:
      An IvySettings object

    • resolveMavenCoordinates

      public static scala.collection.immutable.Seq<String> resolveMavenCoordinates(String coordinates, org.apache.ivy.core.settings.IvySettings ivySettings, scala.Option<org.apache.ivy.core.settings.IvySettings> noCacheIvySettings, boolean transitive, scala.collection.immutable.Seq<String> exclusions, boolean isTest, PrintStream printStream)
      Resolves any dependencies that were supplied through maven coordinates

      Parameters:
      coordinates - Comma-delimited string of maven coordinates
      ivySettings - An IvySettings containing resolvers to use
      noCacheIvySettings - An no-cache(local-m2-cache) IvySettings containing resolvers to use
      transitive - Whether resolving transitive dependencies, default is true
      exclusions - Exclusions to apply when resolving transitive dependencies
      isTest - (undocumented)
      printStream - (undocumented)
      Returns:
      Seq of path to the jars of the given maven artifacts including their transitive dependencies

    • parseQueryParams

      public static scala.Tuple3<Object,String,String> parseQueryParams(URI uri)
      Parse URI query string's parameter value of transitive, exclude and repos. Other invalid parameters will be ignored.

      Parameters:
      uri - Ivy URI need to be downloaded.
      Returns:
      Tuple value of parameter transitive, exclude and repos value.

      1. transitive: whether to download dependency jar of Ivy URI, default value is true and this parameter value is case-insensitive. This mimics Hive's behaviour for parsing the transitive parameter. Invalid value will be treat as false. Example: Input: exclude=org.mortbay.jetty:jetty&transitive=true Output: true

      2. exclude: comma separated exclusions to apply when resolving transitive dependencies, consists of group:module pairs separated by commas. Example: Input: excludeorg.mortbay.jetty:jetty,org.eclipse.jetty:jetty-http Output: [org.mortbay.jetty:jetty,org.eclipse.jetty:jetty-http]

      3. repos: comma separated repositories to use when resolving dependencies.

    • org$apache$spark$internal$Logging$$log_

      public static org.slf4j.Logger org$apache$spark$internal$Logging$$log_()

    • org$apache$spark$internal$Logging$$log__$eq

      public static void org$apache$spark$internal$Logging$$log__$eq(org.slf4j.Logger x$1)

    • LogStringContext

      public static org.apache.spark.internal.Logging.LogStringContext LogStringContext(scala.StringContext sc)