public abstract class AbstractLauncher<T extends AbstractLauncher<T>>
extends Object
| Modifier and Type | Method and Description |
|---|---|
T |
addAppArgs(String... args)
Adds command line arguments for the application.
|
T |
addFile(String file)
Adds a file to be submitted with the application.
|
T |
addJar(String jar)
Adds a jar file to be submitted with the application.
|
T |
addPyFile(String file)
Adds a python file / zip / egg to be submitted with the application.
|
T |
addSparkArg(String arg)
Adds a no-value argument to the Spark invocation.
|
T |
addSparkArg(String name,
String value)
Adds an argument with a value to the Spark invocation.
|
T |
setAppName(String appName)
Set the application name.
|
T |
setAppResource(String resource)
Set the main application resource.
|
T |
setConf(String key,
String value)
Set a single configuration value for the application.
|
T |
setDeployMode(String mode)
Set the deploy mode for the application.
|
T |
setMainClass(String mainClass)
Sets the application class name for Java/Scala applications.
|
T |
setMaster(String master)
Set the Spark master for the application.
|
T |
setPropertiesFile(String path)
Set a custom properties file with Spark configuration for the application.
|
T |
setRemote(String remote)
Set the Spark master for the application.
|
T |
setVerbose(boolean verbose)
Enables verbose reporting for SparkSubmit.
|
abstract SparkAppHandle |
startApplication(SparkAppHandle.Listener... listeners)
Starts a Spark application.
|
public T setPropertiesFile(String path)
path - Path to custom properties file to use.public T setConf(String key, String value)
key - Configuration key.value - The value to use.public T setAppName(String appName)
appName - Application name.public T setMaster(String master)
master - Spark master.public T setRemote(String remote)
remote - Spark remote url.public T setDeployMode(String mode)
mode - Deploy mode.public T setAppResource(String resource)
resource - Path to the main application resource.public T setMainClass(String mainClass)
mainClass - Application's main class.public T addSparkArg(String arg)
Use this method with caution. It is possible to create an invalid Spark command by passing unknown arguments to this method, since those are allowed for forward compatibility.
arg - Argument to add.public T addSparkArg(String name, String value)
It is safe to add arguments modified by other methods in this class (such as
setMaster(String) - the last invocation will be the one to take effect.
Use this method with caution. It is possible to create an invalid Spark command by passing unknown arguments to this method, since those are allowed for forward compatibility.
name - Name of argument to add.value - Value of the argument.public T addAppArgs(String... args)
args - Arguments to pass to the application's main class.public T addJar(String jar)
jar - Path to the jar file.public T addFile(String file)
file - Path to the file.public T addPyFile(String file)
file - Path to the file.public T setVerbose(boolean verbose)
verbose - Whether to enable verbose output.public abstract SparkAppHandle startApplication(SparkAppHandle.Listener... listeners) throws java.io.IOException
This method returns a handle that provides information about the running application and can be used to do basic interaction with it.
The returned handle assumes that the application will instantiate a single SparkContext
during its lifetime. Once that context reports a final state (one that indicates the
SparkContext has stopped), the handle will not perform new state transitions, so anything
that happens after that cannot be monitored. If the underlying application is launched as
a child process, SparkAppHandle.kill() can still be used to kill the child process.
listeners - Listeners to add to the handle before the app is launched.java.io.IOException