package launcher
Type Members
- abstract class AbstractLauncher[T <: AbstractLauncher[T]] extends AnyRef
Base class for launcher implementations.
Base class for launcher implementations.
- Since
2.3.0
- class InProcessLauncher extends AbstractLauncher[InProcessLauncher]
In-process launcher for Spark applications.
In-process launcher for Spark applications.
Use this class to start Spark applications programmatically. Applications launched using this class will run in the same process as the caller.
Because Spark only supports a single active instance of
SparkContextper JVM, code that uses this class should be careful about which applications are launched. It's recommended that this launcher only be used to launch applications in cluster mode.Also note that, when running applications in client mode, JVM-related configurations (like driver memory or configs which modify the driver's class path) do not take effect. Logging configuration is also inherited from the parent application.
- Since
2.3.0
- class JavaModuleOptions extends AnyRef
This helper class is used to place some JVM runtime options(eg:
--add-opens) required by Spark when using Java 17.This helper class is used to place some JVM runtime options(eg:
--add-opens) required by Spark when using Java 17.DEFAULT_MODULE_OPTIONShas added-XX:+IgnoreUnrecognizedVMOptionsto be robust.- Since
3.3.0
- trait SparkAppHandle extends AnyRef
A handle to a running Spark application.
A handle to a running Spark application.
Provides runtime information about the underlying Spark application, and actions to control it.
- Since
1.6.0
- class SparkLauncher extends AbstractLauncher[SparkLauncher]
Launcher for Spark applications.
Launcher for Spark applications.
Use this class to start Spark applications programmatically. The class uses a builder pattern to allow clients to configure the Spark application and launch it as a child process.