org.apache.spark

Class SparkEnv

Object

org.apache.spark.SparkEnv

All Implemented Interfaces:

org.apache.spark.internal.Logging

public class SparkEnv
extends Object
implements org.apache.spark.internal.Logging

:: DeveloperApi :: Holds all the runtime environment objects for a running Spark instance (either master or worker), including the serializer, RpcEnv, block manager, map output tracker, etc. Currently Spark code finds the SparkEnv through a global variable, so all the threads can access the same SparkEnv. It can be accessed by SparkEnv.get (e.g. after creating a SparkContext).

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.spark.internal.Logging

org.apache.spark.internal.Logging.SparkShellLoggingFilter

Constructor Summary

Constructors

Constructor and Description

SparkEnv(String executorId, org.apache.spark.rpc.RpcEnv rpcEnv, Serializer serializer, Serializer closureSerializer, org.apache.spark.serializer.SerializerManager serializerManager, org.apache.spark.MapOutputTracker mapOutputTracker, org.apache.spark.shuffle.ShuffleManager shuffleManager, org.apache.spark.broadcast.BroadcastManager broadcastManager, org.apache.spark.storage.BlockManager blockManager, org.apache.spark.SecurityManager securityManager, org.apache.spark.metrics.MetricsSystem metricsSystem, org.apache.spark.memory.MemoryManager memoryManager, org.apache.spark.scheduler.OutputCommitCoordinator outputCommitCoordinator, SparkConf conf)

Method Summary

Modifier and Type	Method and Description
org.apache.spark.storage.BlockManager	blockManager()
org.apache.spark.broadcast.BroadcastManager	broadcastManager()
Serializer	closureSerializer()
SparkConf	conf()
String	executorId()
static SparkEnv	get() Returns the SparkEnv.
org.apache.spark.MapOutputTracker	mapOutputTracker()
org.apache.spark.memory.MemoryManager	memoryManager()
org.apache.spark.metrics.MetricsSystem	metricsSystem()
static void	org$apache$spark$internal$Logging$$log__$eq(org.slf4j.Logger x$1)
static org.slf4j.Logger	org$apache$spark$internal$Logging$$log_()
org.apache.spark.scheduler.OutputCommitCoordinator	outputCommitCoordinator()
org.apache.spark.SecurityManager	securityManager()
Serializer	serializer()
org.apache.spark.serializer.SerializerManager	serializerManager()
static void	set(SparkEnv e)
org.apache.spark.shuffle.ShuffleManager	shuffleManager()

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.spark.internal.Logging

$init$, initializeForcefully, initializeLogIfNecessary, initializeLogIfNecessary, initializeLogIfNecessary$default$2, initLock, isTraceEnabled, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logName, logTrace, logTrace, logWarning, logWarning, org$apache$spark$internal$Logging$$log__$eq, org$apache$spark$internal$Logging$$log_, uninitialize

Constructor Detail

SparkEnv

public SparkEnv(String executorId,
                org.apache.spark.rpc.RpcEnv rpcEnv,
                Serializer serializer,
                Serializer closureSerializer,
                org.apache.spark.serializer.SerializerManager serializerManager,
                org.apache.spark.MapOutputTracker mapOutputTracker,
                org.apache.spark.shuffle.ShuffleManager shuffleManager,
                org.apache.spark.broadcast.BroadcastManager broadcastManager,
                org.apache.spark.storage.BlockManager blockManager,
                org.apache.spark.SecurityManager securityManager,
                org.apache.spark.metrics.MetricsSystem metricsSystem,
                org.apache.spark.memory.MemoryManager memoryManager,
                org.apache.spark.scheduler.OutputCommitCoordinator outputCommitCoordinator,
                SparkConf conf)

Method Detail

set

public static void set(SparkEnv e)

get

public static SparkEnv get()

Returns the SparkEnv.

Returns:

(undocumented)

org$apache$spark$internal$Logging$$log_

public static org.slf4j.Logger org$apache$spark$internal$Logging$$log_()

org$apache$spark$internal$Logging$$log__$eq

public static void org$apache$spark$internal$Logging$$log__$eq(org.slf4j.Logger x$1)

executorId

public String executorId()

serializer

public Serializer serializer()

closureSerializer

public Serializer closureSerializer()

serializerManager

public org.apache.spark.serializer.SerializerManager serializerManager()

mapOutputTracker

public org.apache.spark.MapOutputTracker mapOutputTracker()

shuffleManager

public org.apache.spark.shuffle.ShuffleManager shuffleManager()

broadcastManager

public org.apache.spark.broadcast.BroadcastManager broadcastManager()

blockManager

public org.apache.spark.storage.BlockManager blockManager()

securityManager

public org.apache.spark.SecurityManager securityManager()

metricsSystem

public org.apache.spark.metrics.MetricsSystem metricsSystem()

memoryManager

public org.apache.spark.memory.MemoryManager memoryManager()

outputCommitCoordinator

public org.apache.spark.scheduler.OutputCommitCoordinator outputCommitCoordinator()

conf

public SparkConf conf()