Class SparkEnv

Object
org.apache.spark.SparkEnv
All Implemented Interfaces:
org.apache.spark.internal.Logging

public class SparkEnv extends Object implements org.apache.spark.internal.Logging
:: DeveloperApi :: Holds all the runtime environment objects for a running Spark instance (either master or worker), including the serializer, RpcEnv, block manager, map output tracker, etc. Currently Spark code finds the SparkEnv through a global variable, so all the threads can access the same SparkEnv. It can be accessed by SparkEnv.get (e.g. after creating a SparkContext).
  • Nested Class Summary

    Nested classes/interfaces inherited from interface org.apache.spark.internal.Logging

    org.apache.spark.internal.Logging.LogStringContext, org.apache.spark.internal.Logging.SparkShellLoggingFilter
  • Constructor Summary

    Constructors
    Constructor
    Description
    SparkEnv(String executorId, org.apache.spark.rpc.RpcEnv rpcEnv, Serializer serializer, Serializer closureSerializer, org.apache.spark.serializer.SerializerManager serializerManager, org.apache.spark.MapOutputTracker mapOutputTracker, org.apache.spark.broadcast.BroadcastManager broadcastManager, org.apache.spark.storage.BlockManager blockManager, org.apache.spark.SecurityManager securityManager, org.apache.spark.metrics.MetricsSystem metricsSystem, org.apache.spark.scheduler.OutputCommitCoordinator outputCommitCoordinator, SparkConf conf)
     
  • Method Summary

    Modifier and Type
    Method
    Description
    org.apache.spark.storage.BlockManager
     
    org.apache.spark.broadcast.BroadcastManager
     
     
     
     
    static SparkEnv
    get()
    Returns the SparkEnv.
    static org.apache.spark.internal.Logging.LogStringContext
    LogStringContext(scala.StringContext sc)
     
    org.apache.spark.MapOutputTracker
     
    org.apache.spark.memory.MemoryManager
     
    org.apache.spark.metrics.MetricsSystem
     
    static org.slf4j.Logger
     
    static void
     
    org.apache.spark.scheduler.OutputCommitCoordinator
     
    org.apache.spark.SecurityManager
     
     
    org.apache.spark.serializer.SerializerManager
     
    static void
     
    org.apache.spark.shuffle.ShuffleManager
     

    Methods inherited from class java.lang.Object

    equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    Methods inherited from interface org.apache.spark.internal.Logging

    initializeForcefully, initializeLogIfNecessary, initializeLogIfNecessary, initializeLogIfNecessary$default$2, isTraceEnabled, log, logDebug, logDebug, logDebug, logDebug, logError, logError, logError, logError, logInfo, logInfo, logInfo, logInfo, logName, LogStringContext, logTrace, logTrace, logTrace, logTrace, logWarning, logWarning, logWarning, logWarning, org$apache$spark$internal$Logging$$log_, org$apache$spark$internal$Logging$$log__$eq, withLogContext
  • Constructor Details

    • SparkEnv

      public SparkEnv(String executorId, org.apache.spark.rpc.RpcEnv rpcEnv, Serializer serializer, Serializer closureSerializer, org.apache.spark.serializer.SerializerManager serializerManager, org.apache.spark.MapOutputTracker mapOutputTracker, org.apache.spark.broadcast.BroadcastManager broadcastManager, org.apache.spark.storage.BlockManager blockManager, org.apache.spark.SecurityManager securityManager, org.apache.spark.metrics.MetricsSystem metricsSystem, org.apache.spark.scheduler.OutputCommitCoordinator outputCommitCoordinator, SparkConf conf)
  • Method Details

    • set

      public static void set(SparkEnv e)
    • get

      public static SparkEnv get()
      Returns the SparkEnv.
      Returns:
      (undocumented)
    • org$apache$spark$internal$Logging$$log_

      public static org.slf4j.Logger org$apache$spark$internal$Logging$$log_()
    • org$apache$spark$internal$Logging$$log__$eq

      public static void org$apache$spark$internal$Logging$$log__$eq(org.slf4j.Logger x$1)
    • LogStringContext

      public static org.apache.spark.internal.Logging.LogStringContext LogStringContext(scala.StringContext sc)
    • executorId

      public String executorId()
    • serializer

      public Serializer serializer()
    • closureSerializer

      public Serializer closureSerializer()
    • serializerManager

      public org.apache.spark.serializer.SerializerManager serializerManager()
    • mapOutputTracker

      public org.apache.spark.MapOutputTracker mapOutputTracker()
    • broadcastManager

      public org.apache.spark.broadcast.BroadcastManager broadcastManager()
    • blockManager

      public org.apache.spark.storage.BlockManager blockManager()
    • securityManager

      public org.apache.spark.SecurityManager securityManager()
    • metricsSystem

      public org.apache.spark.metrics.MetricsSystem metricsSystem()
    • outputCommitCoordinator

      public org.apache.spark.scheduler.OutputCommitCoordinator outputCommitCoordinator()
    • conf

      public SparkConf conf()
    • shuffleManager

      public org.apache.spark.shuffle.ShuffleManager shuffleManager()
    • memoryManager

      public org.apache.spark.memory.MemoryManager memoryManager()