Package org.apache.spark.storage
Class StorageUtils
Object
org.apache.spark.storage.StorageUtils
Helper methods for storage-related objects.
-
Constructor Summary
-
Method Summary
Modifier and TypeMethodDescriptionstatic void
dispose
(ByteBuffer buffer) Attempt to clean up a ByteBuffer if it is direct or memory-mapped.static int
Get the port used by the external shuffle service.static org.apache.spark.internal.Logging.LogStringContext
LogStringContext
(scala.StringContext sc) static org.slf4j.Logger
static void
org$apache$spark$internal$Logging$$log__$eq
(org.slf4j.Logger x$1)
-
Constructor Details
-
StorageUtils
public StorageUtils()
-
-
Method Details
-
dispose
Attempt to clean up a ByteBuffer if it is direct or memory-mapped. This uses an *unsafe* Sun API that will cause errors if one attempts to read from the disposed buffer. However, neither the bytes allocated to direct buffers nor file descriptors opened for memory-mapped buffers put pressure on the garbage collector. Waiting for garbage collection may lead to the depletion of off-heap memory or huge numbers of open files. There's unfortunately no standard API to manually dispose of these kinds of buffers.- Parameters:
buffer
- (undocumented)
-
externalShuffleServicePort
Get the port used by the external shuffle service. In Yarn mode, this may be already be set through the Hadoop configuration as the server is launched in the Yarn NM.- Parameters:
conf
- (undocumented)- Returns:
- (undocumented)
-
org$apache$spark$internal$Logging$$log_
public static org.slf4j.Logger org$apache$spark$internal$Logging$$log_() -
org$apache$spark$internal$Logging$$log__$eq
public static void org$apache$spark$internal$Logging$$log__$eq(org.slf4j.Logger x$1) -
LogStringContext
public static org.apache.spark.internal.Logging.LogStringContext LogStringContext(scala.StringContext sc)
-