Package org.apache.spark.util
package org.apache.spark.util
Spark utilities.
-
ClassDescriptionAn internal class used to track accumulators by Spark itself.AccumulatorV2<IN,
OUT> The base class for accumulators, that can accumulate inputs of typeIN, and produce output of typeOUT.Implicit methods related to Scala Array.Extractor Object for pulling out the root cause of an error.A mutable class loader that gives preference to its own URLs over the parent class loader when loading classes and resources.An interface to represent clocks, so that they can be mocked out in unit tests.A cleaner that renders closures serializable if they can be done so safely.Anaccumulatorfor collecting a list of elements.Contains basic command line parsing functionality and methods to parse some common Spark CLI options.Anaccumulatorfor computing sum, count, and averages for double precision floating numbers.Utility functions to simplify and speed-up file listing.An extractor object for parsing strings into integers.Serializes SparkListener events to/from JSON.A trait that allows a class to giveSizeEstimatormore accurate size estimation.ListenerBus<L,E> An event bus which posts events to its listeners.:: : DeveloperApi :: Utils for querying Spark logs with Spark SQL.Anaccumulatorfor computing sum, count, and average of 64-bit integers.Provides utility functions to be used inside SparkSubmit.An extractor object for parsing JVM memory strings, such as "10g", into an Int representing the number of megabytes.Helper class to identify a method.MutablePair<T1,T2> :: DeveloperApi :: A tuple of 2 elements.URL class loader that exposes the `addURL` method in URLClassLoader.A class loader which makes some protected methods in ClassLoader accessible.Various utility methods used by Spark Security.Hadoop configuration but serializable.Various utility methods used by Spark.Contains utilities for working with posix signals.:: DeveloperApi :: Estimates the sizes of Java objects (number of bytes of memory they occupy), for use in memory-aware caches.Utils for handling schemas.A class for tracking the statistics of a set of numbers (count, mean and variance) in a numerically robust way.:: DeveloperApi :::: DeveloperApi ::Various utility methods used by Spark.Utilities for working with Spark version strings