public class AccumulatorContext
extends Object
Constructor and Description |
---|
AccumulatorContext() |
Modifier and Type | Method and Description |
---|---|
static void |
clear()
Clears all registered
AccumulatorV2 s. |
static scala.Option<AccumulatorV2<?,?>> |
get(long id)
Returns the
AccumulatorV2 registered with the given ID, if any. |
static scala.Option<Object> |
internOption(scala.Option<Object> value)
Naive way to reduce the duplicate Some objects for values 0 and -1
TODO: Eventually if this spreads out to more values then using
Guava's weak interner would be a better solution.
|
static long |
newId()
Returns a globally unique ID for a new
AccumulatorV2 . |
static int |
numAccums()
Returns the number of accumulators registered.
|
static void |
org$apache$spark$internal$Logging$$log__$eq(org.slf4j.Logger x$1) |
static org.slf4j.Logger |
org$apache$spark$internal$Logging$$log_() |
static void |
register(AccumulatorV2<?,?> a)
Registers an
AccumulatorV2 created on the driver such that it can be used on the executors. |
static void |
remove(long id)
Unregisters the
AccumulatorV2 with the given ID, if any. |
public static long newId()
AccumulatorV2
.
Note: Once you copy the AccumulatorV2
the ID is no longer unique.public static int numAccums()
public static void register(AccumulatorV2<?,?> a)
AccumulatorV2
created on the driver such that it can be used on the executors.
All accumulators registered here can later be used as a container for accumulating partial
values across multiple tasks. This is what org.apache.spark.scheduler.DAGScheduler
does.
Note: if an accumulator is registered here, it should also be registered with the active
context cleaner for cleanup so as to avoid memory leaks.
If an AccumulatorV2
with the same ID was already registered, this does nothing instead
of overwriting it. We will never register same accumulator twice, this is just a sanity check.
a
- (undocumented)public static void remove(long id)
AccumulatorV2
with the given ID, if any.id
- (undocumented)public static scala.Option<AccumulatorV2<?,?>> get(long id)
AccumulatorV2
registered with the given ID, if any.id
- (undocumented)public static void clear()
AccumulatorV2
s. For testing only.public static scala.Option<Object> internOption(scala.Option<Object> value)
value
- (undocumented)public static org.slf4j.Logger org$apache$spark$internal$Logging$$log_()
public static void org$apache$spark$internal$Logging$$log__$eq(org.slf4j.Logger x$1)