A trait that can be mixed into a subclass of AccumulatorV2 to track the "logical" value of the "last attempt" of the execution using the accumulator - aggregated from the last attempts of any Task that calculated some RDD partitions and used this accumulator, and discarding any values coming from earlier attempts that have been recomputed. If the accumulator is used by multiple RDDs, the last attempt value is tracked separately for each, and can be retrieved for each or all of them separately, see lastAttemptValueForX methods. If the accumulator is used directly on the Spark Driver using AccumulatorV2#add, that value is considered the last attempt value. If the accumulator was both used in Tasks and updated directly on the driver, it can't determine what should be considered the last attempt, and lastAttemptValueForX methods will return None.

Contract for driver-only updates: A driver-side value (set via AccumulatorV2#add on the driver, outside any Task) is only returned by methods that do not narrow by RDD, namely lastAttemptValueForAllRDDs and lastAttemptValueForHighestRDDId. Methods that narrow to specific RDDs or RDD scopes (lastAttemptValueForRDDId, lastAttemptValueForRDDIds, lastAttemptValueForRDDScopes) return the zero value when a driver-only value is present, because a driver-side update cannot be attributed to any particular RDD or scope.

LastAttemptAccumulator is not reset by the AccumulatorV2#reset method implementation, and its state is not copied by the AccumulatorV2#copy method implementation, and it should not be serialized to the Executors. The internal state should only be initialized by the initializeLastAttemptAccumulator method on the "main" instance of the accumulator, that was created and registered with AccumulatorContext with AccumulatorV2#register. All the interfaces of LastAttemptAccumulator: mergeLastAttempt (used only by DAGScheduler) and lastAttemptValueForX, logAccumulatorState (used by the using code) should only be invoked on that instance, on the Spark Driver.

The LastAttemptAccumulator is not thread-safe. mergeLastAttempt should only be used by DAGScheduler, by the scheduler thread. Retrieving the value using lastAttemptValueForXXX while it is concurrently updated (execution is running) can produce some inconsistencies, but should not crash. If an RDD using the LastAttemptAccumulator is used concurrently by multiple actions that all try to recompute it, it may produce unexpected results and the semantics of what is "last attempt" becomes ambiguous. This should not be done in practice, and will likely result in more unexpected behaviours in Spark.

Implementations must implement partialMergeVal and partialMerge methods operating on PARTIAL type. In regular AccumulatorV2 implementations, the AccumulatorV2 object itself holds the intermediate value of the accumulator, and AccumulatorV2#merge method is used to merge these objects together. LastAttemptAccumulator needs to keep track of partial values of every partition of every RDD that used the accumulator, and holding a full AccumulatorV2 object for each would have a high overhead. Therefore, an implementation should be able to return PARTIAL value from partialMergeVal that represents an intermediate mergeable value, and a partialMerge method that can merge that value into the accumulator. Implementations must also implement an isMergeable method that checks if the other AccumulatorV2 is of a compatible type to be merged with this using partialMergeVal. In regular AccumulatorV2 implementations, this check is normally done inside the AccumulatorV2#merge method, which is not used here.

If an implementation is used to keep user data in the accumulator, it should override accumulatorStoresUserData to return true, to ensure correct structured logging annotation. Otherwise it should override it to false.