public class SparkThreadUtils
extends Object
| Constructor and Description |
|---|
SparkThreadUtils() |
| Modifier and Type | Method and Description |
|---|---|
static <T> T |
awaitResult(scala.concurrent.Awaitable<T> awaitable,
scala.concurrent.duration.Duration atMost)
Preferred alternative to
Await.result(). |
public static <T> T awaitResult(scala.concurrent.Awaitable<T> awaitable,
scala.concurrent.duration.Duration atMost)
throws SparkException
Await.result().
This method wraps and re-throws any exceptions thrown by the underlying Await call, ensuring
that this thread's stack trace appears in logs.
In addition, it calls Awaitable.result directly to avoid using ForkJoinPool's
BlockingContext. Codes running in the user's thread may be in a thread of Scala ForkJoinPool.
As concurrent executions in ForkJoinPool may see some ThreadLocal value unexpectedly, this
method basically prevents ForkJoinPool from running other tasks in the current waiting thread.
In general, we should use this method because many places in Spark use ThreadLocal and it's
hard to debug when ThreadLocals leak to other tasks.
awaitable - (undocumented)atMost - (undocumented)SparkException