Class SparkThreadUtils


public class SparkThreadUtils extends Object
  • Constructor Details

    • SparkThreadUtils

      public SparkThreadUtils()
  • Method Details

    • awaitResult

      public static <T> T awaitResult(scala.concurrent.Awaitable<T> awaitable, scala.concurrent.duration.Duration atMost) throws SparkException
      Preferred alternative to Await.result().

      This method wraps and re-throws any exceptions thrown by the underlying Await call, ensuring that this thread's stack trace appears in logs.

      In addition, it calls Awaitable.result directly to avoid using ForkJoinPool's BlockingContext. Codes running in the user's thread may be in a thread of Scala ForkJoinPool. As concurrent executions in ForkJoinPool may see some ThreadLocal value unexpectedly, this method basically prevents ForkJoinPool from running other tasks in the current waiting thread. In general, we should use this method because many places in Spark use ThreadLocal and it's hard to debug when ThreadLocals leak to other tasks.

      awaitable - (undocumented)
      atMost - (undocumented)