Package org.apache.spark.status

Class KVUtils

Object
org.apache.spark.status.KVUtils
public class KVUtils extends Object

  • Constructor Summary

    Constructors
    Constructor
    Description
    KVUtils()
     

  • Method Summary

    Modifier and Type
    Method
    Description
    static <T> int
    count(org.apache.spark.util.kvstore.KVStoreView<T> view, scala.Function1<T,Object> countFunc)
    Counts the number of elements in the KVStoreView which satisfy a predicate.
    static org.apache.spark.util.kvstore.KVStore
    createKVStore(scala.Option<File> storePath, boolean live, SparkConf conf)
     
    static <T> void
    foreach(org.apache.spark.util.kvstore.KVStoreView<T> view, scala.Function1<T,scala.runtime.BoxedUnit> foreachFunc)
    Applies a function f to all values produced by KVStoreView.
    static org.apache.spark.internal.Logging.LogStringContext
    LogStringContext(scala.StringContext sc)
     
    static <T, B> scala.collection.immutable.Seq<B>
    mapToSeq(org.apache.spark.util.kvstore.KVStoreView<T> view, scala.Function1<T,B> mapFunc)
    Maps all values of KVStoreView to new values using a transformation function.
    static <T, B> scala.collection.immutable.Seq<B>
    mapToSeqWithFilter(org.apache.spark.util.kvstore.KVStoreView<T> view, int max, scala.Function1<T,B> mapFunc, scala.Function1<B,Object> filterFunc)
    Maps all values of KVStoreView to new values using a transformation function and filtered by a filter function.
    static <M> org.apache.spark.util.kvstore.KVStore
    open(File path, M metadata, SparkConf conf, boolean live, scala.reflect.ClassTag<M> evidence$1)
    Open or create a disk-based KVStore.
    static org.slf4j.Logger
    org$apache$spark$internal$Logging$$log_()
     
    static void
    org$apache$spark$internal$Logging$$log__$eq(org.slf4j.Logger x$1)
     
    static org.apache.spark.status.KVUtils.KVStoreScalaSerializer
    serializerForHistoryServer(SparkConf conf)
     
    static <T> int
    size(org.apache.spark.util.kvstore.KVStoreView<T> view)
     
    static <T> scala.collection.immutable.Seq<T>
    viewToSeq(org.apache.spark.util.kvstore.KVStoreView<T> view)
    Turns a KVStoreView into a Scala sequence.
    static <T> scala.collection.immutable.Seq<T>
    viewToSeq(org.apache.spark.util.kvstore.KVStoreView<T> view, int from, int until, scala.Function1<T,Object> filter)
    Turns an interval of KVStoreView into a Scala sequence, applying a filter.
    static <T> scala.collection.immutable.Seq<T>
    viewToSeq(org.apache.spark.util.kvstore.KVStoreView<T> view, int max, scala.Function1<T,Object> filter)
    Turns a KVStoreView into a Scala sequence, applying a filter.

    Methods inherited from class java.lang.Object

    equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Constructor Details

    • KVUtils

      public KVUtils()

  • Method Details

    • open

      public static <M> org.apache.spark.util.kvstore.KVStore open(File path, M metadata, SparkConf conf, boolean live, scala.reflect.ClassTag<M> evidence$1)
      Open or create a disk-based KVStore.

      Parameters:
      path - Location of the store.
      metadata - Metadata value to compare to the data in the store. If the store does not contain any metadata (e.g. it's a new store), this value is written as the store's metadata.
      conf - SparkConf use to get HYBRID_STORE_DISK_BACKEND
      live - (undocumented)
      evidence$1 - (undocumented)
      Returns:
      (undocumented)

    • serializerForHistoryServer

      public static org.apache.spark.status.KVUtils.KVStoreScalaSerializer serializerForHistoryServer(SparkConf conf)

    • createKVStore

      public static org.apache.spark.util.kvstore.KVStore createKVStore(scala.Option<File> storePath, boolean live, SparkConf conf)

    • viewToSeq

      public static <T> scala.collection.immutable.Seq<T> viewToSeq(org.apache.spark.util.kvstore.KVStoreView<T> view, int max, scala.Function1<T,Object> filter)
      Turns a KVStoreView into a Scala sequence, applying a filter.

    • viewToSeq

      public static <T> scala.collection.immutable.Seq<T> viewToSeq(org.apache.spark.util.kvstore.KVStoreView<T> view, int from, int until, scala.Function1<T,Object> filter)
      Turns an interval of KVStoreView into a Scala sequence, applying a filter.

    • viewToSeq

      public static <T> scala.collection.immutable.Seq<T> viewToSeq(org.apache.spark.util.kvstore.KVStoreView<T> view)
      Turns a KVStoreView into a Scala sequence.

    • count

      public static <T> int count(org.apache.spark.util.kvstore.KVStoreView<T> view, scala.Function1<T,Object> countFunc)
      Counts the number of elements in the KVStoreView which satisfy a predicate.

    • foreach

      public static <T> void foreach(org.apache.spark.util.kvstore.KVStoreView<T> view, scala.Function1<T,scala.runtime.BoxedUnit> foreachFunc)
      Applies a function f to all values produced by KVStoreView.

    • mapToSeq

      public static <T, B> scala.collection.immutable.Seq<B> mapToSeq(org.apache.spark.util.kvstore.KVStoreView<T> view, scala.Function1<T,B> mapFunc)
      Maps all values of KVStoreView to new values using a transformation function.

    • mapToSeqWithFilter

      public static <T, B> scala.collection.immutable.Seq<B> mapToSeqWithFilter(org.apache.spark.util.kvstore.KVStoreView<T> view, int max, scala.Function1<T,B> mapFunc, scala.Function1<B,Object> filterFunc)
      Maps all values of KVStoreView to new values using a transformation function and filtered by a filter function.
      Parameters:
      view - (undocumented)
      max - (undocumented)
      mapFunc - (undocumented)
      filterFunc - (undocumented)
      Returns:
      (undocumented)

    • size

      public static <T> int size(org.apache.spark.util.kvstore.KVStoreView<T> view)

    • org$apache$spark$internal$Logging$$log_

      public static org.slf4j.Logger org$apache$spark$internal$Logging$$log_()

    • org$apache$spark$internal$Logging$$log__$eq

      public static void org$apache$spark$internal$Logging$$log__$eq(org.slf4j.Logger x$1)

    • LogStringContext

      public static org.apache.spark.internal.Logging.LogStringContext LogStringContext(scala.StringContext sc)