package util
- Alphabetic
- Public
- Protected
Type Members
- class CaseInsensitiveStringMap extends Map[String, String]
Case-insensitive map of string keys to string values.
Case-insensitive map of string keys to string values.
This is used to pass options to v2 implementations to ensure consistent case insensitivity.
Methods that return keys in this map, like
#entrySet()and#keySet(), return keys converted to lower case. This map doesn't allow null key.- Annotations
- @Experimental()
- Since
3.0.0
- class ExecutionListenerManager extends Logging
Manager for QueryExecutionListener.
Manager for QueryExecutionListener. See
org.apache.spark.sql.SQLContext.listenerManager. - class MapperRowCounter extends AccumulatorV2[Long, List[(Integer, Long)]]
An AccumulatorV2 counter for collecting a list of (mapper index, row count).
An AccumulatorV2 counter for collecting a list of (mapper index, row count).
- Since
3.4.0
- class NumericHistogram extends AnyRef
A generic, re-usable histogram class that supports partial aggregations.
A generic, re-usable histogram class that supports partial aggregations. The algorithm is a heuristic adapted from the following paper: Yael Ben-Haim and Elad Tom-Tov, "A streaming parallel decision tree algorithm", J. Machine Learning Research 11 (2010), pp. 849--872. Although there are no approximation guarantees, it appears to work well with adequate data and a large (e.g., 20-80) number of histogram bins.
Adapted from Hive's NumericHistogram. Can refer to https://github.com/apache/hive/blob/master/ql/src/ java/org/apache/hadoop/hive/ql/udf/generic/NumericHistogram.java
Differences:
- Declaring Coord and it's variables as public types for easy access in the HistogramNumeric class. 2. Add method getNumBins() for serialize NumericHistogram in NumericHistogramSerializer. 3. Add method addBin() for deserialize NumericHistogram in NumericHistogramSerializer. 4. In Hive's code, the method pass a serialized histogram, in Spark, this method pass a deserialized histogram. Here we change the code about merge bins.
- Since
3.3.0
- trait QueryExecutionListener extends AnyRef
The interface of query execution listener that can be used to analyze execution metrics.
The interface of query execution listener that can be used to analyze execution metrics.
- Note
Implementations should guarantee thread-safety as they can be invoked by multiple different threads.
- class SQLOpenHashSet[T] extends AnyRef
- Annotations
- @Private()
Value Members
- object ArtifactUtils
- object ProtobufUtils
- object SQLOpenHashSet