Package org.apache.spark.sql

Class SQLImplicits

Object
org.apache.spark.sql.SQLImplicits
All Implemented Interfaces:
LowPrioritySQLImplicits
Direct Known Subclasses:
SparkSession.implicits$, SQLContext.implicits$
public abstract class SQLImplicits extends Object implements LowPrioritySQLImplicits
A collection of implicit methods for converting common Scala objects into Datasets.

Since:
1.6.0

  • Constructor Details

    • SQLImplicits

      public SQLImplicits()

  • Method Details

    • localSeqToDatasetHolder

      public <T> DatasetHolder<T> localSeqToDatasetHolder(scala.collection.Seq<T> s, Encoder<T> evidence$8)
      Creates a Dataset from a local Seq.
      Parameters:
      s - (undocumented)
      evidence$8 - (undocumented)
      Returns:
      (undocumented)
      Since:
      1.6.0

    • newBooleanArrayEncoder

      public Encoder<boolean[]> newBooleanArrayEncoder()
      Since:
      1.6.1

    • newBooleanEncoder

      public Encoder<Object> newBooleanEncoder()
      Since:
      1.6.0

    • newBooleanSeqEncoder

      public Encoder<scala.collection.Seq<Object>> newBooleanSeqEncoder()
      Deprecated.
      use <T>newSequenceEncoder(scala.reflect.api.TypeTags.TypeTag<T>)
      Returns:
      (undocumented)
      Since:
      1.6.1

    • newBoxedBooleanEncoder

      public Encoder<Boolean> newBoxedBooleanEncoder()
      Since:
      2.0.0

    • newBoxedByteEncoder

      public Encoder<Byte> newBoxedByteEncoder()
      Since:
      2.0.0

    • newBoxedDoubleEncoder

      public Encoder<Double> newBoxedDoubleEncoder()
      Since:
      2.0.0

    • newBoxedFloatEncoder

      public Encoder<Float> newBoxedFloatEncoder()
      Since:
      2.0.0

    • newBoxedIntEncoder

      public Encoder<Integer> newBoxedIntEncoder()
      Since:
      2.0.0

    • newBoxedLongEncoder

      public Encoder<Long> newBoxedLongEncoder()
      Since:
      2.0.0

    • newBoxedShortEncoder

      public Encoder<Short> newBoxedShortEncoder()
      Since:
      2.0.0

    • newByteArrayEncoder

      public Encoder<byte[]> newByteArrayEncoder()
      Since:
      1.6.1

    • newByteEncoder

      public Encoder<Object> newByteEncoder()
      Since:
      1.6.0

    • newByteSeqEncoder

      public Encoder<scala.collection.Seq<Object>> newByteSeqEncoder()
      Deprecated.
      use <T>newSequenceEncoder(scala.reflect.api.TypeTags.TypeTag<T>)
      Returns:
      (undocumented)
      Since:
      1.6.1

    • newDateEncoder

      public Encoder<Date> newDateEncoder()
      Since:
      2.2.0

    • newDoubleArrayEncoder

      public Encoder<double[]> newDoubleArrayEncoder()
      Since:
      1.6.1

    • newDoubleEncoder

      public Encoder<Object> newDoubleEncoder()
      Since:
      1.6.0

    • newDoubleSeqEncoder

      public Encoder<scala.collection.Seq<Object>> newDoubleSeqEncoder()
      Deprecated.
      use <T>newSequenceEncoder(scala.reflect.api.TypeTags.TypeTag<T>)
      Returns:
      (undocumented)
      Since:
      1.6.1

    • newDurationEncoder

      public Encoder<Duration> newDurationEncoder()
      Since:
      3.2.0

    • newFloatArrayEncoder

      public Encoder<float[]> newFloatArrayEncoder()
      Since:
      1.6.1

    • newFloatEncoder

      public Encoder<Object> newFloatEncoder()
      Since:
      1.6.0

    • newFloatSeqEncoder

      public Encoder<scala.collection.Seq<Object>> newFloatSeqEncoder()
      Deprecated.
      use <T>newSequenceEncoder(scala.reflect.api.TypeTags.TypeTag<T>)
      Returns:
      (undocumented)
      Since:
      1.6.1

    • newInstantEncoder

      public Encoder<Instant> newInstantEncoder()
      Since:
      3.0.0

    • newIntArrayEncoder

      public Encoder<int[]> newIntArrayEncoder()
      Since:
      1.6.1

    • newIntEncoder

      public Encoder<Object> newIntEncoder()
      Since:
      1.6.0

    • newIntSeqEncoder

      public Encoder<scala.collection.Seq<Object>> newIntSeqEncoder()
      Deprecated.
      use <T>newSequenceEncoder(scala.reflect.api.TypeTags.TypeTag<T>)
      Returns:
      (undocumented)
      Since:
      1.6.1

    • newJavaDecimalEncoder

      public Encoder<BigDecimal> newJavaDecimalEncoder()
      Since:
      2.2.0

    • newJavaEnumEncoder

      public <A extends Enum<?>> Encoder<A> newJavaEnumEncoder(scala.reflect.api.TypeTags.TypeTag<A> evidence$1)
      Since:
      3.2.0

    • newLocalDateEncoder

      public Encoder<LocalDate> newLocalDateEncoder()
      Since:
      3.0.0

    • newLocalDateTimeEncoder

      public Encoder<LocalDateTime> newLocalDateTimeEncoder()
      Since:
      3.4.0

    • newLongArrayEncoder

      public Encoder<long[]> newLongArrayEncoder()
      Since:
      1.6.1

    • newLongEncoder

      public Encoder<Object> newLongEncoder()
      Since:
      1.6.0

    • newLongSeqEncoder

      public Encoder<scala.collection.Seq<Object>> newLongSeqEncoder()
      Deprecated.
      use <T>newSequenceEncoder(scala.reflect.api.TypeTags.TypeTag<T>)
      Returns:
      (undocumented)
      Since:
      1.6.1

    • newMapEncoder

      public <T extends scala.collection.Map<?, ?>> Encoder<T> newMapEncoder(scala.reflect.api.TypeTags.TypeTag<T> evidence$4)
      Since:
      2.3.0

    • newPeriodEncoder

      public Encoder<Period> newPeriodEncoder()
      Since:
      3.2.0

    • newProductArrayEncoder

      public <A extends scala.Product> Encoder<Object> newProductArrayEncoder(scala.reflect.api.TypeTags.TypeTag<A> evidence$6)
      Since:
      1.6.1

    • newProductSeqEncoder

      public <A extends scala.Product> Encoder<scala.collection.Seq<A>> newProductSeqEncoder(scala.reflect.api.TypeTags.TypeTag<A> evidence$2)
      Deprecated.
      use <T>newSequenceEncoder(scala.reflect.api.TypeTags.TypeTag<T>)
      Parameters:
      evidence$2 - (undocumented)
      Returns:
      (undocumented)
      Since:
      1.6.1

    • newScalaDecimalEncoder

      public Encoder<scala.math.BigDecimal> newScalaDecimalEncoder()
      Since:
      2.2.0

    • newSequenceEncoder

      public <T extends scala.collection.Seq<?>> Encoder<T> newSequenceEncoder(scala.reflect.api.TypeTags.TypeTag<T> evidence$3)
      Since:
      2.2.0

    • newSetEncoder

      public <T extends scala.collection.immutable.Set<?>> Encoder<T> newSetEncoder(scala.reflect.api.TypeTags.TypeTag<T> evidence$5)
      Notice that we serialize Set to Catalyst array. The set property is only kept when manipulating the domain objects. The serialization format doesn't keep the set property. When we have a Catalyst array which contains duplicated elements and convert it to Dataset[Set[T} by using the encoder, the elements will be de-duplicated.

      Parameters:
      evidence$5 - (undocumented)
      Returns:
      (undocumented)
      Since:
      2.3.0

    • newShortArrayEncoder

      public Encoder<short[]> newShortArrayEncoder()
      Since:
      1.6.1

    • newShortEncoder

      public Encoder<Object> newShortEncoder()
      Since:
      1.6.0

    • newShortSeqEncoder

      public Encoder<scala.collection.Seq<Object>> newShortSeqEncoder()
      Deprecated.
      use <T>newSequenceEncoder(scala.reflect.api.TypeTags.TypeTag<T>)
      Returns:
      (undocumented)
      Since:
      1.6.1

    • newStringArrayEncoder

      public Encoder<String[]> newStringArrayEncoder()
      Since:
      1.6.1

    • newStringEncoder

      public Encoder<String> newStringEncoder()
      Since:
      1.6.0

    • newStringSeqEncoder

      public Encoder<scala.collection.Seq<String>> newStringSeqEncoder()
      Deprecated.
      use <T>newSequenceEncoder(scala.reflect.api.TypeTags.TypeTag<T>)
      Returns:
      (undocumented)
      Since:
      1.6.1

    • newTimeStampEncoder

      public Encoder<Timestamp> newTimeStampEncoder()
      Since:
      2.2.0

    • rddToDatasetHolder

      public <T> DatasetHolder<T> rddToDatasetHolder(RDD<T> rdd, Encoder<T> evidence$7)
      Creates a Dataset from an RDD.

      Parameters:
      rdd - (undocumented)
      evidence$7 - (undocumented)
      Returns:
      (undocumented)
      Since:
      1.6.0

    • symbolToColumn

      public ColumnName symbolToColumn(scala.Symbol s)
      An implicit conversion that turns a Scala Symbol into a Column.
      Parameters:
      s - (undocumented)
      Returns:
      (undocumented)
      Since:
      1.3.0