org.apache.spark

sql

package sql

Allows the execution of relational queries, including those expressed in SQL using Spark.

Source
package.scala
Linear Supertypes
AnyRef, Any
Ordering
  1. Grouped
  2. Alphabetic
  3. By inheritance
Inherited
  1. sql
  2. AnyRef
  3. Any
  1. Hide All
  2. Show all
Learn more about member selection
Visibility
  1. Public
  2. All

Type Members

  1. class AnalysisException extends Exception with Serializable

    :: DeveloperApi :: Thrown when a query fails to analyze, usually because the query itself is invalid.

  2. class Column extends Logging

    :: Experimental :: A column that will be computed based on the data in a DataFrame.

  3. class ColumnName extends Column

    :: Experimental :: A convenient class used for constructing schema.

  4. class DataFrame extends Queryable with Serializable

    :: Experimental :: A distributed collection of data organized into named columns.

  5. case class DataFrameHolder extends Product with Serializable

    A container for a DataFrame, used for implicit conversions.

  6. final class DataFrameNaFunctions extends AnyRef

    :: Experimental :: Functionality for working with missing data in DataFrames.

  7. class DataFrameReader extends Logging

    :: Experimental :: Interface used to load a DataFrame from external storage systems (e.

  8. final class DataFrameStatFunctions extends AnyRef

    :: Experimental :: Statistic functions for DataFrames.

  9. final class DataFrameWriter extends AnyRef

    :: Experimental :: Interface used to write a DataFrame to external storage systems (e.

  10. class Dataset[T] extends Queryable with Serializable with Logging

    :: Experimental :: A Dataset is a strongly typed collection of objects that can be transformed in parallel using functional or relational operations.

  11. case class DatasetHolder[T] extends Product with Serializable

    A container for a Dataset, used for implicit conversions.

  12. trait Encoder[T] extends Serializable

    :: Experimental :: Used to convert a JVM object of type T to and from the internal Spark SQL representation.

  13. class ExperimentalMethods extends AnyRef

    :: Experimental :: Holder for experimental methods for the bravest.

  14. class GroupedData extends AnyRef

    :: Experimental :: A set of methods for aggregations on a DataFrame, created by DataFrame.groupBy.

  15. class GroupedDataset[K, V] extends Serializable

    :: Experimental :: A Dataset has been logically grouped by a user specified grouping key.

  16. trait Row extends Serializable

    Represents one row of output from a relational operator.

  17. class RowFactory extends AnyRef

  18. class SQLContext extends Logging with Serializable

    The entry point for working with structured data (rows and columns) in Spark.

  19. abstract class SQLImplicits extends AnyRef

    A collection of implicit methods for converting common Scala objects into DataFrames.

  20. class SaveMode extends Enum[SaveMode]

  21. type Strategy = GenericStrategy[SparkPlan]

    Converts a logical plan into zero or more SparkPlans.

    Converts a logical plan into zero or more SparkPlans. This API is exposed for experimenting with the query planner and is not designed to be stable across spark releases. Developers writing libraries should instead consider using the stable APIs provided in org.apache.spark.sql.sources

    Annotations
    @DeveloperApi()
  22. class TypedColumn[-T, U] extends Column

    A Column where an Encoder has been given for the expected input and return type.

  23. class UDFRegistration extends Logging

    Functions for registering user-defined functions.

  24. case class UserDefinedFunction(f: AnyRef, dataType: DataType, inputTypes: Seq[DataType] = immutable.this.Nil) extends Product with Serializable

    A user-defined function.

  25. type SchemaRDD = DataFrame

    Type alias for DataFrame.

    Type alias for DataFrame. Kept here for backward source compatibility for Scala.

    Annotations
    @deprecated
    Deprecated

    (Since version 1.3.0) use DataFrame

Value Members

  1. object Encoders

    :: Experimental :: Methods for creating an Encoder.

  2. object Row extends Serializable

  3. object SQLContext extends Serializable

    This SQLContext object contains utility functions to create a singleton SQLContext instance, or to get the created SQLContext instance.

  4. package api

    Contains API classes that are specific to a single language (i.

  5. package expressions

  6. object functions extends LegacyFunctions

    :: Experimental :: Functions available for DataFrame.

  7. package hive

    Support for running Spark SQL queries using functionality from Apache Hive (does not require an existing Hive installation).

  8. package jdbc

  9. package sources

    A set of APIs for adding data sources to Spark SQL.

  10. package types

    Contains a type system for attributes produced by relations, including complex types like structs, arrays and maps.

  11. package util

Inherited from AnyRef

Inherited from Any

Row

Ungrouped