org.apache.spark.sql

execution

package execution

:: DeveloperApi :: An execution engine for relational query plans that runs on top Spark and returns RDDs.

Note that the operators in this package are created automatically by a query planner using a SQLContext and are not intended to be used directly by end users of Spark SQL. They are documented here in order to make it easier for others to understand the performance characteristics of query plans that are generated by Spark SQL.

Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. execution
  2. AnyRef
  3. Any
  1. Hide All
  2. Show all
Learn more about member selection
Visibility
  1. Public
  2. All

Type Members

  1. case class Aggregate(partial: Boolean, groupingExpressions: Seq[Expression], aggregateExpressions: Seq[NamedExpression], child: SparkPlan) extends SparkPlan with UnaryNode with Product with Serializable

    :: DeveloperApi :: Groups input data by groupingExpressions and computes the aggregateExpressions for each group.

  2. case class AggregateEvaluation(schema: Seq[Attribute], initialValues: Seq[Expression], update: Seq[Expression], result: Expression) extends Product with Serializable

  3. case class BatchPythonEvaluation(udf: PythonUDF, output: Seq[Attribute], child: SparkPlan) extends SparkPlan with Product with Serializable

    :: DeveloperApi :: Uses PythonRDD to evaluate a PythonUDF, one partition of tuples at a time.

  4. case class CacheTableCommand(tableName: String, plan: Option[LogicalPlan], isLazy: Boolean) extends SparkPlan with LeafNode with Command with Product with Serializable

    :: DeveloperApi ::

  5. trait Command extends AnyRef

  6. case class DescribeCommand(child: SparkPlan, output: Seq[Attribute])(context: SQLContext) extends SparkPlan with LeafNode with Command with Product with Serializable

    :: DeveloperApi ::

  7. case class Distinct(partial: Boolean, child: SparkPlan) extends SparkPlan with UnaryNode with Product with Serializable

    :: DeveloperApi :: Computes the set of distinct input rows using a HashSet.

  8. case class EvaluatePython(udf: PythonUDF, child: LogicalPlan, resultAttribute: AttributeReference) extends catalyst.plans.logical.UnaryNode with Product with Serializable

    :: DeveloperApi :: Evaluates a PythonUDF, appending the result to the end of the input tuple.

  9. case class Except(left: SparkPlan, right: SparkPlan) extends SparkPlan with BinaryNode with Product with Serializable

    :: DeveloperApi :: Returns a table with the elements from left that are not in right using the built-in spark subtract function.

  10. case class Exchange(newPartitioning: Partitioning, child: SparkPlan) extends SparkPlan with UnaryNode with Product with Serializable

    :: DeveloperApi ::

  11. case class ExecutedCommand(cmd: RunnableCommand) extends SparkPlan with Product with Serializable

  12. case class ExplainCommand(logicalPlan: LogicalPlan, output: Seq[Attribute], extended: Boolean)(context: SQLContext) extends SparkPlan with LeafNode with Command with Product with Serializable

    An explain command for users to see how a command will be executed.

  13. case class ExternalSort(sortOrder: Seq[SortOrder], global: Boolean, child: SparkPlan) extends SparkPlan with UnaryNode with Product with Serializable

    :: DeveloperApi :: Performs a sort, spilling to disk as needed.

  14. case class Filter(condition: Expression, child: SparkPlan) extends SparkPlan with UnaryNode with Product with Serializable

    :: DeveloperApi ::

  15. case class Generate(generator: Generator, join: Boolean, outer: Boolean, child: SparkPlan) extends SparkPlan with UnaryNode with Product with Serializable

    :: DeveloperApi :: Applies a Generator to a stream of input rows, combining the output of each into a new stream of rows.

  16. case class GeneratedAggregate(partial: Boolean, groupingExpressions: Seq[Expression], aggregateExpressions: Seq[NamedExpression], child: SparkPlan) extends SparkPlan with UnaryNode with Product with Serializable

    :: DeveloperApi :: Alternate version of aggregation that leverages projection and thus code generation.

  17. case class Intersect(left: SparkPlan, right: SparkPlan) extends SparkPlan with BinaryNode with Product with Serializable

    :: DeveloperApi :: Returns the rows in left that also appear in right using the built in spark intersection function.

  18. case class Limit(limit: Int, child: SparkPlan) extends SparkPlan with UnaryNode with Product with Serializable

    :: DeveloperApi :: Take the first limit elements.

  19. case class LogicalRDD(output: Seq[Attribute], rdd: RDD[Row])(sqlContext: SQLContext) extends LogicalPlan with MultiInstanceRelation with Product with Serializable

  20. case class OutputFaker(output: Seq[Attribute], child: SparkPlan) extends SparkPlan with Product with Serializable

    :: DeveloperApi :: A plan node that does nothing but lie about the output of its child.

  21. case class PhysicalRDD(output: Seq[Attribute], rdd: RDD[Row]) extends SparkPlan with LeafNode with Product with Serializable

  22. case class Project(projectList: Seq[NamedExpression], child: SparkPlan) extends SparkPlan with UnaryNode with Product with Serializable

    :: DeveloperApi ::

  23. class QueryExecutionException extends Exception

  24. trait RunnableCommand extends catalyst.plans.logical.Command

  25. case class Sample(fraction: Double, withReplacement: Boolean, seed: Long, child: SparkPlan) extends SparkPlan with UnaryNode with Product with Serializable

    :: DeveloperApi ::

  26. case class SetCommand(kv: Option[(String, Option[String])], output: Seq[Attribute])(context: SQLContext) extends SparkPlan with LeafNode with Command with Logging with Product with Serializable

    :: DeveloperApi ::

  27. case class Sort(sortOrder: Seq[SortOrder], global: Boolean, child: SparkPlan) extends SparkPlan with UnaryNode with Product with Serializable

    :: DeveloperApi :: Performs a sort on-heap.

  28. abstract class SparkPlan extends QueryPlan[SparkPlan] with Logging with Serializable

    :: DeveloperApi ::

  29. case class TakeOrdered(limit: Int, sortOrder: Seq[SortOrder], child: SparkPlan) extends SparkPlan with UnaryNode with Product with Serializable

    :: DeveloperApi :: Take the first limit elements as defined by the sortOrder.

  30. case class UncacheTableCommand(tableName: String) extends SparkPlan with LeafNode with Command with Product with Serializable

    :: DeveloperApi ::

  31. case class Union(children: Seq[SparkPlan]) extends SparkPlan with Product with Serializable

    :: DeveloperApi ::

  32. case class ExistingRdd(output: Seq[Attribute], rdd: RDD[Row]) extends SparkPlan with LeafNode with Product with Serializable

    Annotations
    @deprecated
    Deprecated

    (Since version 1.2.0) Use LogicalRDD

  33. case class SparkLogicalPlan(alreadyPlanned: SparkPlan)(sqlContext: SQLContext) extends LogicalPlan with MultiInstanceRelation with Product with Serializable

    Annotations
    @deprecated
    Deprecated

    (Since version 1.2.0) Use LogicalRDD

Value Members

  1. object EvaluatePython extends Serializable

  2. object RDDConversions

    :: DeveloperApi ::

  3. object SparkPlan extends Serializable

  4. package debug

    :: DeveloperApi :: Contains methods for debugging query execution.

  5. package joins

    :: DeveloperApi :: Physical execution operators for join operations.

Inherited from AnyRef

Inherited from Any

Ungrouped