org.apache.spark.sql.connector.distributions

Class Distributions

Object

org.apache.spark.sql.connector.distributions.Distributions

@Experimental
public class Distributions
extends Object

Helper methods to create distributions to pass into Spark.

Since:

3.2.0

Method Summary

Modifier and Type	Method and Description
static ClusteredDistribution	clustered(Expression[] clustering) Creates a distribution where tuples that share the same values for clustering expressions are co-located in the same partition.
static OrderedDistribution	ordered(SortOrder[] ordering) Creates a distribution where tuples have been ordered across partitions according to ordering expressions, but not necessarily within a given partition.
static UnspecifiedDistribution	unspecified() Creates a distribution where no promises are made about co-location of data.

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

unspecified

public static UnspecifiedDistribution unspecified()

Creates a distribution where no promises are made about co-location of data.

clustered

public static ClusteredDistribution clustered(Expression[] clustering)

Creates a distribution where tuples that share the same values for clustering expressions are co-located in the same partition.

ordered

public static OrderedDistribution ordered(SortOrder[] ordering)

Creates a distribution where tuples have been ordered across partitions according to ordering expressions, but not necessarily within a given partition.