| repartition {SparkR} | R Documentation |
The following options for repartition are possible:
"Option 1" Return a new SparkDataFrame partitioned by the given columns into 'numPartitions'.
"Option 2" Return a new SparkDataFrame that has exactly 'numPartitions'.
"Option 3" Return a new SparkDataFrame partitioned by the given column(s), using 'spark.sql.shuffle.partitions' as number of partitions.
## S4 method for signature 'SparkDataFrame' repartition(x, numPartitions = NULL, col = NULL, ...)
x |
A SparkDataFrame |
numPartitions |
The number of partitions to use. |
col |
The column by which the partitioning will be performed. |
Other SparkDataFrame functions: SparkDataFrame-class,
[[, agg,
arrange, as.data.frame,
attach, cache,
collect, colnames,
coltypes, columns,
count, dapply,
describe, dim,
distinct, dropDuplicates,
dropna, drop,
dtypes, except,
explain, filter,
first, group_by,
head, histogram,
insertInto, intersect,
isLocal, join,
limit, merge,
mutate, ncol,
persist, printSchema,
registerTempTable, rename,
sample, saveAsTable,
selectExpr, select,
showDF, show,
str, take,
unionAll, unpersist,
withColumn, write.df,
write.jdbc, write.json,
write.parquet, write.text
## Not run:
##D sc <- sparkR.init()
##D sqlContext <- sparkRSQL.init(sc)
##D path <- "path/to/file.json"
##D df <- read.json(sqlContext, path)
##D newDF <- repartition(df, 2L)
##D newDF <- repartition(df, numPartitions = 2L)
##D newDF <- repartition(df, col = df$"col1", df$"col2")
##D newDF <- repartition(df, 3L, col = df$"col1", df$"col2")
## End(Not run)