R Front End for 'Apache Spark'


[Up] [Top]

Documentation for package ‘SparkR’ version 3.2.4

Help Pages

A B C D E F G H I J K L M N O P Q R S T U V W X Y misc

-- A --

abs Math functions for Column operations
abs-method Math functions for Column operations
acos Math functions for Column operations
acos-method Math functions for Column operations
acosh Math functions for Column operations
acosh-method Math functions for Column operations
add_months Date time arithmetic functions for Column operations
add_months-method Date time arithmetic functions for Column operations
AFTSurvivalRegressionModel-class S4 class that represents a AFTSurvivalRegressionModel
agg summarize
agg-method summarize
alias alias
alias-method alias
ALSModel-class S4 class that represents an ALSModel
approxCountDistinct Aggregate functions for Column operations
approxCountDistinct-method Aggregate functions for Column operations
approxQuantile Calculates the approximate quantiles of numerical columns of a SparkDataFrame
approxQuantile-method Calculates the approximate quantiles of numerical columns of a SparkDataFrame
approx_count_distinct Aggregate functions for Column operations
approx_count_distinct-method Aggregate functions for Column operations
arrange Arrange Rows by Variables
arrange-method Arrange Rows by Variables
arrays_overlap Collection functions for Column operations
arrays_overlap-method Collection functions for Column operations
arrays_zip Collection functions for Column operations
arrays_zip-method Collection functions for Column operations
arrays_zip_with Collection functions for Column operations
arrays_zip_with-method Collection functions for Column operations
array_aggregate Collection functions for Column operations
array_aggregate-method Collection functions for Column operations
array_contains Collection functions for Column operations
array_contains-method Collection functions for Column operations
array_distinct Collection functions for Column operations
array_distinct-method Collection functions for Column operations
array_except Collection functions for Column operations
array_except-method Collection functions for Column operations
array_exists Collection functions for Column operations
array_exists-method Collection functions for Column operations
array_filter Collection functions for Column operations
array_filter-method Collection functions for Column operations
array_forall Collection functions for Column operations
array_forall-method Collection functions for Column operations
array_intersect Collection functions for Column operations
array_intersect-method Collection functions for Column operations
array_join Collection functions for Column operations
array_join-method Collection functions for Column operations
array_max Collection functions for Column operations
array_max-method Collection functions for Column operations
array_min Collection functions for Column operations
array_min-method Collection functions for Column operations
array_position Collection functions for Column operations
array_position-method Collection functions for Column operations
array_remove Collection functions for Column operations
array_remove-method Collection functions for Column operations
array_repeat Collection functions for Column operations
array_repeat-method Collection functions for Column operations
array_sort Collection functions for Column operations
array_sort-method Collection functions for Column operations
array_to_vector ML functions for Column operations
array_to_vector-method ML functions for Column operations
array_transform Collection functions for Column operations
array_transform-method Collection functions for Column operations
array_union Collection functions for Column operations
array_union-method Collection functions for Column operations
as.data.frame Download data from a SparkDataFrame into a R data.frame
as.data.frame-method Download data from a SparkDataFrame into a R data.frame
as.DataFrame Create a SparkDataFrame
asc A set of operations working with SparkDataFrame columns
ascii String functions for Column operations
ascii-method String functions for Column operations
asc_nulls_first A set of operations working with SparkDataFrame columns
asc_nulls_last A set of operations working with SparkDataFrame columns
asin Math functions for Column operations
asin-method Math functions for Column operations
asinh Math functions for Column operations
asinh-method Math functions for Column operations
assert_true Miscellaneous functions for Column operations
assert_true-method Miscellaneous functions for Column operations
associationRules-method FP-growth
atan Math functions for Column operations
atan-method Math functions for Column operations
atan2 Math functions for Column operations
atan2-method Math functions for Column operations
atanh Math functions for Column operations
atanh-method Math functions for Column operations
attach Attach SparkDataFrame to R search path
attach-method Attach SparkDataFrame to R search path
avg avg
avg-method avg
awaitTermination awaitTermination
awaitTermination-method awaitTermination

-- B --

base64 String functions for Column operations
base64-method String functions for Column operations
between between
between-method between
bin Math functions for Column operations
bin-method Math functions for Column operations
BisectingKMeansModel-class S4 class that represents a BisectingKMeansModel
bitwiseNOT Non-aggregate functions for Column operations
bitwiseNOT-method Non-aggregate functions for Column operations
bitwise_not Non-aggregate functions for Column operations
bitwise_not-method Non-aggregate functions for Column operations
broadcast broadcast
broadcast-method broadcast
bround Math functions for Column operations
bround-method Math functions for Column operations

-- C --

cache Cache
cache-method Cache
cacheTable Cache Table
cancelJobGroup Cancel active jobs for the specified group
cast Casts the column to a different data type.
cast-method Casts the column to a different data type.
cbrt Math functions for Column operations
cbrt-method Math functions for Column operations
ceil Math functions for Column operations
ceil-method Math functions for Column operations
ceiling Math functions for Column operations
ceiling-method Math functions for Column operations
checkpoint checkpoint
checkpoint-method checkpoint
clearCache Clear Cache
clearJobGroup Clear current job group ID and its description
coalesce Coalesce
coalesce-method Coalesce
coalesce-method Non-aggregate functions for Column operations
collect Collects all the elements of a SparkDataFrame and coerces them into an R data.frame.
collect-method Collects all the elements of a SparkDataFrame and coerces them into an R data.frame.
collect_list Aggregate functions for Column operations
collect_list-method Aggregate functions for Column operations
collect_set Aggregate functions for Column operations
collect_set-method Aggregate functions for Column operations
colnames Column Names of SparkDataFrame
colnames-method Column Names of SparkDataFrame
colnames<- Column Names of SparkDataFrame
colnames<--method Column Names of SparkDataFrame
coltypes coltypes
coltypes-method coltypes
coltypes<- coltypes
coltypes<--method coltypes
column S4 class that represents a SparkDataFrame column
Column-class S4 class that represents a SparkDataFrame column
column-method S4 class that represents a SparkDataFrame column
columnfunctions A set of operations working with SparkDataFrame columns
columns Column Names of SparkDataFrame
columns-method Column Names of SparkDataFrame
column_aggregate_functions Aggregate functions for Column operations
column_avro_functions Avro processing functions for Column operations
column_collection_functions Collection functions for Column operations
column_datetime_diff_functions Date time arithmetic functions for Column operations
column_datetime_functions Date time functions for Column operations
column_math_functions Math functions for Column operations
column_misc_functions Miscellaneous functions for Column operations
column_ml_functions ML functions for Column operations
column_nonaggregate_functions Non-aggregate functions for Column operations
column_string_functions String functions for Column operations
column_window_functions Window functions for Column operations
concat Collection functions for Column operations
concat-method Collection functions for Column operations
concat_ws String functions for Column operations
concat_ws-method String functions for Column operations
contains A set of operations working with SparkDataFrame columns
conv Math functions for Column operations
conv-method Math functions for Column operations
corr corr
corr-method corr
cos Math functions for Column operations
cos-method Math functions for Column operations
cosh Math functions for Column operations
cosh-method Math functions for Column operations
count Count
count-method Count
count-method Returns the number of rows in a SparkDataFrame
countDistinct Aggregate functions for Column operations
countDistinct-method Aggregate functions for Column operations
count_distinct Aggregate functions for Column operations
count_distinct-method Aggregate functions for Column operations
cov cov
cov-method cov
covar_pop cov
covar_pop-method cov
covar_samp cov
covar_samp-method cov
crc32 Miscellaneous functions for Column operations
crc32-method Miscellaneous functions for Column operations
createDataFrame Create a SparkDataFrame
createExternalTable (Deprecated) Create an external table
createOrReplaceTempView Creates a temporary view using the given name.
createOrReplaceTempView-method Creates a temporary view using the given name.
createTable Creates a table based on the dataset in a data source
create_array Non-aggregate functions for Column operations
create_array-method Non-aggregate functions for Column operations
create_lambda Create o.a.s.sql.expressions.LambdaFunction corresponding to transformation described by func. Used by higher order functions.
create_map Non-aggregate functions for Column operations
create_map-method Non-aggregate functions for Column operations
crossJoin CrossJoin
crossJoin-method CrossJoin
crosstab Computes a pair-wise frequency table of the given columns
crosstab-method Computes a pair-wise frequency table of the given columns
cube cube
cube-method cube
cume_dist Window functions for Column operations
cume_dist-method Window functions for Column operations
currentDatabase Returns the current default database
current_date Date time functions for Column operations
current_date-method Date time functions for Column operations
current_timestamp Date time functions for Column operations
current_timestamp-method Date time functions for Column operations

-- D --

dapply dapply
dapply-method dapply
dapplyCollect dapplyCollect
dapplyCollect-method dapplyCollect
datediff Date time arithmetic functions for Column operations
datediff-method Date time arithmetic functions for Column operations
date_add Date time arithmetic functions for Column operations
date_add-method Date time arithmetic functions for Column operations
date_format Date time arithmetic functions for Column operations
date_format-method Date time arithmetic functions for Column operations
date_sub Date time arithmetic functions for Column operations
date_sub-method Date time arithmetic functions for Column operations
date_trunc Date time functions for Column operations
date_trunc-method Date time functions for Column operations
dayofmonth Date time functions for Column operations
dayofmonth-method Date time functions for Column operations
dayofweek Date time functions for Column operations
dayofweek-method Date time functions for Column operations
dayofyear Date time functions for Column operations
dayofyear-method Date time functions for Column operations
DecisionTreeClassificationModel-class S4 class that represents a DecisionTreeClassificationModel
DecisionTreeRegressionModel-class S4 class that represents a DecisionTreeRegressionModel
decode String functions for Column operations
decode-method String functions for Column operations
degrees Math functions for Column operations
degrees-method Math functions for Column operations
dense_rank Window functions for Column operations
dense_rank-method Window functions for Column operations
desc A set of operations working with SparkDataFrame columns
describe describe
describe-method describe
desc_nulls_first A set of operations working with SparkDataFrame columns
desc_nulls_last A set of operations working with SparkDataFrame columns
dim Returns the dimensions of SparkDataFrame
dim-method Returns the dimensions of SparkDataFrame
distinct Distinct
distinct-method Distinct
drop drop
drop-method drop
dropDuplicates dropDuplicates
dropDuplicates-method dropDuplicates
dropFields dropFields
dropFields-method dropFields
dropna A set of SparkDataFrame functions working with NA values
dropna-method A set of SparkDataFrame functions working with NA values
dropTempTable (Deprecated) Drop Temporary Table
dropTempView Drops the temporary view with the given view name in the catalog.
dtypes DataTypes
dtypes-method DataTypes

-- E --

element_at Collection functions for Column operations
element_at-method Collection functions for Column operations
encode String functions for Column operations
encode-method String functions for Column operations
endsWith endsWith
endsWith-method endsWith
except except
except-method except
exceptAll exceptAll
exceptAll-method exceptAll
exp Math functions for Column operations
exp-method Math functions for Column operations
explain Explain
explain-method Explain
explode Collection functions for Column operations
explode-method Collection functions for Column operations
explode_outer Collection functions for Column operations
explode_outer-method Collection functions for Column operations
expm1 Math functions for Column operations
expm1-method Math functions for Column operations
expr Non-aggregate functions for Column operations
expr-method Non-aggregate functions for Column operations

-- F --

factorial Math functions for Column operations
factorial-method Math functions for Column operations
fillna A set of SparkDataFrame functions working with NA values
fillna-method A set of SparkDataFrame functions working with NA values
filter Filter
filter-method Filter
findFrequentSequentialPatterns-method PrefixSpan
first Return the first row of a SparkDataFrame
first-method Return the first row of a SparkDataFrame
fitted Get fitted result from a k-means model
fitted-method Get fitted result from a k-means model
fitted-method Bisecting K-Means Clustering Model
flatten Collection functions for Column operations
flatten-method Collection functions for Column operations
floor Math functions for Column operations
floor-method Math functions for Column operations
FMClassificationModel-class S4 class that represents a FMClassificationModel
FMRegressionModel-class S4 class that represents a FMRegressionModel
format_number String functions for Column operations
format_number-method String functions for Column operations
format_string String functions for Column operations
format_string-method String functions for Column operations
FPGrowthModel-class S4 class that represents a FPGrowthModel
freqItems Finding frequent items for columns, possibly with false positives
freqItems-method Finding frequent items for columns, possibly with false positives
freqItemsets-method FP-growth
from_avro Avro processing functions for Column operations
from_avro-method Avro processing functions for Column operations
from_csv Collection functions for Column operations
from_csv-method Collection functions for Column operations
from_json Collection functions for Column operations
from_json-method Collection functions for Column operations
from_unixtime Date time functions for Column operations
from_unixtime-method Date time functions for Column operations
from_utc_timestamp Date time arithmetic functions for Column operations
from_utc_timestamp-method Date time arithmetic functions for Column operations

-- G --

gapply gapply
gapply-method gapply
gapplyCollect gapplyCollect
gapplyCollect-method gapplyCollect
GaussianMixtureModel-class S4 class that represents a GaussianMixtureModel
GBTClassificationModel-class S4 class that represents a GBTClassificationModel
GBTRegressionModel-class S4 class that represents a GBTRegressionModel
GeneralizedLinearRegressionModel-class S4 class that represents a generalized linear model
getField A set of operations working with SparkDataFrame columns
getItem A set of operations working with SparkDataFrame columns
getLocalProperty Get a local property set in this thread, or 'NULL' if it is missing. See 'setLocalProperty'.
getNumPartitions getNumPartitions
getNumPartitions-method getNumPartitions
glm Generalized Linear Models (R-compliant)
glm-method Generalized Linear Models (R-compliant)
greatest Non-aggregate functions for Column operations
greatest-method Non-aggregate functions for Column operations
groupBy GroupBy
groupBy-method GroupBy
groupedData S4 class that represents a GroupedData
GroupedData-class S4 class that represents a GroupedData
grouping_bit Aggregate functions for Column operations
grouping_bit-method Aggregate functions for Column operations
grouping_id Aggregate functions for Column operations
grouping_id-method Aggregate functions for Column operations
group_by GroupBy
group_by-method GroupBy

-- H --

hash Miscellaneous functions for Column operations
hash-method Miscellaneous functions for Column operations
hashCode Compute the hashCode of an object
head Head
head-method Head
hex Math functions for Column operations
hex-method Math functions for Column operations
hint hint
hint-method hint
histogram Compute histogram statistics for given column
histogram-method Compute histogram statistics for given column
hour Date time functions for Column operations
hour-method Date time functions for Column operations
hypot Math functions for Column operations
hypot-method Math functions for Column operations

-- I --

ifelse Non-aggregate functions for Column operations
ifelse-method Non-aggregate functions for Column operations
initcap String functions for Column operations
initcap-method String functions for Column operations
input_file_name Non-aggregate functions for Column operations
input_file_name-method Non-aggregate functions for Column operations
insertInto insertInto
insertInto-method insertInto
install.spark Download and Install Apache Spark to a Local Directory
instr String functions for Column operations
instr-method String functions for Column operations
intersect Intersect
intersect-method Intersect
intersectAll intersectAll
intersectAll-method intersectAll
invoke_higher_order_function Invokes higher order function expression identified by name, (relative to o.a.s.sql.catalyst.expressions)
is.nan Non-aggregate functions for Column operations
is.nan-method Non-aggregate functions for Column operations
isActive isActive
isActive-method isActive
isLocal isLocal
isLocal-method isLocal
isNaN A set of operations working with SparkDataFrame columns
isnan Non-aggregate functions for Column operations
isnan-method Non-aggregate functions for Column operations
isNotNull A set of operations working with SparkDataFrame columns
isNull A set of operations working with SparkDataFrame columns
IsotonicRegressionModel-class S4 class that represents an IsotonicRegressionModel
isStreaming isStreaming
isStreaming-method isStreaming

-- J --

join Join
join-method Join

-- K --

KMeansModel-class S4 class that represents a KMeansModel
KSTest-class S4 class that represents an KSTest
kurtosis Aggregate functions for Column operations
kurtosis-method Aggregate functions for Column operations

-- L --

lag Window functions for Column operations
lag-method Window functions for Column operations
last last
last-method last
lastProgress lastProgress
lastProgress-method lastProgress
last_day Date time functions for Column operations
last_day-method Date time functions for Column operations
LDAModel-class S4 class that represents an LDAModel
lead Window functions for Column operations
lead-method Window functions for Column operations
least Non-aggregate functions for Column operations
least-method Non-aggregate functions for Column operations
length String functions for Column operations
length-method String functions for Column operations
levenshtein String functions for Column operations
levenshtein-method String functions for Column operations
like A set of operations working with SparkDataFrame columns
limit Limit
limit-method Limit
LinearRegressionModel-class S4 class that represents a LinearRegressionModel
LinearSVCModel-class S4 class that represents an LinearSVCModel
listColumns Returns a list of columns for the given table/view in the specified database
listDatabases Returns a list of databases available
listFunctions Returns a list of functions registered in the specified database
listTables Returns a list of tables or views in the specified database
lit Non-aggregate functions for Column operations
lit-method Non-aggregate functions for Column operations
loadDF Load a SparkDataFrame
localCheckpoint localCheckpoint
localCheckpoint-method localCheckpoint
locate String functions for Column operations
locate-method String functions for Column operations
log Math functions for Column operations
log-method Math functions for Column operations
log10 Math functions for Column operations
log10-method Math functions for Column operations
log1p Math functions for Column operations
log1p-method Math functions for Column operations
log2 Math functions for Column operations
log2-method Math functions for Column operations
LogisticRegressionModel-class S4 class that represents an LogisticRegressionModel
lower String functions for Column operations
lower-method String functions for Column operations
lpad String functions for Column operations
lpad-method String functions for Column operations
ltrim String functions for Column operations
ltrim-method String functions for Column operations

-- M --

map_concat Collection functions for Column operations
map_concat-method Collection functions for Column operations
map_entries Collection functions for Column operations
map_entries-method Collection functions for Column operations
map_filter Collection functions for Column operations
map_filter-method Collection functions for Column operations
map_from_arrays Collection functions for Column operations
map_from_arrays-method Collection functions for Column operations
map_from_entries Collection functions for Column operations
map_from_entries-method Collection functions for Column operations
map_keys Collection functions for Column operations
map_keys-method Collection functions for Column operations
map_values Collection functions for Column operations
map_values-method Collection functions for Column operations
map_zip_with Collection functions for Column operations
map_zip_with-method Collection functions for Column operations
max Aggregate functions for Column operations
max-method Aggregate functions for Column operations
md5 Miscellaneous functions for Column operations
md5-method Miscellaneous functions for Column operations
mean Aggregate functions for Column operations
mean-method Aggregate functions for Column operations
merge Merges two data frames
merge-method Merges two data frames
min Aggregate functions for Column operations
min-method Aggregate functions for Column operations
minute Date time functions for Column operations
minute-method Date time functions for Column operations
monotonically_increasing_id Non-aggregate functions for Column operations
monotonically_increasing_id-method Non-aggregate functions for Column operations
month Date time functions for Column operations
month-method Date time functions for Column operations
months_between Date time arithmetic functions for Column operations
months_between-method Date time arithmetic functions for Column operations
MultilayerPerceptronClassificationModel-class S4 class that represents a MultilayerPerceptronClassificationModel
mutate Mutate
mutate-method Mutate

-- N --

n Count
n-method Count
na.omit A set of SparkDataFrame functions working with NA values
na.omit-method A set of SparkDataFrame functions working with NA values
NaiveBayesModel-class S4 class that represents a NaiveBayesModel
names Column Names of SparkDataFrame
names-method Column Names of SparkDataFrame
names<- Column Names of SparkDataFrame
names<--method Column Names of SparkDataFrame
nanvl Non-aggregate functions for Column operations
nanvl-method Non-aggregate functions for Column operations
ncol Returns the number of columns in a SparkDataFrame
ncol-method Returns the number of columns in a SparkDataFrame
negate Non-aggregate functions for Column operations
negate-method Non-aggregate functions for Column operations
next_day Date time arithmetic functions for Column operations
next_day-method Date time arithmetic functions for Column operations
not !
not-method !
nrow Returns the number of rows in a SparkDataFrame
nrow-method Returns the number of rows in a SparkDataFrame
nth_value Window functions for Column operations
nth_value-method Window functions for Column operations
ntile Window functions for Column operations
ntile-method Window functions for Column operations
n_distinct Aggregate functions for Column operations
n_distinct-method Aggregate functions for Column operations

-- O --

orderBy Ordering Columns in a WindowSpec
orderBy-method Arrange Rows by Variables
orderBy-method Ordering Columns in a WindowSpec
otherwise otherwise
otherwise-method otherwise
over over
over-method over
overlay String functions for Column operations
overlay-method String functions for Column operations

-- P --

partitionBy partitionBy
partitionBy-method partitionBy
percentile_approx Aggregate functions for Column operations
percentile_approx-method Aggregate functions for Column operations
percent_rank Window functions for Column operations
percent_rank-method Window functions for Column operations
persist Persist
persist-method Persist
pivot Pivot a column of the GroupedData and perform the specified aggregation.
pivot-method Pivot a column of the GroupedData and perform the specified aggregation.
pmod Math functions for Column operations
pmod-method Math functions for Column operations
posexplode Collection functions for Column operations
posexplode-method Collection functions for Column operations
posexplode_outer Collection functions for Column operations
posexplode_outer-method Collection functions for Column operations
PowerIterationClustering-class S4 class that represents a PowerIterationClustering
predict Makes predictions from a MLlib model
predict-method Alternating Least Squares (ALS) for Collaborative Filtering
predict-method Bisecting K-Means Clustering Model
predict-method Decision Tree Model for Regression and Classification
predict-method Factorization Machines Classification Model
predict-method Factorization Machines Regression Model
predict-method FP-growth
predict-method Multivariate Gaussian Mixture Model (GMM)
predict-method Gradient Boosted Tree Model for Regression and Classification
predict-method Generalized Linear Models
predict-method Isotonic Regression Model
predict-method K-Means Clustering Model
predict-method Linear Regression Model
predict-method Logistic Regression Model
predict-method Multilayer Perceptron Classification Model
predict-method Naive Bayes Models
predict-method Random Forest Model for Regression and Classification
predict-method Accelerated Failure Time (AFT) Survival Regression Model
predict-method Linear SVM Model
PrefixSpan-class S4 class that represents a PrefixSpan
print.jobj Print a JVM object reference.
print.structField Print a Spark StructField.
print.structType Print a Spark StructType.
print.summary.DecisionTreeClassificationModel Decision Tree Model for Regression and Classification
print.summary.DecisionTreeRegressionModel Decision Tree Model for Regression and Classification
print.summary.GBTClassificationModel Gradient Boosted Tree Model for Regression and Classification
print.summary.GBTRegressionModel Gradient Boosted Tree Model for Regression and Classification
print.summary.GeneralizedLinearRegressionModel Generalized Linear Models
print.summary.KSTest (One-Sample) Kolmogorov-Smirnov Test
print.summary.RandomForestClassificationModel Random Forest Model for Regression and Classification
print.summary.RandomForestRegressionModel Random Forest Model for Regression and Classification
printSchema Print Schema of a SparkDataFrame
printSchema-method Print Schema of a SparkDataFrame
product Aggregate functions for Column operations
product-method Aggregate functions for Column operations

-- Q --

quarter Date time functions for Column operations
quarter-method Date time functions for Column operations
queryName queryName
queryName-method queryName

-- R --

radians Math functions for Column operations
radians-method Math functions for Column operations
raise_error Miscellaneous functions for Column operations
raise_error-method Miscellaneous functions for Column operations
rand Non-aggregate functions for Column operations
rand-method Non-aggregate functions for Column operations
randn Non-aggregate functions for Column operations
randn-method Non-aggregate functions for Column operations
RandomForestClassificationModel-class S4 class that represents a RandomForestClassificationModel
RandomForestRegressionModel-class S4 class that represents a RandomForestRegressionModel
randomSplit randomSplit
randomSplit-method randomSplit
rangeBetween rangeBetween
rangeBetween-method rangeBetween
rank Window functions for Column operations
rank-method Window functions for Column operations
rbind Union two or more SparkDataFrames
rbind-method Union two or more SparkDataFrames
read.df Load a SparkDataFrame
read.jdbc Create a SparkDataFrame representing the database table accessible via JDBC URL
read.json Create a SparkDataFrame from a JSON file.
read.ml Load a fitted MLlib model from the input path.
read.orc Create a SparkDataFrame from an ORC file.
read.parquet Create a SparkDataFrame from a Parquet file.
read.stream Load a streaming SparkDataFrame
read.text Create a SparkDataFrame from a text file.
recoverPartitions Recovers all the partitions in the directory of a table and update the catalog
refreshByPath Invalidates and refreshes all the cached data and metadata for SparkDataFrame containing path
refreshTable Invalidates and refreshes all the cached data and metadata of the given table
regexp_extract String functions for Column operations
regexp_extract-method String functions for Column operations
regexp_replace String functions for Column operations
regexp_replace-method String functions for Column operations
registerTempTable (Deprecated) Register Temporary Table
registerTempTable-method (Deprecated) Register Temporary Table
rename rename
rename-method rename
repartition Repartition
repartition-method Repartition
repartitionByRange Repartition by range
repartitionByRange-method Repartition by range
repeat_string String functions for Column operations
repeat_string-method String functions for Column operations
reverse Collection functions for Column operations
reverse-method Collection functions for Column operations
rint Math functions for Column operations
rint-method Math functions for Column operations
rlike A set of operations working with SparkDataFrame columns
rollup rollup
rollup-method rollup
round Math functions for Column operations
round-method Math functions for Column operations
rowsBetween rowsBetween
rowsBetween-method rowsBetween
row_number Window functions for Column operations
row_number-method Window functions for Column operations
rpad String functions for Column operations
rpad-method String functions for Column operations
rtrim String functions for Column operations
rtrim-method String functions for Column operations

-- S --

sample Sample
sample-method Sample
sampleBy Returns a stratified sample without replacement
sampleBy-method Returns a stratified sample without replacement
sample_frac Sample
sample_frac-method Sample
saveAsTable Save the contents of the SparkDataFrame to a data source as a table
saveAsTable-method Save the contents of the SparkDataFrame to a data source as a table
saveDF Save the contents of SparkDataFrame to a data source.
saveDF-method Save the contents of SparkDataFrame to a data source.
schema Get schema object
schema-method Get schema object
schema_of_csv Collection functions for Column operations
schema_of_csv-method Collection functions for Column operations
schema_of_json Collection functions for Column operations
schema_of_json-method Collection functions for Column operations
sd Aggregate functions for Column operations
sd-method Aggregate functions for Column operations
second Date time functions for Column operations
second-method Date time functions for Column operations
select Select
select-method Select
selectExpr SelectExpr
selectExpr-method SelectExpr
setCheckpointDir Set checkpoint directory
setCurrentDatabase Sets the current default database
setJobDescription Set a human readable description of the current job.
setJobGroup Assigns a group ID to all the jobs started by this thread until the group ID is set to a different value or cleared.
setLocalProperty Set a local property that affects jobs submitted from this thread, such as the Spark fair scheduler pool.
setLogLevel Set new log level
sha1 Miscellaneous functions for Column operations
sha1-method Miscellaneous functions for Column operations
sha2 Miscellaneous functions for Column operations
sha2-method Miscellaneous functions for Column operations
shiftLeft Math functions for Column operations
shiftleft Math functions for Column operations
shiftLeft-method Math functions for Column operations
shiftleft-method Math functions for Column operations
shiftRight Math functions for Column operations
shiftright Math functions for Column operations
shiftRight-method Math functions for Column operations
shiftright-method Math functions for Column operations
shiftRightUnsigned Math functions for Column operations
shiftrightunsigned Math functions for Column operations
shiftRightUnsigned-method Math functions for Column operations
shiftrightunsigned-method Math functions for Column operations
show show
show-method show
showDF showDF
showDF-method showDF
shuffle Collection functions for Column operations
shuffle-method Collection functions for Column operations
sign Math functions for Column operations
sign-method Math functions for Column operations
signum Math functions for Column operations
signum-method Math functions for Column operations
sin Math functions for Column operations
sin-method Math functions for Column operations
sinh Math functions for Column operations
sinh-method Math functions for Column operations
size Collection functions for Column operations
size-method Collection functions for Column operations
skewness Aggregate functions for Column operations
skewness-method Aggregate functions for Column operations
slice Collection functions for Column operations
slice-method Collection functions for Column operations
sort_array Collection functions for Column operations
sort_array-method Collection functions for Column operations
soundex String functions for Column operations
soundex-method String functions for Column operations
spark.addFile Add a file or directory to be downloaded with this Spark job on every node.
spark.als Alternating Least Squares (ALS) for Collaborative Filtering
spark.als-method Alternating Least Squares (ALS) for Collaborative Filtering
spark.assignClusters PowerIterationClustering
spark.assignClusters-method PowerIterationClustering
spark.associationRules FP-growth
spark.associationRules-method FP-growth
spark.bisectingKmeans Bisecting K-Means Clustering Model
spark.bisectingKmeans-method Bisecting K-Means Clustering Model
spark.decisionTree Decision Tree Model for Regression and Classification
spark.decisionTree-method Decision Tree Model for Regression and Classification
spark.findFrequentSequentialPatterns PrefixSpan
spark.findFrequentSequentialPatterns-method PrefixSpan
spark.fmClassifier Factorization Machines Classification Model
spark.fmClassifier-method Factorization Machines Classification Model
spark.fmRegressor Factorization Machines Regression Model
spark.fmRegressor-method Factorization Machines Regression Model
spark.fpGrowth FP-growth
spark.fpGrowth-method FP-growth
spark.freqItemsets FP-growth
spark.freqItemsets-method FP-growth
spark.gaussianMixture Multivariate Gaussian Mixture Model (GMM)
spark.gaussianMixture-method Multivariate Gaussian Mixture Model (GMM)
spark.gbt Gradient Boosted Tree Model for Regression and Classification
spark.gbt-method Gradient Boosted Tree Model for Regression and Classification
spark.getSparkFiles Get the absolute path of a file added through spark.addFile.
spark.getSparkFilesRootDirectory Get the root directory that contains files added through spark.addFile.
spark.glm Generalized Linear Models
spark.glm-method Generalized Linear Models
spark.isoreg Isotonic Regression Model
spark.isoreg-method Isotonic Regression Model
spark.kmeans K-Means Clustering Model
spark.kmeans-method K-Means Clustering Model
spark.kstest (One-Sample) Kolmogorov-Smirnov Test
spark.kstest-method (One-Sample) Kolmogorov-Smirnov Test
spark.lapply Run a function over a list of elements, distributing the computations with Spark
spark.lda Latent Dirichlet Allocation
spark.lda-method Latent Dirichlet Allocation
spark.lm Linear Regression Model
spark.lm-method Linear Regression Model
spark.logit Logistic Regression Model
spark.logit-method Logistic Regression Model
spark.mlp Multilayer Perceptron Classification Model
spark.mlp-method Multilayer Perceptron Classification Model
spark.naiveBayes Naive Bayes Models
spark.naiveBayes-method Naive Bayes Models
spark.perplexity Latent Dirichlet Allocation
spark.perplexity-method Latent Dirichlet Allocation
spark.posterior Latent Dirichlet Allocation
spark.posterior-method Latent Dirichlet Allocation
spark.randomForest Random Forest Model for Regression and Classification
spark.randomForest-method Random Forest Model for Regression and Classification
spark.survreg Accelerated Failure Time (AFT) Survival Regression Model
spark.survreg-method Accelerated Failure Time (AFT) Survival Regression Model
spark.svmLinear Linear SVM Model
spark.svmLinear-method Linear SVM Model
SparkDataFrame-class S4 class that represents a SparkDataFrame
sparkR.callJMethod Call Java Methods
sparkR.callJStatic Call Static Java Methods
sparkR.conf Get Runtime Config from the current active SparkSession
sparkR.init (Deprecated) Initialize a new Spark Context
sparkR.newJObject Create Java Objects
sparkR.session Get the existing SparkSession or initialize a new SparkSession.
sparkR.session.stop Stop the Spark Session and Spark Context
sparkR.stop Stop the Spark Session and Spark Context
sparkR.uiWebUrl Get the URL of the SparkUI instance for the current active SparkSession
sparkR.version Get version of Spark on which this application is running
sparkRHive.init (Deprecated) Initialize a new HiveContext
sparkRSQL.init (Deprecated) Initialize a new SQLContext
spark_partition_id Non-aggregate functions for Column operations
spark_partition_id-method Non-aggregate functions for Column operations
split_string String functions for Column operations
split_string-method String functions for Column operations
sql SQL Query
sqrt Math functions for Column operations
sqrt-method Math functions for Column operations
startsWith startsWith
startsWith-method startsWith
status status
status-method status
stddev Aggregate functions for Column operations
stddev-method Aggregate functions for Column operations
stddev_pop Aggregate functions for Column operations
stddev_pop-method Aggregate functions for Column operations
stddev_samp Aggregate functions for Column operations
stddev_samp-method Aggregate functions for Column operations
stopQuery stopQuery
stopQuery-method stopQuery
storageLevel StorageLevel
storageLevel-method StorageLevel
str Compactly display the structure of a dataset
str-method Compactly display the structure of a dataset
StreamingQuery-class S4 class that represents a StreamingQuery
struct Non-aggregate functions for Column operations
struct-method Non-aggregate functions for Column operations
structField structField
structField.character structField
structField.jobj structField
structType structType
structType.character structType
structType.jobj structType
structType.structField structType
subset Subset
subset-method Subset
substr substr
substr-method substr
substring_index String functions for Column operations
substring_index-method String functions for Column operations
sum Aggregate functions for Column operations
sum-method Aggregate functions for Column operations
sumDistinct Aggregate functions for Column operations
sumDistinct-method Aggregate functions for Column operations
summarize summarize
summarize-method summarize
summary summary
summary-method Alternating Least Squares (ALS) for Collaborative Filtering
summary-method Bisecting K-Means Clustering Model
summary-method Decision Tree Model for Regression and Classification
summary-method Factorization Machines Classification Model
summary-method Factorization Machines Regression Model
summary-method Multivariate Gaussian Mixture Model (GMM)
summary-method Gradient Boosted Tree Model for Regression and Classification
summary-method Generalized Linear Models
summary-method Isotonic Regression Model
summary-method K-Means Clustering Model
summary-method (One-Sample) Kolmogorov-Smirnov Test
summary-method Latent Dirichlet Allocation
summary-method Linear Regression Model
summary-method Logistic Regression Model
summary-method Multilayer Perceptron Classification Model
summary-method Naive Bayes Models
summary-method Random Forest Model for Regression and Classification
summary-method Accelerated Failure Time (AFT) Survival Regression Model
summary-method Linear SVM Model
summary-method summary
sum_distinct Aggregate functions for Column operations
sum_distinct-method Aggregate functions for Column operations

-- T --

tableNames Table Names
tables Tables
tableToDF Create a SparkDataFrame from a SparkSQL table or view
take Take the first NUM rows of a SparkDataFrame and return the results as a R data.frame
take-method Take the first NUM rows of a SparkDataFrame and return the results as a R data.frame
tan Math functions for Column operations
tan-method Math functions for Column operations
tanh Math functions for Column operations
tanh-method Math functions for Column operations
timestamp_seconds Date time functions for Column operations
timestamp_seconds-method Date time functions for Column operations
toDegrees Math functions for Column operations
toDegrees-method Math functions for Column operations
toJSON toJSON
toJSON-method toJSON
toRadians Math functions for Column operations
toRadians-method Math functions for Column operations
to_avro Avro processing functions for Column operations
to_avro-method Avro processing functions for Column operations
to_csv Collection functions for Column operations
to_csv-method Collection functions for Column operations
to_date Date time functions for Column operations
to_date-method Date time functions for Column operations
to_json Collection functions for Column operations
to_json-method Collection functions for Column operations
to_timestamp Date time functions for Column operations
to_timestamp-method Date time functions for Column operations
to_utc_timestamp Date time arithmetic functions for Column operations
to_utc_timestamp-method Date time arithmetic functions for Column operations
transform Mutate
transform-method Mutate
transform_keys Collection functions for Column operations
transform_keys-method Collection functions for Column operations
transform_values Collection functions for Column operations
transform_values-method Collection functions for Column operations
translate String functions for Column operations
translate-method String functions for Column operations
trim String functions for Column operations
trim-method String functions for Column operations
trunc Date time functions for Column operations
trunc-method Date time functions for Column operations

-- U --

unbase64 String functions for Column operations
unbase64-method String functions for Column operations
uncacheTable Uncache Table
unhex Math functions for Column operations
unhex-method Math functions for Column operations
union Return a new SparkDataFrame containing the union of rows
union-method Return a new SparkDataFrame containing the union of rows
unionAll Return a new SparkDataFrame containing the union of rows.
unionAll-method Return a new SparkDataFrame containing the union of rows.
unionByName Return a new SparkDataFrame containing the union of rows, matched by column names
unionByName-method Return a new SparkDataFrame containing the union of rows, matched by column names
unique Distinct
unique-method Distinct
unix_timestamp Date time functions for Column operations
unix_timestamp-method Date time functions for Column operations
unpersist Unpersist
unpersist-method Unpersist
unresolved_named_lambda_var Create o.a.s.sql.expressions.UnresolvedNamedLambdaVariable, convert it to o.s.sql.Column and wrap with R Column. Used by higher order functions.
upper String functions for Column operations
upper-method String functions for Column operations

-- V --

var Aggregate functions for Column operations
var-method Aggregate functions for Column operations
variance Aggregate functions for Column operations
variance-method Aggregate functions for Column operations
var_pop Aggregate functions for Column operations
var_pop-method Aggregate functions for Column operations
var_samp Aggregate functions for Column operations
var_samp-method Aggregate functions for Column operations
vector_to_array ML functions for Column operations
vector_to_array-method ML functions for Column operations

-- W --

weekofyear Date time functions for Column operations
weekofyear-method Date time functions for Column operations
when Non-aggregate functions for Column operations
when-method Non-aggregate functions for Column operations
where Filter
where-method Filter
window Date time functions for Column operations
window-method Date time functions for Column operations
windowOrderBy windowOrderBy
windowOrderBy-method windowOrderBy
windowPartitionBy windowPartitionBy
windowPartitionBy-method windowPartitionBy
WindowSpec-class S4 class that represents a WindowSpec
with Evaluate a R expression in an environment constructed from a SparkDataFrame
with-method Evaluate a R expression in an environment constructed from a SparkDataFrame
withColumn WithColumn
withColumn-method WithColumn
withColumnRenamed rename
withColumnRenamed-method rename
withField withField
withField-method withField
withWatermark withWatermark
withWatermark-method withWatermark
write.df Save the contents of SparkDataFrame to a data source.
write.df-method Save the contents of SparkDataFrame to a data source.
write.jdbc Save the content of SparkDataFrame to an external database table via JDBC.
write.jdbc-method Save the content of SparkDataFrame to an external database table via JDBC.
write.json Save the contents of SparkDataFrame as a JSON file
write.json-method Save the contents of SparkDataFrame as a JSON file
write.ml Saves the MLlib model to the input path
write.ml-method Alternating Least Squares (ALS) for Collaborative Filtering
write.ml-method Bisecting K-Means Clustering Model
write.ml-method Decision Tree Model for Regression and Classification
write.ml-method Factorization Machines Classification Model
write.ml-method Factorization Machines Regression Model
write.ml-method FP-growth
write.ml-method Multivariate Gaussian Mixture Model (GMM)
write.ml-method Gradient Boosted Tree Model for Regression and Classification
write.ml-method Generalized Linear Models
write.ml-method Isotonic Regression Model
write.ml-method K-Means Clustering Model
write.ml-method Latent Dirichlet Allocation
write.ml-method Linear Regression Model
write.ml-method Logistic Regression Model
write.ml-method Multilayer Perceptron Classification Model
write.ml-method Naive Bayes Models
write.ml-method Random Forest Model for Regression and Classification
write.ml-method Accelerated Failure Time (AFT) Survival Regression Model
write.ml-method Linear SVM Model
write.orc Save the contents of SparkDataFrame as an ORC file, preserving the schema.
write.orc-method Save the contents of SparkDataFrame as an ORC file, preserving the schema.
write.parquet Save the contents of SparkDataFrame as a Parquet file, preserving the schema.
write.parquet-method Save the contents of SparkDataFrame as a Parquet file, preserving the schema.
write.stream Write the streaming SparkDataFrame to a data source.
write.stream-method Write the streaming SparkDataFrame to a data source.
write.text Save the content of SparkDataFrame in a text file at the specified path.
write.text-method Save the content of SparkDataFrame in a text file at the specified path.

-- X --

xxhash64 Miscellaneous functions for Column operations
xxhash64-method Miscellaneous functions for Column operations

-- Y --

year Date time functions for Column operations
year-method Date time functions for Column operations

-- misc --

!-method !
$ Select
$-method Select
$<- Select
$<--method Select
%<=>% %<=>%
%<=>%-method %<=>%
%in% Match a column with given values.
%in%-method Match a column with given values.
[ Subset
[-method Subset
[[ Subset
[[-method Subset
[[<- Subset
[[<--method Subset