R frontend for Spark


[Up] [Top]

Documentation for package ‘SparkR’ version 1.6.0

Help Pages

A B C D E F G H I J K L M N O P Q R S T U V W Y misc

-- A --

abs abs
acos acos
add_months add_months
agg Summarize data across columns
agg summarize
alias alias
approxCountDistinct approxCountDistinct
arrange Arrange
array_contains array_contains
as.data.frame Download data from a DataFrame into a data.frame
as.data.frame-method Download data from a DataFrame into a data.frame
as.DataFrame Create a DataFrame
asc S4 class that represents a DataFrame column
ascii ascii
asin asin
atan atan
atan2 atan2
attach Attach DataFrame to R search path
attach-method Attach DataFrame to R search path
avg avg

-- B --

base64 base64
between between
between S4 class that represents a DataFrame column
bin bin
bitwiseNOT bitwiseNOT

-- C --

cache Cache
cacheTable Cache Table
cancelJobGroup Cancel active jobs for the specified group
cast Casts the column to a different data type.
cast S4 class that represents a DataFrame column
cbrt cbrt
ceil ceil
ceiling ceil
clearCache Clear Cache
clearJobGroup Clear current job group ID and its description
col Though scala functions has "col" function, we don't expose it in SparkR because we don't want to conflict with the "col" function in the R base package and we also have "column" function exported which is an alias of "col".
collect Collects all the elements of a Spark DataFrame and coerces them into an R data.frame.
colnames Column names
colnames<- Column names
coltypes coltypes
coltypes<- coltypes
column Though scala functions has "col" function, we don't expose it in SparkR because we don't want to conflict with the "col" function in the R base package and we also have "column" function exported which is an alias of "col".
Column-class S4 class that represents a DataFrame column
columns Column names
columns Get schema object
concat concat
concat_ws concat_ws
contains S4 class that represents a DataFrame column
conv conv
corr corr
corr crosstab
cos cos
cosh cosh
count count
count nrow
count-method Summarize data across columns
countDistinct Count Distinct
cov crosstab
crc32 crc32
createDataFrame Create a DataFrame
createExternalTable Create an external table
crosstab crosstab
cume_dist cume_dist

-- D --

dataFrame S4 class that represents a DataFrame
DataFrame-class S4 class that represents a DataFrame
datediff datediff
date_add date_add
date_format date_format
date_sub date_sub
dayofmonth dayofmonth
dayofyear dayofyear
decode decode
dense_rank dense_rank
desc S4 class that represents a DataFrame column
describe summary
dim Returns the dimentions (number of rows and columns) of a DataFrame
distinct Distinct
dropna dropna
dropTempTable Drop Temporary Table
dtypes DataTypes
dtypes Get schema object

-- E --

encode encode
endsWith S4 class that represents a DataFrame column
except except
exp exp
explain Explain
explode explode
expm1 expm1
expr expr

-- F --

factorial factorial
fillna dropna
filter Filter
first Return the first row of a DataFrame
floor floor
format_number format_number
format_string format_string
freqItems crosstab
from_unixtime from_unixtime
from_utc_timestamp from_utc_timestamp

-- G --

generateAliasesForIntersectedCols Creates a list of columns by replacing the intersected ones with aliases. The name of the alias column is formed by concatanating the original column name and a suffix.
getField S4 class that represents a DataFrame column
getItem S4 class that represents a DataFrame column
glm Fits a generalized linear model
glm-method Fits a generalized linear model
greatest greatest
groupBy GroupBy
groupedData S4 class that represents a DataFrame
GroupedData-class S4 class that represents a GroupedData
group_by GroupBy

-- H --

hashCode Compute the hashCode of an object
head Head
hex hex
hour hour
hypot hypot

-- I --

ifelse ifelse
infer_type infer the SQL type
initcap initcap
insertInto insertInto
instr instr
intersect Intersect
is.nan is.nan
isLocal isLocal
isNaN S4 class that represents a DataFrame column
isnan is.nan
isNotNull S4 class that represents a DataFrame column
isNull S4 class that represents a DataFrame column

-- J --

join Join
jsonFile Create a DataFrame from a JSON file.

-- K --

kurtosis kurtosis

-- L --

lag lag
last last
last_day last_day
lead lead
least least
length length
levenshtein levenshtein
like S4 class that represents a DataFrame column
limit Limit
lit lit
loadDF Load an DataFrame
locate locate
log log
log10 log10
log1p log1p
log2 log2
lower lower
lpad lpad
ltrim ltrim

-- M --

max max
md5 md5
mean mean
merge Merges two data frames
min min
minute minute
month month
months_between months_between
mutate Mutate

-- N --

n count
na.omit dropna
names Column names
names<- Column names
nanvl nanvl
ncol Returns the number of columns in a DataFrame
negate negate
next_day next_day
nrow nrow
ntile ntile
n_distinct Count Distinct

-- O --

orderBy Arrange
otherwise S4 class that represents a DataFrame column
otherwise otherwise

-- P --

parquetFile Create a DataFrame from a Parquet file.
percent_rank percent_rank
persist Persist
PipelineModel-class S4 class that represents a PipelineModel
pmod pmod
predict Make predictions from a model
predict-method Make predictions from a model
print.jobj Print a JVM object reference.
print.structField Print a Spark StructField.
print.structType Print a Spark StructType.
printSchema Print Schema of a DataFrame
printSchema Get schema object

-- Q --

quarter quarter

-- R --

rand rand
randn randn
rank rank
rbind rbind
read.df Load an DataFrame
read.json Create a DataFrame from a JSON file.
read.parquet Create a DataFrame from a Parquet file.
regexp_extract regexp_extract
regexp_replace regexp_replace
registerTempTable Register Temporary Table
rename rename
repartition Repartition
reverse reverse
rint rint
rlike S4 class that represents a DataFrame column
round round
row_number row_number
rpad rpad
rtrim rtrim

-- S --

sample Sample
sampleBy crosstab
sample_frac Sample
saveAsParquetFile write.parquet
saveAsTable saveAsTable
saveDF Save the contents of the DataFrame to a data source
schema Get schema object
sd sd
second second
select Select
select-method Select
selectExpr Select
selectExpr SelectExpr
setJobGroup Assigns a group ID to all the jobs started by this thread until the group ID is set to a different value or cleared.
sha1 sha1
sha2 sha2
shiftLeft shiftLeft
shiftRight shiftRight
shiftRightUnsigned shiftRightUnsigned
show show
show-method show
showDF showDF
sign signum
signum signum
sin sin
sinh sinh
size size
skewness skewness
sort_array sort_array
soundex soundex
sparkR.init Initialize a new Spark Context.
sparkR.stop Stop the Spark context.
sparkRHive.init Initialize a new HiveContext.
sparkRSQL.init Initialize a new SQLContext.
sql SQL Query
sqrt sqrt
startsWith S4 class that represents a DataFrame column
stddev sd
stddev_pop stddev_pop
stddev_samp stddev_samp
struct struct
structField structField
structType structType
subset Subset
substr substr
substring_index substring_index
sum sum
sumDistinct sumDistinct
summarize Summarize data across columns
summarize summarize
summary summary
summary-method summary

-- T --

table Create a DataFrame from a SparkSQL Table
tableNames Table Names
tables Tables
take Take the first NUM rows of a DataFrame and return a the results as a data.frame
tan tan
tanh tanh
toDegrees toDegrees
toRadians toRadians
to_date to_date
to_utc_timestamp to_utc_timestamp
transform Mutate
transform WithColumn
translate translate
trim trim

-- U --

unbase64 unbase64
uncacheTable Uncache Table
unhex unhex
unionAll rbind
unique Distinct
unix_timestamp unix_timestamp
unpersist Unpersist
upper upper

-- V --

var var
variance var
var_pop var_pop
var_samp var_samp

-- W --

weekofyear weekofyear
when S4 class that represents a DataFrame column
when when
where Filter
with Evaluate a R expression in an environment constructed from a DataFrame
with-method Evaluate a R expression in an environment constructed from a DataFrame
withColumn WithColumn
withColumnRenamed rename
write.df Save the contents of the DataFrame to a data source
write.json write.json
write.parquet write.parquet

-- Y --

year year

-- misc --

$ Select
$<- Select
%in% Match a column with given values.
[ Subset
[[ Subset