public class HiveContext extends SQLContext
SQLContext.implicits
Constructor and Description |
---|
HiveContext(SparkContext sc) |
Modifier and Type | Method and Description |
---|---|
void |
analyze(String tableName)
Analyzes the given table in the current database to generate statistics, which will be
used in query optimizations.
|
void |
refreshTable(String tableName)
Invalidate and refresh all the cached the metadata of the given table.
|
void |
setConf(String key,
String value)
Set the given Spark SQL configuration property.
|
DataFrame |
sql(String sqlText)
Executes a SQL query using Spark, returning the result as a
DataFrame . |
applySchema, applySchema, applySchema, applySchema, baseRelationToDataFrame, cacheTable, clearCache, createDataFrame, createDataFrame, createDataFrame, createDataFrame, createDataFrame, createDataFrame, createDataFrame, createDataFrame, createExternalTable, createExternalTable, createExternalTable, createExternalTable, createExternalTable, createExternalTable, dropTempTable, emptyDataFrame, experimental, getAllConfs, getConf, getConf, implicits, isCached, jdbc, jdbc, jdbc, jsonFile, jsonFile, jsonFile, jsonRDD, jsonRDD, jsonRDD, jsonRDD, jsonRDD, jsonRDD, load, load, load, load, load, load, parquetFile, parquetFile, registerDataFrameAsTable, setConf, sparkContext, table, tableNames, tableNames, tables, tables, udf, uncacheTable
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
initializeIfNecessary, initializeLogging, isTraceEnabled, log_, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logName, logTrace, logTrace, logWarning, logWarning
public HiveContext(SparkContext sc)
public DataFrame sql(String sqlText)
SQLContext
DataFrame
. The dialect that is
used for SQL parsing can be configured with 'spark.sql.dialect'.
sql
in class SQLContext
public void refreshTable(String tableName)
public void analyze(String tableName)
Right now, it only supports Hive tables and it only updates the size of a Hive table in the Hive metastore.
public void setConf(String key, String value)
SQLContext
setConf
in class SQLContext