org.apache.spark.sql.hive
Class HiveContext

Object
  org.apache.spark.sql.SQLContext
      org.apache.spark.sql.hive.HiveContext

All Implemented Interfaces:

java.io.Serializable, Logging

public class HiveContext
extends SQLContext

An instance of the Spark SQL execution engine that integrates with data stored in Hive. Configuration for Hive is read from hive-site.xml on the classpath.

Since:

1.0.0

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.spark.sql.SQLContext

SQLContext.implicits$

Constructor Summary

HiveContext(SparkContext sc)

Method Summary

void	analyze(String tableName) Analyzes the given table in the current database to generate statistics, which will be used in query optimizations.
static String	HIVE_METASTORE_JARS()
static String	HIVE_METASTORE_VERSION()
static String	hiveExecutionVersion() The version of hive used internally by Spark SQL.
static scala.collection.immutable.Map<String,String>	newTemporaryConfiguration() Constructs a configuration for hive, where the metastore is located in a temp directory.
void	refreshTable(String tableName) Invalidate and refresh all the cached the metadata of the given table.
void	setConf(String key, String value) Set the given Spark SQL configuration property.

Methods inherited from class org.apache.spark.sql.SQLContext

applySchema, applySchema, applySchema, applySchema, baseRelationToDataFrame, cacheTable, clearCache, createDataFrame, createDataFrame, createDataFrame, createDataFrame, createDataFrame, createDataFrame, createExternalTable, createExternalTable, createExternalTable, createExternalTable, createExternalTable, createExternalTable, dropTempTable, emptyDataFrame, experimental, getAllConfs, getConf, getConf, getOrCreate, implicits, isCached, jdbc, jdbc, jdbc, jsonFile, jsonFile, jsonFile, jsonRDD, jsonRDD, jsonRDD, jsonRDD, jsonRDD, jsonRDD, load, load, load, load, load, load, parquetFile, parquetFile, range, range, range, read, setConf, sparkContext, sql, table, tableNames, tableNames, tables, tables, udf, uncacheTable

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.spark.Logging

initializeIfNecessary, initializeLogging, isTraceEnabled, log_, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logName, logTrace, logTrace, logWarning, logWarning

Constructor Detail

HiveContext

public HiveContext(SparkContext sc)

Method Detail

hiveExecutionVersion

public static String hiveExecutionVersion()

The version of hive used internally by Spark SQL.

HIVE_METASTORE_VERSION

public static String HIVE_METASTORE_VERSION()

HIVE_METASTORE_JARS

public static String HIVE_METASTORE_JARS()

newTemporaryConfiguration

public static scala.collection.immutable.Map<String,String> newTemporaryConfiguration()

Constructs a configuration for hive, where the metastore is located in a temp directory.

refreshTable

public void refreshTable(String tableName)

Invalidate and refresh all the cached the metadata of the given table. For performance reasons, Spark SQL or the external data source library it uses might cache certain metadata about a table, such as the location of blocks. When those change outside of Spark SQL, users should call this function to invalidate the cache.

Parameters:

tableName - (undocumented)

Since:

1.3.0

analyze

public void analyze(String tableName)

Analyzes the given table in the current database to generate statistics, which will be used in query optimizations.

Right now, it only supports Hive tables and it only updates the size of a Hive table in the Hive metastore.

Parameters:

tableName - (undocumented)

Since:

1.2.0

setConf

public void setConf(String key,
                    String value)

Description copied from class: SQLContext

Set the given Spark SQL configuration property.

Overrides:

setConf in class SQLContext

Parameters:

key - (undocumented)

value - (undocumented)