org.apache.spark.sql.api.java

JavaSQLContext

class JavaSQLContext extends UDFRegistration

The entry point for executing Spark SQL queries from a Java program.

Linear Supertypes
UDFRegistration, AnyRef, Any
Known Subclasses
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. JavaSQLContext
  2. UDFRegistration
  3. AnyRef
  4. Any
  1. Hide All
  2. Show all
Learn more about member selection
Visibility
  1. Public
  2. All

Instance Constructors

  1. new JavaSQLContext(sparkContext: JavaSparkContext)

  2. new JavaSQLContext(sqlContext: SQLContext)

Value Members

  1. final def !=(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  2. final def !=(arg0: Any): Boolean

    Definition Classes
    Any
  3. final def ##(): Int

    Definition Classes
    AnyRef → Any
  4. final def ==(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  5. final def ==(arg0: Any): Boolean

    Definition Classes
    Any
  6. def applySchema(rowRDD: JavaRDD[Row], schema: StructType): JavaSchemaRDD

    :: DeveloperApi :: Creates a JavaSchemaRDD from an RDD containing Rows by applying a schema to this RDD.

    :: DeveloperApi :: Creates a JavaSchemaRDD from an RDD containing Rows by applying a schema to this RDD. It is important to make sure that the structure of every Row of the provided RDD matches the provided schema. Otherwise, there will be runtime exception.

    Annotations
    @DeveloperApi()
  7. def applySchema(rdd: JavaRDD[_], beanClass: Class[_]): JavaSchemaRDD

    Applies a schema to an RDD of Java Beans.

    Applies a schema to an RDD of Java Beans.

    WARNING: Since there is no guaranteed ordering for fields in a Java Bean, SELECT * queries will return the columns in an undefined order.

  8. final def asInstanceOf[T0]: T0

    Definition Classes
    Any
  9. def baseRelationToSchemaRDD(baseRelation: BaseRelation): JavaSchemaRDD

  10. def clone(): AnyRef

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  11. def createParquetFile(beanClass: Class[_], path: String, allowExisting: Boolean = true, conf: Configuration = new Configuration()): JavaSchemaRDD

    :: Experimental :: Creates an empty parquet file with the schema of class beanClass, which can be registered as a table.

    :: Experimental :: Creates an empty parquet file with the schema of class beanClass, which can be registered as a table. This registered table can be used as the target of future insertInto operations.

    JavaSQLContext sqlCtx = new JavaSQLContext(...)
    
    sqlCtx.createParquetFile(Person.class, "path/to/file.parquet").registerTempTable("people")
    sqlCtx.sql("INSERT INTO people SELECT 'michael', 29")
    beanClass

    A java bean class object that will be used to determine the schema of the parquet file.

    path

    The path where the directory containing parquet metadata should be created. Data inserted into this table will also be stored at this location.

    allowExisting

    When false, an exception will be thrown if this directory already exists.

    conf

    A Hadoop configuration object that can be used to specific options to the parquet output format.

    Annotations
    @Experimental()
  12. final def eq(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  13. def equals(arg0: Any): Boolean

    Definition Classes
    AnyRef → Any
  14. def finalize(): Unit

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  15. final def getClass(): Class[_]

    Definition Classes
    AnyRef → Any
  16. def getSchema(beanClass: Class[_]): Seq[AttributeReference]

    Returns a Catalyst Schema for the given java bean class.

    Returns a Catalyst Schema for the given java bean class.

    Attributes
    protected
  17. def hashCode(): Int

    Definition Classes
    AnyRef → Any
  18. final def isInstanceOf[T0]: Boolean

    Definition Classes
    Any
  19. def jsonFile(path: String, schema: StructType): JavaSchemaRDD

    :: Experimental :: Loads a JSON file (one object per line) and applies the given schema, returning the result as a JavaSchemaRDD.

    :: Experimental :: Loads a JSON file (one object per line) and applies the given schema, returning the result as a JavaSchemaRDD.

    Annotations
    @Experimental()
  20. def jsonFile(path: String): JavaSchemaRDD

    Loads a JSON file (one object per line), returning the result as a JavaSchemaRDD.

    Loads a JSON file (one object per line), returning the result as a JavaSchemaRDD. It goes through the entire dataset once to determine the schema.

  21. def jsonRDD(json: JavaRDD[String], schema: StructType): JavaSchemaRDD

    :: Experimental :: Loads an RDD[String] storing JSON objects (one object per record) and applies the given schema, returning the result as a JavaSchemaRDD.

    :: Experimental :: Loads an RDD[String] storing JSON objects (one object per record) and applies the given schema, returning the result as a JavaSchemaRDD.

    Annotations
    @Experimental()
  22. def jsonRDD(json: JavaRDD[String]): JavaSchemaRDD

    Loads an RDD[String] storing JSON objects (one object per record), returning the result as a JavaSchemaRDD.

    Loads an RDD[String] storing JSON objects (one object per record), returning the result as a JavaSchemaRDD. It goes through the entire dataset once to determine the schema.

  23. final def ne(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  24. final def notify(): Unit

    Definition Classes
    AnyRef
  25. final def notifyAll(): Unit

    Definition Classes
    AnyRef
  26. def parquetFile(path: String): JavaSchemaRDD

    Loads a parquet file, returning the result as a JavaSchemaRDD.

  27. def registerFunction(name: String, f: UDF22[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _], dataType: DataType): Unit

    Definition Classes
    UDFRegistration
  28. def registerFunction(name: String, f: UDF21[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _], dataType: DataType): Unit

    Definition Classes
    UDFRegistration
  29. def registerFunction(name: String, f: UDF20[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _], dataType: DataType): Unit

    Definition Classes
    UDFRegistration
  30. def registerFunction(name: String, f: UDF19[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _], dataType: DataType): Unit

    Definition Classes
    UDFRegistration
  31. def registerFunction(name: String, f: UDF18[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _], dataType: DataType): Unit

    Definition Classes
    UDFRegistration
  32. def registerFunction(name: String, f: UDF17[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _], dataType: DataType): Unit

    Definition Classes
    UDFRegistration
  33. def registerFunction(name: String, f: UDF16[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _], dataType: DataType): Unit

    Definition Classes
    UDFRegistration
  34. def registerFunction(name: String, f: UDF15[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _], dataType: DataType): Unit

    Definition Classes
    UDFRegistration
  35. def registerFunction(name: String, f: UDF14[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _], dataType: DataType): Unit

    Definition Classes
    UDFRegistration
  36. def registerFunction(name: String, f: UDF13[_, _, _, _, _, _, _, _, _, _, _, _, _, _], dataType: DataType): Unit

    Definition Classes
    UDFRegistration
  37. def registerFunction(name: String, f: UDF12[_, _, _, _, _, _, _, _, _, _, _, _, _], dataType: DataType): Unit

    Definition Classes
    UDFRegistration
  38. def registerFunction(name: String, f: UDF11[_, _, _, _, _, _, _, _, _, _, _, _], dataType: DataType): Unit

    Definition Classes
    UDFRegistration
  39. def registerFunction(name: String, f: UDF10[_, _, _, _, _, _, _, _, _, _, _], dataType: DataType): Unit

    Definition Classes
    UDFRegistration
  40. def registerFunction(name: String, f: UDF9[_, _, _, _, _, _, _, _, _, _], dataType: DataType): Unit

    Definition Classes
    UDFRegistration
  41. def registerFunction(name: String, f: UDF8[_, _, _, _, _, _, _, _, _], dataType: DataType): Unit

    Definition Classes
    UDFRegistration
  42. def registerFunction(name: String, f: UDF7[_, _, _, _, _, _, _, _], dataType: DataType): Unit

    Definition Classes
    UDFRegistration
  43. def registerFunction(name: String, f: UDF6[_, _, _, _, _, _, _], dataType: DataType): Unit

    Definition Classes
    UDFRegistration
  44. def registerFunction(name: String, f: UDF5[_, _, _, _, _, _], dataType: DataType): Unit

    Definition Classes
    UDFRegistration
  45. def registerFunction(name: String, f: UDF4[_, _, _, _, _], dataType: DataType): Unit

    Definition Classes
    UDFRegistration
  46. def registerFunction(name: String, f: UDF3[_, _, _, _], dataType: DataType): Unit

    Definition Classes
    UDFRegistration
  47. def registerFunction(name: String, f: UDF2[_, _, _], dataType: DataType): Unit

    Definition Classes
    UDFRegistration
  48. def registerFunction(name: String, f: UDF1[_, _], dataType: DataType): Unit

    Definition Classes
    UDFRegistration
  49. def registerRDDAsTable(rdd: JavaSchemaRDD, tableName: String): Unit

    Registers the given RDD as a temporary table in the catalog.

    Registers the given RDD as a temporary table in the catalog. Temporary tables exist only during the lifetime of this instance of SQLContext.

  50. def sql(sqlText: String): JavaSchemaRDD

    Executes a SQL query using Spark, returning the result as a SchemaRDD.

    Executes a SQL query using Spark, returning the result as a SchemaRDD. The dialect that is used for SQL parsing can be configured with 'spark.sql.dialect'.

  51. val sqlContext: SQLContext

  52. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes
    AnyRef
  53. def toString(): String

    Definition Classes
    AnyRef → Any
  54. final def wait(): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  55. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  56. final def wait(arg0: Long): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from UDFRegistration

Inherited from AnyRef

Inherited from Any

userf

Ungrouped