Package org.apache.spark.sql.avro

Class SchemaConverters

Object

org.apache.spark.sql.avro.SchemaConverters

public class SchemaConverters
extends Object

This object contains method that are used to convert sparkSQL schemas to avro schemas and vice versa.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	SchemaConverters.SchemaType	Internal wrapper for SQL data type and nullability.
static class	SchemaConverters.SchemaType$

Constructor Summary

Constructors

Constructor	Description
SchemaConverters()

Method Summary

Modifier and Type	Method	Description
static org.apache.avro.Schema	toAvroType(DataType catalystType, boolean nullable, String recordName, String nameSpace)	Converts a Spark SQL schema to a corresponding Avro schema.
static SchemaConverters.SchemaType	toSqlType(org.apache.avro.Schema avroSchema)	Converts an Avro schema to a corresponding Spark SQL schema.
static SchemaConverters.SchemaType	toSqlType(org.apache.avro.Schema avroSchema, boolean useStableIdForUnionType, String stableIdPrefixForUnionType)	Converts an Avro schema to a corresponding Spark SQL schema.
static SchemaConverters.SchemaType	toSqlType(org.apache.avro.Schema avroSchema, scala.collection.immutable.Map<String,String> options)	Deprecated. using toSqlType(..., useStableIdForUnionType: Boolean) instead.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

SchemaConverters

public SchemaConverters()

Method Details

toSqlType

public static SchemaConverters.SchemaType toSqlType(org.apache.avro.Schema avroSchema,
 boolean useStableIdForUnionType,
 String stableIdPrefixForUnionType)

Converts an Avro schema to a corresponding Spark SQL schema.

Parameters:

avroSchema - (undocumented)

useStableIdForUnionType - (undocumented)

stableIdPrefixForUnionType - (undocumented)

Returns:

(undocumented)

Since:

4.0.0

toSqlType

public static SchemaConverters.SchemaType toSqlType(org.apache.avro.Schema avroSchema)

Converts an Avro schema to a corresponding Spark SQL schema.

Parameters:

avroSchema - (undocumented)

Returns:

(undocumented)

Since:

2.4.0

toSqlType

public static SchemaConverters.SchemaType toSqlType(org.apache.avro.Schema avroSchema,
 scala.collection.immutable.Map<String,String> options)

Deprecated.

using toSqlType(..., useStableIdForUnionType: Boolean) instead. Since 4.0.0.

toAvroType

public static org.apache.avro.Schema toAvroType(DataType catalystType,
 boolean nullable,
 String recordName,
 String nameSpace)

Converts a Spark SQL schema to a corresponding Avro schema.

Parameters:

catalystType - (undocumented)

nullable - (undocumented)

recordName - (undocumented)

nameSpace - (undocumented)

Returns:

(undocumented)

Since:

2.4.0