org.apache.spark.sql.avro

Class SchemaConverters

Object

org.apache.spark.sql.avro.SchemaConverters

public class SchemaConverters
extends Object

This object contains method that are used to convert sparkSQL schemas to avro schemas and vice versa.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	SchemaConverters.SchemaType Internal wrapper for SQL data type and nullability.
static class	SchemaConverters.SchemaType$

Constructor Summary

Constructors

Constructor and Description
SchemaConverters()

Method Summary

Modifier and Type	Method and Description
static org.apache.avro.Schema	toAvroType(DataType catalystType, boolean nullable, String recordName, String nameSpace) Converts a Spark SQL schema to a corresponding Avro schema.
static SchemaConverters.SchemaType	toSqlType(org.apache.avro.Schema avroSchema) Converts an Avro schema to a corresponding Spark SQL schema.
static SchemaConverters.SchemaType	toSqlType(org.apache.avro.Schema avroSchema, boolean useStableIdForUnionType) Converts an Avro schema to a corresponding Spark SQL schema.
static SchemaConverters.SchemaType	toSqlType(org.apache.avro.Schema avroSchema, scala.collection.immutable.Map<String,String> options)

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SchemaConverters

public SchemaConverters()

Method Detail

toSqlType

public static SchemaConverters.SchemaType toSqlType(org.apache.avro.Schema avroSchema,
                                                    boolean useStableIdForUnionType)

Converts an Avro schema to a corresponding Spark SQL schema.

Parameters:

avroSchema - (undocumented)

useStableIdForUnionType - (undocumented)

Returns:

(undocumented)

Since:

4.0.0

toSqlType

public static SchemaConverters.SchemaType toSqlType(org.apache.avro.Schema avroSchema)

Converts an Avro schema to a corresponding Spark SQL schema.

Parameters:

avroSchema - (undocumented)

Returns:

(undocumented)

Since:

2.4.0

toSqlType

public static SchemaConverters.SchemaType toSqlType(org.apache.avro.Schema avroSchema,
                                                    scala.collection.immutable.Map<String,String> options)

toAvroType

public static org.apache.avro.Schema toAvroType(DataType catalystType,
                                                boolean nullable,
                                                String recordName,
                                                String nameSpace)

Converts a Spark SQL schema to a corresponding Avro schema.

Parameters:

catalystType - (undocumented)

nullable - (undocumented)

recordName - (undocumented)

nameSpace - (undocumented)

Returns:

(undocumented)

Since:

2.4.0