Constructor and Description |
---|
JsonRDD() |
Modifier and Type | Method and Description |
---|---|
static DataType |
compatibleType(DataType t1,
DataType t2)
Returns the most general data type for two given data types.
|
static Object |
enforceCorrectType(Object value,
DataType desiredType) |
static StructType |
inferSchema(RDD<String> json,
double samplingRatio,
String columnNameOfCorruptRecords) |
static RDD<Row> |
jsonStringToRow(RDD<String> json,
StructType schema,
String columnNameOfCorruptRecords) |
static StructType |
nullTypeToStringType(StructType struct) |
static void |
rowToJSON(StructType rowSchema,
com.fasterxml.jackson.core.JsonGenerator gen,
Row row)
Transforms a single Row to JSON using Jackson
|
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
initializeIfNecessary, initializeLogging, isTraceEnabled, log_, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logName, logTrace, logTrace, logWarning, logWarning
public static RDD<Row> jsonStringToRow(RDD<String> json, StructType schema, String columnNameOfCorruptRecords)
public static StructType inferSchema(RDD<String> json, double samplingRatio, String columnNameOfCorruptRecords)
public static StructType nullTypeToStringType(StructType struct)
public static DataType compatibleType(DataType t1, DataType t2)
public static Object enforceCorrectType(Object value, DataType desiredType)
public static void rowToJSON(StructType rowSchema, com.fasterxml.jackson.core.JsonGenerator gen, Row row)
rowSchema
- the schema object used for conversiongen
- a JsonGenerator objectrow
- The row to convert