Package org.apache.spark.sql

Interface Row

All Superinterfaces:
Serializable
All Known Implementing Classes:
MutableAggregationBuffer
public interface Row extends Serializable
Represents one row of output from a relational operator. Allows both generic access by ordinal, which will incur boxing overhead for primitives, as well as native primitive access.

It is invalid to use the native primitive interface to retrieve a value that is null, instead a user must check isNullAt before attempting to retrieve a value that might be null.

To create a new Row, use RowFactory.create() in Java or Row.apply() in Scala.

A Row object can be constructed by providing field values. Example: 


 import org.apache.spark.sql._

 // Create a Row from values.
 Row(value1, value2, value3, ...)
 // Create a Row from a Seq of values.
 Row.fromSeq(Seq(value1, value2, ...))

A value of a row can be accessed through both generic access by ordinal, which will incur boxing overhead for primitives, as well as native primitive access. An example of generic access by ordinal: 


 import org.apache.spark.sql._

 val row = Row(1, true, "a string", null)
 // row: Row = [1,true,a string,null]
 val firstValue = row(0)
 // firstValue: Any = 1
 val fourthValue = row(3)
 // fourthValue: Any = null

For native primitive access, it is invalid to use the native primitive interface to retrieve a value that is null, instead a user must check isNullAt before attempting to retrieve a value that might be null. An example of native primitive access: 


 // using the row from the previous example.
 val firstValue = row.getInt(0)
 // firstValue: Int = 1
 val isNull = row.isNullAt(3)
 // isNull: Boolean = true

In Scala, fields in a Row object can be extracted in a pattern match. Example: 


 import org.apache.spark.sql._

 val pairs = sql("SELECT key, value FROM src").rdd.map {
   case Row(key: Int, value: String) =>
     key -> value
 }

Since:
1.3.0

  • Method Summary

    Modifier and Type
    Method
    Description
    boolean
    anyNull()
    Returns true if there are any NULL values in this row.
    Object
    apply(int i)
    Returns the value at position i.
    Row
    copy()
    Make a copy of the current Row object.
    boolean
    equals(Object o)
     
    int
    fieldIndex(String name)
    Returns the index of a given field name.
    Object
    get(int i)
    Returns the value at position i.
    <T> T
    getAnyValAs(int i)
    Returns the value at position i.
    <T> T
    getAs(int i)
    Returns the value at position i.
    <T> T
    getAs(String fieldName)
    Returns the value of a given fieldName.
    boolean
    getBoolean(int i)
    Returns the value at position i as a primitive boolean.
    byte
    getByte(int i)
    Returns the value at position i as a primitive byte.
    Date
    getDate(int i)
    Returns the value at position i of date type as java.sql.Date.
    BigDecimal
    getDecimal(int i)
    Returns the value at position i of decimal type as java.math.BigDecimal.
    double
    getDouble(int i)
    Returns the value at position i as a primitive double.
    float
    getFloat(int i)
    Returns the value at position i as a primitive float.
    Instant
    getInstant(int i)
    Returns the value at position i of date type as java.time.Instant.
    int
    getInt(int i)
    Returns the value at position i as a primitive int.
    <K, V> Map<K,V>
    getJavaMap(int i)
    Returns the value at position i of array type as a java.util.Map.
    <T> List<T>
    getList(int i)
    Returns the value at position i of array type as java.util.List.
    LocalDate
    getLocalDate(int i)
    Returns the value at position i of date type as java.time.LocalDate.
    long
    getLong(int i)
    Returns the value at position i as a primitive long.
    <K, V> scala.collection.Map<K,V>
    getMap(int i)
    Returns the value at position i of map type as a Scala Map.
    <T> scala.collection.immutable.Seq<T>
    getSeq(int i)
    Returns the value at position i of array type as a Scala Seq.
    short
    getShort(int i)
    Returns the value at position i as a primitive short.
    String
    getString(int i)
    Returns the value at position i as a String object.
    Row
    getStruct(int i)
    Returns the value at position i of struct type as a Row object.
    Timestamp
    getTimestamp(int i)
    Returns the value at position i of date type as java.sql.Timestamp.
    <T> scala.collection.immutable.Map<String,T>
    getValuesMap(scala.collection.immutable.Seq<String> fieldNames)
    Returns a Map consisting of names and values for the requested fieldNames For primitive types if value is null it returns 'zero value' specific for primitive i.e.
    int
    hashCode()
     
    boolean
    isNullAt(int i)
    Checks whether the value at position i is null.
    String
    json()
    The compact JSON representation of this row.
    org.json4s.JValue
    jsonValue()
    JSON representation of the row.
    int
    length()
    Number of elements in the Row.
    String
    mkString()
    Displays all elements of this sequence in a string (without a separator).
    String
    mkString(String sep)
    Displays all elements of this sequence in a string using a separator string.
    String
    mkString(String start, String sep, String end)
    Displays all elements of this traversable or iterator in a string using start, end, and separator strings.
    String
    prettyJson()
    The pretty (i.e.
    StructType
    schema()
    Schema for the row.
    int
    size()
    Number of elements in the Row.
    scala.collection.immutable.Seq<Object>
    toSeq()
    Return a Scala Seq representing the row.
    String
    toString()
     

  • Method Details

    • size

      int size()
      Number of elements in the Row.

    • length

      int length()
      Number of elements in the Row.

    • schema

      StructType schema()
      Schema for the row.
      Returns:
      (undocumented)

    • apply

      Object apply(int i)
      Returns the value at position i. If the value is null, null is returned. The following is a mapping between Spark SQL types and return types: 

      
   BooleanType -> java.lang.Boolean
   ByteType -> java.lang.Byte
   ShortType -> java.lang.Short
   IntegerType -> java.lang.Integer
   LongType -> java.lang.Long
   FloatType -> java.lang.Float
   DoubleType -> java.lang.Double
   StringType -> String
   DecimalType -> java.math.BigDecimal

   DateType -> java.sql.Date if spark.sql.datetime.java8API.enabled is false
   DateType -> java.time.LocalDate if spark.sql.datetime.java8API.enabled is true

   TimestampType -> java.sql.Timestamp if spark.sql.datetime.java8API.enabled is false
   TimestampType -> java.time.Instant if spark.sql.datetime.java8API.enabled is true

   BinaryType -> byte array
   ArrayType -> scala.collection.Seq (use getList for java.util.List)
   MapType -> scala.collection.Map (use getJavaMap for java.util.Map)
   StructType -> org.apache.spark.sql.Row
      Parameters:
      i - (undocumented)
      Returns:
      (undocumented)

    • get

      Object get(int i)
      Returns the value at position i. If the value is null, null is returned. The following is a mapping between Spark SQL types and return types: 

      
   BooleanType -> java.lang.Boolean
   ByteType -> java.lang.Byte
   ShortType -> java.lang.Short
   IntegerType -> java.lang.Integer
   LongType -> java.lang.Long
   FloatType -> java.lang.Float
   DoubleType -> java.lang.Double
   StringType -> String
   DecimalType -> java.math.BigDecimal

   DateType -> java.sql.Date if spark.sql.datetime.java8API.enabled is false
   DateType -> java.time.LocalDate if spark.sql.datetime.java8API.enabled is true

   TimestampType -> java.sql.Timestamp if spark.sql.datetime.java8API.enabled is false
   TimestampType -> java.time.Instant if spark.sql.datetime.java8API.enabled is true

   BinaryType -> byte array
   ArrayType -> scala.collection.Seq (use getList for java.util.List)
   MapType -> scala.collection.Map (use getJavaMap for java.util.Map)
   StructType -> org.apache.spark.sql.Row
      Parameters:
      i - (undocumented)
      Returns:
      (undocumented)

    • isNullAt

      boolean isNullAt(int i)
      Checks whether the value at position i is null.

    • getBoolean

      boolean getBoolean(int i)
      Returns the value at position i as a primitive boolean.

      Parameters:
      i - (undocumented)
      Returns:
      (undocumented)
      Throws:
      ClassCastException - when data type does not match.
      org.apache.spark.SparkRuntimeException - when value is null.

    • getByte

      byte getByte(int i)
      Returns the value at position i as a primitive byte.

      Parameters:
      i - (undocumented)
      Returns:
      (undocumented)
      Throws:
      ClassCastException - when data type does not match.
      org.apache.spark.SparkRuntimeException - when value is null.

    • getShort

      short getShort(int i)
      Returns the value at position i as a primitive short.

      Parameters:
      i - (undocumented)
      Returns:
      (undocumented)
      Throws:
      ClassCastException - when data type does not match.
      org.apache.spark.SparkRuntimeException - when value is null.

    • getInt

      int getInt(int i)
      Returns the value at position i as a primitive int.

      Parameters:
      i - (undocumented)
      Returns:
      (undocumented)
      Throws:
      ClassCastException - when data type does not match.
      org.apache.spark.SparkRuntimeException - when value is null.

    • getLong

      long getLong(int i)
      Returns the value at position i as a primitive long.

      Parameters:
      i - (undocumented)
      Returns:
      (undocumented)
      Throws:
      ClassCastException - when data type does not match.
      org.apache.spark.SparkRuntimeException - when value is null.

    • getFloat

      float getFloat(int i)
      Returns the value at position i as a primitive float. Throws an exception if the type mismatches or if the value is null.

      Parameters:
      i - (undocumented)
      Returns:
      (undocumented)
      Throws:
      ClassCastException - when data type does not match.
      org.apache.spark.SparkRuntimeException - when value is null.

    • getDouble

      double getDouble(int i)
      Returns the value at position i as a primitive double.

      Parameters:
      i - (undocumented)
      Returns:
      (undocumented)
      Throws:
      ClassCastException - when data type does not match.
      org.apache.spark.SparkRuntimeException - when value is null.

    • getString

      String getString(int i)
      Returns the value at position i as a String object.

      Parameters:
      i - (undocumented)
      Returns:
      (undocumented)
      Throws:
      ClassCastException - when data type does not match.

    • getDecimal

      BigDecimal getDecimal(int i)
      Returns the value at position i of decimal type as java.math.BigDecimal.

      Parameters:
      i - (undocumented)
      Returns:
      (undocumented)
      Throws:
      ClassCastException - when data type does not match.

    • getDate

      Date getDate(int i)
      Returns the value at position i of date type as java.sql.Date.

      Parameters:
      i - (undocumented)
      Returns:
      (undocumented)
      Throws:
      ClassCastException - when data type does not match.

    • getLocalDate

      LocalDate getLocalDate(int i)
      Returns the value at position i of date type as java.time.LocalDate.

      Parameters:
      i - (undocumented)
      Returns:
      (undocumented)
      Throws:
      ClassCastException - when data type does not match.

    • getTimestamp

      Timestamp getTimestamp(int i)
      Returns the value at position i of date type as java.sql.Timestamp.

      Parameters:
      i - (undocumented)
      Returns:
      (undocumented)
      Throws:
      ClassCastException - when data type does not match.

    • getInstant

      Instant getInstant(int i)
      Returns the value at position i of date type as java.time.Instant.

      Parameters:
      i - (undocumented)
      Returns:
      (undocumented)
      Throws:
      ClassCastException - when data type does not match.

    • getSeq

      <T> scala.collection.immutable.Seq<T> getSeq(int i)
      Returns the value at position i of array type as a Scala Seq.

      Parameters:
      i - (undocumented)
      Returns:
      (undocumented)
      Throws:
      ClassCastException - when data type does not match.

    • getList

      <T> List<T> getList(int i)
      Returns the value at position i of array type as java.util.List.

      Parameters:
      i - (undocumented)
      Returns:
      (undocumented)
      Throws:
      ClassCastException - when data type does not match.

    • getMap

      <K, V> scala.collection.Map<K,V> getMap(int i)
      Returns the value at position i of map type as a Scala Map.

      Parameters:
      i - (undocumented)
      Returns:
      (undocumented)
      Throws:
      ClassCastException - when data type does not match.

    • getJavaMap

      <K, V> Map<K,V> getJavaMap(int i)
      Returns the value at position i of array type as a java.util.Map.

      Parameters:
      i - (undocumented)
      Returns:
      (undocumented)
      Throws:
      ClassCastException - when data type does not match.

    • getStruct

      Row getStruct(int i)
      Returns the value at position i of struct type as a Row object.

      Parameters:
      i - (undocumented)
      Returns:
      (undocumented)
      Throws:
      ClassCastException - when data type does not match.

    • getAs

      <T> T getAs(int i)
      Returns the value at position i. For primitive types if value is null it returns 'zero value' specific for primitive i.e. 0 for Int - use isNullAt to ensure that value is not null

      Parameters:
      i - (undocumented)
      Returns:
      (undocumented)
      Throws:
      ClassCastException - when data type does not match.

    • getAs

      <T> T getAs(String fieldName)
      Returns the value of a given fieldName. For primitive types if value is null it returns 'zero value' specific for primitive i.e. 0 for Int - use isNullAt to ensure that value is not null

      Parameters:
      fieldName - (undocumented)
      Returns:
      (undocumented)
      Throws:
      UnsupportedOperationException - when schema is not defined.
      IllegalArgumentException - when fieldName do not exist.
      ClassCastException - when data type does not match.

    • fieldIndex

      int fieldIndex(String name)
      Returns the index of a given field name.

      Parameters:
      name - (undocumented)
      Returns:
      (undocumented)
      Throws:
      UnsupportedOperationException - when schema is not defined.
      IllegalArgumentException - when a field name does not exist.

    • getValuesMap

      <T> scala.collection.immutable.Map<String,T> getValuesMap(scala.collection.immutable.Seq<String> fieldNames)
      Returns a Map consisting of names and values for the requested fieldNames For primitive types if value is null it returns 'zero value' specific for primitive i.e. 0 for Int - use isNullAt to ensure that value is not null

      Parameters:
      fieldNames - (undocumented)
      Returns:
      (undocumented)
      Throws:
      UnsupportedOperationException - when schema is not defined.
      IllegalArgumentException - when fieldName do not exist.
      ClassCastException - when data type does not match.

    • toString

      String toString()
      Overrides:
      toString in class Object

    • copy

      Row copy()
      Make a copy of the current Row object.
      Returns:
      (undocumented)

    • anyNull

      boolean anyNull()
      Returns true if there are any NULL values in this row.

    • equals

      boolean equals(Object o)
      Overrides:
      equals in class Object

    • hashCode

      int hashCode()
      Overrides:
      hashCode in class Object

    • toSeq

      scala.collection.immutable.Seq<Object> toSeq()
      Return a Scala Seq representing the row. Elements are placed in the same order in the Seq.
      Returns:
      (undocumented)

    • mkString

      String mkString()
      Displays all elements of this sequence in a string (without a separator).

    • mkString

      String mkString(String sep)
      Displays all elements of this sequence in a string using a separator string.

    • mkString

      String mkString(String start, String sep, String end)
      Displays all elements of this traversable or iterator in a string using start, end, and separator strings.
      Parameters:
      start - (undocumented)
      sep - (undocumented)
      end - (undocumented)
      Returns:
      (undocumented)

    • getAnyValAs

      <T> T getAnyValAs(int i)
      Returns the value at position i.

      Parameters:
      i - (undocumented)
      Returns:
      (undocumented)
      Throws:
      UnsupportedOperationException - when schema is not defined.
      ClassCastException - when data type does not match.
      org.apache.spark.SparkRuntimeException - when value is null.

    • json

      String json()
      The compact JSON representation of this row.
      Returns:
      (undocumented)
      Since:
      3.0

    • prettyJson

      String prettyJson()
      The pretty (i.e. indented) JSON representation of this row.
      Returns:
      (undocumented)
      Since:
      3.0

    • jsonValue

      org.json4s.JValue jsonValue()
      JSON representation of the row.

      Note that this only supports the data types that are also supported by RowEncoder.

      Returns:
      the JSON representation of the row.