class ArrowColumnVector extends ColumnVector
A column vector backed by Apache Arrow.
- Annotations
- @DeveloperApi()
- Source
- ArrowColumnVector.java
- Alphabetic
- By Inheritance
- ArrowColumnVector
- ColumnVector
- AutoCloseable
- AnyRef
- Any
- Hide All
- Show All
- Public
- Protected
Instance Constructors
- new ArrowColumnVector(vector: ValueVector)
Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def ##: Int
- Definition Classes
- AnyRef → Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- def clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.CloneNotSupportedException]) @IntrinsicCandidate() @native()
- def close(): Unit
Cleans up memory for this column vector.
Cleans up memory for this column vector. The column vector is not usable after this.
This overwrites
AutoCloseable#close
to remove thethrows
clause, as column vector is in-memory and we don't expect any exception to happen during closing.- Definition Classes
- ArrowColumnVector → ColumnVector → AutoCloseable
- Annotations
- @Override()
- final def dataType(): DataType
Returns the data type of this column vector.
Returns the data type of this column vector.
- Definition Classes
- ColumnVector
- final def eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def equals(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef → Any
- def getArray(rowId: Int): ColumnarArray
Returns the array type value for
rowId
.Returns the array type value for
rowId
. If the slot forrowId
is null, it should return null.To support array type, implementations must construct an
ColumnarArray
and return it in this method.ColumnarArray
requires aColumnVector
that stores the data of all the elements of all the arrays in this vector, and an offset and length which points to a range in thatColumnVector
, and the range represents the array for rowId. Implementations are free to decide where to put the data vector and offsets and lengths. For example, we can use the first child vector as the data vector, and store offsets and lengths in 2 int arrays in this vector.- Definition Classes
- ArrowColumnVector → ColumnVector
- Annotations
- @Override()
- def getBinary(rowId: Int): Array[Byte]
Returns the binary type value for
rowId
.Returns the binary type value for
rowId
. If the slot forrowId
is null, it should return null.- Definition Classes
- ArrowColumnVector → ColumnVector
- Annotations
- @Override()
- def getBoolean(rowId: Int): Boolean
Returns the boolean type value for
rowId
.Returns the boolean type value for
rowId
. The return value is undefined and can be anything, if the slot forrowId
is null.- Definition Classes
- ArrowColumnVector → ColumnVector
- Annotations
- @Override()
- def getBooleans(rowId: Int, count: Int): Array[Boolean]
Gets boolean type values from
[rowId, rowId + count)
.Gets boolean type values from
[rowId, rowId + count)
. The return values for the null slots are undefined and can be anything.- Definition Classes
- ColumnVector
- def getByte(rowId: Int): Byte
Returns the byte type value for
rowId
.Returns the byte type value for
rowId
. The return value is undefined and can be anything, if the slot forrowId
is null.- Definition Classes
- ArrowColumnVector → ColumnVector
- Annotations
- @Override()
- def getBytes(rowId: Int, count: Int): Array[Byte]
Gets byte type values from
[rowId, rowId + count)
.Gets byte type values from
[rowId, rowId + count)
. The return values for the null slots are undefined and can be anything.- Definition Classes
- ColumnVector
- def getChild(ordinal: Int): ArrowColumnVector
- returns
child
ColumnVector
at the given ordinal.
- Definition Classes
- ArrowColumnVector → ColumnVector
- Annotations
- @Override()
- final def getClass(): Class[_ <: AnyRef]
- Definition Classes
- AnyRef → Any
- Annotations
- @IntrinsicCandidate() @native()
- def getDecimal(rowId: Int, precision: Int, scale: Int): Decimal
Returns the decimal type value for
rowId
.Returns the decimal type value for
rowId
. If the slot forrowId
is null, it should return null.- Definition Classes
- ArrowColumnVector → ColumnVector
- Annotations
- @Override()
- def getDouble(rowId: Int): Double
Returns the double type value for
rowId
.Returns the double type value for
rowId
. The return value is undefined and can be anything, if the slot forrowId
is null.- Definition Classes
- ArrowColumnVector → ColumnVector
- Annotations
- @Override()
- def getDoubles(rowId: Int, count: Int): Array[Double]
Gets double type values from
[rowId, rowId + count)
.Gets double type values from
[rowId, rowId + count)
. The return values for the null slots are undefined and can be anything.- Definition Classes
- ColumnVector
- def getFloat(rowId: Int): Float
Returns the float type value for
rowId
.Returns the float type value for
rowId
. The return value is undefined and can be anything, if the slot forrowId
is null.- Definition Classes
- ArrowColumnVector → ColumnVector
- Annotations
- @Override()
- def getFloats(rowId: Int, count: Int): Array[Float]
Gets float type values from
[rowId, rowId + count)
.Gets float type values from
[rowId, rowId + count)
. The return values for the null slots are undefined and can be anything.- Definition Classes
- ColumnVector
- def getInt(rowId: Int): Int
Returns the int type value for
rowId
.Returns the int type value for
rowId
. The return value is undefined and can be anything, if the slot forrowId
is null.- Definition Classes
- ArrowColumnVector → ColumnVector
- Annotations
- @Override()
- def getInterval(rowId: Int): CalendarInterval
Returns the calendar interval type value for
rowId
.Returns the calendar interval type value for
rowId
. If the slot forrowId
is null, it should return null.In Spark, calendar interval type value is basically two integer values representing the number of months and days in this interval, and a long value representing the number of microseconds in this interval. An interval type vector is the same as a struct type vector with 3 fields:
months
,days
andmicroseconds
.To support interval type, implementations must implement
#getChild(int)
and define 3 child vectors: the first child vector is an int type vector, containing all the month values of all the interval values in this vector. The second child vector is an int type vector, containing all the day values of all the interval values in this vector. The third child vector is a long type vector, containing all the microsecond values of all the interval values in this vector. Note that the ArrowColumnVector leverages its built-in IntervalMonthDayNanoVector instead of above-mentioned protocol.- Definition Classes
- ArrowColumnVector → ColumnVector
- Annotations
- @Override()
- def getInts(rowId: Int, count: Int): Array[Int]
Gets int type values from
[rowId, rowId + count)
.Gets int type values from
[rowId, rowId + count)
. The return values for the null slots are undefined and can be anything.- Definition Classes
- ColumnVector
- def getLong(rowId: Int): Long
Returns the long type value for
rowId
.Returns the long type value for
rowId
. The return value is undefined and can be anything, if the slot forrowId
is null.- Definition Classes
- ArrowColumnVector → ColumnVector
- Annotations
- @Override()
- def getLongs(rowId: Int, count: Int): Array[Long]
Gets long type values from
[rowId, rowId + count)
.Gets long type values from
[rowId, rowId + count)
. The return values for the null slots are undefined and can be anything.- Definition Classes
- ColumnVector
- def getMap(rowId: Int): ColumnarMap
Returns the map type value for
rowId
.Returns the map type value for
rowId
. If the slot forrowId
is null, it should return null.In Spark, map type value is basically a key data array and a value data array. A key from the key array with a index and a value from the value array with the same index contribute to an entry of this map type value.
To support map type, implementations must construct a
ColumnarMap
and return it in this method.ColumnarMap
requires aColumnVector
that stores the data of all the keys of all the maps in this vector, and anotherColumnVector
that stores the data of all the values of all the maps in this vector, and a pair of offset and length which specify the range of the key/value array that belongs to the map type value at rowId.- Definition Classes
- ArrowColumnVector → ColumnVector
- Annotations
- @Override()
- def getShort(rowId: Int): Short
Returns the short type value for
rowId
.Returns the short type value for
rowId
. The return value is undefined and can be anything, if the slot forrowId
is null.- Definition Classes
- ArrowColumnVector → ColumnVector
- Annotations
- @Override()
- def getShorts(rowId: Int, count: Int): Array[Short]
Gets short type values from
[rowId, rowId + count)
.Gets short type values from
[rowId, rowId + count)
. The return values for the null slots are undefined and can be anything.- Definition Classes
- ColumnVector
- final def getStruct(rowId: Int): ColumnarRow
Returns the struct type value for
rowId
.Returns the struct type value for
rowId
. If the slot forrowId
is null, it should return null.To support struct type, implementations must implement
#getChild(int)
and make this vector a tree structure. The number of child vectors must be same as the number of fields of the struct type, and each child vector is responsible to store the data for its corresponding struct field.- Definition Classes
- ColumnVector
- def getUTF8String(rowId: Int): UTF8String
Returns the string type value for
rowId
.Returns the string type value for
rowId
. If the slot forrowId
is null, it should return null.Note that the returned
UTF8String
may point to the data of this column vector, please copy it if you want to keep it after this column vector is freed.- Definition Classes
- ArrowColumnVector → ColumnVector
- Annotations
- @Override()
- def getValueVector(): ValueVector
- final def getVariant(rowId: Int): VariantVal
Returns the Variant value for
rowId
.Returns the Variant value for
rowId
. Similar to#getInterval(int)
, the implementation must implement#getChild(int)
and define 2 child vectors of binary type for the Variant value and metadata.- Definition Classes
- ColumnVector
- def hasNull(): Boolean
Returns true if this column vector contains any null values.
Returns true if this column vector contains any null values.
- Definition Classes
- ArrowColumnVector → ColumnVector
- Annotations
- @Override()
- def hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @IntrinsicCandidate() @native()
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- def isNullAt(rowId: Int): Boolean
Returns whether the value at
rowId
is NULL.Returns whether the value at
rowId
is NULL.- Definition Classes
- ArrowColumnVector → ColumnVector
- Annotations
- @Override()
- final def ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- final def notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @IntrinsicCandidate() @native()
- final def notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @IntrinsicCandidate() @native()
- def numNulls(): Int
Returns the number of nulls in this column vector.
Returns the number of nulls in this column vector.
- Definition Classes
- ArrowColumnVector → ColumnVector
- Annotations
- @Override()
- final def synchronized[T0](arg0: => T0): T0
- Definition Classes
- AnyRef
- def toString(): String
- Definition Classes
- AnyRef → Any
- final def wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException]) @native()
- final def wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
Deprecated Value Members
- def finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.Throwable]) @Deprecated
- Deprecated
(Since version 9)