public class ContextAwareIterator<T>
extends Object
implements scala.collection.Iterator<T>
As the Python evaluation consumes the parent iterator in a separate thread,
it could consume more data from the parent even after the task ends and the parent is closed.
If an off-heap access exists in the parent iterator, it could cause segmentation fault
which crashes the executor.
Thus, we should use ContextAwareIterator to stop consuming after the task ends.
scala.collection.Iterator.GroupedIterator<B>, scala.collection.Iterator.SliceIterator<A>scala.collection.TraversableOnce.BufferedCanBuildFrom<A,CC extends scala.collection.TraversableOnce<Object>>, scala.collection.TraversableOnce.FlattenOps<A>, scala.collection.TraversableOnce.ForceImplicitAmbiguity, scala.collection.TraversableOnce.MonadOps<A>, scala.collection.TraversableOnce.OnceCanBuildFrom<A>| Constructor and Description |
|---|
ContextAwareIterator(TaskContext context,
scala.collection.Iterator<T> delegate) |
| Modifier and Type | Method and Description |
|---|---|
TaskContext |
context() |
scala.collection.Iterator<T> |
delegate() |
boolean |
hasNext() |
T |
next() |
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait$init$, $plus$plus, apply, buffered, collect, contains, continually, copyToArray, corresponds, drop, dropWhile, duplicate, empty, exists, fill, filter, filterNot, find, flatMap, forall, foreach, from, from, grouped, hasDefiniteSize, indexOf, indexOf, indexWhere, indexWhere, isEmpty, isTraversableAgain, iterate, IteratorCanBuildFrom, length, map, padTo, partition, patch, range, range, sameElements, scanLeft, scanRight, seq, single, slice, sliceIterator, sliding, sliding$default$2, span, tabulate, take, takeWhile, toIterator, toStream, toString, toTraversable, withFilter, zip, zipAll, zipWithIndex$colon$bslash, $div$colon, $init$, addString, addString, addString, aggregate, alternateImplicit, collectFirst, copyToArray, copyToArray, copyToBuffer, count, flattenTraversableOnce, fold, foldLeft, foldRight, max, maxBy, min, minBy, mkString, mkString, mkString, MonadOps, nonEmpty, OnceCanBuildFrom, product, reduce, reduceLeft, reduceLeftOption, reduceOption, reduceRight, reduceRightOption, reversed, size, sum, to, toArray, toBuffer, toIndexedSeq, toIterable, toList, toMap, toSeq, toSet, toVectorpublic ContextAwareIterator(TaskContext context, scala.collection.Iterator<T> delegate)
public TaskContext context()
public scala.collection.Iterator<T> delegate()
public boolean hasNext()
hasNext in interface scala.collection.Iterator<T>