StreamingContext.queueStream(rdds: List[pyspark.rdd.RDD[T]], oneAtATime: bool = True, default: Optional[pyspark.rdd.RDD[T]] = None) → pyspark.streaming.dstream.DStream[T][source]

Create an input stream from a queue of RDDs or list. In each batch, it will process either one or all of the RDDs returned by the queue.


Queue of RDDs

oneAtATimebool, optional

pick one rdd each time or pick all of them once.

defaultpyspark.RDD, optional

The default rdd if no more in rdds


Changes to the queue after the stream is created will not be recognized.