|
|||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
public interface JavaDStreamLike<T,This extends JavaDStreamLike<T,This,R>,R extends JavaRDDLike<T,R>>
Method Summary | ||
---|---|---|
DStream<T> |
checkpoint(Duration interval)
Enable periodic checkpointing of RDDs of this DStream. |
|
scala.reflect.ClassTag<T> |
classTag()
|
|
StreamingContext |
context()
Return the StreamingContext associated with this DStream |
|
JavaDStream<Long> |
count()
Return a new DStream in which each RDD has a single element generated by counting each RDD of this DStream. |
|
JavaPairDStream<T,Long> |
countByValue()
Return a new DStream in which each RDD contains the counts of each distinct value in each RDD of this DStream. |
|
JavaPairDStream<T,Long> |
countByValue(int numPartitions)
Return a new DStream in which each RDD contains the counts of each distinct value in each RDD of this DStream. |
|
JavaPairDStream<T,Long> |
countByValueAndWindow(Duration windowDuration,
Duration slideDuration)
Return a new DStream in which each RDD contains the count of distinct elements in RDDs in a sliding window over this DStream. |
|
JavaPairDStream<T,Long> |
countByValueAndWindow(Duration windowDuration,
Duration slideDuration,
int numPartitions)
Return a new DStream in which each RDD contains the count of distinct elements in RDDs in a sliding window over this DStream. |
|
JavaDStream<Long> |
countByWindow(Duration windowDuration,
Duration slideDuration)
Return a new DStream in which each RDD has a single element generated by counting the number of elements in a window over this DStream. |
|
DStream<T> |
dstream()
|
|
|
flatMap(FlatMapFunction<T,U> f)
Return a new DStream by applying a function to all elements of this DStream, and then flattening the results |
|
|
flatMapToPair(PairFlatMapFunction<T,K2,V2> f)
Return a new DStream by applying a function to all elements of this DStream, and then flattening the results |
|
void |
foreach(Function<R,Void> foreachFunc)
Deprecated. As of release 0.9.0, replaced by foreachRDD |
|
void |
foreach(Function2<R,Time,Void> foreachFunc)
Deprecated. As of release 0.9.0, replaced by foreachRDD |
|
void |
foreachRDD(Function<R,Void> foreachFunc)
Apply a function to each RDD in this DStream. |
|
void |
foreachRDD(Function2<R,Time,Void> foreachFunc)
Apply a function to each RDD in this DStream. |
|
JavaDStream<java.util.List<T>> |
glom()
Return a new DStream in which each RDD is generated by applying glom() to each RDD of this DStream. |
|
|
map(Function<T,R> f)
Return a new DStream by applying a function to all elements of this DStream. |
|
|
mapPartitions(FlatMapFunction<java.util.Iterator<T>,U> f)
Return a new DStream in which each RDD is generated by applying mapPartitions() to each RDDs of this DStream. |
|
|
mapPartitionsToPair(PairFlatMapFunction<java.util.Iterator<T>,K2,V2> f)
Return a new DStream in which each RDD is generated by applying mapPartitions() to each RDDs of this DStream. |
|
|
mapToPair(PairFunction<T,K2,V2> f)
Return a new DStream by applying a function to all elements of this DStream. |
|
void |
print()
Print the first ten elements of each RDD generated in this DStream. |
|
void |
print(int num)
Print the first num elements of each RDD generated in this DStream. |
|
JavaDStream<T> |
reduce(Function2<T,T,T> f)
Return a new DStream in which each RDD has a single element generated by reducing each RDD of this DStream. |
|
DStream<T> |
reduceByWindow(scala.Function2<T,T,T> reduceFunc,
Duration windowDuration,
Duration slideDuration)
Deprecated. As this API is not Java compatible. |
|
JavaDStream<T> |
reduceByWindow(Function2<T,T,T> reduceFunc,
Duration windowDuration,
Duration slideDuration)
Return a new DStream in which each RDD has a single element generated by reducing all elements in a sliding window over this DStream. |
|
JavaDStream<T> |
reduceByWindow(Function2<T,T,T> reduceFunc,
Function2<T,T,T> invReduceFunc,
Duration windowDuration,
Duration slideDuration)
Return a new DStream in which each RDD has a single element generated by reducing all elements in a sliding window over this DStream. |
|
JavaDStream<Long> |
scalaIntToJavaLong(DStream<Object> in)
|
|
java.util.List<R> |
slice(Time fromTime,
Time toTime)
Return all the RDDs between 'fromDuration' to 'toDuration' (both included) |
|
|
transform(Function<R,JavaRDD<U>> transformFunc)
Return a new DStream in which each RDD is generated by applying a function on each RDD of 'this' DStream. |
|
|
transform(Function2<R,Time,JavaRDD<U>> transformFunc)
Return a new DStream in which each RDD is generated by applying a function on each RDD of 'this' DStream. |
|
|
transformToPair(Function<R,JavaPairRDD<K2,V2>> transformFunc)
Return a new DStream in which each RDD is generated by applying a function on each RDD of 'this' DStream. |
|
|
transformToPair(Function2<R,Time,JavaPairRDD<K2,V2>> transformFunc)
Return a new DStream in which each RDD is generated by applying a function on each RDD of 'this' DStream. |
|
|
transformWith(JavaDStream<U> other,
Function3<R,JavaRDD<U>,Time,JavaRDD<W>> transformFunc)
Return a new DStream in which each RDD is generated by applying a function on each RDD of 'this' DStream and 'other' DStream. |
|
|
transformWith(JavaPairDStream<K2,V2> other,
Function3<R,JavaPairRDD<K2,V2>,Time,JavaRDD<W>> transformFunc)
Return a new DStream in which each RDD is generated by applying a function on each RDD of 'this' DStream and 'other' DStream. |
|
|
transformWithToPair(JavaDStream<U> other,
Function3<R,JavaRDD<U>,Time,JavaPairRDD<K2,V2>> transformFunc)
Return a new DStream in which each RDD is generated by applying a function on each RDD of 'this' DStream and 'other' DStream. |
|
|
transformWithToPair(JavaPairDStream<K2,V2> other,
Function3<R,JavaPairRDD<K2,V2>,Time,JavaPairRDD<K3,V3>> transformFunc)
Return a new DStream in which each RDD is generated by applying a function on each RDD of 'this' DStream and 'other' DStream. |
|
R |
wrapRDD(RDD<T> in)
|
Method Detail |
---|
scala.reflect.ClassTag<T> classTag()
DStream<T> dstream()
R wrapRDD(RDD<T> in)
JavaDStream<Long> scalaIntToJavaLong(DStream<Object> in)
void print()
void print(int num)
num
- (undocumented)JavaDStream<Long> count()
JavaPairDStream<T,Long> countByValue()
JavaPairDStream<T,Long> countByValue(int numPartitions)
numPartitions
partitions.
numPartitions
- number of partitions of each RDD in the new DStream.
JavaDStream<Long> countByWindow(Duration windowDuration, Duration slideDuration)
windowDuration
- (undocumented)slideDuration
- (undocumented)
JavaPairDStream<T,Long> countByValueAndWindow(Duration windowDuration, Duration slideDuration)
windowDuration
- width of the window; must be a multiple of this DStream's
batching intervalslideDuration
- sliding interval of the window (i.e., the interval after which
the new DStream will generate RDDs); must be a multiple of this
DStream's batching interval
JavaPairDStream<T,Long> countByValueAndWindow(Duration windowDuration, Duration slideDuration, int numPartitions)
numPartitions
partitions.
windowDuration
- width of the window; must be a multiple of this DStream's
batching intervalslideDuration
- sliding interval of the window (i.e., the interval after which
the new DStream will generate RDDs); must be a multiple of this
DStream's batching intervalnumPartitions
- number of partitions of each RDD in the new DStream.
JavaDStream<java.util.List<T>> glom()
StreamingContext context()
StreamingContext
associated with this DStream
<R> JavaDStream<R> map(Function<T,R> f)
<K2,V2> JavaPairDStream<K2,V2> mapToPair(PairFunction<T,K2,V2> f)
<U> JavaDStream<U> flatMap(FlatMapFunction<T,U> f)
f
- (undocumented)
<K2,V2> JavaPairDStream<K2,V2> flatMapToPair(PairFlatMapFunction<T,K2,V2> f)
f
- (undocumented)
<U> JavaDStream<U> mapPartitions(FlatMapFunction<java.util.Iterator<T>,U> f)
f
- (undocumented)
<K2,V2> JavaPairDStream<K2,V2> mapPartitionsToPair(PairFlatMapFunction<java.util.Iterator<T>,K2,V2> f)
f
- (undocumented)
JavaDStream<T> reduce(Function2<T,T,T> f)
f
- (undocumented)
DStream<T> reduceByWindow(scala.Function2<T,T,T> reduceFunc, Duration windowDuration, Duration slideDuration)
reduceFunc
- associative reduce functionwindowDuration
- width of the window; must be a multiple of this DStream's
batching intervalslideDuration
- sliding interval of the window (i.e., the interval after which
the new DStream will generate RDDs); must be a multiple of this
DStream's batching interval
JavaDStream<T> reduceByWindow(Function2<T,T,T> reduceFunc, Duration windowDuration, Duration slideDuration)
reduceFunc
- associative reduce functionwindowDuration
- width of the window; must be a multiple of this DStream's
batching intervalslideDuration
- sliding interval of the window (i.e., the interval after which
the new DStream will generate RDDs); must be a multiple of this
DStream's batching interval
JavaDStream<T> reduceByWindow(Function2<T,T,T> reduceFunc, Function2<T,T,T> invReduceFunc, Duration windowDuration, Duration slideDuration)
reduceFunc
- associative reduce functioninvReduceFunc
- inverse reduce functionwindowDuration
- width of the window; must be a multiple of this DStream's
batching intervalslideDuration
- sliding interval of the window (i.e., the interval after which
the new DStream will generate RDDs); must be a multiple of this
DStream's batching interval
java.util.List<R> slice(Time fromTime, Time toTime)
fromTime
- (undocumented)toTime
- (undocumented)
void foreach(Function<R,Void> foreachFunc)
foreachFunc
- (undocumented)void foreach(Function2<R,Time,Void> foreachFunc)
foreachFunc
- (undocumented)void foreachRDD(Function<R,Void> foreachFunc)
foreachFunc
- (undocumented)void foreachRDD(Function2<R,Time,Void> foreachFunc)
foreachFunc
- (undocumented)<U> JavaDStream<U> transform(Function<R,JavaRDD<U>> transformFunc)
transformFunc
- (undocumented)
<U> JavaDStream<U> transform(Function2<R,Time,JavaRDD<U>> transformFunc)
transformFunc
- (undocumented)
<K2,V2> JavaPairDStream<K2,V2> transformToPair(Function<R,JavaPairRDD<K2,V2>> transformFunc)
transformFunc
- (undocumented)
<K2,V2> JavaPairDStream<K2,V2> transformToPair(Function2<R,Time,JavaPairRDD<K2,V2>> transformFunc)
transformFunc
- (undocumented)
<U,W> JavaDStream<W> transformWith(JavaDStream<U> other, Function3<R,JavaRDD<U>,Time,JavaRDD<W>> transformFunc)
other
- (undocumented)transformFunc
- (undocumented)
<U,K2,V2> JavaPairDStream<K2,V2> transformWithToPair(JavaDStream<U> other, Function3<R,JavaRDD<U>,Time,JavaPairRDD<K2,V2>> transformFunc)
other
- (undocumented)transformFunc
- (undocumented)
<K2,V2,W> JavaDStream<W> transformWith(JavaPairDStream<K2,V2> other, Function3<R,JavaPairRDD<K2,V2>,Time,JavaRDD<W>> transformFunc)
other
- (undocumented)transformFunc
- (undocumented)
<K2,V2,K3,V3> JavaPairDStream<K3,V3> transformWithToPair(JavaPairDStream<K2,V2> other, Function3<R,JavaPairRDD<K2,V2>,Time,JavaPairRDD<K3,V3>> transformFunc)
other
- (undocumented)transformFunc
- (undocumented)
DStream<T> checkpoint(Duration interval)
interval
- Time interval after which generated RDD will be checkpointed
|
|||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |