public class RawTextHelper
extends Object
Constructor and Description |
---|
RawTextHelper() |
Modifier and Type | Method and Description |
---|---|
static long |
add(long v1,
long v2) |
static long |
max(long v1,
long v2) |
static scala.collection.Iterator<scala.Tuple2<String,Object>> |
splitAndCountPartitions(scala.collection.Iterator<String> iter)
Splits lines and counts the words.
|
static long |
subtract(long v1,
long v2) |
static scala.collection.Iterator<scala.Tuple2<String,Object>> |
topK(scala.collection.Iterator<scala.Tuple2<String,Object>> data,
int k)
Gets the top k words in terms of word counts.
|
static void |
warmUp(SparkContext sc)
Warms up the SparkContext in master and executor by running tasks to force JIT kick in
before real workload starts.
|
public static scala.collection.Iterator<scala.Tuple2<String,Object>> splitAndCountPartitions(scala.collection.Iterator<String> iter)
iter
- (undocumented)public static scala.collection.Iterator<scala.Tuple2<String,Object>> topK(scala.collection.Iterator<scala.Tuple2<String,Object>> data, int k)
data
iterator (that is, the counts have been reduced).data
- (undocumented)k
- (undocumented)public static void warmUp(SparkContext sc)
sc
- (undocumented)public static long add(long v1, long v2)
public static long subtract(long v1, long v2)
public static long max(long v1, long v2)