public class RawTextHelper
extends Object
Constructor and Description |
---|
RawTextHelper() |
Modifier and Type | Method and Description |
---|---|
static long |
add(long v1,
long v2) |
static long |
max(long v1,
long v2) |
static scala.collection.Iterator<scala.Tuple2<String,Object>> |
splitAndCountPartitions(scala.collection.Iterator<String> iter)
Splits lines and counts the words.
|
static long |
subtract(long v1,
long v2) |
static scala.collection.Iterator<scala.Tuple2<String,Object>> |
topK(scala.collection.Iterator<scala.Tuple2<String,Object>> data,
int k)
Gets the top k words in terms of word counts.
|
static void |
warmUp(SparkContext sc)
Warms up the SparkContext in master and slave by running tasks to force JIT kick in
before real workload starts.
|
public static scala.collection.Iterator<scala.Tuple2<String,Object>> splitAndCountPartitions(scala.collection.Iterator<String> iter)
public static scala.collection.Iterator<scala.Tuple2<String,Object>> topK(scala.collection.Iterator<scala.Tuple2<String,Object>> data, int k)
data
iterator (that is, the counts have been reduced).public static void warmUp(SparkContext sc)
public static long add(long v1, long v2)
public static long subtract(long v1, long v2)
public static long max(long v1, long v2)