public class Word2VecModel extends Model<Word2VecModel> implements Word2VecBase, MLWritable
Word2Vec
.Modifier and Type | Class and Description |
---|---|
static class |
Word2VecModel.Data$ |
static class |
Word2VecModel.Word2VecModelWriter$ |
Modifier and Type | Method and Description |
---|---|
Word2VecModel |
copy(ParamMap extra)
Creates a copy of this instance with the same UID and some extra params.
|
Dataset<Row> |
findSynonyms(String word,
int num)
Find "num" number of words closest in similarity to the given word, not
including the word itself.
|
Dataset<Row> |
findSynonyms(Vector vec,
int num)
Find "num" number of words whose vector representation is most similar to the supplied vector.
|
scala.Tuple2<String,Object>[] |
findSynonymsArray(String word,
int num)
Find "num" number of words closest in similarity to the given word, not
including the word itself.
|
scala.Tuple2<String,Object>[] |
findSynonymsArray(Vector vec,
int num)
Find "num" number of words whose vector representation is most similar to the supplied vector.
|
Dataset<Row> |
getVectors() |
Param<String> |
inputCol()
Param for input column name.
|
static Word2VecModel |
load(String path) |
IntParam |
maxIter()
Param for maximum number of iterations (>= 0).
|
IntParam |
maxSentenceLength()
Sets the maximum length (in words) of each sentence in the input data.
|
IntParam |
minCount()
The minimum number of times a token must appear to be included in the word2vec model's
vocabulary.
|
IntParam |
numPartitions()
Number of partitions for sentences of words.
|
Param<String> |
outputCol()
Param for output column name.
|
static MLReader<Word2VecModel> |
read() |
LongParam |
seed()
Param for random seed.
|
Word2VecModel |
setInputCol(String value) |
Word2VecModel |
setOutputCol(String value) |
DoubleParam |
stepSize()
Param for Step size to be used for each iteration of optimization (> 0).
|
String |
toString() |
Dataset<Row> |
transform(Dataset<?> dataset)
Transform a sentence column to a vector column to represent the whole sentence.
|
StructType |
transformSchema(StructType schema)
Check transform validity and derive the output schema from the input schema.
|
String |
uid()
An immutable unique ID for the object and its derivatives.
|
IntParam |
vectorSize()
The dimension of the code that you want to transform from words.
|
IntParam |
windowSize()
The window size (context words from [-window, window]).
|
MLWriter |
write()
Returns an
MLWriter instance for this ML instance. |
transform, transform, transform
params
getMaxSentenceLength, getMinCount, getNumPartitions, getVectorSize, getWindowSize, validateAndTransformSchema
getInputCol
getOutputCol
getMaxIter
getStepSize
clear, copyValues, defaultCopy, defaultParamMap, explainParam, explainParams, extractParamMap, extractParamMap, get, getDefault, getOrDefault, getParam, hasDefault, hasParam, isDefined, isSet, onParamChange, paramMap, params, set, set, set, setDefault, setDefault, shouldOwn
save
$init$, initializeForcefully, initializeLogIfNecessary, initializeLogIfNecessary, initializeLogIfNecessary$default$2, initLock, isTraceEnabled, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logName, logTrace, logTrace, logWarning, logWarning, org$apache$spark$internal$Logging$$log__$eq, org$apache$spark$internal$Logging$$log_, uninitialize
public static MLReader<Word2VecModel> read()
public static Word2VecModel load(String path)
public final IntParam vectorSize()
Word2VecBase
vectorSize
in interface Word2VecBase
public final IntParam windowSize()
Word2VecBase
windowSize
in interface Word2VecBase
public final IntParam numPartitions()
Word2VecBase
numPartitions
in interface Word2VecBase
public final IntParam minCount()
Word2VecBase
minCount
in interface Word2VecBase
public final IntParam maxSentenceLength()
Word2VecBase
maxSentenceLength
size.
Default: 1000maxSentenceLength
in interface Word2VecBase
public final LongParam seed()
HasSeed
public DoubleParam stepSize()
HasStepSize
stepSize
in interface HasStepSize
public final IntParam maxIter()
HasMaxIter
maxIter
in interface HasMaxIter
public final Param<String> outputCol()
HasOutputCol
outputCol
in interface HasOutputCol
public final Param<String> inputCol()
HasInputCol
inputCol
in interface HasInputCol
public String uid()
Identifiable
uid
in interface Identifiable
public Dataset<Row> findSynonyms(String word, int num)
word
- (undocumented)num
- (undocumented)public Dataset<Row> findSynonyms(Vector vec, int num)
vec
- (undocumented)num
- (undocumented)public scala.Tuple2<String,Object>[] findSynonymsArray(Vector vec, int num)
vec
- (undocumented)num
- (undocumented)public scala.Tuple2<String,Object>[] findSynonymsArray(String word, int num)
word
- (undocumented)num
- (undocumented)public Word2VecModel setInputCol(String value)
public Word2VecModel setOutputCol(String value)
public Dataset<Row> transform(Dataset<?> dataset)
transform
in class Transformer
dataset
- (undocumented)public StructType transformSchema(StructType schema)
PipelineStage
We check validity for interactions between parameters during transformSchema
and
raise an exception if any parameter value is invalid. Parameter value checks which
do not depend on other parameters are handled by Param.validate()
.
Typical implementation should first conduct verification on schema change and parameter validity, including complex parameter interaction checks.
transformSchema
in class PipelineStage
schema
- (undocumented)public Word2VecModel copy(ParamMap extra)
Params
defaultCopy()
.copy
in interface Params
copy
in class Model<Word2VecModel>
extra
- (undocumented)public MLWriter write()
MLWritable
MLWriter
instance for this ML instance.write
in interface MLWritable
public String toString()
toString
in interface Identifiable
toString
in class Object