public class VectorSizeHint extends Transformer implements HasInputCol, HasHandleInvalid, DefaultParamsWritable
Note: VectorSizeHint modifies inputCol to include size metadata and does not have an outputCol.
| Constructor and Description |
|---|
VectorSizeHint() |
VectorSizeHint(String uid) |
| Modifier and Type | Method and Description |
|---|---|
VectorSizeHint |
copy(ParamMap extra)
Creates a copy of this instance with the same UID and some extra params.
|
int |
getSize()
group getParam
|
Param<String> |
handleInvalid()
Param for how to handle invalid entries.
|
Param<String> |
inputCol()
Param for input column name.
|
static VectorSizeHint |
load(String path) |
static MLReader<T> |
read() |
VectorSizeHint |
setHandleInvalid(String value) |
VectorSizeHint |
setInputCol(String value) |
VectorSizeHint |
setSize(int value) |
IntParam |
size()
The size of Vectors in
inputCol. |
String |
toString() |
Dataset<Row> |
transform(Dataset<?> dataset)
Transforms the input dataset.
|
StructType |
transformSchema(StructType schema)
Check transform validity and derive the output schema from the input schema.
|
String |
uid()
An immutable unique ID for the object and its derivatives.
|
transform, transform, transformparamsgetInputColgetHandleInvalidclear, copyValues, defaultCopy, defaultParamMap, explainParam, explainParams, extractParamMap, extractParamMap, get, getDefault, getOrDefault, getParam, hasDefault, hasParam, isDefined, isSet, onParamChange, paramMap, params, set, set, set, setDefault, setDefault, shouldOwnwritesave$init$, initializeForcefully, initializeLogIfNecessary, initializeLogIfNecessary, initializeLogIfNecessary$default$2, initLock, isTraceEnabled, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logName, logTrace, logTrace, logWarning, logWarning, org$apache$spark$internal$Logging$$log__$eq, org$apache$spark$internal$Logging$$log_, uninitializepublic VectorSizeHint(String uid)
public VectorSizeHint()
public static VectorSizeHint load(String path)
public static MLReader<T> read()
public final Param<String> inputCol()
HasInputColinputCol in interface HasInputColpublic String uid()
Identifiableuid in interface Identifiablepublic IntParam size()
inputCol.public int getSize()
public VectorSizeHint setSize(int value)
public VectorSizeHint setInputCol(String value)
public Param<String> handleInvalid()
skip (filter out rows with invalid vectors), error (throw an
error) and optimistic (do not check the vector size, and keep all rows). error by default.
Note: Users should take care when setting this param to optimistic. The use of the
optimistic option will prevent the transformer from validating the sizes of vectors in
inputCol. A mismatch between the metadata of a column and its contents could result in
unexpected behaviour or errors when using that column.
handleInvalid in interface HasHandleInvalidpublic VectorSizeHint setHandleInvalid(String value)
public Dataset<Row> transform(Dataset<?> dataset)
Transformertransform in class Transformerdataset - (undocumented)public StructType transformSchema(StructType schema)
PipelineStage
We check validity for interactions between parameters during transformSchema and
raise an exception if any parameter value is invalid. Parameter value checks which
do not depend on other parameters are handled by Param.validate().
Typical implementation should first conduct verification on schema change and parameter validity, including complex parameter interaction checks.
transformSchema in class PipelineStageschema - (undocumented)public VectorSizeHint copy(ParamMap extra)
ParamsdefaultCopy().copy in interface Paramscopy in class Transformerextra - (undocumented)public String toString()
toString in interface IdentifiabletoString in class Object