pyspark.sql.DataFrame.hint

DataFrame.hint(name: str, *parameters: Union[PrimitiveType, List[PrimitiveType]]) → DataFrame

Specifies some hint on the current DataFrame.

New in version 2.2.0.

Changed in version 3.4.0: Supports Spark Connect.

Parameters

namestr

A name of the hint.

parametersstr, list, float or int

Optional parameters.

Returns

DataFrame

Hinted DataFrame

Examples

>>> df = spark.createDataFrame([(2, "Alice"), (5, "Bob")], schema=["age", "name"])
>>> df2 = spark.createDataFrame([Row(height=80, name="Tom"), Row(height=85, name="Bob")])
>>> df.join(df2, "name").explain()  
== Physical Plan ==
...
... +- SortMergeJoin ...
...

Explicitly trigger the broadcast hashjoin by providing the hint in df2.

>>> df.join(df2.hint("broadcast"), "name").explain()
== Physical Plan ==
...
... +- BroadcastHashJoin ...
...

previous

pyspark.sql.DataFrame.head

next

pyspark.sql.DataFrame.inputFiles