Return thread target wrapper which is recommended to be used in PySpark when the
pinned thread mode is enabled. The wrapper function, before calling original
thread target, it inherits the inheritable properties specific
to JVM thread such as InheritableThreadLocal.
Also, note that pinned thread mode does not close the connection from Python
to JVM when the thread is finished in the Python side. With this wrapper, Python
garbage-collects the Python thread instance and also closes the connection
which finishes JVM thread correctly.
When the pinned thread mode is off, it return the original f.
New in version 3.2.0.
the original thread target.
This API is experimental.
It is important to know that it captures the local properties when you decorate it
whereas InheritableThread captures when the thread is started.
Therefore, it is encouraged to decorate it when you want to capture the local
For example, the local properties from the current Spark context is captured
when you define a function here instead of the invocation:
... def target_func():
... pass # your codes.
If you have any updates on local properties afterwards, it would not be reflected to
the Spark context in target_func().
The example below mimics the behavior of JVM threads as close as possible: