pyspark.BarrierTaskContext.getTaskInfos#

BarrierTaskContext.getTaskInfos()[source]#

Returns BarrierTaskInfo for all tasks in this barrier stage, ordered by partition ID.

New in version 2.4.0.

Notes

This API is experimental

Examples

>>> from pyspark import BarrierTaskContext
>>> rdd = spark.sparkContext.parallelize([1])
>>> barrier_info = rdd.barrier().mapPartitions(
...     lambda _: [BarrierTaskContext.get().getTaskInfos()]).collect()[0][0]
>>> barrier_info.address
'...:...'