pyspark.pandas.window.Expanding.median#

Expanding.median()[source]#

Calculate the expanding median.

Added in version 4.3.0.

Note

the current implementation of this API uses Spark’s Window without specifying partition specification. This leads to move all data into single partition in single machine and could cause serious performance degradation. Avoid this method against very large dataset.

Returns:
Series or DataFrame

Returned object type is determined by the caller of the expanding calculation.

See also

pyspark.pandas.Series.expanding

Calling object with Series data.

pyspark.pandas.DataFrame.expanding

Calling object with DataFrames.

pyspark.pandas.Series.median

Equivalent method for Series.

pyspark.pandas.DataFrame.median

Equivalent method for DataFrame.

Notes

median in pandas-on-Spark uses distributed percentile approximation algorithm unlike pandas, the result might be different with pandas.

Examples

>>> s = ps.Series([4, 3, 5, 2, 6])
>>> s
0    4
1    3
2    5
3    2
4    6
dtype: int64

>>> s.expanding(2).median()
0    NaN
1    3.0
2    4.0
3    3.0
4    4.0
dtype: float64

>>> s.expanding(3).median()
0    NaN
1    NaN
2    4.0
3    3.0
4    4.0
dtype: float64