pyspark.sql.DataFrame.foreachPartition¶

DataFrame.foreachPartition(f)[source]¶

Applies the f function to each partition of this DataFrame.

This a shorthand for df.rdd.foreachPartition().

New in version 1.3.0.

Examples

>>> def f(people):
...     for person in people:
...         print(person.name)
>>> df.foreachPartition(f)

pyspark.sql.DataFrame.foreach

pyspark.sql.DataFrame.freqItems