AttributeError: ‘DataFrame’ object has no attribute ‘map’ is an error which occurs in Python when you try to map a dataframe.
In this article I am going to show you why this error is happening and how you can solve this error. I am also going to share with you other possible solutions which have worked for many developers who suffered from this error.
Explaining the Error : AttributeError: ‘DataFrame’ object has no attribute ‘map’
First, we need to reproduce the error, let us start by checking out the message of the error. Bellow is the message displayed with the error.
#
AttributeError Traceback (most recent call last)
<ipython-input-11-a19a1763d3ac> in <module>()
1 from pyspark.mllib.clustering import KMeans
2 spark_df = sqlContext.createDataFrame(pandas_df)
845 jc = self._jdf.apply(name)
846 return Column(jc)
AttributeError: 'DataFrame' object has no attribute 'map'
#
Bellow we will describe how the error can be solved. With a possible solution for your issue.
Solution : use explode instead of flatMap
First, using explode instead of flatmap solves the issue for at least 90 percent of people who have a similar error message.
Bellow is an example of how you can use explode in a code instead of using flatmap.
#
df.select($"name",explode($"knownLanguages"))
.show(false)
#
The solution above should be enough to fix your solution.
It is unfortunate that I spent a long time looking for other alternative solutions without even finding a useful solution.
I wish you good luck with your research, but this is the only solution that I have tested and have worked for me.
if you like our effort make sure to consider donating to our Kofi account, there is a red button that you can use if you are feeling generous.
Summing-up
The article is over guys, I hope my effort did not go to waste and I helped some of you solve this error, keep learning Python and keep coding, cheers.
If you want to learn more about Python, please check out the Python Documentation : https://docs.python.org/3/