Unsupported class file major version is an error which happens because of Spark and Java compatibility issues.
In this article, I am going to explain what happens when you get this error and how you can solve it with a main solution, we will also explore other solutions which can possibly solve the issue.
Explaining the Error : Unsupported class file major version
First of all, we should understand that the error occurs when launching pySpark because of Spark and Java compatibility issues.
When you reproduce the error, it should look like this.
#
Traceback (most recent call last):
File "<input>", line 4, in <module>
sock_info = self.ctx._jvm.PythonRDD.collectAndServe(self._jrdd.rdd())
answer, self.gateway_client, self.target_id, self.name)
format(target_id, ".", name), value)
py4j.protocol.Py4JJavaError: An error occurred while calling z:org.apache.spark.api.python.PythonRDD.collectAndServe.
: java.lang.IllegalArgumentException: Unsupported class file major version 55
#
To solve the problem above, I have a couple of solutions which have worked for me, bellow is a detailed explanation of the solutions.
Solution 1 : only for Jupyter Notebook users
To be able to run Spark successfully in Jupyter Notebook, let us try the instructions bellow.
The most common solution to this error is to downgrade your java, for example from java 11 to java 8.
First, we should install java 8. You can use the command bellow.
#
sudo apt install openjdk-8-jdk
#
Setting a version of Java is important since Now we have two versions of java installed.
Launch the following command. Choose java 8 and save.
#
sudo update-alternatives --config java
#
Now, the error above should be gone forever. If not, follow the solution bellow.
Solution 2 : use findspark to add the spark location
You can use findspark in order to add the spark location to the beginning of your file by adding the lines of code bellow to the beginning of your file.
#
import findspark
import os
spark_location='/opt/spark-2.4.3/' # Set your own
java8_location= '/usr/lib/jvm/java-8-openjdk-amd64' # Set your own
os.environ['JAVA_HOME'] = java8_location
findspark.init(spark_home=spark_location)
#
If this article has been useful for your particular case, consider donating to our Kofi account, there is a red button at the top of this page.
Summing-up
This is the end of our article, I hope the solutions I presented worked for you, Learning Python is a fun journey, do not let the erros discourage you. Keep coding and cheers. If you want to learn more about Python, please check out the Python Documentation : https://docs.python.org/3/