Python Scrapy TypeError: Object of type ‘bytes’ is not JSON serializable is an error which occurs when you try to serialise objects and do it the wrong way.
Today, I will explain in detail why this error is happening and how to fix it in the most efficient way possible.
Exploring the Python Scrapy TypeError: Object of type ‘bytes’ is not JSON serializable
This is an error which occurs when you try to serialise objects and do it the wrong way.
Please double check the error message in this blog post so you can avoid mixing between different python issues.
#
TypeError: Object of type 'bytes' is not JSON serializable
#
Bellow we will describe how the error can be solved. With multiple possible solutions.
The Method that fixed the issue for me : Correctly use .encode(‘ ‘)
In python when you try to serialise objects that are not json serializable, you are going to end up with the error we are trying to solve.
The error takes place when you do something like this
#
total['brand'] = [i.encode('utf-8') for i in brand]
total.append(total)
#
The solution is to stop doing that and let the JSON format take care of the task of serialising the objects..
The error can also take place, if you do
#
for i in brand:
total['i'] = i.strip()
#
The proper solution is to use .decode() on each iteration of i ( each object in total )
#
for i in brand:
total['i'] = i.strip().decode()
#
Do not forget to use .strip() to remove any leading (spaces at the beginning) and trailing (spaces at the end).
You can also try doing this which might solve the annoying issue
#
for i in brand:
total['i'] = i.strip().decode("utf-8").
#
This should be all for this method, these are all the options we can try to solve this issue.
I hope the fixes above fixed your problem. Thank you for reading and reaching the end of this blog post.
Summing-up :
This is the end of this article, I hope we helped you solve the error : Python Scrapy TypeError Object of type bytes is not JSON serializable or at least show you why it happens and guided you in the right direction. If you want to help, you can donate to our Kofi account.
Thank you for reading my blog post to the end, If you want to learn more about the Python programming language, check out the official Python Documentation : https://docs.python.org/3/