UnicodeDecodeError: ‘ascii’ codec can’t decode byte when Loading UTF-8 file in numpy is a numpy error which occurs when you try to load a CSV file into a numpy array and you do not know how to encode and decode the data properly.
I will explain why this error takes place and how to fix it, while also trying to add other solutions that could solve the error.
Exploring UnicodeDecodeError: ‘ascii’ codec can’t decode byte when Loading UTF-8 file in numpy
This is a numpy error which occurs when you try to load a CSV file into a numpy array and you do not know how to encode and decode the data properly.
Do not mix between errors. Make sure the error message looks like the error message bellow after double checking.
#
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 2: ordinal not in range(128).
#
In the section bellow we will explain the root of the error more and propose the best possible fix.
Solution : use genfromtxt with numpy ‘correctly’
When you load a CSV file into a numpy array you should know to correctly encode and decode the data.
First we should store the CSV file in a variable, in this case we will call the variable y.
The solution is to use genfromtxt.
#
np.genfromtxt([txt.encode()],delimiter=',',dtype='U20',
converters={0:lambda y: y.decode()})
#
genfromtxt takes a dtype parameter which is U20 in this case.
converters is very important in order to convert the data of each column to a value.
We use 0:lambda y so that the value can be either the value of y or 0.
Then we use y.decode() to decode the data.
After this the error should be gone for good. This the easiest and smartest way to load a csv file into a numpy array with no problems.
All of this happens before converters converts the whole file.
I hope the fix above fixed your problem, good luck with the scripts to come.
The solutions above should be enough to solve the problem, if you like our effort make sure to consider donating to our Kofi account, there is a red button that you can use if you are feeling generous.
Summing-up
The error above can be very confusing, thank you for reading and cheers. If you want to learn more about Python, please check out the Python Documentation : https://docs.python.org/3/