Fixing Python multiprocessing.Pool: AttributeError after using pool.map()

Python multiprocessing.Pool: AttributeError after using pool.map() is an error which occurs when you start the pool before defining the classes/function(s).

In the blog post I attempt to explain why this error takes place and how you can solve it, I will also add other solutions that could solve the error if possible.

Exploring the Error : Python multiprocessing.Pool: AttributeError after using pool.map()

This is an error which occurs when you start the pool before defining the classes/function(s).

Please double check so you can avoid mixing between errors. The error message should look like the error message bellow.

                                                                       #
AttributeError: Can't pickle local object 'myClass.mymethod..single'
# you can also get
AttributeError: Can't get attribute 'values' on module 'myscript' from '.../script1.py'
                                                                       #

Bellow I make my best attempt at solving the error and present multiple possible fixes.

The only way to solve this problem : use multiprocessing.Pool() the right way

The AttributeError happens when you start the pool before you define the classes

The first solution is to put the pool start up in the bottom of your code. Bellow the function and classes not before the function.

So, instead of putting this at the beginning

                                                                       #
import multiprocessing
pool = multiprocessing.Pool(multiprocessing.cpu_count() - 1)
... # the rest of your code
                                                                       #

You should put it at the end like this, you should also add name == ‘main’:

                                                                       #
... # the rest of your code
if __name__ == '__main__':  
    with multiprocessing.Pool(multiprocessing.cpu_count() - 1) as pool:
        print(myClass().mymethod())
                                                                       #

You should also move the nested target-function, in my case ‘values’ to the top instead of leaving it inside the classes, this will most likely solve the second error.

By now, both errors should be gone. Everything should work as intended.

The solutions above should be enough to solve the problem.

Summing-up : 

The error in this article is one of the most confusing errors in Python, I hope this article has helped you solve the issue or at least pointed you in the right direction. Keep learning guys, keep coding and cheers.

If you want to learn more about Python, please check out the Python Documentation : https://docs.python.org/3/