Skip to content

How to Parallelize a for Loop in Python

If you need to run a for loop in parallel, then you can do one of the following:

Option 1 – Using multiprocessing

import multiprocessing def sumall(value): return sum(range(1, value + 1)) pool_obj = multiprocessing.Pool() answer = pool_obj.map(sumall,range(,5)) print(answer)
Code language: Python (python)

Option 2 – Using joblib module

from joblib import Parallel, delayed import math def sqrt_func(i, j): time.sleep(1) return math.sqrt(i**j) Parallel(n_jobs=2)(delayed(sqrt_func)(i, j) for i in range(5) for j in range(2))
Code language: Python (python)

Option 3 – Using asyncio

import asyncio import time def background(f): def wrapped(*args, **kwargs): return asyncio.get_event_loop().run_in_executor(None, f, *args, **kwargs) return wrapped @background def your_function(argument): time.sleep(2) print('function finished for '+str(argument)) for i in range(10): your_function(i) print('loop finished')
Code language: Python (python)

See also  How to Write Unmaintainable Code
Tags:
Subscribe
Notify of
guest
0 Comments
Inline Feedbacks
View all comments
0
Would love your thoughts, please comment.x
()
x