sync/async/multithreading/multiprocessing in Python
Parallel and concurrent processing in Python is confusig, mainly due to the Global Interpreter Lock and many options to overcome it:
and more ...
Let's explore differences between these various approaches with a practical example - Broken Link Checker.
With the broken link checker application written in 4 different ways. The project is organized as follows:
- multitasking - notebook where I explore various aspects of this project. Check it if you're looking for more explanations
- multiprocessing_check.py - needed to explore multiprocessing in Jupyter Notebook
- benchmark.py - main performance comparison of various approaches
- sequential_checker.py
- multithreading_checker.py
- multiprocessing_checker.py
- asyncio_checker.py