You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Inputs are queued into the queue straight way and n (future) workers are created that immediately start de-queueing. The following loop collects available results at each round, essentially waiting for the slowest of the n workers to return their result. It then starts another round of n concurrent workers until the queue is empty.
Like core.pmap, this approach to parallelism works well when workers are roughly doing the same amount of work. If some item in the queue requires substantially more effort to produce, then it's not particularly efficient as every other worker in the queue needs to wait. But regardless, the existence of this parallel.pmap is more about independence from the size of sequential chunks (normally 32) and the number of cores, so you can control exactly how many threads are running in parallel independently from the platform and the input sequence type.
Activity
reborg commentedon Mar 18, 2023
Yes, you can see here https://github.com/reborg/parallel/blob/master/src/parallel/core.clj#L527
Inputs are queued into the queue straight way and n (
future) workers are created that immediately start de-queueing. The following loop collects available results at each round, essentially waiting for the slowest of the n workers to return their result. It then starts another round of n concurrent workers until the queue is empty.Like
core.pmap, this approach to parallelism works well when workers are roughly doing the same amount of work. If some item in the queue requires substantially more effort to produce, then it's not particularly efficient as every other worker in the queue needs to wait. But regardless, the existence of thisparallel.pmapis more about independence from the size of sequential chunks (normally 32) and the number of cores, so you can control exactly how many threads are running in parallel independently from the platform and the input sequence type.