Hi @ecpolley! Hope all is well.
I'm reaching out today to see if you'd be interested in receiving a PR that contains a {future} backend for SuperLearner. future is part of a collection of R packages by Henrik Bengtsson that seeks to unify the underlying parallelization frameworks within R. The reason for preferring a future-oriented backend is it allows us to easily work with remote and cluster-based resources for intense computations.
For a future backend to exist, very few changes are required. For example, the following diff based on a slight code change with mcSuperLearner (based on multicore) seems to work out quite well.
Let me know if you would be interested in receiving this backend.
Moreover, glancing at the code differences between snow*(), mc*(), and future*() SuperLearner backends, there is a lot of similar code that could potentially be combined with minimal disruption.
/ cc @SG-TLR
Hi @ecpolley! Hope all is well.
I'm reaching out today to see if you'd be interested in receiving a PR that contains a
{future}backend for SuperLearner.futureis part of a collection of R packages by Henrik Bengtsson that seeks to unify the underlying parallelization frameworks within R. The reason for preferring afuture-oriented backend is it allows us to easily work with remote and cluster-based resources for intense computations.For a future backend to exist, very few changes are required. For example, the following diff based on a slight code change with
mcSuperLearner(based onmulticore) seems to work out quite well.Let me know if you would be interested in receiving this backend.
Moreover, glancing at the code differences between
snow*(),mc*(), andfuture*()SuperLearner backends, there is a lot of similar code that could potentially be combined with minimal disruption./ cc @SG-TLR