Add missing files for docker container building

[le-ander]

also updated the dockerfile to the latest version we have.
closes #51

[LuckyMD]

This is a lot more than needed to run the tutorial, but maybe that's good so that people can use the env for further analysis as well. Have you tested that the whole workflow runs on this?

[le-ander]

Yes, it's indeed a lot more. Haven't run the tutorial with this in a while, but I can just start a testrun now 👍

[LuckyMD]

I like how everyone will have sfaira, episcanpy, and BED installed in their tutorial environment ;). Do you think it's worth it to separate mandatory from non-mandatory installs? Maybe just by comments in the dockerfile?

[le-ander]

I don't mind, either way is fine for me. 🤷‍♂️
The majority of the build time will go into compiling R and python anyways so I don't think commenting packages will make much of a difference.

[LuckyMD]

I don't mean commenting out packages, but just putting a comment in the Dockerfile saying "required install" and "optional extension" as headings?

[LuckyMD]

Also, I think that R package install will also take quite some time ^^

[LuckyMD]

Btw... if you rerun the workflow, we could save that and update the "latest notebook" with an updated version of all dependencies! Possible you think? I would ofc go through results manually as well to check if it's more than just running through.

[le-ander]

yeah, I can just add some comments in the python-packages.txt and Dockerfile, will also add version pinnings again.

sure, we can save the run and have a new "latest notebook", could also address #20 and #39 when we're doing that, don't you think?

[le-ander]

Oh dear, looks like the scran installation in my current container fails 🙄
I get a C compilation error when installing ‘BiocNeighbors’ that just doesn't give me anything on google:

g++ -std=gnu++11 -I"/opt/R/lib/R/include" -DNDEBUG  -I"/opt/R/lib/R/library/Rcpp/include" -I"/opt/R/lib/R/library/RcppAnnoy/include" -I"/opt/R/lib/R/library/RcppHNSW/include" -I/usr/local/include  -fpic  -g -O2  -c annoy.cpp -o annoy.o
In file included from annoy.cpp:1:
annoy.h:33:63: error: wrong number of template arguments (4, should be 5)
     typedef AnnoyIndex<Index_t, Data_t, Distance, Kiss64Random> _index;

I'm tempted to just switch my containers to R 4.0 and Bioconductor 3.12
Do you have any experience with R 4 already?

[LuckyMD]

I have successfully avoided R 4.0 so far... this looks fairly painful. It might make sense to update, but then there will be more fun things coming up in the workflow, I imagine.

[LuckyMD]

sure, we can save the run and have a new "latest notebook", could also address #20 and #39 when we're doing that, don't you think?

Yes, should def do that... and remove some code that was just added in a PR to make the notebook compatible with latest tools. I would then tweet about it and @ you... off to the next 500 stars ;).

[le-ander]

Ok cool, I'll put building a R4 container on my list then. will wait until next week though as volker is releasing a new version of scVelo which annotates hvgs instead of filtering, want to have that in the container as well.

Can you, as a first step, send me a cleaned version of the latest notebook that has the extra code removed and implements #20 and #39 ? I'll then give it a go and run it with the R4 container 🤞

[LuckyMD]

I can do that... but maybe only on the weekend, as I have to finish these revisions and have to prep sth for a meeting on Friday as well. Although it shouldn't take too long... maybe I can fit in tomorrow night as well.

[le-ander]

No rush! As I said, I'll wait with the container building until the next scVelo release. So if you prep the notebook until some time next week that's fine :)

[kuang-da]

Oh dear, looks like the scran installation in my current container fails
I get a C compilation error when installing ‘BiocNeighbors’ that just doesn't give me anything on google:

g++ -std=gnu++11 -I"/opt/R/lib/R/include" -DNDEBUG  -I"/opt/R/lib/R/library/Rcpp/include" -I"/opt/R/lib/R/library/RcppAnnoy/include" -I"/opt/R/lib/R/library/RcppHNSW/include" -I/usr/local/include  -fpic  -g -O2  -c annoy.cpp -o annoy.o
In file included from annoy.cpp:1:
annoy.h:33:63: error: wrong number of template arguments (4, should be 5)
     typedef AnnoyIndex<Index_t, Data_t, Distance, Kiss64Random> _index;

I'm tempted to just switch my containers to R 4.0 and Bioconductor 3.12
Do you have any experience with R 4 already?

Hi, @le-ander. I just met the exact same error as yours while installing scater in the rocker and totally lost about it.

annoy.h:33:63: error: wrong number of template arguments (4, should be 5)

It seems the error is about the calculation of the nearest neighbor in the package BiocNeighbors. But I am really not sure how to fix it.....

May I ask how did you bypass the error? Thank you very much!!!

My dockerfile is as follows:

FROM rocker/tidyverse:3.6.3

RUN apt-get update \
    && apt-get install -y libboost-all-dev \
    && apt-get install -y libbz2-dev \
    && apt-get install -y libjpeg-dev

RUN R -e "install.packages(\"latticeExtra\")"
RUN R -e "BiocManager::install(\"GO.db\")" 
Run R -e "BiocManager::install(\"HSMMSingleCell\")"
RUN R -e "BiocManager::install(\"org.Mm.eg.db\")"
RUN R -e "BiocManager::install(\"org.Hs.eg.db\")"
RUN R -e "BiocManager::install(\"DESeq2\")"
RUN R -e "BiocManager::install(\"SingleCellExperiment\")"
# RUN R -e "BiocManager::install(\"BiocNeighbors\")"
RUN R -e "BiocManager::install(\"scater\")"

[LuckyMD]

@le-ander check out #58 for an updated notebook to test.

[arkal]

FYI @le-ander @kuang-da I had this error come up in a different context, but what helped me was

install.packages('devtools', repos="https://cloud.r-project.org")
devtools::install_version("RcppAnnoy", version="0.0.16", repos="https://cloud.r-project.org")

It looks like something changed between 0.0.16 and 0.0.18 that brings up this error.

[LuckyMD]

Thanks! So some package is now using approximate nearest neighbour calculation that wasn't before I guess.