Repository navigation
Add CDEPS Dev Container to repo #447
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
danrosen25
wants to merge
1
commit into
ESCOMP:main
Choose a base branch
from
danrosen25:feature/devcontainer
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,161 @@ | ||
| FROM ubuntu:24.04 | ||
|
|
||
| # Define arguments for image | ||
| ARG CTNR_NAME="cdepsdev_ubuntu-24.04_gcc-13" \ | ||
| CTNR_DESCRIPTION="CDEPS development container" \ | ||
| CTNR_VERSION="1.0.0" \ | ||
| CTNR_USER="cdepsdev" \ | ||
| SPACK_VERSION="v1.2.1" \ | ||
| SPACK_REPO_VERSION="v2026.06.0" | ||
|
|
||
| # Set system environment variables | ||
| ENV DEBIAN_FRONTEND=noninteractive \ | ||
| LANG=en_US.UTF-8 \ | ||
| LANGUAGE=en_US:en \ | ||
| LC_ALL=en_US.UTF-8 | ||
|
|
||
| # Install system packages via apt | ||
| RUN <<INSTALL_PACKAGES | ||
| apt-get update | ||
| apt-get install -yqq \ | ||
| automake \ | ||
| autotools-dev \ | ||
| build-essential \ | ||
| bzip2 \ | ||
| ca-certificates \ | ||
| cmake \ | ||
| curl \ | ||
| file \ | ||
| g++-13 \ | ||
| gcc-13 \ | ||
| gfortran-13 \ | ||
| git \ | ||
| gpg \ | ||
| gzip \ | ||
| libcurl4-gnutls-dev \ | ||
| libjpeg-dev \ | ||
| libssl-dev \ | ||
| locales \ | ||
| lsb-release \ | ||
| m4 \ | ||
| make \ | ||
| openssh-client \ | ||
| pkg-config \ | ||
| python3 \ | ||
| python3-dev \ | ||
| python3-venv \ | ||
| python3-pip \ | ||
| sudo \ | ||
| tar \ | ||
| tcl-dev \ | ||
| unzip \ | ||
| vim \ | ||
| wget | ||
| locale-gen en_US.UTF-8 | ||
| update-locale LANG=en_US.UTF-8 LC_ALL=en_US.UTF-8 | ||
| ln -sf gcc-13 /usr/bin/gcc | ||
| ln -sf g++-13 /usr/bin/g++ | ||
| ln -sf gfortran-13 /usr/bin/gfortran | ||
| ln -sf python3 /usr/bin/python | ||
| arch=$(dpkg-architecture -qDEB_HOST_MULTIARCH) | ||
| find "/usr/lib/${arch}" -name "libjpeg*" \ | ||
| -exec ln -sf {} /usr/lib/. \; | ||
| apt-get clean | ||
| INSTALL_PACKAGES | ||
|
|
||
| # Create non-root user with sudo privileges | ||
| ENV HOME=/home/${CTNR_USER} | ||
| RUN <<SETUP_USER | ||
| groupadd ${CTNR_USER} | ||
| useradd -m -d ${HOME} -s /bin/bash -g ${CTNR_USER} ${CTNR_USER} | ||
| adduser ${CTNR_USER} sudo | ||
| echo '%sudo ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers | ||
| SETUP_USER | ||
| USER ${CTNR_USER} | ||
| WORKDIR ${HOME} | ||
|
|
||
| # Install spack via git clone | ||
| ENV SPACK_ROOT=${HOME}/spack | ||
| RUN <<INSTALL_SPACK | ||
| mkdir -p ${SPACK_ROOT} | ||
| git clone --depth=2 --branch=${SPACK_VERSION} --no-single-branch \ | ||
| https://github.com/spack/spack.git ${SPACK_ROOT} | ||
| . ${SPACK_ROOT}/share/spack/setup-env.sh | ||
| spack repo update --tag ${SPACK_REPO_VERSION} builtin | ||
| INSTALL_SPACK | ||
| ENV PATH="${SPACK_ROOT}/bin:${PATH}" | ||
|
|
||
| # Configure external spack packages | ||
| RUN <<SPACK_PKG_EXTERNAL | ||
| . ${SPACK_ROOT}/share/spack/setup-env.sh | ||
| target=$(dpkg-architecture -qDEB_HOST_MULTIARCH | cut -d- -f1) | ||
| spack compiler find | ||
| echo "Discovering external packages, this may take a few minutes..." | ||
| spack external find --all | ||
| # Install essential packages | ||
| spack install gcc || exit 1 | ||
| spack install python || exit 1 | ||
| spack install cmake || exit 1 | ||
| SPACK_PKG_EXTERNAL | ||
|
|
||
| # Install spack packages | ||
| RUN <<SPACK_PKG_INSTALL | ||
| . ${SPACK_ROOT}/share/spack/setup-env.sh | ||
| spack install --fail-fast \ | ||
| openmpi@4.1.6 \ | ||
| esmf@8.9.0 +debug \ | ||
| hdf5@1.14.6 +fortran \ | ||
| netcdf-c@4.10.0 \ | ||
| netcdf-fortran@4.6.2 \ | ||
| parallelio@2.6.6 \ | ||
| parallel-netcdf@1.14.1 +fortran || exit 1 | ||
| spack clean --all | ||
| SPACK_PKG_INSTALL | ||
|
|
||
| # Environment setup | ||
| ENV OPENMPI_ROOT=${HOME}/stack/openmpi \ | ||
| ESMF_ROOT=${HOME}/stack/esmf \ | ||
| HDF5_ROOT=${HOME}/stack/hdf5 \ | ||
| NETCDF_ROOT=${HOME}/stack/netcdf \ | ||
| PNETCDF_ROOT=${HOME}/stack/pnetcdf \ | ||
| PIO_ROOT=${HOME}/stack/pio | ||
|
|
||
| RUN <<SPACK_CREATE_VIEWS | ||
| . ${SPACK_ROOT}/share/spack/setup-env.sh | ||
| spack view -d false symlink ${OPENMPI_ROOT} openmpi | ||
| spack view -d false symlink ${ESMF_ROOT} esmf | ||
| spack view -d false symlink ${HDF5_ROOT} hdf5 | ||
| spack view -d false symlink ${NETCDF_ROOT} netcdf-c netcdf-fortran | ||
| spack view -d false symlink ${PNETCDF_ROOT} parallel-netcdf | ||
| spack view -d false symlink ${PIO_ROOT} parallelio | ||
| SPACK_CREATE_VIEWS | ||
|
|
||
| # Copy user environment setup script | ||
| COPY --chown=$CTNR_USER:$CTNR_USER --chmod=0755 envsetup.sh ${HOME}/.envsetup.sh | ||
|
|
||
| # Source the user environment setup script | ||
| RUN <<SETUPENV | ||
| tmp=$(mktemp) | ||
| printf '%s\n' \ | ||
| '# Source the user environment setup script' \ | ||
| 'if [ -f "${HOME}/.envsetup.sh" ]; then' \ | ||
| ' . "${HOME}/.envsetup.sh"' \ | ||
| 'fi' '' > "$tmp" | ||
| [ -f ${HOME}/.bashrc ] && cat ${HOME}/.bashrc >> "$tmp" | ||
| mv "$tmp" ${HOME}/.bashrc | ||
| SETUPENV | ||
|
|
||
| # Copy README file | ||
| COPY --chown=$CTNR_USER:$CTNR_USER --chmod=0644 README.md ${HOME}/README.md | ||
|
|
||
| # Default command | ||
| CMD ["/bin/bash", "-l"] | ||
|
|
||
| # Set environment variable for devcontainer title | ||
| ENV DEVCONTAINER_NAME=${CTNR_NAME} | ||
|
|
||
| # Add OCI labels for image metadata | ||
| # https://specs.opencontainers.org/image-spec/annotations/ | ||
| LABEL org.opencontainers.image.version="${CTNR_VERSION}" \ | ||
| org.opencontainers.image.title="${CTNR_NAME}" \ | ||
| org.opencontainers.image.description="${CTNR_DESCRIPTION}" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,77 @@ | ||
| # CDEPS Ubuntu 24.04 Development Container | ||
|
|
||
| This Dev Container provides a reproducible CDEPS development environment based | ||
| on Ubuntu 24.04, GCC / GFortran 13, an MPI library, and other packages needed | ||
| to build CDEPS. | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| - Docker or another Docker-compatible container runtime | ||
| - Visual Studio Code | ||
| - The **Dev Containers** extension for Visual Studio Code | ||
|
|
||
| Open the CDEPS repository in Visual Studio Code and run **Dev Containers: | ||
| Reopen in Container**. The image is built from `Dockerfile` in this directory, | ||
| and the repository is mounted at `/home/cdepsdev/CDEPS`. | ||
|
|
||
| ## Installed Toolchain | ||
|
|
||
| The image includes the following build tools and packages: | ||
|
|
||
| | Software | Version | | ||
| | --- | --- | | ||
| | GCC / G++ / GNU Fortran | 13 | | ||
| | Spack | 1.2.1 | | ||
| | OpenMPI | 4.1.6 | | ||
| | ESMF | 8.9.0 | | ||
| | HDF5 | 1.14.6 | | ||
| | NetCDF-C | 4.10.0 | | ||
| | NetCDF-Fortran | 4.6.2 | | ||
| | ParallelIO | 2.6.6 | | ||
| | Parallel-netCDF | 1.14.1 | | ||
|
|
||
| Spack is installed at `/home/cdepsdev/spack`. | ||
|
|
||
| ## Environment | ||
|
|
||
| The bash login shell sources `.envsetup.sh` automatically. This initializes | ||
| Spack and loads the software stack, then sets environment variables needed | ||
| for the CDEPS build: | ||
|
|
||
| - CC | ||
| - FC | ||
| - CXX | ||
| - CPPFLAGS | ||
| - LDFLAGS | ||
| - FFLAGS | ||
| - ESMF_VERSION | ||
| - ParallelIO_VERSION | ||
| - PIO | ||
| - CDEPS_CMAKE_FLAGS | ||
|
|
||
| To reinitialize the environment in an existing shell, run: | ||
|
|
||
| ```bash | ||
| source ~/.envsetup.sh | ||
| ``` | ||
|
|
||
| ## Build CDEPS | ||
|
|
||
| Use an out-of-source build directory. From the repository root inside the | ||
| container: | ||
|
|
||
| Replace CDEPS_BUILD_DIRECTORY and CDEPS_SOURCE_DIRECTORY with their | ||
| respective locations. | ||
|
|
||
| ```bash | ||
| mkdir -p CDEPS_BUILD_DIRECTORY | ||
| cd CDEPS_BUILD_DIRECTORY | ||
| cmake CDEPS_SOURCE_DIRECTORY ${CDEPS_CMAKE_FLAGS} | ||
| make -j4 VERBOSE=1 | ||
| ``` | ||
|
|
||
| To disable the bundled FoX XML library, configure with: | ||
|
|
||
| ```bash | ||
| cmake /home/cdepsdev/CDEPS ${CDEPS_CMAKE_FLAGS} -DDISABLE_FoX=ON | ||
| ``` | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| { | ||
| "name": "cdepsdev_ubuntu-24.04_gcc-13", | ||
| "build": { "dockerfile": "Dockerfile" }, | ||
| "customizations": { | ||
| "vscode": { | ||
| "settings": { | ||
| "terminal.integrated.defaultProfile.linux": "bash", | ||
| "terminal.integrated.profiles.linux": { | ||
| "bash": { | ||
| "path": "bash", | ||
| "args": ["-l"] | ||
| } | ||
| } | ||
| }, | ||
| "extensions": [ | ||
| "ms-vscode.cmake-tools", | ||
| "ms-vscode.cpptools", | ||
| "fortran-lang.modern-fortran" | ||
| ] | ||
| } | ||
| }, | ||
| "workspaceMount": "source=${localWorkspaceFolder},target=/home/cdepsdev/CDEPS,type=bind,consistency=cached", | ||
| "workspaceFolder": "/home/cdepsdev/CDEPS" | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,36 @@ | ||
| #!/bin/bash | ||
|
|
||
| # Spack setup | ||
| . ${SPACK_ROOT}/share/spack/setup-env.sh | ||
| spack load openmpi | ||
| spack load esmf | ||
| spack load hdf5 | ||
| spack load netcdf-c | ||
| spack load netcdf-fortran | ||
| spack load parallelio | ||
| spack load parallel-netcdf | ||
|
|
||
| # CDEPS environment variables | ||
| export CC=mpicc | ||
| export FC=mpifort | ||
| export CXX=mpicxx | ||
| export CPPFLAGS="-I/usr/include -I/usr/local/include" | ||
| export LDFLAGS="-L/usr/lib/$(dpkg-architecture -qDEB_HOST_MULTIARCH)" | ||
| export FFLAGS="-DCPRGNU -g -Wall -ffree-form -ffree-line-length-none -fallow-argument-mismatch" | ||
| export ESMF_VERSION=v$(spack find --format "{version}" esmf) | ||
| export ParallelIO_VERSION=pio$(spack find --format "{version}" parallelio) | ||
| export PIO=$PIO_ROOT | ||
| export CDEPS_CMAKE_FLAGS="-Wno-dev -DCMAKE_BUILD_TYPE=DEBUG -DWERROR=ON" | ||
|
|
||
| # Print Welcome Message | ||
| echo "Welcome to the CDEPS Development Container!" | ||
| echo "*** ${DEVCONTAINER_NAME} ***" | ||
| echo "" | ||
| echo "The following packages have been pre-loaded:" | ||
| spack find --loaded --format "{name}@{version}" | ||
| echo "" | ||
| echo "Build CDEPS using the following steps:" | ||
| echo "1. Create directory: mkdir -p <build_directory> && cd <build_directory>" | ||
| echo "2. Generate build files: cmake <source_directory> \${CDEPS_CMAKE_FLAGS}" | ||
| echo "3. Build CDEPS: make -j 4 VERBOSE=1" | ||
| echo "" |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For licensing reasons we've been endorsing using podman. And podman is available on Derecho and other machines, while Docker is not.
So I think it should explicitly say podman here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The podman daemon also needs to be running, which should be mentioned somewhere.