diff --git a/.github/workflows/deploy.yml b/.github/workflows/deploy.yml index bde84f5..340e9d3 100644 --- a/.github/workflows/deploy.yml +++ b/.github/workflows/deploy.yml @@ -16,7 +16,7 @@ jobs: steps: - name: Checkout code - uses: actions/checkout@v4 + uses: actions/checkout@v6 with: submodules: 'true' diff --git a/.gitignore b/.gitignore index 7246ae9..cbaadd6 100644 --- a/.gitignore +++ b/.gitignore @@ -1,2 +1,3 @@ src/slides_files *.html +.jupyter_cache/ diff --git a/README.md b/README.md index f466b86..319bdf0 100644 --- a/README.md +++ b/README.md @@ -37,18 +37,12 @@ The exercises for this course are listed in the quarto markdown files which can divided into those that can be answered/solved during the session, and also "Extension" tasks which can be worked on outside of the session. There is a small group of python scripts in the [problem](exercises/problem/) sub-directory. These act as an example of a non-reproducible workflow. The tasks involve understanding and -improving the reproducibility of the scripts using what is covered in the session. It is recommended to download these scripts to a seperate location on your local machine for +improving the reproducibility of the scripts using what is covered in the session. It is recommended to download these scripts to a separate location on your local machine for this purpose. -### Worked Solution - -Worked solutions for all of the exercises can be found in the [worked solutions](worked-solutions/) directory. -These cover the tasks that require any modifications/improvements to the example workflow provided. They are not exhaustive, but use common technologies and approaches -to solving the tasks. - ### Mini ReproHack -The [Mini ReproHack repository](https://github.com/Cambridge-ICCS/MiniReproHack) can be used before, alongside or after this course, to learn hands-on what makes software reproducible. It contains instructions on how do a [reproducibility hackathon](www.reprohack.org) by trying to reproduce the results of real climate science papers. +The [Mini ReproHack repository](https://github.com/Cambridge-ICCS/MiniReproHack) can be used before, alongside or after this course, to learn hands-on what makes software reproducible. It contains instructions on how to do a [reproducibility hackathon](https://www.reprohack.org) by trying to reproduce the results of real climate science papers. ## Preparation and prerequisites @@ -70,6 +64,7 @@ If you want to follow along with the exercises, basic Python coding skills and a ## Installation and setup To complete the exercises, you will need to download the dataset used by the python scripts. This can be found [here](https://www.metoffice.gov.uk/hadobs/hadcrut5/data/HadCRUT.5.0.0.0/download.html) +See [exercises/README.md](exercises/README.md) for which files to download and where to put them. As part of the exercises will be assessing the data dependencies of the scripts, it will be left as a task to find out which of the datasets you need! It is recommended to download the python scripts from [exercises/problem](exercises/problem/) to your local machine, in a new directory. @@ -96,7 +91,7 @@ To contribute back to the repository please first [fork it](https://github.com/Cambridge-ICCS/ReproducibilityInComputingCourse/fork), make the necessary changes to fix the problem, and then open a pull request back to this repository clearly describing the changes you have made. -We will then preform a review and merge once ready. +We will then perform a review and merge once ready. If you would like support using these materials, adapting them to your needs, or delivering them please get in touch either via GitHub or via diff --git a/exercises/version_control_exercises.qmd b/exercises/01_version_control_exercises.qmd similarity index 100% rename from exercises/version_control_exercises.qmd rename to exercises/01_version_control_exercises.qmd diff --git a/exercises/dependencies_exercises.qmd b/exercises/02_dependencies_exercises.qmd similarity index 89% rename from exercises/dependencies_exercises.qmd rename to exercises/02_dependencies_exercises.qmd index 12b5bde..f976510 100644 --- a/exercises/dependencies_exercises.qmd +++ b/exercises/02_dependencies_exercises.qmd @@ -11,14 +11,15 @@ Returning to the Python scripts, try to answer the following questions: ::::: {.incremental .column width=40%} - `xarray` - `matplotlib` - - `numpy` - `netcdf4` / `h5netcdf`! ::::: ::::: {.fragment .column width=60%} The last dependency is tricky, because it is not required by `xarray`, unless you want to read/write `netcdf` files! -If we include the dependecies of the dependencies, the list is even longer! +If we include the dependencies of the dependencies (e.g. `numpy`, which `xarray` +and `matplotlib` both rely on without importing it directly in our scripts), the +list is even longer! ``` ValueError: found the following matches with the input file in xarray's IO backends: ['netcdf4', 'h5netcdf']. But their dependencies may not be installed, see: https://docs.xarray.dev/en/stable/user-guide/io.html diff --git a/exercises/automation_exercises.qmd b/exercises/03_automation_exercises.qmd similarity index 100% rename from exercises/automation_exercises.qmd rename to exercises/03_automation_exercises.qmd diff --git a/exercises/testing_exercises.qmd b/exercises/04_testing_exercises.qmd similarity index 100% rename from exercises/testing_exercises.qmd rename to exercises/04_testing_exercises.qmd diff --git a/exercises/documentation_exercises.qmd b/exercises/05_documentation_exercises.qmd similarity index 100% rename from exercises/documentation_exercises.qmd rename to exercises/05_documentation_exercises.qmd diff --git a/exercises/final_exercises.qmd b/exercises/06_final_exercises.qmd similarity index 93% rename from exercises/final_exercises.qmd rename to exercises/06_final_exercises.qmd index 437862f..e816d79 100644 --- a/exercises/final_exercises.qmd +++ b/exercises/06_final_exercises.qmd @@ -7,9 +7,9 @@ Tasks: ### Extensions -1. Share your repository with someone, see if they can reproduce the plotd +1. Share your repository with someone, see if they can reproduce the plots 2. Try to reproduce the results using the repo that you have been shared -3. Try to use a different dataset (e.g. a different version of the HadCrut data) +3. Try to use a different dataset (e.g. a different version of the HadCRUT data) - Does it work? - What changes did you need to make? diff --git a/exercises/README.md b/exercises/README.md index 8b77cdc..446444c 100644 --- a/exercises/README.md +++ b/exercises/README.md @@ -8,13 +8,35 @@ reproducibility into account. ## Data -If you are trying to work through these exercises, you will need the data that -the scripts rely on, which can be found here: https://www.metoffice.gov.uk/hadobs/hadcrut5/data/HadCRUT.5.0.0.0/download.html. +The scripts rely on the HadCRUT.5.0.0.0 dataset, available from +https://www.metoffice.gov.uk/hadobs/hadcrut5/data/HadCRUT.5.0.0.0/download.html + +Only the "HadCRUT5 analysis time series: ensemble means and uncertainties" +NetCDF files are needed: + +- `HadCRUT.5.0.0.0_analysis_summary-series_global_annual.nc` +- `HadCRUT.5.0.0.0_analysis_summary-series_northern-hemisphere_annual.nc` +- `HadCRUT.5.0.0.0_analysis_summary-series_southern-hemisphere_annual.nc` +- `HadCRUT.5.0.0.0_analysis_summary-series_global_monthly.nc` + +Download these four files from the "Summary series" table (NetCDF → Annual/Monthly per region), +and place them in a `data/` directory next to the `problem/` scripts, so the +scripts can find them automatically. +This is better that downloading the whole dataset which is rather large! ## Instructions: 1. Create a new repository on your local machine with `git init repro-examples` 2. Copy the files in the `problem` directory into the new repository -3. Download the data needed for the analysis from https://doi.org/10.5281/zenodo.7014332 -4. Try to complete the tasks in the files named `*_exercises.qmd`. Some of these are -achievable during the session, others are tasks that could be worked on in your own time. +3. Download the four NetCDF files listed above into a `data/` directory next to the scripts +4. Work through the exercise files in order. + During the workshop the slides will tell you when to tackle each one. + - `01_version_control_exercises.qmd` - version control + - `02_dependencies_exercises.qmd` - dependencies + - `03_automation_exercises.qmd` - automation + - `04_testing_exercises.qmd` - testing + - `05_documentation_exercises.qmd` - documentation + - `06_final_exercises.qmd` - try it all out + + Some tasks are achievable during the session, others are "Extension" tasks that + could be worked on in your own time. diff --git a/exercises/problem/plot2.py b/exercises/problem/plot2.py index 0537204..23243e9 100644 --- a/exercises/problem/plot2.py +++ b/exercises/problem/plot2.py @@ -6,9 +6,9 @@ southern_ds = xarray.open_dataset("../data/HadCRUT.5.0.0.0_analysis_summary-series_southern-hemisphere_annual.nc") def plot_temp_with_error(ax, ds, label): - axs.fill_between(ds['time'].data, ds['tas_lower'].data, ds['tas_upper'].data, - color="grey", alpha=0.3, edgecolor=None) - axs.plot(ds['time'].data, ds['tas_mean'].data) + ax.fill_between(ds['time'].data, ds['tas_lower'].data, ds['tas_upper'].data, + color="grey", alpha=0.3, edgecolor=None) + ax.plot(ds['time'].data, ds['tas_mean'].data, label=label) fig, axs = plt.subplots(1, 1, figsize=(10,6), layout='constrained') diff --git a/exercises/problem/plot4.py b/exercises/problem/plot4.py index afa599a..2ef5595 100644 --- a/exercises/problem/plot4.py +++ b/exercises/problem/plot4.py @@ -9,7 +9,7 @@ # Absolute mean temperature between 1961 and 1990 # https://crudata.uea.ac.uk/cru/data/temperature/abs_glnhsh.txt -absolute_value = 13.974 + 273.5 +absolute_value = 13.974 + 273.15 time = global_ds['time'].data temperature = global_ds['tas_mean'].data + absolute_value albedo = 0.3 diff --git a/src/README.md b/src/README.md new file mode 100644 index 0000000..20a6878 --- /dev/null +++ b/src/README.md @@ -0,0 +1,48 @@ +# Building the slides + +The slide deck (`slides.qmd`) is generated with [quarto](https://quarto.org) and outputs `index.html` in this directory. + +## Requirements + +- The [quarto CLI](https://quarto.org/docs/get-started/) +- A Python installation with the packages from [requirements.txt](../requirements.txt) (Quarto executes the Python code cells with `python3`). +- The `Summer-school-Intro-Git` submodule checked out (some slides are included from it). + +## Setup + +Fetch the summer school submodule: +```bash +git submodule update --init --recursive +``` +(or clone with `git clone --recurse-submodules `). + +Set up the a Python environment from the repository root. +With a Python venv this would be: + +```bash +python3 -m venv .venv +source .venv/bin/activate +pip install -r requirements.txt +``` + +If you use [nix](https://nixos.org/), the root [flake.nix](../flake.nix) provides a shell with quarto and all Python dependencies: + +```bash +nix develop +``` + +## Render + +From this directory (`src/`): + +```bash +quarto render slides.qmd +``` + +If you used a virtual environment, make sure its `bin` directory is on your `PATH` so that `python3` resolves to it: + +```bash +PATH="$PWD/../.venv/bin:$PATH" quarto render slides.qmd +``` + +The rendered deck is written to `index.html` in this directory. diff --git a/src/automation.qmd b/src/_automation.qmd similarity index 100% rename from src/automation.qmd rename to src/_automation.qmd diff --git a/src/dependencies.qmd b/src/_dependencies.qmd similarity index 95% rename from src/dependencies.qmd rename to src/_dependencies.qmd index b34a0f0..7b0222f 100644 --- a/src/dependencies.qmd +++ b/src/_dependencies.qmd @@ -29,7 +29,7 @@ - e.g. add a "Dependencies" section to your README.md - Specify: - Versions of each dependency e.g. `numpy` 2.3.9 - - Where/how to aquire the dependency + - Where/how to acquire the dependency ## How to declare data dependencies @@ -44,7 +44,7 @@ - Ideally automate the fetching of the data ::: :::{.column width="40%"} -![Example of a zenodo listing. Note how each version of the dataset has a different reference](images/zenodo.png) +![Example of a zenodo listing. Note how each version of the dataset has a different reference](images/zenodo.png){.light-bg} ::: ::: @@ -92,7 +92,7 @@ ::: :::{.column width="40%"} ![](images/docker-logo-ocean-blue.svg) -![](images/apptainer.svg) +![](images/apptainer.svg){.light-bg} ![](images/podman-logo-source.svg) ::: ::: diff --git a/src/documentation.qmd b/src/_documentation.qmd similarity index 96% rename from src/documentation.qmd rename to src/_documentation.qmd index 04565d2..4d64e0d 100644 --- a/src/documentation.qmd +++ b/src/_documentation.qmd @@ -23,7 +23,7 @@ :::: ::::{.column width=50%} -![](images/ftorch-readme.png) +![](images/ftorch-readme.png){.light-bg} :::: ::: @@ -64,7 +64,7 @@ def energy_balanced_temperature(emissivity, albedo): Parameters ---------- emissivity: float - Effective emmissivity of the Earth + Effective emissivity of the Earth albedo: float Effective albedo of the Earth diff --git a/src/fair_principles.qmd b/src/_fair_principles.qmd similarity index 100% rename from src/fair_principles.qmd rename to src/_fair_principles.qmd diff --git a/src/introduction.qmd b/src/_introduction.qmd similarity index 98% rename from src/introduction.qmd rename to src/_introduction.qmd index 883831b..3f4631c 100644 --- a/src/introduction.qmd +++ b/src/_introduction.qmd @@ -4,7 +4,7 @@ In scientific research, a result is: -![](/images/reproducibility_matrix.jpg){width=60%, fig-align="center"} +![](images/reproducibility_matrix.jpg){width=60%, fig-align="center"} :::{.footer} We will use the definitions used in [The Turing Way](https://book.the-turing-way.org/reproducible-research/overview/overview-definitions/) diff --git a/src/introduction_walkthrough.qmd b/src/_introduction_walkthrough.qmd similarity index 100% rename from src/introduction_walkthrough.qmd rename to src/_introduction_walkthrough.qmd diff --git a/src/_pytest.qmd b/src/_pytest.qmd index 71888cb..c02a463 100644 --- a/src/_pytest.qmd +++ b/src/_pytest.qmd @@ -61,7 +61,7 @@ tests/test.py . What if we implemented `add` wrong? -```{.python filename="src/add,py"} +```{.python filename="src/add.py"} def add(a, b): return a-b # Incorrect ``` diff --git a/src/reproducibility_initiatives.qmd b/src/_reproducibility_initiatives.qmd similarity index 100% rename from src/reproducibility_initiatives.qmd rename to src/_reproducibility_initiatives.qmd diff --git a/src/testing.qmd b/src/_testing.qmd similarity index 100% rename from src/testing.qmd rename to src/_testing.qmd diff --git a/src/version_control.qmd b/src/_version_control.qmd similarity index 100% rename from src/version_control.qmd rename to src/_version_control.qmd diff --git a/src/custom.scss b/src/custom.scss index fd3aa3a..f63c232 100644 --- a/src/custom.scss +++ b/src/custom.scss @@ -93,6 +93,18 @@ $presentation-line-height: 1.5em; margin-top: 1em; } +/* light backdrop for images with dark content, so they stay visible on the + dark theme - soft radial gradient ~ gaussian glow behind the image */ +.reveal img.light-bg, +/* Git logo lives in the Summer-school-Intro-Git submodule as a remote URL, + so style it by src instead of editing the submodule */ +.reveal img[src*="Git-Logo"], +.reveal img[data-src*="Git-Logo"] { + background: radial-gradient(ellipse at center, #f2f2f2 0%, #d9d9d9 55%, #b3b3b3 100%); + padding: 0.5em; + border-radius: 12px; +} + /*-- vertically center columns (or any container) --*/ .v-center-container { display: flex; diff --git a/src/slides.qmd b/src/slides.qmd index 0e90e49..a4a8326 100644 --- a/src/slides.qmd +++ b/src/slides.qmd @@ -34,27 +34,27 @@ execute: cache: true --- -{{< include introduction.qmd >}} +{{< include _introduction.qmd >}} -{{< include version_control.qmd >}} -{{< include ../exercises/version_control_exercises.qmd >}} +{{< include _version_control.qmd >}} +{{< include ../exercises/01_version_control_exercises.qmd >}} -{{< include dependencies.qmd >}} -{{< include ../exercises/dependencies_exercises.qmd >}} +{{< include _dependencies.qmd >}} +{{< include ../exercises/02_dependencies_exercises.qmd >}} -{{< include automation.qmd >}} -{{< include ../exercises/automation_exercises.qmd >}} +{{< include _automation.qmd >}} +{{< include ../exercises/03_automation_exercises.qmd >}} -{{< include testing.qmd >}} +{{< include _testing.qmd >}} {{< include _pytest.qmd >}} -{{< include ../exercises/testing_exercises.qmd >}} +{{< include ../exercises/04_testing_exercises.qmd >}} -{{< include documentation.qmd >}} -{{< include ../exercises/documentation_exercises.qmd >}} +{{< include _documentation.qmd >}} +{{< include ../exercises/05_documentation_exercises.qmd >}} -{{< include fair_principles.qmd >}} +{{< include _fair_principles.qmd >}} -{{< include reproducibility_initiatives.qmd >}} +{{< include _reproducibility_initiatives.qmd >}} # Conclusion/Outlook @@ -107,6 +107,6 @@ Register your interest here: ![](images/iccs-ss-qr.svg) :::: ::::{.column width=70%} -![](images/iccs-summer-school-2026.jpg) +![](images/iccs-summer-school-2026.jpg){.light-bg} :::: :::