Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
3593fe9
Update README.md to warn of transition state
jlchang Oct 2, 2024
7e1aea2
test simpleminded xfer of epcc file
jlchang Oct 3, 2024
b2a3ad8
get file in right place
jlchang Oct 3, 2024
7f6bf13
trivial change to try out repo's PR process
jlchang Oct 3, 2024
e177305
Merge pull request #1 from jlchang/jlc_test
jlchang Oct 3, 2024
8a88334
pick a different epcc episode
jlchang Oct 3, 2024
55773d9
indicate EPCC episode
jlchang Oct 3, 2024
27a1bf1
add the EPCC snippets
jlchang Oct 3, 2024
a4f2cca
first step converting 13-scheduler.md to Rmd for HPCC
jlchang Oct 17, 2024
52967a9
update workflows
jlchang Jan 15, 2025
0b1a78e
Merge pull request #4 from jlchang/fix_workflows
jlchang Jan 15, 2025
700d7ec
[actions] update sandpaper workflow to version 0.16.11
jlchang Jan 23, 2025
0da50ed
Merge pull request #6 from jlchang/update/workflows
jlchang Jan 23, 2025
2a3a4b2
Use a configuration script to control things
ocaisa Mar 5, 2025
025c734
Update sandpaper-main.yaml
ocaisa Mar 5, 2025
4bd329a
Comments! ... and POSIX compliance
Mar 5, 2025
8c47205
Merge pull request #1 from tkphd/snippetification
ocaisa Mar 5, 2025
a704f15
Try to fix lesson building
ocaisa Mar 5, 2025
3c6959c
Update lesson config
ocaisa Mar 5, 2025
9cd9187
Tweak a few things
ocaisa Mar 5, 2025
1940ebb
Explore the configuration a bit
ocaisa Mar 5, 2025
313a9c9
Workbenchify 10 & 11, including Portal conditional
Mar 20, 2025
93b261d
Workbenched Ep. 12
Mar 20, 2025
b077bd7
Workbenched Ep. 13
Mar 20, 2025
4f4fb85
Workbenched Ep. 14
Mar 20, 2025
5098d86
Workbenched Ep. 15, without snippets
Mar 20, 2025
2140d2f
Workbenched Ep. 16, without snippets
Mar 20, 2025
f29cda2
Workbenched Ep. 17, without snippets
Mar 20, 2025
94f77cb
Workbenched Ep. 18, without snippets
Mar 20, 2025
b91909d
Workbenched Ep. 19, without snippets
Mar 20, 2025
63d31f1
Merge remote-tracking branch 'origin/main' into default-snippets
ocaisa Mar 20, 2025
ac9b0bc
Merge pull request #2 from tkphd/default-snippets
ocaisa Mar 20, 2025
34b46ad
Rmd conversion
May 15, 2025
c287cd7
Initialize a testing snippet library with clearly-wrong data
Aug 21, 2025
5067601
Snippetized Ep. 15
tkphd Nov 5, 2025
8b9e0e0
Snippetized Ep. 17
tkphd Nov 5, 2025
2c924f8
Snippetized Ep. 18
tkphd Nov 5, 2025
f91826f
Ghastly Mistakes appears to work
tkphd Nov 5, 2025
2bfd247
Working snippetized version.
tkphd Nov 5, 2025
264df4d
Merge pull request #4 from tkphd/snippet-library
ocaisa Nov 7, 2025
c0a0d98
Merge branch 'main' into main
ocaisa Nov 20, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions .github/workflows/sandpaper-main.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,6 @@ on:
branches:
- main
- master
schedule:
- cron: '0 0 * * 2'
workflow_dispatch:
inputs:
name:
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/sandpaper-version.txt
Original file line number Diff line number Diff line change
@@ -1 +1 @@
0.16.10
0.16.11
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ site/*
/*.Rcheck/
# RStudio files
.Rproj.user/
*.Rproj
# produced vignettes
vignettes/*.html
vignettes/*.pdf
Expand Down
4 changes: 3 additions & 1 deletion MAINTENANCE.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
# Site Structure and Maintenance
---
title: "Site Structure and Maintenance"
---

This site depends strongly on GitHub infrastructure, and a number of
operational decisions have been made during the development process
Expand Down
16 changes: 9 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,12 @@ This lesson teaches the basics of interacting with high-performance computing

## Using this material

NOTE: This is *not* Carpentries boilerplate! Please read carefully.
NOTE: This lesson is being transitioned to Carpentries Workbench but is still *not* Carpentries boilerplate!
Instructions below still need to be edited to reflect the transition. Proceed with caution.

1. Follow the instructions found in The Carpentries' [example lesson][ex-lesson]
1. ~~Follow the instructions found in The Carpentries' [example lesson][ex-lesson]
to create a repository for your lesson. Install Ruby, Make, and Jekyll
following the instructions [here][setup].
following the instructions [here][setup].~~

2. For easier portability, we use snippets of text and code to capture inputs
and outputs that are host- or site-specific and cannot be scripted. These
Expand All @@ -37,17 +38,17 @@ NOTE: This is *not* Carpentries boilerplate! Please read carefully.
or omit episodes, by editing the configuration block under `episode_names`
in this file.

4. Set the environment variable `HPC_JEKYLL_CONFIG` to the relative path of the
configuration file in your snippets folder:
4. ~~Set the environment variable `HPC_JEKYLL_CONFIG` to the relative path of the
configuration file in your snippets folder:~~

```bash
export HPC_JEKYLL_CONFIG=_includes/snippets_library/.../_config_options.yml
```

5. Preview the lesson locally, by running `make serve`. You can then view the
5. ~~Preview the lesson locally, by running `make serve`. You can then view the
website in your browser, following the links in the output (usually,
<https://localhost:4000>). Pages will be automatically regenerated every
time you write to them.
time you write to them.~~

6. If there are discrepancies in the output, edit the snippet file containing
it, or create a new one and customize.
Expand All @@ -67,6 +68,7 @@ NOTE: This is *not* Carpentries boilerplate! Please read carefully.
future will probably not be accepted.

## Deploying a Customized Lesson
**While this lesson is in transition, we do not recommend customizing**

The steps above will help you port the default HPC Intro lesson to your
specific cluster, but the changes will only be visible on your local machine.
Expand Down
37 changes: 20 additions & 17 deletions config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ carpentry: 'cp'
title: 'Introduction to High-Performance Computing'

# Date the lesson was created (YYYY-MM-DD, this is empty by default)
created: 2017-03-24
created: 2017-03-24

# Comma-separated list of keywords for the lesson
keywords: 'software, data, lesson, The Carpentries'
Expand All @@ -33,7 +33,7 @@ life_cycle: 'alpha'
license: 'CC-BY 4.0'

# Link to the source repository for this lesson
source: 'https://github.com/tobyhodges/probable-pancake'
source: 'https://github.com/ocaisa/probable-pancake'

# Default branch of your lesson
branch: 'main'
Expand Down Expand Up @@ -64,26 +64,29 @@ contact: 'team@carpentries.org'
# - another-learner.md

# Order of episodes in your lesson
episodes:
- 10-hpc-intro.md
- 11-connecting.md
- 12-cluster.md
- 13-scheduler.md
- 14-environment-variables.md
- 15-modules.md
- 16-transferring-files.md
- 17-parallel.md
- 18-resources.md
- 19-responsibility.md
episodes:
- 10-hpc-intro.Rmd
- 11-connecting.Rmd
- 12-cluster.Rmd
- 13-scheduler.Rmd
- 14-environment-variables.Rmd
- 15-modules.Rmd
- 16-transferring-files.Rmd
- 17-parallel.Rmd
- 18-resources.Rmd
- 19-responsibility.Rmd

# Information for Learners
learners:
learners:
- setup.md

# Information for Instructors
instructors:
instructors:
- instructor-notes.Rmd

# Learner Profiles
profiles:
profiles:
- learner-profiles.md

# Customisation ---------------------------------------------
#
Expand All @@ -92,6 +95,6 @@ profiles:


carpentry_description: Lesson Description
url: 'https://tkphd.github.io/hpc-intro-online'
url: 'https://ocaisa.github.io/probable_pancake'
analytics: carpentries
lang: en
59 changes: 30 additions & 29 deletions episodes/10-hpc-intro.md → episodes/10-hpc-intro.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,11 @@ teaching: 15
exercises: 5
---

```{r, echo=FALSE}
# Source the external configuration script
source("load_config.R")
```

::::::::::::::::::::::::::::::::::::::: objectives

- Describe what an HPC system is
Expand All @@ -22,15 +27,15 @@ Frequently, research problems that use computing can outgrow the capabilities
of the desktop or laptop computer where they started:

- A statistics student wants to cross-validate a model. This involves running
the model 1000 times -- but each run takes an hour. Running the model on
the model 1000 times but each run takes an hour. Running the model on
a laptop will take over a month! In this research problem, final results are
calculated after all 1000 models have run, but typically only one model is
run at a time (in **serial**) on the laptop. Since each of the 1000 runs is
independent of all others, and given enough computers, it's theoretically
possible to run them all at once (in **parallel**).
- A genomics researcher has been using small datasets of sequence data, but
soon will be receiving a new type of sequencing data that is 10 times as
large. It's already challenging to open the datasets on a computer --
large. It's already challenging to open the datasets on a computer
analyzing these larger datasets will probably crash it. In this research
problem, the calculations required might be impossible to parallelize, but a
computer with **more memory** would be required to analyze the much larger
Expand All @@ -54,7 +59,7 @@ problems in parallel**.

## Jargon Busting Presentation

Open the [HPC Jargon Buster]({{ site.url }}files/jargon#p1)
Open the [HPC Jargon Buster](files/jargon.html#p1)
in a new tab. To present the content, press `C` to open a **c**lone in a
separate window, then press `P` to toggle **p**resentation mode.

Expand All @@ -71,48 +76,44 @@ results.
## Some Ideas

- Checking email: your computer (possibly in your pocket) contacts a remote
machine, authenticates, and downloads a list of new messages; it also
uploads changes to message status, such as whether you read, marked as
junk, or deleted the message. Since yours is not the only account, the
mail server is probably one of many in a data center.
- Searching for a phrase online involves comparing your search term against
a massive database of all known sites, looking for matches. This "query"
machine, authenticates, and downloads a list of new messages; it also uploads
changes to message status, such as whether you read, marked as junk, or
deleted the message. Since yours is not the only account, the mail server is
probably one of many in a data center.
- Searching for a phrase online involves comparing your search term against a
massive database of all known sites, looking for matches. This "query"
operation can be straightforward, but building that database is a
[monumental task][mapreduce]! Servers are involved at every step.
- Searching for directions on a mapping website involves connecting your
(A) starting and (B) end points by [traversing a graph][dijkstra] in
search of the "shortest" path by distance, time, expense, or another
metric. Converting a map into the right form is relatively simple, but
calculating all the possible routes between A and B is expensive.
- Searching for directions on a mapping website involves connecting your (A)
starting and (B) end points by [traversing a graph][dijkstra] in search of
the "shortest" path by distance, time, expense, or another metric. Converting
a map into the right form is relatively simple, but calculating all the
possible routes between A and B is expensive.

Checking email could be serial: your machine connects to one server and
exchanges data. Searching by querying the database for your search term (or
endpoints) could also be serial, in that one machine receives your query
and returns the result. However, assembling and storing the full database
is far beyond the capability of any one machine. Therefore, these functions
are served in parallel by a large, ["hyperscale"][hyperscale] collection of
servers working together.


endpoints) could also be serial, in that one machine receives your query and
returns the result. However, assembling and storing the full database is far
beyond the capability of any one machine. Therefore, these functions are served
in parallel by a large, ["hyperscale"][hyperscale] collection of servers
working together.

:::::::::::::::::::::::::

::::::::::::::::::::::::::::::::::::::::::::::::::



[mapreduce]: https://en.wikipedia.org/wiki/MapReduce
[dijkstra]: https://en.wikipedia.org/wiki/Dijkstra%27s_algorithm
[hyperscale]: https://en.wikipedia.org/wiki/Hyperscale_computing


:::::::::::::::::::::::::::::::::::::::: keypoints

- High Performance Computing (HPC) typically involves connecting to very large computing systems elsewhere in the world.
- These other systems can be used to do work that would either be impossible or much slower on smaller systems.
- High Performance Computing (HPC) typically involves connecting to very large
computing systems elsewhere in the world.
- These other systems can be used to do work that would either be impossible or
much slower on smaller systems.
- HPC resources are shared by multiple users.
- The standard method of interacting with such systems is via a command line interface.
- The standard method of interacting with such systems is via a command line
interface.

::::::::::::::::::::::::::::::::::::::::::::::::::


Loading
Loading