Skip to content

Commit e70834d

Browse files
dougborgclaude
dougborg
and
claude
committed
Add Gatling load test suite for overpass.deflock.org
Four simulation scenarios to validate server performance (~512 Overpass compute slots) before switching app users to the self-hosted instance: - Baseline: deterministic single-user zoom progression (36 requests) - Concurrent: ramp 1→50 users to find degradation inflection point - Stress: spike to 500 users to exceed server capacity - Burst: realistic app sessions (10-20 requests each) in waves Queries use the exact same per-profile tag filters as the app's NodeProfile.getDefaults() (all 11 built-in profiles). Pure query logic lives in a shared source set with 22 ScalaTest unit tests. Includes GitHub Actions workflow with scenario picker dropdown, matrix strategy for parallel runs, and PR comment with report download links. Dev container with pinned Coursier (SHA256 verified) for IDE support. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent de65cec commit e70834d

24 files changed

Lines changed: 1632 additions & 1 deletion

‎.github/workflows/load-test.yml‎

Lines changed: 151 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,151 @@
1+
# Manual-trigger workflow to run Gatling load tests against overpass.deflock.org.
2+
#
3+
# Trigger from the Actions tab → "Load Test" → "Run workflow" → pick a scenario.
4+
# The HTML report is uploaded as a downloadable artifact (retained 30 days).
5+
#
6+
# When "all" is selected, all 4 simulations run in parallel on separate runners,
7+
# hitting the server simultaneously for distributed load. A summary job then
8+
# collects all reports and posts a PR comment with download links.
9+
10+
name: Load Test
11+
12+
on:
13+
workflow_dispatch:
14+
inputs:
15+
scenario:
16+
description: 'Test scenario to run'
17+
required: true
18+
default: 'baseline'
19+
type: choice
20+
options:
21+
- baseline
22+
- concurrent
23+
- stress
24+
- burst
25+
- all
26+
27+
concurrency:
28+
group: load-test
29+
cancel-in-progress: true
30+
31+
jobs:
32+
# Build the matrix dynamically based on the selected scenario.
33+
# This avoids the `matrix` context parsing issue in job-level `if`.
34+
resolve-matrix:
35+
runs-on: ubuntu-latest
36+
outputs:
37+
matrix: ${{ steps.set-matrix.outputs.matrix }}
38+
steps:
39+
- id: set-matrix
40+
run: |
41+
if [ "${{ inputs.scenario }}" = "all" ]; then
42+
echo 'matrix={"include":[{"name":"baseline","class":"deflock.OverpassSimulation"},{"name":"concurrent","class":"deflock.ConcurrentSimulation"},{"name":"stress","class":"deflock.StressSimulation"},{"name":"burst","class":"deflock.BurstSimulation"}]}' >> "$GITHUB_OUTPUT"
43+
elif [ "${{ inputs.scenario }}" = "baseline" ]; then
44+
echo 'matrix={"include":[{"name":"baseline","class":"deflock.OverpassSimulation"}]}' >> "$GITHUB_OUTPUT"
45+
elif [ "${{ inputs.scenario }}" = "concurrent" ]; then
46+
echo 'matrix={"include":[{"name":"concurrent","class":"deflock.ConcurrentSimulation"}]}' >> "$GITHUB_OUTPUT"
47+
elif [ "${{ inputs.scenario }}" = "stress" ]; then
48+
echo 'matrix={"include":[{"name":"stress","class":"deflock.StressSimulation"}]}' >> "$GITHUB_OUTPUT"
49+
elif [ "${{ inputs.scenario }}" = "burst" ]; then
50+
echo 'matrix={"include":[{"name":"burst","class":"deflock.BurstSimulation"}]}' >> "$GITHUB_OUTPUT"
51+
fi
52+
53+
load-test:
54+
needs: resolve-matrix
55+
runs-on: ubuntu-latest
56+
strategy:
57+
fail-fast: false
58+
matrix: ${{ fromJson(needs.resolve-matrix.outputs.matrix) }}
59+
steps:
60+
- uses: actions/checkout@v5
61+
62+
- name: Set up JDK 21
63+
uses: actions/setup-java@v4
64+
with:
65+
distribution: temurin
66+
java-version: 21
67+
68+
# Caches Gradle wrapper and dependencies between runs
69+
- name: Setup Gradle
70+
uses: gradle/actions/setup-gradle@v4
71+
72+
- name: Run Gatling simulation (${{ matrix.name }})
73+
working-directory: load-tests
74+
run: ./gradlew gatlingRun --simulation ${{ matrix.class }} --non-interactive
75+
76+
- name: Upload Gatling report
77+
if: always()
78+
uses: actions/upload-artifact@v4
79+
with:
80+
name: gatling-report-${{ matrix.name }}
81+
path: load-tests/build/reports/gatling/
82+
retention-days: 30
83+
84+
# Post a summary comment on the PR (if triggered from a PR branch) with
85+
# links to download each report artifact.
86+
summary:
87+
needs: load-test
88+
if: always()
89+
runs-on: ubuntu-latest
90+
steps:
91+
- uses: actions/checkout@v5
92+
93+
- name: Find associated PR
94+
id: find-pr
95+
run: |
96+
# Search for a PR with this branch as the head ref. Check the
97+
# current repo first, then the parent (upstream) repo if this is a fork.
98+
PR=$(gh pr list --repo "${{ github.repository }}" --head "${{ github.ref_name }}" --json number --jq '.[0].number' 2>/dev/null || echo "")
99+
if [ -z "$PR" ]; then
100+
PARENT=$(gh api "repos/${{ github.repository }}" --jq '.parent.full_name // empty' 2>/dev/null || echo "")
101+
if [ -n "$PARENT" ]; then
102+
PR=$(gh pr list --repo "$PARENT" --head "${{ github.ref_name }}" --json number --jq '.[0].number' 2>/dev/null || echo "")
103+
if [ -n "$PR" ]; then
104+
echo "pr_repo=$PARENT" >> "$GITHUB_OUTPUT"
105+
fi
106+
fi
107+
else
108+
echo "pr_repo=${{ github.repository }}" >> "$GITHUB_OUTPUT"
109+
fi
110+
echo "pr_number=$PR" >> "$GITHUB_OUTPUT"
111+
env:
112+
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
113+
114+
- name: Post PR comment with report links
115+
if: steps.find-pr.outputs.pr_number != ''
116+
env:
117+
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
118+
RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
119+
run: |
120+
SCENARIO="${{ inputs.scenario }}"
121+
RUN_ID="${{ github.run_id }}"
122+
123+
# Build artifact links from the matrix
124+
ARTIFACTS=""
125+
if [ "$SCENARIO" = "all" ] || [ "$SCENARIO" = "baseline" ]; then
126+
ARTIFACTS="$ARTIFACTS\n| Baseline | [Download]($RUN_URL/artifacts) |"
127+
fi
128+
if [ "$SCENARIO" = "all" ] || [ "$SCENARIO" = "concurrent" ]; then
129+
ARTIFACTS="$ARTIFACTS\n| Concurrent | [Download]($RUN_URL/artifacts) |"
130+
fi
131+
if [ "$SCENARIO" = "all" ] || [ "$SCENARIO" = "stress" ]; then
132+
ARTIFACTS="$ARTIFACTS\n| Stress | [Download]($RUN_URL/artifacts) |"
133+
fi
134+
if [ "$SCENARIO" = "all" ] || [ "$SCENARIO" = "burst" ]; then
135+
ARTIFACTS="$ARTIFACTS\n| Burst | [Download]($RUN_URL/artifacts) |"
136+
fi
137+
138+
BODY=$(cat <<EOF
139+
## Load Test Results — \`$SCENARIO\`
140+
141+
| Scenario | Report |
142+
|----------|--------|
143+
$(echo -e "$ARTIFACTS")
144+
145+
[View full run]($RUN_URL)
146+
147+
> Download the artifact ZIP → extract → open \`index.html\` for interactive charts.
148+
EOF
149+
)
150+
151+
gh pr comment "${{ steps.find-pr.outputs.pr_number }}" --repo "${{ steps.find-pr.outputs.pr_repo }}" --body "$BODY"

‎.gitignore‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -103,4 +103,4 @@ build_keys.conf
103103
linux/
104104
macos/
105105
web/
106-
windows/
106+
windows/

‎load-tests/.devcontainer/Dockerfile‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
# Dev container image for running Gatling load tests.
2+
#
3+
# Based on Microsoft's Java 21 dev container, which includes:
4+
# - JDK 21 (Eclipse Temurin)
5+
# - Gradle (via the wrapper in the project)
6+
# - Standard dev tools (git, curl, etc.)
7+
#
8+
# We add Coursier (the Scala package manager) for Scala tooling support
9+
# in VS Code via the Metals extension.
10+
11+
FROM mcr.microsoft.com/devcontainers/java:21
12+
13+
# Install Coursier for Scala tooling (used by the Metals VS Code extension).
14+
# Note: the Scala compiler itself is managed by Gradle via the Gatling plugin,
15+
# so we only need Coursier for IDE support.
16+
#
17+
# Pinned to a specific version with checksum verification to prevent
18+
# supply chain attacks via mutable "latest" release URLs.
19+
ARG COURSIER_VERSION=v2.1.24
20+
ARG COURSIER_SHA256=1517a0b6c4b9608dc45da8f34bc8290707ed50104ee92662f57808d2c012be54
21+
RUN curl -fL "https://github.com/coursier/coursier/releases/download/${COURSIER_VERSION}/cs-x86_64-pc-linux.gz" \
22+
| gzip -d > /usr/local/bin/cs \
23+
&& echo "${COURSIER_SHA256} /usr/local/bin/cs" | sha256sum -c - \
24+
&& chmod +x /usr/local/bin/cs

‎load-tests/.devcontainer/devcontainer.json‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
// Dev container for the Gatling load test project.
2+
//
3+
// This provides a ready-to-go JVM + Scala environment without needing
4+
// to install anything locally. Open the load-tests/ folder in VS Code
5+
// and select "Dev Containers: Reopen in Container".
6+
//
7+
// Includes:
8+
// - JDK 21 (Temurin)
9+
// - Scala via Coursier
10+
// - VS Code extensions: Scala Metals (IDE support) + Gradle (build tasks)
11+
{
12+
"name": "Deflock Load Tests",
13+
"build": {
14+
"dockerfile": "Dockerfile"
15+
},
16+
"workspaceFolder": "/workspaces/deflock-app/load-tests",
17+
"customizations": {
18+
"vscode": {
19+
"extensions": [
20+
"scalameta.metals",
21+
"vscjava.vscode-gradle"
22+
]
23+
}
24+
},
25+
// Pre-download all Gradle + Gatling dependencies so the first
26+
// `./gradlew gatlingRun` is fast.
27+
"postCreateCommand": "./gradlew dependencies"
28+
}

‎load-tests/.gitignore‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
# Gradle build outputs (includes Gatling HTML reports in build/reports/gatling/)
2+
build/
3+
4+
# Gradle cache
5+
.gradle/

‎load-tests/CLAUDE.md‎

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
# Load Tests
2+
3+
Gatling load tests for `overpass.deflock.org`. Scala 2.13, Gradle build, JDK 21.
4+
5+
## Architecture
6+
7+
Three source sets to avoid circular dependencies:
8+
9+
- **`src/shared/`** — Pure Scala logic (no Gatling dependency). `OverpassQuery` (query builder, tag filters, timeouts) and `TestData` (cities, viewports, feeders).
10+
- **`src/gatling/`** — Gatling simulations. Depends on `shared`. Contains `OverpassRequests` (HTTP request def) and four simulations.
11+
- **`src/test/`** — ScalaTest unit tests. Depends on `shared`. Tests pure logic without Gatling.
12+
13+
The `shared` source set exists because the Gatling Gradle plugin creates a circular dependency if `test` depends on `gatling` output directly.
14+
15+
## Key files
16+
17+
| File | Purpose |
18+
|---|---|
19+
| `src/shared/scala/deflock/OverpassQuery.scala` | Query builder, tag filters (must match app), timeouts |
20+
| `src/shared/scala/deflock/TestData.scala` | Cities, viewports, feeders (`randomFeeder`, `weightedZoomFeeder`) |
21+
| `src/gatling/scala/deflock/OverpassRequests.scala` | Gatling HTTP request definition, `feederForZoom` |
22+
| `src/gatling/scala/deflock/OverpassSimulation.scala` | Baseline: 1 user, all cities x all zooms, deterministic |
23+
| `src/gatling/scala/deflock/ConcurrentSimulation.scala` | Ramp to 50 users, find degradation point |
24+
| `src/gatling/scala/deflock/StressSimulation.scala` | Spike to 500 users, exceed server capacity |
25+
| `src/gatling/scala/deflock/BurstSimulation.scala` | Realistic app sessions in waves |
26+
| `build.gradle.kts` | Gradle config with `shared` source set, ScalaTest deps |
27+
28+
## Commands
29+
30+
```bash
31+
./gradlew gatlingRun # baseline
32+
./gradlew gatlingRun --simulation deflock.ConcurrentSimulation # concurrent
33+
./gradlew gatlingRun --simulation deflock.StressSimulation # stress
34+
./gradlew gatlingRun --simulation deflock.BurstSimulation # burst
35+
./gradlew test # unit tests
36+
./gradlew compileGatlingScala # compile check
37+
```
38+
39+
Do NOT use `gatlingRun-deflock.ClassName` syntax — it doesn't work with the Gatling Gradle plugin v3.15.0. Use `--simulation` flag instead.
40+
41+
## Tag filter parity
42+
43+
`OverpassQuery.tagFilters` must exactly match the app's `NodeProfile.getDefaults()` in `lib/models/node_profile.dart`. There are 11 built-in profiles. Empty tag values (e.g., `camera:mount: ''`) are filtered out, matching what `OverpassService._buildQuery()` does in `lib/services/overpass_service.dart`.
44+
45+
When profiles change in the app, update `tagFilters` to match.
46+
47+
## Conventions
48+
49+
- Baseline simulation must be deterministic (no randomization) for reproducible results.
50+
- Concurrent/stress/burst simulations use randomized feeders for realistic traffic.
51+
- `ThreadLocalRandom` (not `scala.util.Random`) for feeders used by concurrent simulations.
52+
- Gatling session keys are constants in `OverpassQuery` (`CityName`, `ZoomLevel`, `QueryBody`).
53+
- User-Agent headers identify load test traffic: `DeFlock/LoadTest-{Scenario}`.
54+
- Client timeout = server timeout + 5s so we receive server-side timeout responses.
55+
56+
## GitHub Actions
57+
58+
The `Load Test` workflow (`.github/workflows/load-test.yml`) has a scenario picker dropdown. Reports are uploaded as artifacts. A summary job posts a comment on the PR with download links.
59+
60+
Trigger via Actions tab or API:
61+
```bash
62+
gh api repos/{owner}/{repo}/actions/workflows/{id}/dispatches \
63+
-f ref=feat/load-tests -f 'inputs[scenario]=baseline'
64+
```

0 commit comments

Comments
 (0)