feat(mattermost): add mattermost bootstrap — role, BoxTemplate, catalog_try#191
Closed
t0kubetsu wants to merge 82 commits into
Closed
feat(mattermost): add mattermost bootstrap — role, BoxTemplate, catalog_try#191t0kubetsu wants to merge 82 commits into
t0kubetsu wants to merge 82 commits into
Conversation
…gs (#153) Critical fixes: - C1: Replace host-wide `docker system prune` + `docker ps -aq` clean target with scoped `docker compose down --rmi all --volumes --remove-orphans` - C2: Fix wrong SERVICE in CVE-2024-6387 (was openssh-cve-2018-15473) - C3: Fix wrong SERVICE in CVE-2019-11043 (was apache-cve-2021-42013) High fixes: - H1: Eliminate $(shell docker ps -aq) parse-time expansion in all files - H2: Fix stop-debug-build hardcoded 'debug' literal → $(DEBUG_SERVICE) - H3: Scope docker compose build/up to $(SERVICE) in all build targets - H4: Moot — resolved by C1 fix Medium fixes: - M1: Add .PHONY declarations to all 18 Makefiles - M2: Add `print: help` alias (was advertised but missing) - M3: Update vite CVE-2023-34092 SERVICE + compose.yml service name (stale vite-cve-2022-44615 from pre-rename) - M4: Remove dead 5-line commented block from term-debug-build - M5: Fix stop-debug-build comment ("stop only" → "stop and remove") - M6: Set blank_template SERVICE = REPLACE_ME (was tomcat service name) - M7: Fix lpe-01 duplicate help entry; expose make term-user properly - M8: Moot — resolved by C1 fix Low fixes: - L1: Normalize ISSUE header formats to `# ISSUE N` across all files - L2: Standardize term shell to /bin/bash (vite CVEs used /bin/sh) - L3: Remove trailing whitespace on target definition lines
New symbolic, parametric templates consumed by the r42topo compiler
(range42-playbooks) to author + compile topology.json into deployable
scenarios. Directory-per-version layout: <id>/v<MAJOR.MINOR.PATCH>/template.yml.
- subnet_layouts/default-3zone — admin/student/ctf bridge plan
- box_templates/{admin-wazuh,deployer,vuln-box,student-box} — VM archetypes
(role, inventory group, spec, default catalog role attachments)
- network_policies/air-gap-ctf — symbolic generalization of demo_lab_network's
hardcoded 05_network_isolation (zones + services + allow/deny matrix + air-gap)
- network_policies/flat-open — no-isolation baseline
- manifest.json: register the topology layer + categories
Box templates can now declare their base OS, so a box's clone image is chosen at authoring time while the warmup roles keep self-detecting the OS at runtime via ansible_facts.distribution (feat/local-apt-mirror). - os: ubuntu added explicitly to admin-wazuh, deployer, student-box, vuln-box (optional, defaults to ubuntu in the loader — no behaviour change). - New debian-jump box: os: debian, Debian 13 (trixie), 2cpu/4gb/32gb student jump box — the reference example of a non-ubuntu box. Values match ansible_facts.distribution lower-cased (ubuntu/debian/fedora). See the generator's docs/box-template-os-field.md for the full rationale + the template-image gate (debian images exist as trixie; fedora not yet).
…ename>) os: ubuntu/debian -> image: ubuntu_noble/debian_trixie. The field now carries the version (image-set name = the 01_init_proxmox templates/<image>/ dir), so a box is never ambiguous about which Debian. debian-jump -> image: debian_trixie.
The renderer now emits attachment params into stage_01, so the box catalog is the parametric source of a VM's config: software.configure.firewalls carries its firewall_rules (22/80/8080 tcp), and a container attachment (cve/crypto/openssl/ CVE-2014-0160) wires a docker-compose play. Replaces the concrete copy in the create-vms-vuln bundle.
…ry box All box templates now own their host firewall rules via the software.configure.firewalls attachment params (emitted into stage_01 by the generator), matching the demo_lab rules: - admin-wazuh : 22, 443, 1514, 1515 (ssh, dashboard, wazuh agent events/enroll) - deployer : 22, 80, 443 (ssh, http, https) - student-box : 22 (ssh) - debian-jump : 22 (ssh) - vuln-box : 22, 80, 8080 + docker CTF stack (prior commit) IP-dependent role config (wazuh indexer IPs, etc.) intentionally stays at the role defaults / deploy time — not catalog-appropriate. The catalog box_template is now the parametric source of a VM's host config, replacing the concrete copies in the create-vms-* bundles.
… debian_trixie Each image now carries the cloud_image.url + cloud_image.filename needed by the generator to render stage_00-download_cloudinit_files/<image>.yml. Only the image actually used by templates is listed (noble-minimal, not noble-server).
… debian_trixie (2 VMs) Each image now carries the full Proxmox template VM specs (vm_id, vm_name, spec, ip, bridge) needed by the generator to render stage_01-create_templates without any hardcoded data in the playbooks repo.
… convention
- box_templates: replace spec/image with template_vm (direct ProxmoxTemplateSpec ref)
- proxmox_templates: replace ip/bridge with ip_octet; remove bridge (both now
come from the subnet layout's template_subnet at generation time)
- vm_names renamed to template-vm-{distro}-{codename}-* for global uniqueness
(ubuntu: template-vm-ubuntu-noble-*, debian: template-vm-debian-trixie-*)
- default-3zone: add template_subnet: {cidr, bridge} — keeps "3-zone" accurate
(templates subnet is infrastructure, not a lab zone)
…sed/, update when conditions and docker repo URLs - software.configure.firewalls: ubuntu/ → debian-based/, when includes Debian - software.install.warmup.basic_packages: all ubuntu/ subdirs → debian-based/, all dispatcher _main.yaml when conditions updated, docker GPG/repo URLs now use ansible_facts.distribution | lower for Ubuntu and Debian
…ta to layout Box templates no longer carry role. Subnet layouts gain base_octet, section, and label per subnet so the generator derives placement purely from the layout.
Group is now derived from subnet name at generation time (r42_{subnet}_group).
…l-lan layout Section directory names and Ansible task labels are now derived automatically from each subnet's list index and name — no need to declare them explicitly in template YAML. Also adds dual-lan layout (lan1/lan2, vmbr150/vmbr151).
Mutual DROP isolation between lan1 and lan2 for the dual-lan layout. Zone names match subnet names so compile_network_policy_from_alloc resolves concrete CIDRs (192.168.150.0/24 <-> 192.168.151.0/24).
default_action: drop was blocking all internet/NAT traffic from VMs (cloud-init apt updates, internet reachability). The isolation intent is only to block lan1<->lan2 cross traffic, not all forwarded packets. Matrix DROP rules at w=500 fire before the default ACCEPT at w=900, so the lan1/lan2 mutual block still holds.
Revert default_action to drop (deny-by-default posture).
Add explicit {src: lan1/lan2, dst: wan, action: accept} rows for
internet egress. These compile at W_EGRESS_ACCEPT=700, AFTER the
cross-LAN DROP rules (500), so isolation is preserved.
Adds a clean apt-mirror/cache implementation scoped to the catalog layer: - software.install.apt_cacher_ng: installs, configures, and starts apt-cacher-ng (port 3142, passthrough enabled, ufw rule opened) - software.configure.apt_mirror_client: deploys /etc/apt/apt.conf.d/01proxy, requires apt_proxy_url to be set, triggers apt cache refresh via handler - apt-mirror box_template: debian-trixie-small base, firewall (22+3142), basic_packages warmup, apt_cacher_ng server role attached Rebased from feat/topology-layer-templates; replaces old airgapped implementation with a simpler proxy-cache-only approach.
…full mirror Extends the apt-mirror box template to support both operating modes: - Proxy-cache mode (default): apt-cacher-ng on port 3142, transparent caching - Airgapped full-mirror mode (apt_mirror_airgapped: true): apt-mirror + nginx on port 80 Server role (software.install.apt_cacher_ng): - defaults: add airgapped vars (apt_mirror_root, apt_mirror_threads, suite flags) - tasks/main.yml: conditional dispatch between proxy and airgapped task sets - tasks/install_mirror.yml: install apt-mirror + nginx - tasks/configure_mirror.yml: create mirror dirs, deploy mirror.list - tasks/prewarm_mirror.yml: optional async fire-and-forget apt-mirror run - tasks/serve_mirror.yml: deploy nginx config, enable service, open ufw port - templates/mirror.list.j2: apt-mirror mirror.list with Debian trixie + Ubuntu suite conditionals - templates/nginx-aptmirror.conf.j2: serve apt_mirror_root/mirror on apt_mirror_http_port + /healthz - handlers/main.yml: add restart nginx handler Client role (software.configure.apt_mirror_client): - defaults: replace old proxy-only vars with apt_mirror_enabled, apt_mirror_airgapped, apt_mirror_vm_ip, apt_mirror_http_port - tasks/main.yml: URL-rewriting approach — computes _apt_effective_url and rewrites /etc/apt/mirrors/debian.list, debian-security.list (Debian) and ubuntu.sources URIs (Ubuntu) Box template: update description, add port 80 to firewall_rules
…gap is a network policy concern
…her_ng and software.install.apt_mirror - software.install.apt_cacher_ng: proxy-cache only, no mode flag - software.install.apt_mirror: new role, full mirror (apt-mirror + nginx) - apt-mirror box template now references software.install.apt_mirror with no params - task/template files moved via git mv to preserve history
…e-based names; add resolute
- Remove ip_octet from all proxmox_templates in debian_trixie and ubuntu_noble
- Add ubuntu_resolute image layer (Ubuntu 26.04 LTS, 12 VM sizes)
- Rename apt_mirror suite variables to codename-based names:
apt_mirror_debian_stable → apt_mirror_debian_trixie
apt_mirror_ubuntu_2204 → apt_mirror_ubuntu_jammy
apt_mirror_ubuntu_2404 → apt_mirror_ubuntu_noble
- Add apt_mirror_debian_bookworm and apt_mirror_ubuntu_resolute flags
- Add Ubuntu resolute section to mirror.list.j2; fix duplicate Ubuntu clean entries
- Add ubuntu-jump box template (Ubuntu 26.04 Resolute example with apt integration docs)
…_packages In an egress-filtered / air-gapped range udp/123 is blocked (even the Proxmox node can't NTP-sync), so software.install.warmup.basic_packages' hard "wait for NTPSynchronized == yes" never succeeds and fails stage_01. Don't enable NTP sync for this box; VMs take host (kvm-clock) time.
…er reqs software-properties-common is absent from Debian trixie 'main' (Unable to locate package), so the docker prerequisites install failed on Debian hosts. It's an Ubuntu-ism and unused here anyway — the Docker CE repo is added via the apt_repository/apt_key modules, not add-apt-repository. Removing it unblocks docker install on Debian; Ubuntu is unaffected (repo-add path is identical).
…r repo (Debian) apt-key is gone on Debian 12+/trixie (and deprecated on Ubuntu), so the apt_key module failed: "Failed to find required executable apt-key". Switch to the modern keyring method: fetch the armored Docker GPG key into /etc/apt/keyrings/docker.asc (get_url) and pin the repo with signed-by=. Also derive arch (amd64/arm64) instead of hardcoding amd64. Works on both Debian and Ubuntu.
…ompose path) The compose install path had three problems that would break the Rocket.Chat stack deploy (community.docker.docker_compose_v2 / docker_container_info): - listed software-properties-common (absent on Debian trixie) → apt failure; - downloaded the standalone docker-compose binary from a v1-era asset name (docker-compose-Linux-x86_64) that 404s for v2, and that binary does not provide the `docker compose` plugin docker_compose_v2 invokes; - no python Docker SDK for the community.docker modules. Now: install docker-compose-plugin from the docker-ce repo (provides `docker compose`), add python3-docker, drop software-properties-common, and check via `docker compose version`. Works on Debian and Ubuntu.
…ires >=8.0)
rocketchat/rocket.chat:latest is now 8.4.3, which refuses to start against
MongoDB 6.0 ("YOUR CURRENT MONGODB VERSION IS NOT SUPPORTED ... UPGRADE TO 8.0
OR LATER") — the container crash-loops and the compose deploy fails with
"container rocketchat is unhealthy". Pin mongo:8.0 for both the mongodb and
mongo-init-replica services. Fresh deploys only (no in-place 6.0->8.0 data
migration); the lab recreates the volume each deploy.
…meouts On egress-filtered ranges, image pulls intermittently time out against CDN anycast endpoints (auth.docker.io / registry-1.docker.io, debian-security, etc.) from the VM's NAT path, failing the whole deploy even though a retry would connect to a working PoP. `docker compose up` is idempotent, so wrap the run with until/retries (5x, 20s) so transient pull failures self-heal; a persistent failure still surfaces after retries are exhausted.
…nt in image) The rocket.chat image (Alpine) ships neither curl nor wget, which broke two things: - the rocketchat healthcheck (curl -f .../api/v1/info) never passed → container stuck "health: starting" forever → provisioner (depends_on service_healthy) never started → `docker compose up` hung indefinitely; - the provisioner init.sh drives the REST API with curl → would fail on "curl: not found" once it ran. Fixes: - healthcheck now uses Node's built-in fetch (Node 18+; the image runs node main.js); - provisioner Dockerfile installs curl (apk add --no-cache curl) in the runtime stage. Rocket.Chat 8.4.3 itself boots fine (SERVER RUNNING, MongoDB 8.0.23).
Rocket.Chat 8.4.3 removed the unauthenticated /api/v1/info endpoint (returns 404), which broke two readiness checks against a server that is actually up: - compose healthcheck -> hit /health (200) instead; - provisioner init.sh readiness wait -> hit /api/info (200) instead. Confirmed live: /health and /api/info both return 200; /api/v1/info returns 404.
… BoxTemplate, catalog_try.yml
# Conflicts: # 03_container_layer/docker/admin/nextcloud/catalog_try.yml
…ost BoxTemplate, catalog_try.yml
4 tasks
…YPOINT The provisioner container failed with "exec /provisioning/init.sh: no such file or directory" because mattermost-team-edition:latest is shell-less — /usr/bin/env and /bin/sh are absent, so the kernel returns ENOENT when trying to resolve the #!/usr/bin/env sh shebang. Three changes: - Pin runtime FROM to 10.9.5 (matches compose.yml; latest has the mmctl panic bug on non-root UID resolution that breaks the healthcheck) - COPY /bin/sh from builder (golang:alpine busybox) into the runtime image - Change ENTRYPOINT to exec form ["/bin/sh", "/provisioning/init.sh"] so Docker invokes sh directly without going through /usr/bin/env
4 tasks
…me stage Alpine /bin/sh is a symlink to /bin/busybox; COPY of the symlink produced a dangling link in the mattermost runtime image, causing ENOENT on exec.
mattermost-team-edition:10.9.5 is Debian-based (glibc). Copying Alpine's musl-linked busybox or jq into it fails because the musl ELF interpreter is absent. The image already ships sh (dash) and jq natively. - Remove COPY /bin/busybox → use native /usr/bin/sh in ENTRYPOINT - Remove COPY /usr/bin/jq → image already has it - Remove apk add jq from builder (no longer needed) - yq is a static Go binary, safe to copy from either base
…leases Removed jq in the previous commit assuming it was native to the mattermost Debian image — it was ours from a prior COPY. Downloading the static jq-linux-amd64 release binary (no musl/glibc dependency).
Contributor
Author
|
Superseded by #193 (feat/mattermost-docker-stack) which is scoped to 03_container_layer/docker/admin/mattermost/ only. The mixed history on this branch made a scoped PR impossible without a dedicated branch. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
software.install.mattermostAnsible role (delegates tosoftware.configure.docker-composewithMATTERMOST_*vars)admin-mattermostBoxTemplate (05_topology_layer/box_templates/admin-mattermost/v1.0.0/template.yml) — Debian Trixie medium, ports 22+8065, warmup.basic_packages + Docker + Compose, software.install.mattermostcatalog_try.ymlfor Mattermost (03_container_layer/docker/admin/mattermost/) — port 8065,/api/v4/system/ping, 120s init timeoutdev_ada(Debian Trixie Docker fixes) into branch to avoid Docker-never-installed issueTest plan
test_mattermostscenario (range42-playbooks PR feature: php - local file inclusion rce via phpinfo #92) on Proxmox via deployercatalog-tryL2 poll returns 200 on/api/v4/system/ping