WP-CLI smoke test, and keep the root key across multisite conversion

.github/workflows/ci.yml

@@ -264,3 +264,46 @@ jobs:
           exit 1
 
       - run: make test-examples
+
+  # A dispatch-layer smoke test, not another correctness suite: three real
+  # dispatch bugs (--version=previous returning the current value,
+  # --format rejected for want of a description line, and migrate-legacy
+  # unregistered without @subcommand) reached a green PHPUnit suite because
+  # nothing exercised the real `wp` binary end to end. 7.4 is the floor
+  # this API has to run on; 8.3 is the newest PHP the rest of the matrix
+  # covers. No Composer step: the smoke test provisions its own WP-CLI phar
+  # and WordPress checkout and needs neither `vendor/` nor the WordPress
+  # test suite.
+  smoke:
+    name: "Smoke / PHP ${{ matrix.php }}"
+    needs: static
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        php: ['7.4', '8.3']
+    services:
+      mysql:
+        image: mysql:8.0
+        env:
+          MYSQL_ALLOW_EMPTY_PASSWORD: 'yes'
+          MYSQL_DATABASE: wordpress_smoke
+        ports:
+          - 3306:3306
+        options: >-
+          --health-cmd="mysqladmin ping"
+          --health-interval=10s
+          --health-timeout=5s
+          --health-retries=5
+    steps:
+      - name: Check out
+        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+
+      - name: Set up PHP ${{ matrix.php }}
+        uses: shivammathur/setup-php@f3e473d116dcccaddc5834248c87452386958240 # 2.37.2
+        with:
+          php-version: ${{ matrix.php }}
+          extensions: sodium, mysqli
+          coverage: none
+
+      - run: make smoke DB_HOST=127.0.0.1

.gitignore

@@ -24,3 +24,5 @@ site/.astro/
 # Spacefast CLI link and state. Written wherever sf publish runs from; never commit it.
 .spacefast/
 
+# The WP-CLI smoke test's throwaway install; see bin/smoke-install.sh.
+/.smoke/

Makefile

@@ -13,9 +13,10 @@ DB_NAME ?= wordpress_test
 DB_USER ?= root
 DB_PASS ?=
 DB_HOST ?= 127.0.0.1
+SMOKE_DB_NAME ?= wordpress_smoke
 
 .DEFAULT_GOAL := help
-.PHONY: help install lint lint-fix compat analyse test test-ms test-examples coverage reference reference-check ci clean
+.PHONY: help install lint lint-fix compat analyse test test-ms test-examples coverage reference reference-check ci smoke clean
 
 help: ## Show this help.
 	@grep -hE '^[a-zA-Z_-]+:.*?## ' $(MAKEFILE_LIST) \
@@ -54,7 +55,11 @@ reference: ## Regenerate docs/reference/ from source docblocks.
 reference-check: ## Fail if docs/reference/ is stale relative to the source.
 	php bin/gen-reference.php --check
 
-ci: lint compat analyse reference-check test test-ms ## Everything CI runs.
+smoke: ## Provision the throwaway install and run the WP-CLI smoke test.
+	SMOKE_DB_NAME=$(SMOKE_DB_NAME) DB_USER=$(DB_USER) DB_PASS="$(DB_PASS)" DB_HOST=$(DB_HOST) WP_VERSION=$(WP_VERSION) bin/smoke-install.sh
+	tests/smoke/smoke.sh
+
+ci: lint compat analyse reference-check test test-ms smoke ## Everything CI runs.
 
 # Local Vault dev server for the vault-provider example (pinned digest):
 #   docker run -d --name secrets-api-vault -p 8201:8200 -e VAULT_DEV_ROOT_TOKEN_ID=dev-root --cap-add=IPC_LOCK hashicorp/vault@sha256:47f14a6acb98f48d798a07df7c83f23a6e636e1cf724c5f8ff165cb32667a1e2

README.md

@@ -27,7 +27,8 @@ composer install
 bin/ci-local.sh
 ```
 
-Add `--keep` to leave the environment running between iterations.
+Add `--keep` to leave the environment running between iterations. `bin/ci-local.sh` now also runs
+the WP-CLI smoke test inside wp-env, against its own throwaway install.
 
 If you already have a WordPress test suite and a database, skip wp-env entirely:
 
@@ -49,6 +50,7 @@ target list.
 | `make test-examples` | phpunit against `examples/*/tests`, needs Moto running (see `examples/README.md`); not part of `make ci` |
 | `make coverage` | phpunit with an HTML coverage report (see `docs/journal/test-coverage-gaps.md` re: wp-env) |
 | `make reference` / `make reference-check` | regenerate `docs/reference/` from docblocks / fail if it is stale |
+| `make smoke` | provision a throwaway WordPress in `.smoke/` and drive `wp secret` / `wp network-secret` end to end (needs MySQL and network access) |
 | `make ci` | all of the above |
 
 Runners without egress to wordpress.org can point the installer at a mirror with `WP_MIRROR_BASE`
@@ -167,8 +169,9 @@ stores credentials, and a flaw in it is a flaw in the thing protecting everythin
 
 CI (`.github/workflows/ci.yml`) is a thin wrapper around the `make` targets above, running on
 github.com's hosted runners: static analysis gates a PHP 7.4/8.0/8.3 × WordPress latest/trunk
-matrix plus a multisite job, plus an `examples` job that runs the platform bindings against a
-Vault service container. See [`docs/reference/ci.md`](docs/reference/ci.md).
+matrix plus a multisite job, a PHP 7.4/8.3 smoke job driving a real `wp` binary, and an
+`examples` job that runs the platform bindings against Moto and Vault service containers. See
+[`docs/reference/ci.md`](docs/reference/ci.md).
 
 ## License
 

bin/ci-local.sh

@@ -71,5 +71,16 @@ echo "==> test (multisite)"
 "${WP_ENV[@]}" run --env-cwd="$CONTAINER_CWD" tests-cli \
 	env WP_MULTISITE=1 vendor/bin/phpunit -c phpunit-multisite.xml.dist
 
+# The smoke test provisions its own throwaway install under .smoke/, with its
+# own database (wordpress_smoke), inside wp-env's *development* `cli`
+# container -- never the `tests-cli` container or its wordpress_test
+# database, which the PHPUnit suite above owns. The two scripts run directly,
+# rather than through `make smoke`, because the `cli` container has no `make`.
+echo "==> smoke (WP-CLI against a throwaway install)"
+"${WP_ENV[@]}" run --env-cwd="$CONTAINER_CWD" cli \
+	env DB_HOST=mysql DB_USER=root DB_PASS=password bash bin/smoke-install.sh
+"${WP_ENV[@]}" run --env-cwd="$CONTAINER_CWD" cli \
+	bash tests/smoke/smoke.sh
+
 echo
 echo "All green."

bin/smoke-install.sh

@@ -0,0 +1,162 @@
+#!/usr/bin/env bash
+#
+# Provision a throwaway single-site WordPress install for the WP-CLI smoke test.
+#
+# This script owns everything under .smoke/: a pinned wp-cli.phar, its download
+# cache, and a full WordPress checkout with its own "wordpress_smoke" database.
+# It never touches wp-env's own dev or test environment, because that one shares
+# wp-content with PHPUnit -- and a drop-in the smoke test writes would sit in
+# front of the PHPUnit suite. See tests/smoke/SPEC.md.
+#
+set -euo pipefail
+cd "$(dirname "$0")/.."
+
+# Pinned wp-cli release. Resolved from
+# https://api.github.com/repos/wp-cli/wp-cli/releases/latest on 2026-09-24 and
+# verified against that release's own published
+# wp-cli-2.12.0.phar.sha256 before computing this digest -- the same discipline
+# ci.yml applies to action SHAs.
+WP_CLI_VERSION="2.12.0"
+WP_CLI_SHA256="ce34ddd838f7351d6759068d09793f26755463b4a4610a5a5c0a97b68220d85c"
+WP_CLI_URL="https://github.com/wp-cli/wp-cli/releases/download/v${WP_CLI_VERSION}/wp-cli-${WP_CLI_VERSION}.phar"
+
+SMOKE_DIR="$PWD/.smoke"
+SMOKE_DB_NAME="${SMOKE_DB_NAME:-wordpress_smoke}"
+DB_USER="${DB_USER:-root}"
+DB_PASS="${DB_PASS:-}"
+DB_HOST="${DB_HOST:-127.0.0.1}"
+WP_VERSION="${WP_VERSION:-latest}"
+SMOKE_URL="${SMOKE_URL:-http://smoke.test}"
+
+# The PHPUnit suite drops and recreates wordpress_test on every run; this
+# script drops and recreates whatever SMOKE_DB_NAME names. Refusing when they
+# collide protects the PHPUnit suite's database from the smoke test, and vice
+# versa.
+if [ "$SMOKE_DB_NAME" = "wordpress_test" ]; then
+	echo "SMOKE_DB_NAME must not be wordpress_test: that database belongs to the PHPUnit suite." >&2
+	exit 1
+fi
+
+export WP_CLI_CACHE_DIR="$SMOKE_DIR/cache"
+
+WP=( php -d display_errors=stderr -d log_errors=0 "$SMOKE_DIR/wp-cli.phar" --path="$SMOKE_DIR/wordpress" --allow-root )
+
+download() {
+	# $1 = URL, $2 = destination path.
+	if command -v curl >/dev/null 2>&1; then
+		curl -fsSL -o "$2" "$1"
+	elif command -v wget >/dev/null 2>&1; then
+		wget -nv -O "$2" "$1"
+	else
+		php -r 'copy($argv[1], $argv[2]);' -- "$1" "$2"
+	fi
+}
+
+sha256_of() {
+	if command -v sha256sum >/dev/null 2>&1; then
+		sha256sum "$1" | awk '{print $1}'
+	else
+		shasum -a 256 "$1" | awk '{print $1}'
+	fi
+}
+
+fetch_wp_cli() {
+	mkdir -p "$SMOKE_DIR" "$WP_CLI_CACHE_DIR"
+
+	if [ -f "$SMOKE_DIR/wp-cli.phar" ]; then
+		ACTUAL=$(sha256_of "$SMOKE_DIR/wp-cli.phar")
+		if [ "$ACTUAL" = "$WP_CLI_SHA256" ]; then
+			return
+		fi
+	fi
+
+	download "$WP_CLI_URL" "$SMOKE_DIR/wp-cli.phar"
+
+	ACTUAL=$(sha256_of "$SMOKE_DIR/wp-cli.phar")
+	if [ "$ACTUAL" != "$WP_CLI_SHA256" ]; then
+		rm -f "$SMOKE_DIR/wp-cli.phar"
+		echo "wp-cli.phar checksum mismatch." >&2
+		echo "expected: $WP_CLI_SHA256" >&2
+		echo "actual:   $ACTUAL" >&2
+		exit 1
+	fi
+}
+
+fetch_wp_cli
+
+if [ ! -f "$SMOKE_DIR/wordpress/wp-load.php" ]; then
+	mkdir -p "$SMOKE_DIR/wordpress"
+	"${WP[@]}" core download --version="$WP_VERSION"
+fi
+
+# Every run starts single-site with no drop-in: a leftover secrets.php or
+# multisite wp-config would make this script's own idempotence lie about what
+# state a fresh run leaves behind.
+rm -f "$SMOKE_DIR/wordpress/wp-config.php" "$SMOKE_DIR/wordpress/wp-content/secrets.php"
+
+"${WP[@]}" config create \
+	--dbname="$SMOKE_DB_NAME" \
+	--dbuser="$DB_USER" \
+	--dbpass="$DB_PASS" \
+	--dbhost="$DB_HOST" \
+	--skip-check \
+	--force
+
+# Drop and recreate the database with mysqli directly, rather than a `wp db`
+# subcommand or the `mysql` client binary, so this script stays dependency-free
+# beyond wp and php. Host/port/socket parsing mirrors install_db() in
+# bin/install-wp-tests.sh. Arguments are passed as $argv, never interpolated
+# into the PHP source.
+IFS=':' read -r DB_HOSTNAME DB_SOCK_OR_PORT <<< "$DB_HOST"
+DB_PORT=""
+DB_SOCKET=""
+if [[ "$DB_SOCK_OR_PORT" =~ ^[0-9]+$ ]]; then
+	DB_PORT="$DB_SOCK_OR_PORT"
+elif [ -n "${DB_SOCK_OR_PORT:-}" ]; then
+	DB_SOCKET="$DB_SOCK_OR_PORT"
+fi
+
+php -r '
+	list( $host, $user, $pass, $name, $port, $socket ) = array_slice( $argv, 1 );
+
+	$port   = $port !== "" ? (int) $port : null;
+	$socket = $socket !== "" ? $socket : null;
+
+	mysqli_report( MYSQLI_REPORT_OFF );
+	$link = mysqli_init();
+
+	if ( ! $link || ! @mysqli_real_connect( $link, $host, $user, $pass, "", $port, $socket ) ) {
+		fwrite( STDERR, "mysqli connect error: " . mysqli_connect_error() . "\n" );
+		exit( 1 );
+	}
+
+	if ( ! mysqli_query( $link, "DROP DATABASE IF EXISTS `" . $name . "`" ) ) {
+		fwrite( STDERR, "mysqli error: " . mysqli_error( $link ) . "\n" );
+		exit( 1 );
+	}
+
+	if ( ! mysqli_query( $link, "CREATE DATABASE `" . $name . "`" ) ) {
+		fwrite( STDERR, "mysqli error: " . mysqli_error( $link ) . "\n" );
+		exit( 1 );
+	}
+' -- "$DB_HOSTNAME" "$DB_USER" "$DB_PASS" "$SMOKE_DB_NAME" "$DB_PORT" "$DB_SOCKET"
+
+"${WP[@]}" core install \
+	--url="$SMOKE_URL" \
+	--title="Secrets API smoke" \
+	--admin_user=smoke \
+	--admin_password="$(php -r 'echo bin2hex(random_bytes(16));')" \
+	--admin_email=smoke@example.com \
+	--skip-email
+
+# Relative on purpose: the same link resolves on the host and inside the
+# wp-env container.
+ln -sfn ../../../.. "$SMOKE_DIR/wordpress/wp-content/plugins/secrets-api"
+"${WP[@]}" plugin activate secrets-api
+
+KEY="$("${WP[@]}" secret generate-key)"
+# --quiet: `wp config set` otherwise echoes the constant's value in its
+# "Success" line, which would put the generated key in plain sight.
+"${WP[@]}" config set WP_SECRETS_KEY "$KEY" --type=constant --quiet
+
+echo "Smoke install ready: $SMOKE_DIR/wordpress (database $SMOKE_DB_NAME)"

cli/class-wp-cli-secret-command.php

@@ -114,10 +114,9 @@ public function set( $args, $assoc_args ) {
 	 * [--slot=<slot>]
 	 * : Which stored version to read.
 	 *
-	 * Named --slot rather than --version because WP-CLI consumes `--version`
-	 * itself before a subcommand ever sees it: passing --version=previous
-	 * silently yielded the current value, since the flag was swallowed and the
-	 * synopsis default filled in behind it.
+	 * Named --slot rather than --version to avoid the value being dropped by
+	 * wrappers such as `wp-env run`, which consume --version themselves (the
+	 * original bug), and to avoid confusion with `wp --version`.
 	 * ---
 	 * default: current
 	 * options:

docs/index.md

@@ -71,6 +71,7 @@ directory holds everything longer than that.
 - [`2026-09-04-0-1-0-is-public.md`](journal/2026-09-04-0-1-0-is-public.md) — devlog: what 0.1.0 shipped, what it left out, and the road to 7.2.
 - [`2026-09-24-a-kms-keyring.md`](journal/2026-09-24-a-kms-keyring.md) — devlog: the first real `WP_Secrets_Keyring`, the root-key cache and `rotate --from` it drove, and what it found.
 - [`2026-09-24-a-vault-provider.md`](journal/2026-09-24-a-vault-provider.md) — devlog: the Vault KV v2 example, the AWS site-scope bug it found, and what stayed open.
+- [`2026-09-24-testing-the-cli-for-real.md`](journal/2026-09-24-testing-the-cli-for-real.md) — devlog: the WP-CLI smoke test, the root-key bug it found, and the last 🟡 coverage gap closing.
 - [`open-questions.md`](journal/open-questions.md) — what is still deliberately undecided.
 - [`proposal-questions.md`](journal/proposal-questions.md) — the five questions the proposal asked, and the answers so far.
 - [`test-coverage-gaps.md`](journal/test-coverage-gaps.md) — paths the suite cannot reach and what was verified by hand.