AWX schema - add extraction script and weekly update workflow#461
Open
himdel wants to merge 4 commits into
Open
AWX schema - add extraction script and weekly update workflow#461himdel wants to merge 4 commits into
himdel wants to merge 4 commits into
Conversation
Strips pg-version-dependent output for stable diffs. New *_old text columns from migration 0185 (text-to-jsonb): - main_conf: setting_old - main_activitystream: deleted_actor_old - main_joblaunchconfig: survey_passwords_old, char_prompts_old - main_schedule: survey_passwords_old, char_prompts_old - main_workflowjobtemplatenode: survey_passwords_old, char_prompts_old - main_workflowjobnode: survey_passwords_old, char_prompts_old - main_notification: body_old - main_unifiedjob: job_env_old None of these fields are used by metrics-utility.
Automates schema extraction using our compose postgres + uv, no awx containers needed. Strips pg-version-dependent output to keep diffs stable across postgres upgrades. Issue: AAP-75301 Assisted-by: Claude Opus 4.6 <noreply@anthropic.com>
Uses peter-evans/create-pull-request to open/update a PR when the schema changes. Runs migrations with uv + apt postgres, strips pg-version-dependent output same as the local script. Issue: AAP-75301 Assisted-by: Claude Opus 4.6 <noreply@anthropic.com>
Contributor
📝 WalkthroughWalkthroughAdds a Bash script for extracting an AWX schema dump, updates the extraction docs, and adds a scheduled/manual GitHub Actions workflow that refreshes ChangesAWX schema refresh automation
Sequence Diagram(s)sequenceDiagram
participant GHA as GitHub Actions
participant AWX as ansible/awx devel checkout
participant PG as PostgreSQL
participant Migrate as awx-manage migrate
participant Dump as pg_dump
participant PR as peter-evans/create-pull-request
GHA->>AWX: check out devel
GHA->>PG: create awx role and awx database
GHA->>Migrate: run migrations
Migrate->>PG: apply schema changes
GHA->>Dump: pg_dump -s
Dump-->>GHA: tools/docker/latest.sql
GHA->>PR: open PR when git diff changes
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
cshiels-ie
approved these changes
Jun 25, 2026
|
|
This PR has been automatically linked to AAP-75301 in Jira. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/awx-schema.yml:
- Around line 12-24: The awx-schema workflow runs untrusted ansible/awx@devel
steps while the job already has write-scoped permissions, so split the workflow
into a read-only job for checkout/extraction/migrations and a separate follow-up
job that only opens the PR with write access. In the current job, harden both
actions/checkout usages in the awx-schema workflow by setting
persist-credentials to false so the checked-out repository cannot reuse the
token. Use the existing checkout steps and the PR-creation logic as the key
symbols to relocate permissions cleanly.
In `@tools/docker/extract-awx-schema.sh`:
- Around line 58-60: The postgres readiness loop in the schema extraction script
can hang indefinitely and uses docker compose exec without disabling TTY, which
is brittle in non-interactive shells. Update the wait logic around the
pg_isready call to use docker compose exec with -T, and add a bounded timeout or
retry limit so the script exits deterministically if postgres never becomes
ready.
- Line 69: The requirements filtering in extract-awx-schema.sh can stop the
script under set -e when grep -v finds no uwsgi== match. Update the filtering
step to use a non-failing approach such as sed in place of the current grep -v,
or explicitly handle grep’s no-match exit so the script continues. Keep the fix
localized to the requirements.txt-to-awx-requirements-filtered.txt
transformation.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Enterprise
Run ID: bb61c915-638e-4a24-91ab-cfca8c870fc8
📒 Files selected for processing (4)
.github/workflows/awx-schema.ymldocs/awx.mdtools/docker/extract-awx-schema.shtools/docker/latest.sql
Comment on lines
+12
to
+24
| permissions: | ||
| contents: write | ||
| pull-requests: write | ||
| steps: | ||
| - name: "Checkout metrics-utility" | ||
| uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 | ||
|
|
||
| - name: "Checkout AWX" | ||
| uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 | ||
| with: | ||
| repository: ansible/awx | ||
| ref: devel | ||
| path: awx |
Contributor
There was a problem hiding this comment.
🔒 Security & Privacy | 🟠 Major | 🏗️ Heavy lift
Untrusted AWX code executes with a write-scoped repo token
This job grants write permissions before running ansible/awx@devel code (pip install -e . and awx-manage), so an upstream compromise could use GITHUB_TOKEN to write to this repo. Also, both checkouts persist credentials by default.
Use a two-job design: run extraction/migrations in a read-only job, then open PR in a separate write-permission job; and set persist-credentials: false on both checkout steps as immediate hardening.
Also applies to: 54-87
🧰 Tools
🪛 zizmor (1.26.1)
[warning] 16-17: credential persistence through GitHub Actions artifacts (artipacked): does not set persist-credentials: false
(artipacked)
[warning] 19-24: credential persistence through GitHub Actions artifacts (artipacked): does not set persist-credentials: false
(artipacked)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/awx-schema.yml around lines 12 - 24, The awx-schema
workflow runs untrusted ansible/awx@devel steps while the job already has
write-scoped permissions, so split the workflow into a read-only job for
checkout/extraction/migrations and a separate follow-up job that only opens the
PR with write access. In the current job, harden both actions/checkout usages in
the awx-schema workflow by setting persist-credentials to false so the
checked-out repository cannot reuse the token. Use the existing checkout steps
and the PR-creation logic as the key symbols to relocate permissions cleanly.
Source: Linters/SAST tools
Comment on lines
+58
to
+60
| until $COMPOSE_CMD -f docker-compose.yaml exec postgres pg_isready -U awx 2>/dev/null; do | ||
| sleep 2 | ||
| done |
Contributor
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
Bound the postgres wait loop and disable TTY allocation
Line 58 can hang forever if readiness never succeeds, and docker compose exec without -T is brittle in non-interactive shells. Add -T plus a timeout/retry budget so failures surface deterministically.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@tools/docker/extract-awx-schema.sh` around lines 58 - 60, The postgres
readiness loop in the schema extraction script can hang indefinitely and uses
docker compose exec without disabling TTY, which is brittle in non-interactive
shells. Update the wait logic around the pg_isready call to use docker compose
exec with -T, and add a bounded timeout or retry limit so the script exits
deterministically if postgres never becomes ready.
| cd "$AWX_DIR" | ||
| uv venv .venv 2>/dev/null || true | ||
| # Filter out uwsgi (needs crypt.h) — not needed for migrations | ||
| grep -v '^uwsgi==' requirements/requirements.txt > "$TMPDIR/awx-requirements-filtered.txt" |
Contributor
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
grep -v can abort the script under set -e
At Line 69, if uwsgi== is no longer present, grep -v exits with status 1 and the script stops. Use a non-failing filter (sed '/^uwsgi==/d') or handle the no-match exit explicitly.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@tools/docker/extract-awx-schema.sh` at line 69, The requirements filtering in
extract-awx-schema.sh can stop the script under set -e when grep -v finds no
uwsgi== match. Update the filtering step to use a non-failing approach such as
sed in place of the current grep -v, or explicitly handle grep’s no-match exit
so the script continues. Keep the fix localized to the
requirements.txt-to-awx-requirements-filtered.txt transformation.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue: AAP-75301
Adds automated AWX schema extraction:
tools/docker/latest.sqlto current awx devel (6d665dda33), stripping pg-version-dependent output (version comments, named NOT NULL constraints, pg18 directives) for stable diffstools/docker/extract-awx-schema.sh) that uses our compose postgres + uv, no awx containers needed, just a local ../awx checkoutpeter-evans/create-pull-request)docs/awx.mdwith script referenceSummary by CodeRabbit
New Features
Documentation