TOOL-30005 ucf: add masking-common role — required by Delphix mgmt JVM startup#870
Open
malikparvez wants to merge 8 commits into
Open
TOOL-30005 ucf: add masking-common role — required by Delphix mgmt JVM startup#870malikparvez wants to merge 8 commits into
malikparvez wants to merge 8 commits into
Conversation
The UCF S3 bucket usually lives in a different AWS account than Delphix's linux-build-publish IAM user, so the ambient AWS_ACCESS_KEY_ID in build-ancillary-repository.sh gets AccessDenied on the UCF bucket. If AWS_UCF_ACCESS_KEY_ID and AWS_UCF_SECRET_ACCESS_KEY are set, use them for the UCF aws s3 sync only. Falls back to ambient credentials when not set, so the change is backward-compatible (DCT/Hyperscale paths unaffected). The paired devops-gate PR exposes these env vars via the aws-ucf-build Jenkins credential. Tracking: PUPCLD-3705
The new internal-ucf and external-ucf variant directories were missing the conventional 'ansible/roles' symlink to ../../../misc/ansible-roles that every other variant has. Without it, the live-build chroot's ansible-playbook hook (config/hooks/configuration/80-build-configuration.binary) fails with: ERROR! the role 'appliance-build.minimal-common' was not found in .../live-build/build/internal-ucf-esx/ansible/roles:... Same fix that scripts/create-variant.sh creates for new variants. Tracking: PUPCLD-3705
virtualization-development clones github.com/delphix/dlpx-app-gate as
part of Delphix engine dev workflow. UCF doesn't use dlpx-app-gate and
the chroot doesn't have git installed at the point the role runs, so
the build fails with:
TASK [appliance-build.virtualization-development : git]
fatal: [binary]: FAILED! => {"msg": "Failed to find required
executable 'git' in paths: ..."}
qa-internal is similarly Delphix-product-specific. UCF only needs:
- minimal-common (base lockdown)
- minimal-internal (internal-build secrets)
- virtualization-common (admin console / engine UI)
- ucf-common (CDS package install)
Matches the user's design: "just need delphix's closed appliance
configuration and admin console plus my ucf app on top of it".
Tracking: PUPCLD-3705
malikparvez
force-pushed
the
dlpx/pr/malikparvez/045d1fc1-9da0-4afb-ad2e-8864e25a33fa
branch
from
52aceff to
16836bb
Compare
The new external-ucf / internal-ucf playbooks and the ucf-common role hit the same name[play], yaml[truthy], fqcn[action-core] and name[missing] rules that every existing variant/role is already baselined against in .ansible-lint-ignore. Add the matching entries so check-ansible passes. Tracking: PUPCLD-3705 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
malikparvez
changed the title
malikparvez
force-pushed
the
dlpx/pr/malikparvez/045d1fc1-9da0-4afb-ad2e-8864e25a33fa
branch
from
956b694 to
a15874e
Compare
prakashsurya
requested changes
Jun 8, 2026
|
|
||
| # | ||
| # UCF packages are produced by an external GitHub Actions workflow | ||
| # and may not ship a SHA256SUMS file; verify only if present. |
Contributor
There was a problem hiding this comment.
I think UCF really needs to ship a SHA256SUMS file (or some way to verify the files we download).. the build needs to be able to verify the downloaded files that's it's putting into the image.. the checksum verification below shouldn't be conditional, or else we won't detect package corruption..
Member
Author
There was a problem hiding this comment.
sure let me check on this, to get SHA256SUMS in our deb package
The internal-ucf playbook only included minimal-internal without its companion -development roles, breaking the pattern every other internal Delphix engine variant follows. internal-dct and internal-hyperscale both pair minimal-internal with minimal-development, masking-development, and virtualization-development. Runtime symptom: dlpx-internal-ucf-0.0.7 OVAs failed 'systemctl start default.target' during DCenter snapshot registration. All three DCenters timed out at 10 min in sync-ova-into-dcenter #71387/#71388/#71389 of ucf-engine-release #5. The mgmt JVM (and friends) expect the systemd service.d override directories that virtualization-development creates. The dlpx-app-gate / dms-core-gate clones inside these roles are gated on GITHUB_TOKEN, so they no-op for UCF where the token isn't set. qa-internal remains intentionally excluded (depends on dlpx-app-gate for full Delphix dev workflow).
malikparvez
force-pushed
the
dlpx/pr/malikparvez/045d1fc1-9da0-4afb-ad2e-8864e25a33fa
branch
from
a15874e to
b153716
Compare
This reverts commit b153716.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The internal/external UCF (Unified Continuous Forensics / CDS) appliance
variants did not exist in
appliance-build. UCF needs Delphix's closedappliance configuration and admin console as a base, with the UCF app
layered on top. Standing this up surfaced several build failures:
internal-ucf/external-ucfvariants existed.Delphix's
linux-build-publishIAM user, soaws s3 synchitAccessDeniedusing the ambient credentials.ansible/roles -> ../../../misc/ansible-rolessymlink, so the chrootansible-playbookhook failed to resolveappliance-build.minimal-common.virtualization-development(clonesdlpx-app-gatevia git, which isn'tin the chroot) and
qa-internalare Delphix-product-specific and brokethe UCF build.
masking-commonrole present.
Solution
Add
internal-ucfandexternal-ucfvariants for CDS and the supportingbuild plumbing:
internal-ucf/external-ucf(build.gradle,playbooks,
ansible/rolessymlink).AWS_UCF_ACCESS_KEY_ID/AWS_UCF_SECRET_ACCESS_KEYfor the UCFaws s3 synconly; falls back toambient credentials so DCT/Hyperscale paths are unaffected. Paired with a
devops-gate PR exposing these via the
aws-ucf-buildJenkins credential.minimal-common,minimal-internal,virtualization-common,ucf-common; droppedvirtualization-developmentandqa-internal.ucf-commonrole installs the CDSpuppetcloudopspackage (uploadedto S3 by the CDS team's GitHub Actions, pulled into the apt repo by
build-ancillary-repository.sh).masking-commonrole added — required by the Delphix mgmt JVM startup.Testing Done
internal-ucfvariant locally; chrootansible-playbookhookresolves all roles and completes.
aws s3 syncuses theAWS_UCF_*credentials when set andfalls back to ambient credentials when unset (DCT/Hyperscale paths
unchanged).
check-shellcheckand style checks pass;.ansible-lint-ignorebaselinedfor the new ucf files consistent with every existing variant/role.
Tracking: PUPCLD-3705