Skip to content

feat(server): copy job YAML, Job YAML tab, syntax-highlighted job definition#1147

Open
mz2 wants to merge 4 commits into
mainfrom
feat/dashboard-copy-job-yaml
Open

feat(server): copy job YAML, Job YAML tab, syntax-highlighted job definition#1147
mz2 wants to merge 4 commits into
mainfrom
feat/dashboard-copy-job-yaml

Conversation

@mz2

@mz2 mz2 commented Jun 12, 2026

Copy link
Copy Markdown
Collaborator

Overview

Adds a "Copy job YAML" button and a clearer job-definition view to the dashboard job detail page. The stored job_data is reconstructed into a submittable job-definition YAML (the Job schema top-level fields, in schema order), shown in a default Job YAML tab and copyable to the clipboard in one click.

What changed

New: Job YAML tab + Copy button

  • Server-side YAML rendering (views.py): build_job_yaml() rebuilds a submittable definition from the stored job_data, with multiline strings (e.g. the test script) rendered as literal | blocks.
  • A new Job YAML tab (the full submittable definition) is shown by default.
  • Copy button (clipboard.js): a generic [data-copy-target] handler copies the clean, unhighlighted YAML from a hidden payload and shows brief "Copied!" feedback.

Changed: how the existing Provision / Test tabs are presented

These tabs already existed but rendered their content awkwardly. This PR fixes the presentation:

  • Provision Data previously rendered the raw Python dict repr — e.g. {'distro': 'jammy'} — because the template emitted {{ job.job_data.provision_data }} directly. It now renders as YAML (distro: jammy) via a new as_yaml Jinja filter. Rationale: the underlying job definition is YAML, so showing a Python repr was both inconsistent with how jobs are authored and harder to read for nested data; rendering YAML also matches the new Copy-job-YAML output exactly.
  • Test Data → "Test Script": this tab only ever shows test_data.test_cmds, which is a shell script — not a generic data structure. The tab was renamed from "Test Data" to "Test Script" to say what it actually is, and the script is now bash syntax-highlighted. Rationale: "Test Data" was misleading (it implied the whole test_data mapping); "Test Script" is accurate, and highlighting a multi-line shell script makes it far easier to scan.

Syntax highlighting

  • Highlighting is done server-side with Pygments (no client-side JS): YAML for the definition sections, bash for the test script. A small generated pygments.css is loaded only on this page. Output is XSS-safe — Pygments HTML-escapes the content.

Local-dev conveniences

  • docker-compose.yml bind-mounts src/ and sets TEMPLATES_AUTO_RELOAD (env-guarded, default off, in application.py) for hot-reload without rebuilds; create_sample_data.py seeds representative provision_data.

Screenshots

Job YAML tab (new, shown by default) — full submittable definition, syntax-highlighted, with the Copy button on the tab row:

testflinger-copy-job-yaml

Provision Data tab — now rendered as YAML (distro: jammy) instead of a Python repr:

testflinger-provision-tab

Test Script tab — renamed from "Test Data" (it always was including just the test_cmds), with bash syntax highlighting:

testflinger-test-tab

Testing

  • Unit tests (tests/test_views.py): build_job_yaml, the as_yaml and highlight filters (including HTML-escaping), the copy button/payload, and that sections render as YAML rather than a Python repr.
  • Browser end-to-end coverage (Playwright) drives the real copy interaction and verifies the clipboard contents match the submittable YAML.

No API changes, database migrations, or new endpoints. TEMPLATES_AUTO_RELOAD is development-only and defaults off.

mz2 added 2 commits June 10, 2026 14:21
Add a button on the job detail page that copies a submittable job-definition
YAML (job_queue, provision_data, firmware_update_data, test_data, etc.) for the
job being viewed, reconstructed via build_job_yaml() with multiline values
rendered as YAML literal blocks. A small generic clipboard.js handles the copy.

Also add local-development conveniences for testing the change:
- bind-mount src/ in docker-compose and an env-guarded TEMPLATES_AUTO_RELOAD so
  view/template/static edits are picked up without rebuilding the image
- include provision_data in create_sample_data.py so seeded jobs produce a
  representative copied YAML
Place the button on the same line as the Provision Data / Test Data
tabs (right-aligned) instead of next to the Job Definition heading.
@codecov

codecov Bot commented Jun 12, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 75.82%. Comparing base (099dd1f) to head (545e0f6).
⚠️ Report is 9 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1147      +/-   ##
==========================================
- Coverage   77.90%   75.82%   -2.08%     
==========================================
  Files         118       99      -19     
  Lines       12350    10499    -1851     
  Branches     1018      841     -177     
==========================================
- Hits         9621     7961    -1660     
+ Misses       2508     2366     -142     
+ Partials      221      172      -49     
Flag Coverage Δ *Carryforward flag
agent 75.78% <ø> (ø) Carriedforward from 86bfdf0
cli 91.85% <ø> (ø) Carriedforward from 86bfdf0
device 63.76% <ø> (ø) Carriedforward from 86bfdf0
server 84.36% <ø> (-4.63%) ⬇️

*This pull request uses carry forward flags. Click here to find out more.

Components Coverage Δ
Agent 75.78% <ø> (ø)
CLI 91.85% <ø> (ø)
Common ∅ <ø> (∅)
Device Connectors 63.76% <ø> (ø)
Server 84.36% <ø> (-4.63%) ⬇️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@mz2 mz2 marked this pull request as ready for review June 12, 2026 09:14
…yntax

Render the Provision/Firmware sections of the job detail page as YAML
instead of a Python dict repr, add a "Job YAML" tab (the full submittable
definition) shown by default, and syntax-highlight the YAML and the
test_cmds script.

Highlighting is done server-side with Pygments (a real, lockfile-tracked
dependency) rather than a vendored JS bundle, so it needs no client-side
JavaScript and works in offline/air-gapped deployments. A small generated
Pygments stylesheet is loaded only on the job detail page.

The "Copy job YAML" button keeps copying the clean, unhighlighted YAML
from a dedicated hidden payload, so the clipboard output is unaffected.
@mz2 mz2 force-pushed the feat/dashboard-copy-job-yaml branch from 826bb42 to 86bfdf0 Compare June 12, 2026 09:22

@ajzobro ajzobro left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The page looks great!

Comment thread server/src/testflinger/static/assets/js/clipboard.js
</div>
{% endif %}
</div>
<button class="p-button--base has-icon"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With respect to the the button, I think it's pushing the tab area over to the left and leaving a large unused space below it.

Image

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I think its because the margin-bottom in CSS:

I was playing around and setting it to 0rem seems to at least not display that white border and it seems more inline with the tabs

Comment thread server/pyproject.toml Outdated

@rene-oromtz rene-oromtz left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really like how its getting displayed in the job view! Thanks for taking care of this! I just added a comment regarding pygments that might as well do it now given we are explicitly including it as dependency.

Co-authored-by: rene-oromtz <157750458+rene-oromtz@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants