Skip to content

Setup initial end-to-end tests, end-to-end testing infrastructure #42

Description

@tmillr

Why?

  • to make sure that action.yml is still valid on and into the future (e.g. its schema may change, the chosen/selected node version may become outdated/invalid, etc.)
  • to make sure the action does not break when the gh runners (or their interface with js actions) change
  • for end-to-end testing, if wanted or needed
  • provides a way to capture/view the action's actual output from an actual/realistic run (e.g. for docs: include screenshot and link to example of posted comments #1)

How?

  • create a scratch branch or scratch repo for pushing test commits to
    • if branch, the ci yaml itself must still be on the default branch
  • choose any events necessary, but also at least run on a schedule (e.g. daily) since it's possible for this action (or its manifest: action.yml) to break even if the pkg/repo itself doesn't actually change
  • observe workflow runs for failures, esp those due to an invalid action.yml. This can also be used as a way to verify that the action.yml is (still) valid before releasing/tagging new versions.
    • can use gh api to retrieve workflow run results
    • can view results manually in browser anywhere workflow run results are displayed (e.g. Actions tab, commit list/history, branch landing page, pr checks), or in user notifications (if configured)
    • maybe auto-open an issue or pr on failure for extra visibility

NOTE: remember that pushing commits from within gh actions/ci will not trigger further events/workflows unless pushed using a custom token/pat

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions