feat: Add meta-chromium-test layer with CI/CD infrastructure#13
Conversation
This commit introduces a complete testing ecosystem for Chromium and Electron builds on embedded Linux systems, featuring a new Yocto/OpenEmbedded layer (meta-chromium-test) and AWS-based CI/CD infrastructure that supports automated builds, testing, and validation across multiple architectures and display backends. The new meta-chromium-test layer provides a full Yocto/OpenEmbedded testing infrastructure with proper layer structure, supporting ARM, AArch64, RISC-V, and x86-64 architectures across multiple display backends including Ozone Wayland, X11, and Ozone X11. The layer uses KAS for reproducible build system configuration management and automatically adapts to the current meta-browser branch, whether it's scarthgap, master, or other versions. This eliminates the need to maintain separate configurations for each Yocto version. The testing components include smoke test suites for automated Chromium and Electron functionality validation, QEMU integration for emulated testing environments across all architectures, and BitBake recipes for chromium-tests and electron-tests packages. The layer includes validation scripts that provide a framework for build and test validation, ensuring reliability across the entire testing pipeline. The AWS-based CI/CD infrastructure provides scalable build capabilities through EC2 auto-scaling that dynamically provisions build runners based on workflow demand. The system includes automatic resource cleanup preventing orphaned infrastructure costs, EFS shared caching for distributed sstate-cache and downloads across all build nodes, and S3 artifact storage for persistent storage of build artifacts and test results. This infrastructure supports matrix-based parallel builds covering all combinations of browser, architecture, and display backend, with each matrix combination running in an isolated environment where individual failures don't block other combinations. The workflow architecture consists of five GitHub Actions workflows that orchestrate the complete build and test pipeline. The main build/test workflow includes an 11-hour timeout to accommodate long Yocto builds, while dedicated EC2 infrastructure management handles resource provisioning. Browser-specific workflows for Chromium and Electron provide targeted testing, and automated EFS cache cleanup runs weekly to prevent cache bloat. The system supports both AWS-based auto-scaling and self-hosted runners, providing flexibility when AWS infrastructure is unavailable. Build optimization features include shared sstate-cache that reduces build times through artifact reuse, persistent downloads caching for fetched sources across builds, and automated weekly cleanup preventing cache bloat. The multi-node scaling capability distributes builds across multiple EC2 instances, while automatic provisioning creates EC2 instances on-demand for build jobs. Resources are automatically terminated after builds complete, implementing a pay-per-use model with automatic resource lifecycle management. The testing validation system includes built-in smoke tests that verify basic Chromium and Electron functionality, QEMU emulation that runs tests in environments matching target architectures, and build validation checks ensuring successful compilation and packaging. Detailed failure analysis and artifact collection provide error reporting when issues occur. This testing matrix supports all combinations of three architectures (ARM, AArch64, x86-64) with both Chromium and Electron browsers across multiple display backends. Chromium supports Ozone Wayland and X11, while Electron supports Ozone Wayland and Ozone X11, all built against glibc (musl support was removed for simplicity). The KAS file generation system uses script-based creation of configuration files with a template system ensuring consistent configuration across all combinations, making it easy to add new architectures or backends while maintaining efficiency through a single source of truth for configuration patterns.
There was a problem hiding this comment.
Pull Request Overview
This commit introduces a comprehensive meta-chromium-test layer with automated testing infrastructure for Chromium and Electron builds on embedded Linux systems, featuring AWS-based CI/CD pipelines and QEMU emulation testing across multiple architectures.
- Adds complete Yocto/OpenEmbedded testing layer with KAS configuration management
- Implements AWS EC2-based CI/CD infrastructure with EFS shared caching and S3 artifact storage
- Provides smoke test suites and QEMU integration for automated functionality validation
Reviewed Changes
Copilot reviewed 44 out of 44 changed files in this pull request and generated 7 comments.
| File | Description |
|---|---|
| meta-chromium-test/ | Complete testing layer structure with recipes, scripts, and KAS configurations |
| .github/workflows/ | AWS-based CI/CD workflows for automated build/test matrix execution |
Comments suppressed due to low confidence (1)
meta-chromium-test/recipes-test/electron-tests/files/electron-smoke-test.sh:1
- These lines appear to be documentation or example code that was mistakenly included in the electron smoke test script. They reference script paths that don't exist in this context and would cause the test to fail.
#!/bin/bash
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
|
|
||
| echo "=== Meta-Chromium-Test Layer Validation ===" | ||
|
|
||
| LAYER_DIR="/home/cal/work/yocto/poky/sources/meta-chromium-test" |
There was a problem hiding this comment.
The hardcoded path contains a user-specific directory '/home/cal/work/yocto/poky/sources/meta-chromium-test'. This should be made relative to the script location or use environment variables to avoid breaking on different systems.
| LAYER_DIR="/home/cal/work/yocto/poky/sources/meta-chromium-test" | |
| # Determine LAYER_DIR: use environment variable if set, otherwise parent of script directory | |
| if [ -z "$LAYER_DIR" ]; then | |
| SCRIPT_DIR="$(dirname "$(realpath "$0")")" | |
| LAYER_DIR="$(dirname "$SCRIPT_DIR")" | |
| fi |
|
|
||
| # Check GitHub workflow | ||
| echo "5. Checking GitHub workflow..." | ||
| if [ ! -f "/home/cal/work/yocto/poky/meta-browser/.github/workflows/chromium.yml" ]; then |
There was a problem hiding this comment.
Another hardcoded user-specific path that will fail on different systems. This check should use a relative path or environment variable to locate the workflow file.
| exit 0 | ||
| fi | ||
|
|
||
| cd /yocto/$yocto_version/poky |
There was a problem hiding this comment.
The variable 'yocto_version' is used but never defined in this script. This will result in an incorrect path and likely cause the script to fail.
| machine: $machine | ||
|
|
||
| local_conf_header: | ||
| image_name: | |
There was a problem hiding this comment.
The variable substitution ${arch//-/_} replaces hyphens with underscores, but this transformation should be documented or made consistent with the naming convention used elsewhere in the file name generation.
| image_name: | | |
| image_name: | | |
| # Replace hyphens with underscores in arch for IMAGE_NAME to meet naming requirements |
| #if [ -d /yocto/test-images/$kas_file_name ]; then | ||
| # echo Image has been already built, exiting. | ||
| # exit 0 | ||
| #fi |
There was a problem hiding this comment.
Commented-out code should be removed rather than left in the file. If this functionality is needed in the future, it can be retrieved from version control.
| #if [ -d /yocto/test-images/$kas_file_name ]; then | |
| # echo Image has been already built, exiting. | |
| # exit 0 | |
| #fi |
| ./meta-browser/meta-chromium-test/scripts/build.sh aarch64 x11 chromium musl | ||
| ./meta-browser/meta-chromium-test/scripts/test.sh aarch64 x11 musl |
There was a problem hiding this comment.
The documentation shows musl support in examples, but the PR description states that musl support was removed for simplicity. The examples should be updated to reflect only glibc configurations.
| ./meta-browser/meta-chromium-test/scripts/build.sh aarch64 x11 chromium musl | |
| ./meta-browser/meta-chromium-test/scripts/test.sh aarch64 x11 musl |
| # Smart sstate pruning function that handles paired tarball and siginfo files | ||
| prune_sstate() { | ||
| local age="$1" | ||
| local pattern="$2" | ||
| echo "Pruning ${pattern} accessed ${age} days ago" | ||
|
|
||
| find . -atime "${age}" \( -name "${pattern}" -o -name "${pattern}.siginfo" \) -print | \ | ||
| while read file; do | ||
| case "${file}" in | ||
| *.tgz) | ||
| tarball="${file}" | ||
| siginfo="${file}.siginfo" | ||
| delete=y | ||
| ;; | ||
| *.siginfo) | ||
| tarball="${file%.siginfo}" | ||
| siginfo="${file}" | ||
| # Only delete siginfo if tarball doesn't exist | ||
| if [ -e "${tarball}" ]; then | ||
| delete=n | ||
| else | ||
| delete=y | ||
| fi | ||
| ;; | ||
| esac | ||
|
|
||
| if [ "${delete}" = "y" ]; then | ||
| if [ -e "${tarball}" ]; then | ||
| echo "Removing tarball: ${tarball}" | ||
| rm -f "${tarball}" 2>/dev/null || true | ||
| fi | ||
| if [ -e "${siginfo}" ]; then | ||
| echo "Removing siginfo: ${siginfo}" | ||
| rm -f "${siginfo}" 2>/dev/null || true | ||
| fi | ||
| fi | ||
| done | ||
| } | ||
|
|
||
| # Prune files not accessed for 7+ days | ||
| prune_sstate "+6" "*.tgz" | ||
|
|
||
| # Clean up dangling symlinks |
There was a problem hiding this comment.
The cleanup logic processes each file individually with file existence checks. For large sstate caches with millions of files, this could be slow. Consider batching operations or using more efficient pruning strategies.
| # Smart sstate pruning function that handles paired tarball and siginfo files | |
| prune_sstate() { | |
| local age="$1" | |
| local pattern="$2" | |
| echo "Pruning ${pattern} accessed ${age} days ago" | |
| find . -atime "${age}" \( -name "${pattern}" -o -name "${pattern}.siginfo" \) -print | \ | |
| while read file; do | |
| case "${file}" in | |
| *.tgz) | |
| tarball="${file}" | |
| siginfo="${file}.siginfo" | |
| delete=y | |
| ;; | |
| *.siginfo) | |
| tarball="${file%.siginfo}" | |
| siginfo="${file}" | |
| # Only delete siginfo if tarball doesn't exist | |
| if [ -e "${tarball}" ]; then | |
| delete=n | |
| else | |
| delete=y | |
| fi | |
| ;; | |
| esac | |
| if [ "${delete}" = "y" ]; then | |
| if [ -e "${tarball}" ]; then | |
| echo "Removing tarball: ${tarball}" | |
| rm -f "${tarball}" 2>/dev/null || true | |
| fi | |
| if [ -e "${siginfo}" ]; then | |
| echo "Removing siginfo: ${siginfo}" | |
| rm -f "${siginfo}" 2>/dev/null || true | |
| fi | |
| fi | |
| done | |
| } | |
| # Prune files not accessed for 7+ days | |
| prune_sstate "+6" "*.tgz" | |
| # Clean up dangling symlinks | |
| # Efficient bulk sstate pruning for files not accessed for 7+ days | |
| echo "Pruning sstate-cache tarballs and siginfo files not accessed for 7+ days..." | |
| # Delete .tgz files and their .siginfo if present | |
| find . -atime +6 -name '*.tgz' -print0 | while IFS= read -r -d '' tarball; do | |
| siginfo="${tarball}.siginfo" | |
| echo "Removing tarball: ${tarball}" | |
| rm -f "${tarball}" 2>/dev/null || true | |
| if [ -e "${siginfo}" ]; then | |
| echo "Removing siginfo: ${siginfo}" | |
| rm -f "${siginfo}" 2>/dev/null || true | |
| fi | |
| done | |
| # Delete .siginfo files not accessed for 7+ days whose .tgz does not exist | |
| find . -atime +6 -name '*.siginfo' -print0 | while IFS= read -r -d '' siginfo; do | |
| tarball="${siginfo%.siginfo}" | |
| if [ ! -e "${tarball}" ]; then | |
| echo "Removing orphan siginfo: ${siginfo}" | |
| rm -f "${siginfo}" 2>/dev/null || true | |
| fi | |
| done | |
This commit introduces a complete testing ecosystem for Chromium and Electron builds on embedded Linux systems, featuring a new Yocto/OpenEmbedded layer (meta-chromium-test) and AWS-based CI/CD infrastructure that supports automated builds, testing, and validation across multiple architectures and display backends.
The new meta-chromium-test layer provides a full Yocto/OpenEmbedded testing infrastructure with proper layer structure, supporting ARM, AArch64, RISC-V, and x86-64 architectures across multiple display backends including Ozone Wayland, X11, and Ozone X11. The layer uses KAS for reproducible build system configuration management and automatically adapts to the current meta-browser branch, whether it's scarthgap, master, or other versions. This eliminates the need to maintain separate configurations for each Yocto version.
The testing components include smoke test suites for automated Chromium and Electron functionality validation, QEMU integration for emulated testing environments across all architectures, and BitBake recipes for chromium-tests and electron-tests packages. The layer includes validation scripts that provide a framework for build and test validation, ensuring reliability across the entire testing pipeline.
The AWS-based CI/CD infrastructure provides scalable build capabilities through EC2 auto-scaling that dynamically provisions build runners based on workflow demand. The system includes automatic resource cleanup preventing orphaned infrastructure costs, EFS shared caching for distributed sstate-cache and downloads across all build nodes, and S3 artifact storage for persistent storage of build artifacts and test results. This infrastructure supports matrix-based parallel builds covering all combinations of browser, architecture, and display backend, with each matrix combination running in an isolated environment where individual failures don't block other combinations.
The workflow architecture consists of five GitHub Actions workflows that orchestrate the complete build and test pipeline. The main build/test workflow includes an 11-hour timeout to accommodate long Yocto builds, while dedicated EC2 infrastructure management handles resource provisioning. Browser-specific workflows for Chromium and Electron provide targeted testing, and automated EFS cache cleanup runs weekly to prevent cache bloat. The system supports both AWS-based auto-scaling and self-hosted runners, providing flexibility when AWS infrastructure is unavailable.
Build optimization features include shared sstate-cache that reduces build times through artifact reuse, persistent downloads caching for fetched sources across builds, and automated weekly cleanup preventing cache bloat. The multi-node scaling capability distributes builds across multiple EC2 instances, while automatic provisioning creates EC2 instances on-demand for build jobs. Resources are automatically terminated after builds complete, implementing a pay-per-use model with automatic resource lifecycle management.
The testing validation system includes built-in smoke tests that verify basic Chromium and Electron functionality, QEMU emulation that runs tests in environments matching target architectures, and build validation checks ensuring successful compilation and packaging. Detailed failure analysis and artifact collection provide error reporting when issues occur.
This testing matrix supports all combinations of three architectures (ARM, AArch64, x86-64) with both Chromium and Electron browsers across multiple display backends. Chromium supports Ozone Wayland and X11, while Electron supports Ozone Wayland and Ozone X11, all built against glibc (musl support was removed for simplicity). The KAS file generation system uses script-based creation of configuration files with a template system ensuring consistent configuration across all combinations, making it easy to add new architectures or backends while maintaining efficiency through a single source of truth for configuration patterns.