๐ ISMS Alignment: This E2E test plan implements Hack23 Secure Development Policy Section 4.3.3 - End-to-End Testing Requirements.
This End-to-End (E2E) Test Plan provides comprehensive testing coverage for the Black Trigram Korean martial arts combat simulator, ensuring all critical user journeys and game mechanics function correctly across different browsers, screen sizes, and performance scenarios.
Per Hack23 AB's Secure Development Policy, this project maintains:
| ๐ฏ Requirement | ๐ Implementation | โ Status | ๐ ISMS Reference |
|---|---|---|---|
| Critical Path Coverage | Complete game flow testing | โ Implemented | Section 4.3.3.1 |
| Browser Testing | Chrome, Firefox, Edge | โ Validated | Section 4.3.3.2 |
| Automated Execution | Every PR via Cypress | โ Active | Section 4.3.3.3 |
| Public Reporting | Mochawesome reports | โ Published | Section 4.3.3.4 |
| Performance Assertions | Load time & FPS monitoring | โ Tracked | Section 4.3.3.5 |
Evidence Links:
- E2E Test Reports: Available as downloadable CI artifacts from the CI Workflow (see the "Artifacts" section in each workflow run)
- Test Execution Badge
See Also:
- ๐ฅ Validate Critical Game Workflows: Ensure all Korean martial arts gameplay paths work correctly
- ๐ฎ Verify Combat Mechanics: Test trigram system, vital point targeting, and combat calculations
- ๐ Confirm State Management: Validate game state persistence and transitions
- ๐ Cross-Browser Compatibility: Test across Chrome, Firefox, and Edge browsers
- ๐ฑ Responsive Design Validation: Ensure proper functionality on desktop, tablet, and mobile viewports
- โก Performance Verification: Confirm 60fps rendering and acceptable load times
- ๐จ PixiJS Integration: Validate WebGL rendering and canvas interactions
- ๐ต Audio System: Test Korean traditional music and combat sound effects
Test Coverage:
- Complete navigation cycle: Intro โ Training โ Combat โ Intro
- All 8 trigram stances in training mode
- Combat mechanics with movement and techniques
- State management and transitions
- Performance under load
Critical User Journeys:
| Journey | Priority | Test File | Status |
|---|---|---|---|
| Intro โ Training โ Intro | โ High | game-journey.cy.ts | Implemented |
| Intro โ Combat โ Intro | โ High | game-journey.cy.ts | Implemented |
| Training - All Stances | โ High | training-flow.cy.ts | Implemented |
| Combat - Movement | โ High | combat-flow.cy.ts | Implemented |
| Combat - Trigram Techniques | โ High | combat-system-integration.cy.ts | Implemented |
| Combat - Vital Points | โ High | pixi-korean-martial-arts.cy.ts | Implemented |
Combat Mechanics Tested:
- Movement system (WASD + Arrow keys)
- Trigram stance selection (1-8 keys)
- Technique execution (Space bar)
- Vital point targeting (Mouse clicks)
- Damage calculations (VitalPointSystem)
- Health/Ki resource management
- Combat feedback (visual + audio)
- AI opponent behavior
Performance Assertions:
| Metric | Target | Test Method |
|---|---|---|
| Initial Load Time | < 3 seconds | Measure time from visit() to canvas ready |
| Screen Transition | < 500ms | Time between ESC and new screen render |
| Input Response | < 100ms | Key press to visual feedback |
| Combat Action | < 200ms | Technique execution to effect render |
| Frame Rate | 60fps ยฑ5% | Monitor FPS during intense combat |
| Memory Usage | < 512MB | Check heap size after 5 minutes |
Browser Testing Matrix:
| Browser | Version | Platform | WebGL Support | Three.js Compatibility | Priority | Status |
|---|---|---|---|---|---|---|
| Chrome | Latest Stable | Windows/Mac/Linux | โ Excellent (SwiftShader) | โ Excellent | โ High | Tested |
| Firefox | Latest Stable | Windows/Mac/Linux | โ Excellent | โ Excellent | โ High | Tested |
| Edge | Latest Stable | Windows | โ Excellent (Chromium) | โ Excellent | โ High | Tested |
| Safari | Latest Stable | Mac | โ Good | โ Good | Planned |
Chrome Configuration for Three.js (See CHROME_CYPRESS_THREEJS_CONFIG.md for details):
- SwiftShader software rendering for WebGL in headless mode
- Optimized Chrome flags for Three.js performance
- 60fps frame rate capping for consistency
- 4GB Node.js heap for memory-intensive Three.js scenes
Viewport Testing Matrix:
| Device Type | Width x Height | Orientation | Priority |
|---|---|---|---|
| Desktop HD | 1280 x 720 | Landscape | โ High |
| Desktop FHD | 1920 x 1080 | Landscape | โ High |
| Tablet | 768 x 1024 | Portrait | |
| Mobile | 375 x 667 | Portrait | ๐ฑ Low |
Audio Features Tested:
- Traditional Korean music in Training Mode
- Cyberpunk combat music in Combat Mode
- Impact sounds for technique execution
- Bone crack audio for critical hits
- Audio synchronization with visuals
cypress/
โโโ e2e/
โ โโโ game-journey.cy.ts # Complete game flow tests
โ โโโ core-features.cy.ts # Essential gameplay features
โ โโโ combat-system-integration.cy.ts # Combat mechanics
โ โโโ training-system-integration.cy.ts # Training mode
โ โโโ combat-flow.cy.ts # Combat-specific workflows
โ โโโ training-flow.cy.ts # Training-specific workflows
โ โโโ combat-screen-layout.cy.ts # Combat UI validation
โ โโโ combat-mode.cy.ts # Combat mode entry/exit
โ โโโ training-mode.cy.ts # Training mode entry/exit
โ โโโ pixi-korean-martial-arts.cy.ts # PixiJS integration (legacy)
โ โโโ performance-threejs.cy.ts # Three.js FPS and performance
โ โโโ webgl-verification.cy.ts # WebGL context and rendering verification โจ NEW
โ โโโ app.cy.ts # Application smoke tests
โ โโโ game-flow.cy.ts # Navigation flows
โโโ fixtures/
โ โโโ test-data.json # Test data fixtures
โโโ support/
โ โโโ e2e.ts # E2E support file
โ โโโ component.ts # Component support
โ โโโ commands.ts # Custom Cypress commands
โ โโโ performance.ts # Performance monitoring utilities
โ โโโ test-isolation.ts # Test isolation utilities
โ โโโ resource-monitoring.ts # Resource leak detection
Test File: webgl-verification.cy.ts
Comprehensive WebGL and Three.js verification tests to ensure proper rendering in headless CI:
Coverage:
-
WebGL Context Verification
- Verify WebGL/WebGL2 context creation
- Check renderer, version, and capabilities
- Validate texture size and viewport limits
- Test WebGL extensions support
-
Frame Rate Performance
- Measure average FPS (target: 30-60fps)
- Monitor frame timing consistency
- Detect frame drops or stuttering
- Validate software rendering performance
-
Rendering Quality
- Verify active canvas rendering (pixel changes)
- Check for blank/black canvas issues
- Validate visual content is present
- Test canvas state changes
-
Memory and Resource Management
- Detect WebGL context leaks
- Monitor GPU memory usage
- Track memory growth during gameplay
- Verify resource cleanup
-
Mode-Specific Testing
- Combat mode WebGL performance
- Training mode WebGL performance
- Screen transition stability
- Context preservation across scenes
Running WebGL Tests:
# Full WebGL verification
npx cypress run --spec "cypress/e2e/webgl-verification.cy.ts"
# With memory optimization
NODE_OPTIONS="--max-old-space-size=4096" \
npx cypress run --spec "cypress/e2e/webgl-verification.cy.ts"For detailed configuration and troubleshooting, see CHROME_CYPRESS_THREEJS_CONFIG.md.
Available Commands:
cy.visitWithWebGLMock()- Visit page with WebGL mocking for headlesscy.waitForCanvasReady()- Wait for PixiJS canvas initializationcy.gameActions()- Execute game inputs with proper timingcy.annotate()- Add test annotations for reportingcy.enterTrainingMode()- Navigate to training modecy.enterCombatMode()- Navigate to combat modecy.returnToIntro()- Return to intro screency.practiceStance()- Practice specific trigram stance
Test fixtures are defined in cypress/fixtures/test-data.json:
Player Archetypes:
- ๋ฌด์ฌ (Musa) - Traditional Warrior
- ์์ด์ (Amsalja) - Shadow Assassin
- ํด์ปค (Hacker) - Cyber Warrior
- ์ ๋ณด์์ (Jeongbo Yowon) - Intelligence Operative
- ์กฐ์งํญ๋ ฅ๋ฐฐ (Jojik Pokryeokbae) - Organized Crime
Trigram Stances:
- ๊ฑด (Geon) - Heaven
- ํ (Tae) - Lake
- ๋ฆฌ (Li) - Fire
- ์ง (Jin) - Thunder
- ์ (Son) - Wind
- ๊ฐ (Gam) - Water
- ๊ฐ (Gan) - Mountain
- ๊ณค (Gon) - Earth
1. Test Execution:
npm run test:e2e # Local execution with Chrome
npm run test:e2e:ci # CI execution (headless)2. Report Merging:
npm run test:e2ereportmerge # Merge JSON reports
npm run test:e2ereporthtmlall # Generate HTML report3. Report Location (Local):
- Individual JSON:
docs/cypress/mochawesome/*.json - Merged JSON:
docs/cypress/mochawesome-all.json - HTML Report:
docs/cypress/mochawesome/index.html - JUnit XML:
docs/cypress/junit/results-*.xml
4. CI Artifacts:
CI uploads artifacts from the following locations (configured in cypress.config.ts to write to docs/cypress/ but uploaded from standard Cypress paths):
- Videos:
cypress/videos/(failures only) โ uploaded todocs/cypress/videos/ - Screenshots:
cypress/screenshots/(failures only) โ uploaded todocs/cypress/screenshots/ - Results:
cypress/results/โ uploaded todocs/cypress/results/ - Reports: Available as downloadable artifacts from GitHub Actions workflow runs
- โ Test pass/fail summary with percentages
- ๐ Interactive charts showing test distribution
- ๐ผ๏ธ Embedded screenshots for failed tests
- ๐ฌ Video playback for failed test runs
- โฑ๏ธ Execution time for each test suite
- ๐ Detailed error messages and stack traces
E2E Test Job:
e2e-tests:
needs: [prepare, build-validation]
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/setup-node@v6
with:
node-version-file: '.nvmrc'
- name: Install dependencies
run: npm ci
- name: Start app and run Cypress tests
run: |
xvfb-run --auto-servernum --server-args="-screen 0 1280x720x24" npm run test:e2e
- name: Upload Cypress results
if: always()
uses: actions/upload-artifact@v5
with:
name: cypress-results
path: |
cypress/videos
cypress/screenshots
cypress/resultsTrigger Conditions:
- โ
Every push to
mainbranch - โ
Every pull request to
mainbranch - โ๏ธ Manual workflow dispatch (optional)
Environment:
- OS: Ubuntu Latest (Linux)
- Node: from .nvmrc
- Display: Xvfb (virtual framebuffer for headless)
- Resolution: 1280x720x24 bit color depth
- Browser: Chrome (headless)
Dependencies:
- Korean font support: fonts-noto-cjk, fonts-noto-cjk-extra
- System libraries: libgtk-3-0, libgbm-dev, libnss3, etc.
- google-chrome-stable
| Journey | Coverage | Tests | Status |
|---|---|---|---|
| Intro โ Training โ Intro | 100% | game-journey.cy.ts | โ |
| Intro โ Combat โ Intro | 100% | game-journey.cy.ts | โ |
| Training Mode - All Stances | 100% | training-flow.cy.ts | โ |
| Combat - Movement | 100% | combat-flow.cy.ts | โ |
| Combat - Trigram Stances | 100% | combat-system-integration.cy.ts | โ |
| Combat - Vital Points | 100% | pixi-korean-martial-arts.cy.ts | โ |
| Responsive Design | 100% | core-features.cy.ts | โ |
| Cross-Browser | 100% | All tests | โ |
| Metric | Target | Current (Enhanced) | Previous | Status |
|---|---|---|---|---|
| Test Coverage | >90% of user journeys | ~95% | ~95% | โ Excellent |
| Pass Rate | >95% | 100% (10/10) | ~98% | โ Excellent |
| Execution Time | 10-12 minutes | ~10-12 min (optimized) | ~20 min | โ Target Met |
| Browser Compatibility | 100% Chrome/Firefox/Edge | 100% | 100% | โ Complete |
| Performance Compliance | <3s load, <500ms interaction | Measured | Measured | โ Met |
| Flaky Test Rate | 0% | 0% (enhanced monitoring) | <1% | โ Zero Flakes |
| Retry Count | โค1 | 1 (runMode), 0 (openMode) | 2 (runMode), 1 (openMode) | โ Improved |
| Resource Leaks | 0 detected | Monitored (10-30MB growth) | Not monitored | โ Tracked |
| Test Isolation | Complete | โ Implemented | Partial | โ Complete |
Enhanced Reliability Features (Nov 2025):
- โ Test Isolation: Complete state reset between tests (localStorage, sessionStorage, game state)
- โ Resource Monitoring: Tracks audio elements, canvas elements, memory usage, event listeners
- โ Global Cleanup Hooks: Automatic cleanup in beforeEach/afterEach for all tests
- โ Flakiness Detection: CI runs tests 3x on PRs to detect inconsistent behavior
- โ Reliability Reporting: Automated HTML report generation with flaky test detection
- โ Reduced Retries: From 2โ1 (runMode), 1โ0 (openMode) to catch flakes faster
Note: Execution time optimized from ~20 minutes to 10-12 minutes through configuration tuning, wait time reduction, and performance monitoring. Test reliability improvements (Nov 2025) added comprehensive isolation and monitoring to achieve 0% flaky rate. See E2E_OPTIMIZATION_RESULTS.md for details.
| Feature | Unit Tests | E2E Tests | Status |
|---|---|---|---|
| Trigram System | โ TrigramCalculator.test.ts | โ combat-system-integration.cy.ts | Complete |
| Vital Points | โ VitalPointSystem.test.ts | โ pixi-korean-martial-arts.cy.ts | Complete |
| Combat Mechanics | โ GameIntegration.test.ts | โ combat-flow.cy.ts | Complete |
| Player Utils | โ playerUtils.test.ts | โ game-journey.cy.ts | Complete |
| UI Components | โ CombatStatsPanel.test.tsx | โ combat-screen-layout.cy.ts | Complete |
| State Management | โ Integration tests | โ game-journey.cy.ts | Complete |
Test Coverage:
- โ Korean logo rendering (ํ๊ด)
- โ Menu navigation (Training/Combat buttons)
- โ Bilingual text display (Korean/English)
- โ Background visual effects
- โ Audio system initialization
- โ Keyboard navigation (1, 2 keys)
Test Files: app.cy.ts, core-features.cy.ts, game-journey.cy.ts
Test Coverage:
- โ All 8 trigram stance practice
- โ Korean technique names display
- โ Visual technique feedback
- โ Training progression tracking
- โ Return to intro (ESC)
Test Files: training-mode.cy.ts, training-flow.cy.ts, training-system-integration.cy.ts
Test Coverage:
- โ Combat HUD rendering
- โ Health bar display (player + opponent)
- โ Ki resource management
- โ Movement controls (WASD, arrows)
- โ Trigram stance selection (1-8)
- โ Technique execution (Space)
- โ Vital point targeting (Mouse)
- โ Combat log updates
- โ Damage calculations
- โ Visual effects (blood, impacts)
- โ Audio feedback (bone impacts)
Test Files: combat-mode.cy.ts, combat-flow.cy.ts, combat-system-integration.cy.ts, combat-screen-layout.cy.ts, pixi-korean-martial-arts.cy.ts
- ๐ Weekly: Review failing tests and identify flaky tests
- ๐ Bi-weekly: Update test scenarios based on new features
- ๐ Monthly: Comprehensive test coverage analysis
- ๐ฏ Quarterly: Test framework and strategy evaluation
- ๐ Retries: Configured in
cypress.config.tsrunMode: 2- CI environment retriesopenMode: 1- Local development retries
- โฑ๏ธ Timeouts: Optimized for game loading
defaultCommandTimeout: 6000mspageLoadTimeout: 15000ms
- ๐ Memory Management:
experimentalMemoryManagement: true - ๐พ Test Retention:
numTestsKeptInMemory: 5 - ๐ฌ Video Optimization:
videoUploadOnPasses: false(failures only) - ๐ธ Screenshots: Only on failure
- ๐งน Cleanup:
trashAssetsBeforeRuns: true
- ๐ XSS Prevention: Input validation tests
- ๐ก๏ธ CSP Compliance: Content Security Policy validation
- ๐ Dependency Scanning: Automated via npm audit
- ๐ OWASP ZAP: Periodic security scans
- ๐ฏ CodeQL: Security vulnerability scanning
Security Test Files:
- ZAP scan:
.github/workflows/zap-scan.yml - CodeQL:
.github/workflows/codeql.yml - Scorecard:
.github/workflows/scorecards.yml
# Install Cypress
npm run cypress:install
npm run cypress:verify
# Run all E2E tests
npm run test:e2e
# Open Cypress Test Runner (interactive)
npx cypress open
# Run specific test file
npx cypress run --spec "cypress/e2e/game-journey.cy.ts"
# Run tests in specific browser
npx cypress run --browser firefox# Tests run automatically on:
# - Push to main branch
# - Pull requests to main branch
# View test results:
# https://github.com/Hack23/blacktrigram/actionsIssue: Tests fail in headless mode
xvfb-run --auto-servernum --server-args="-screen 0 1280x720x24" npm run test:e2eIssue: Korean fonts not rendering
sudo apt-get install fonts-noto-cjk fonts-noto-cjk-extra| Control | Requirement | Implementation |
|---|---|---|
| A.8.9 | Configuration Management | Version-controlled test plans |
| A.14.2 | Testing in Development | Automated E2E testing |
| A.14.3 | Test Data Protection | Sanitized test data only |
| Function | Category | Implementation |
|---|---|---|
| PR.IP-1 | Baseline Configuration | Cypress config management |
| PR.IP-2 | SDLC Integration | E2E tests in CI/CD |
| DE.CM-4 | Malicious Code Detection | Dependency scanning |
| Control | Description | Implementation |
|---|---|---|
| 2.1 | Software Inventory | Package.json tracking |
| 16.9 | Automated Testing | E2E test automation |
| 16.10 | Test Environment Segregation | Isolated test runs |
- Cypress Documentation
- Mochawesome Reporter
- Three.js Documentation โจ NEW
- WebGL Specification โจ NEW
- Chrome + Cypress + Three.js Configuration โจ NEW
- Black Trigram Architecture
- Unit Test Plan
- Business Continuity Plan
- ISMS Integration Analysis
- Security Architecture
- Threat Model
- E2E Optimization Results - Performance optimization details
- E2E Execution Strategy - Smoke tests and CI strategy
The E2E test suite has been optimized to reduce execution time from ~20 minutes to the target of 10-12 minutes while maintaining full test coverage and reliability.
Key Optimizations:
-
Configuration Tuning (2-3 min savings)
- Video compression optimized for faster encoding
- Memory management improved
- Timeout settings reduced appropriately
-
Code Optimization (3-4 min savings)
- Fixed wait times replaced with assertions
- Command delays reduced
- Test transitions optimized
-
Performance Monitoring (observability)
- Per-test timing metrics
- Slowest test identification
- CI duration reporting
Test Suite Options:
- Smoke Tests (
npm run test:e2e:smoke): 5-6 minutes - Critical path validation - Full Suite (
npm run test:e2e): 10-12 minutes - Comprehensive testing - Reliability Check (
npm run test:reliability): <1 minute - Analyze flaky tests
For detailed optimization analysis and metrics, see E2E_OPTIMIZATION_RESULTS.md.
Global Cleanup Hooks (cypress/support/e2e.ts):
beforeEach(() => {
// Clear browser storage
cy.clearLocalStorage();
cy.clearCookies();
// Reset viewport
cy.viewport(1280, 720);
// Clear game state
// Start resource monitoring
cy.startResourceMonitoring();
});
afterEach(() => {
// Detect resource leaks
cy.detectResourceLeaks();
// Log test metrics
// Force cleanup resources
});Test Isolation Utilities (cypress/support/test-isolation.ts):
cy.isolateTest()- Reset to clean state before testcy.cleanupTest()- Clean up after test completioncy.captureState()- Capture current state for restorationcy.restoreState()- Restore previously captured state- State management: localStorage, sessionStorage, game state, timers, event listeners
Resource Monitoring (cypress/support/resource-monitoring.ts):
cy.startResourceMonitoring()- Begin tracking resourcescy.detectResourceLeaks()- Check for leaks after testcy.logResourceReport()- Log detailed resource usagecy.forceResourceCleanup()- Force cleanup all resources- Tracks: audio elements, canvas elements, memory usage, event listeners
Pull Request Testing (.github/workflows/test-and-report.yml):
- Runs smoke tests 3 times on every PR
- Detects inconsistent behavior (0 < failures < 3 = flaky)
- Reports consistent failures (all 3 runs fail)
- Uploads results as artifacts for analysis
Reliability Reporting (scripts/generate-reliability-report.cjs):
npm run test:reliabilityGenerates:
- HTML report with metrics and flaky test details
- Console summary with pass rates and failures
- Identifies flaky tests with pass rate percentages
- Lists consistently failing tests
- Calculates average test duration
Cypress Retries (cypress.config.ts):
runMode: Reduced from 2 to 1 (better flaky detection)openMode: Reduced from 1 to 0 (immediate feedback)- Encourages fixing tests rather than masking flakiness
Timeout Settings (optimized for reliability):
defaultCommandTimeout: 5000ms (fast failure detection)pageLoadTimeout: 12000ms (sufficient for app load)requestTimeout: 6000ms (reasonable for API calls)responseTimeout: 6000ms (matches request timeout)
Using Test Isolation:
describe('My Feature Tests', () => {
beforeEach(() => {
cy.isolateTest(); // Optional: explicit isolation
cy.visit('/');
});
afterEach(() => {
cy.cleanupTest(); // Optional: explicit cleanup
});
it('should work reliably', () => {
// Test implementation
// Global hooks handle most cleanup automatically
});
});Resource Monitoring:
it('should not leak resources', () => {
cy.startResourceMonitoring(); // Optional: already in global hook
// Perform test actions
cy.get('[data-testid="button"]').click();
cy.detectResourceLeaks(); // Optional: already in global hook
// Logs warnings if leaks detected
});Checking for Flakiness Locally:
# Run tests multiple times
for i in {1..5}; do npm run test:e2e:smoke; done
# Generate reliability report
npm run test:reliabilityCurrent Status (as of Nov 2025):
- โ 0% Flaky Test Rate (target achieved)
- โ 100% Pass Rate (10/10 smoke tests)
- โ Resource Monitoring Active (detects 10-30MB memory growth)
- โ Complete Test Isolation (state reset between tests)
- โ Reduced Retries (1 or 0, down from 2)
- โ CI Flakiness Detection (3x runs on PRs)
Before Improvements:
- Flaky Test Rate: <1% (acceptable but not ideal)
- Retries: 2 (could mask flaky tests)
- Resource Leaks: Not monitored
- Test Isolation: Partial
After Improvements:
- Flaky Test Rate: 0% (zero tolerance)
- Retries: 1 (runMode), 0 (openMode)
- Resource Leaks: Monitored and logged
- Test Isolation: Complete with global hooks
๐ Document Metadata:
โ
Approved by: Development Team
๐ค Distribution: Development Team, QA Team, Security Team
๐ Review Cycle: Monthly
โฐ Last Updated: 2025-11-17
๐ Version: 1.2.0 (Test Reliability Enhanced)
๐ค Maintained by: Test Specialist Agent
๐ฅ ํ๊ด์ ํ์ง์ ์งํค๋ผ - Protect the Quality of Black Trigram