Avoid CLI invocation overhead, use runtime cache invalidation for GRAPHQL WebAPI tests

[ilnytskyi]

Description (*)

• removed CLI overhead of exec("php -f {$appDir}/bin/magento cache:flush full_page", $out);
• moved cache invalidation into runtime approach

        $objectManager
            ->get(\Magento\Framework\App\Cache\TypeListInterface::class)
            ->cleanType(\Magento\PageCache\Model\Cache\Type::TYPE_IDENTIFIER);

Tested on dev/tests/api-functional/testsuite/Magento/GraphQl/Customer testsuite locally, shoved about 15% speed gain

Scenario	Result	Wall time
Current CLI flush	246 tests, 8 failures, 3 skipped	325.173s
ObjectManager cleanType('full_page') only	246 tests, 8 failures, 3 skipped	274.681s

Instrumentation the tearDown method shoved that using in-process object manager was about 100x faster that CLI command invocation for customer testsute.

CLI flush total: 65.036137s
OM clean total:   0.651645s
difference:      64.384492s

Related Pull Requests

Fixed Issues (if relevant)

1. Fixes Extremely slow WebAPI GraphQL functional tests #39725

Manual testing scenarios (*)

1. run graphql API functional tests
2. ...

Questions or comments

• we still need to confirm it improves tests performance
• Let's compare it against PRs where that testsuite is passed:

1. 2h 5m and 29m for GRAPHQL WebAPI tests for CE
   https://github.com/magento/magento2/pull/40787/checks?check_run_id=73193963114
   https://public-results-storage-prod.magento-testing-service.engineering/reports/magento/magento2/pull/40787/7cd9f3f1af8dbf934c3a44a94d5c28af/WebApi/allure-report-ce-WebAPI-GRAPHQL/index.html

2. 1h 46m total and 28m for GRAPHQL WebAPI tests for CE
   https://github.com/magento/magento2/pull/40750/checks?check_run_id=72657132676
   https://public-results-storage-prod.magento-testing-service.engineering/reports/magento/magento2/pull/40750/bc5e4db104623e15f0d5c7d739f3a2ee/WebApi/allure-report-ce-WebAPI-GRAPHQL/index.html

3. 1h 49m total and 32m for GRAPHQL WebAPI tests for CE
   https://github.com/magento/magento2/pull/40742/checks?check_run_id=74627364632
   https://public-results-storage-prod.magento-testing-service.engineering/reports/magento/magento2/pull/40742/96df43391c473e82259d6b7225a47194/WebApi/allure-report-ce-WebAPI-GRAPHQL/index.html

4. 2h 16m total and 25m for GRAPHQL WebAPI tests for CE
   https://github.com/magento/magento2/pull/40685/checks?check_run_id=71209955812
   https://public-results-storage-prod.magento-testing-service.engineering/reports/magento/magento2/pull/40685/69ed0cddb7d8662d651a944e55d7b9fa/WebApi/allure-report-ce-WebAPI-GRAPHQL/index.html

• so about 25-30 min can be set as baseline for that testsute based on 4 other PRs where that testsuite passed

Contribution checklist (*)

• Pull request has a meaningful description of its purpose
• All commits are accompanied by meaningful commit messages
• All new or changed code is covered with unit/integration tests (if applicable)
• README.md files for modified modules are updated and included in the pull request if any README.md predefined sections require an update
• All automated tests passed successfully (all builds are green)

[m2-assistant]

Hi @ilnytskyi. Thank you for your contribution!
Here are some useful tips on how you can test your changes using Magento test environment.
❗ Automated tests can be triggered manually with an appropriate comment:

• @magento run all tests - run or re-run all required tests against the PR changes
• @magento run <test-build(s)> - run or re-run specific test build(s)
  For example: @magento run Unit Tests

<test-build(s)> is a comma-separated list of build names.

Allowed build names are:

1. Database Compare
2. Functional Tests CE
3. Functional Tests EE
4. Functional Tests B2B
5. Integration Tests
6. Magento Health Index
7. Sample Data Tests CE
8. Sample Data Tests EE
9. Sample Data Tests B2B
10. Static Tests
11. Unit Tests
12. WebAPI Tests
13. Semantic Version Checker

You can find more information about the builds here
ℹ️ Run only required test builds during development. Run all test builds before sending your pull request for review.

---

For more details, review the Code Contributions documentation.
Join Magento Community Engineering Slack and ask your questions in #github channel.

[ct-prd-pr-scan]

The security team has been informed about this pull request due to the presence of risky security keywords. For security vulnerability reports, please visit Adobe's vulnerability disclosure program on HackerOne or email psirt@adobe.com.

[ilnytskyi]

@magento run all tests

UPD: 27 minutes

[ilnytskyi]

@magento run WebAPI Tests

UPD: 26m 51s

[ilnytskyi]

Well probably due to how this repo runs its CI the total time of tests might not be the best metric.

However when we compare individual runs

other PR:
https://public-results-storage-prod.magento-testing-service.engineering/reports/magento/magento2/pull/40787/7cd9f3f1af8dbf934c3a44a94d5c28af/WebApi/allure-report-ce-WebAPI-GRAPHQL/index.html#suites

this PR:
https://public-results-storage-prod.magento-testing-service.engineering/reports/magento/magento2/pull/40804/c632f1c45a00ae4ae99ee3b72214b706/WebApi/allure-report-ce-WebAPI-GRAPHQL/index.html#suites

we can see this change easily erases 500-600ms form every API test.

Many tests finish under 1 second.

So the diff shows that about 800 tests finish under a second. Looks like impact is real.