Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 2 additions & 5 deletions .github/scripts/end2end/setup-e2e-env.sh
Original file line number Diff line number Diff line change
Expand Up @@ -126,10 +126,8 @@ export RING_S3C_INGESTION_NON_VERSIONED_OBJECT_COUNT_PER_TYPE
export CRR_SOURCE_LOCATION_NAME CRR_DESTINATION_LOCATION_NAME CRR_ROLE_NAME
export CRR_LOCATION_A_NAME CRR_LOCATION_B_NAME CRR_LOCATION_C_NAME
export CRR_INFO_A CRR_INFO_B CRR_INFO_C
export MOCHA_FILE=${MOCHA_FILE:-_reports/test-results-[hash].xml}

# Ensure test results dir exists for Mocha JUnit reporter
mkdir -p "$(dirname "$MOCHA_FILE")"
# Ensure test results dir exists for Mocha xunit reporter

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need that (mocha don't do it directly?) ? Why this change ?

mkdir -p tests/functional/_reports

# --- 8. TLS CA cert for ingress endpoints ---
ZENKO_CA_CERT_FILE="$(mktemp /tmp/zenko-ca-cert-XXXXXX.pem)"
Expand Down Expand Up @@ -405,7 +403,6 @@ if [ -n "${GITHUB_ENV:-}" ]; then # Don't do it for Codespace
echo "VAULT_AUTH_HOST=$VAULT_AUTH_HOST" >> "$GITHUB_ENV"
echo "KAFKA_CONNECT_URL=$KAFKA_CONNECT_URL" >> "$GITHUB_ENV"
echo "NODE_EXTRA_CA_CERTS=$NODE_EXTRA_CA_CERTS" >> "$GITHUB_ENV"
echo "MOCHA_FILE=$MOCHA_FILE" >> "$GITHUB_ENV"
echo "VERIFY_CERTIFICATES=$VERIFY_CERTIFICATES" >> "$GITHUB_ENV"
echo "ENABLE_RING_TESTS=$ENABLE_RING_TESTS" >> "$GITHUB_ENV"
echo "AWS_ACCESS_KEY=$AWS_ACCESS_KEY" >> "$GITHUB_ENV"
Expand Down
7 changes: 5 additions & 2 deletions .github/scripts/merge-junit-reports.py
Original file line number Diff line number Diff line change
Expand Up @@ -111,8 +111,11 @@ def merge_reports(output_file, input_files):
all_testsuites.extend(testsuites)

except ET.ParseError as e:

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not related to the pr but I just hit this bug :

4 CTST rerun, one of them I cancelled manually : End up with a file thats not parsable and it fails the whole archive step, forcing us to rerun the whole ci.

Possible further improvement that I left : Only continue on parse error when its not the latest run file

print(f"::error::Error parsing {file}: {e}", file=sys.stderr)
sys.exit(1)
# A report can be empty or truncated when its run was cancelled
# before the test runner finished writing it. Skip it rather than
# aborting the whole merge, since later attempts are still valid.
print(f"::warning::Skipping malformed report {file}: {e}", file=sys.stderr)
continue
except FileNotFoundError:
print(f"::warning::File not found: {file}", file=sys.stderr)
continue
Expand Down
8 changes: 4 additions & 4 deletions .github/workflows/end2end.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -118,8 +118,6 @@ env:
TMATE_SERVER_PORT: ${{ secrets.TMATE_SERVER_PORT }}
TMATE_SERVER_RSA_FINGERPRINT: ${{ secrets.TMATE_SERVER_RSA_FINGERPRINT }}
TMATE_SERVER_ED25519_FINGERPRINT: ${{ secrets.TMATE_SERVER_ED25519_FINGERPRINT }}
# Mocha reporter configuration
MOCHA_FILE: ${{ github.workspace }}/_reports/test-results-[hash].xml

jobs:
check-dashboards:
Expand Down Expand Up @@ -439,13 +437,13 @@ jobs:
run: yarn run test:object_api
- name: Run smoke tests
working-directory: tests/functional
run: yarn run test:smoke
run: MOCHA_FILE=_reports/smoke-http.xml yarn run test:smoke
- name: Enable HTTPS
run: bash enable-https.sh
working-directory: ./.github/scripts/end2end
- name: Run smoke tests (HTTPS)
working-directory: tests/functional
run: yarn run test:smoke
run: MOCHA_FILE=_reports/smoke-https.xml yarn run test:smoke
- name: Debug wait
uses: ./.github/actions/debug-wait
timeout-minutes: 60
Expand All @@ -456,6 +454,7 @@ jobs:
user: ${{ secrets.ARTIFACTS_USER }}
password: ${{ secrets.ARTIFACTS_PASSWORD }}
trunk_token: ${{ secrets.TRUNK_TOKEN }}
junit-paths: ${{ github.workspace }}/tests/functional/_reports/*.xml
if: always()

end2end-sharded:
Expand Down Expand Up @@ -514,6 +513,7 @@ jobs:
user: ${{ secrets.ARTIFACTS_USER }}
password: ${{ secrets.ARTIFACTS_PASSWORD }}
trunk_token: ${{ secrets.TRUNK_TOKEN }}
junit-paths: ${{ github.workspace }}/tests/functional/_reports/*.xml
if: always()

ctst-end2end-sharded:
Expand Down
2 changes: 0 additions & 2 deletions tests/functional/mocha/.mocharc.js
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,4 @@
module.exports = {
'pass-on-failing-test-suite': process.env.CI_PASS_ON_TEST_FAILURE === 'true',
'exit': true,
'reporter': 'mocha-multi-reporters',
'reporter-options': 'configFile=mocha/mocha-reporter.json',
};
6 changes: 6 additions & 0 deletions tests/functional/mocha/mocha-reporter.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
module.exports = {
reporterEnabled: 'spec, xunit',
xunitReporterOptions: {
output: process.env.MOCHA_FILE || '_reports/test-results.xml',

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we ok with that ? "testsuitesTitle": "{id}" what was the goal of that ?

},
};
7 changes: 0 additions & 7 deletions tests/functional/mocha/mocha-reporter.json

This file was deleted.

43 changes: 21 additions & 22 deletions tests/functional/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -32,12 +32,11 @@
"eslint": "^10.8.0",
"eslint-config-scality": "scality/Guidelines#8.3.0",
"js-yaml": "^5.2.2",
"mocha": "^11.7.5",
"mocha-junit-reporter": "^2.2.1",
"mocha": "^12.0.2",
"mocha-multi-reporters": "^1.1.7",
"npm-run-all": "^4.1.5",
"prometheus-query": "^3.4.0",
"scubaclient": "git+https://github.com/scality/scubaclient.git#1.1.4",
"scubaclient": "git+https://github.com/scality/scubaclient.git#1.2.0",
"ts-node": "^10.9.2",
"tsconfig-paths": "^4.2.0",
"typescript": "^5.8.3",
Expand All @@ -50,26 +49,26 @@
"build:cucumber": "tsc --build tsconfig.json",
"initializeFunctionalTestsResources": "ts-node -r tsconfig-paths/register testResourcesSetup/index.ts",
"unused-steps": "! yarn cucumber-js --config ctst/cucumber.config.cjs --dry-run --format usage 2>&1 | grep UNUSED",
"test:aws_crr": "mocha --config mocha/.mocharc.js -t 10000 --reporter-options configFile=mocha/mocha-reporter.json,cmrOutput=mocha-junit-reporter+testsuitesTitle+test_aws_crr mocha/backbeat/tests/crr/awsBackend.js",
"test:azure_crr": "mocha --config mocha/.mocharc.js -t 10000 --reporter-options configFile=mocha/mocha-reporter.json,cmrOutput=mocha-junit-reporter+testsuitesTitle+test_azure_crr mocha/backbeat/tests/crr/azureBackend.js",
"test:gcp_crr": "mocha --config mocha/.mocharc.js -t 10000 --reporter-options configFile=mocha/mocha-reporter.json,cmrOutput=mocha-junit-reporter+testsuitesTitle+test_gcp_crr mocha/backbeat/tests/crr/gcpBackend.js",
"test:one_to_many": "mocha --config mocha/.mocharc.js -t 10000 --reporter-options configFile=mocha/mocha-reporter.json,cmrOutput=mocha-junit-reporter+testsuitesTitle+test_one_to_many mocha/backbeat/tests/crr/oneToMany.js",
"test:crr": "mocha --config mocha/.mocharc.js -t 10000 --reporter-options configFile=mocha/mocha-reporter.json,cmrOutput=mocha-junit-reporter+testsuitesTitle+test_crr mocha/backbeat/tests/crr/crr.js",
"test:api": "mocha --config mocha/.mocharc.js -t 10000 --reporter-options configFile=mocha/mocha-reporter.json,cmrOutput=mocha-junit-reporter+testsuitesTitle+test_api --recursive mocha/backbeat/tests/api",
"test:retry": "mocha --config mocha/.mocharc.js -t 10000 --reporter-options configFile=mocha/mocha-reporter.json,cmrOutput=mocha-junit-reporter+testsuitesTitle+test_retry --recursive mocha/backbeat/tests/retry",
"test:crr_pause_resume": "mocha --config mocha/.mocharc.js -t 10000 --reporter-options configFile=mocha/mocha-reporter.json,cmrOutput=mocha-junit-reporter+testsuitesTitle+test_crr_pause_resume --recursive mocha/backbeat/tests/crr-pause-resume",
"test:expiration": "mocha --config mocha/.mocharc.js -t 900000 --reporter-options configFile=mocha/mocha-reporter.json,cmrOutput=mocha-junit-reporter+testsuitesTitle+test_expiration mocha/backbeat/tests/lifecycle/expiration.js",
"test:transition": "mocha --config mocha/.mocharc.js -t 900000 --reporter-options configFile=mocha/mocha-reporter.json,cmrOutput=mocha-junit-reporter+testsuitesTitle+test_transition mocha/backbeat/tests/lifecycle/transition.js",
"test:lifecycle": "mocha --config mocha/.mocharc.js -t 1800000 --reporter-options configFile=mocha/mocha-reporter.json,cmrOutput=mocha-junit-reporter+testsuitesTitle+test_lifecycle --recursive mocha/backbeat/tests/lifecycle",
"test:ingestion_oob_s3c": "mocha --config mocha/.mocharc.js -t 180000 --reporter-options configFile=mocha/mocha-reporter.json,cmrOutput=mocha-junit-reporter+testsuitesTitle+test_ingestion_oob_s3c --recursive mocha/backbeat/tests/ingestion",
"test:location_quota": "mocha --config mocha/.mocharc.js -t 10000 --reporter-options configFile=mocha/mocha-reporter.json,cmrOutput=mocha-junit-reporter+testsuitesTitle+test_location_quota --recursive mocha/cloudserver/locationQuota/tests",
"test:bucket_get_v2": "mocha --config mocha/.mocharc.js -t 10000 --reporter-options configFile=mocha/mocha-reporter.json,cmrOutput=mocha-junit-reporter+testsuitesTitle+test_bucket_get_v2 --recursive mocha/cloudserver/bucketGetV2/tests",
"test:bucket_policy": "mocha --config mocha/.mocharc.js -t 10000 --reporter-options configFile=mocha/mocha-reporter.json,cmrOutput=mocha-junit-reporter+testsuitesTitle+test_bucket_policy --recursive mocha/cloudserver/bucketPolicy/tests",
"test:operator": "mocha --config mocha/.mocharc.js -t 10000 --reporter-options configFile=mocha/mocha-reporter.json,cmrOutput=mocha-junit-reporter+testsuitesTitle+test_operator mocha/init_test.js",
"test:smoke": "mocha --config mocha/.mocharc.js -t 10000 --reporter-options configFile=mocha/mocha-reporter.json,cmrOutput=mocha-junit-reporter+testsuitesTitle+test_smoke --recursive mocha/smoke_tests",
"test:iam_policies": "mocha --config mocha/.mocharc.js -t 15000 --reporter-options configFile=mocha/mocha-reporter.json,cmrOutput=mocha-junit-reporter+testsuitesTitle+test_iam_policies --recursive mocha/iam_policies",
"test:aws_crr": "MOCHA_FILE=_reports/aws-crr.xml mocha --config mocha/.mocharc.js -t 10000 --reporter mocha-multi-reporters --reporter-options configFile=mocha/mocha-reporter.js mocha/backbeat/tests/crr/awsBackend.js",
"test:azure_crr": "MOCHA_FILE=_reports/azure-crr.xml mocha --config mocha/.mocharc.js -t 10000 --reporter mocha-multi-reporters --reporter-options configFile=mocha/mocha-reporter.js mocha/backbeat/tests/crr/azureBackend.js",
"test:gcp_crr": "MOCHA_FILE=_reports/gcp-crr.xml mocha --config mocha/.mocharc.js -t 10000 --reporter mocha-multi-reporters --reporter-options configFile=mocha/mocha-reporter.js mocha/backbeat/tests/crr/gcpBackend.js",
"test:one_to_many": "MOCHA_FILE=_reports/one-to-many.xml mocha --config mocha/.mocharc.js -t 10000 --reporter mocha-multi-reporters --reporter-options configFile=mocha/mocha-reporter.js mocha/backbeat/tests/crr/oneToMany.js",
"test:crr": "MOCHA_FILE=_reports/crr.xml mocha --config mocha/.mocharc.js -t 10000 --reporter mocha-multi-reporters --reporter-options configFile=mocha/mocha-reporter.js mocha/backbeat/tests/crr/crr.js",
"test:api": "MOCHA_FILE=_reports/api.xml mocha --config mocha/.mocharc.js -t 10000 --reporter mocha-multi-reporters --reporter-options configFile=mocha/mocha-reporter.js --recursive mocha/backbeat/tests/api",
"test:retry": "MOCHA_FILE=_reports/retry.xml mocha --config mocha/.mocharc.js -t 10000 --reporter mocha-multi-reporters --reporter-options configFile=mocha/mocha-reporter.js --recursive mocha/backbeat/tests/retry",
"test:crr_pause_resume": "MOCHA_FILE=_reports/crr-pause-resume.xml mocha --config mocha/.mocharc.js -t 10000 --reporter mocha-multi-reporters --reporter-options configFile=mocha/mocha-reporter.js --recursive mocha/backbeat/tests/crr-pause-resume",
"test:expiration": "MOCHA_FILE=_reports/expiration.xml mocha --config mocha/.mocharc.js -t 900000 --reporter mocha-multi-reporters --reporter-options configFile=mocha/mocha-reporter.js mocha/backbeat/tests/lifecycle/expiration.js",
"test:transition": "MOCHA_FILE=_reports/transition.xml mocha --config mocha/.mocharc.js -t 900000 --reporter mocha-multi-reporters --reporter-options configFile=mocha/mocha-reporter.js mocha/backbeat/tests/lifecycle/transition.js",
"test:lifecycle": "MOCHA_FILE=_reports/lifecycle.xml mocha --config mocha/.mocharc.js -t 1800000 --reporter mocha-multi-reporters --reporter-options configFile=mocha/mocha-reporter.js --recursive mocha/backbeat/tests/lifecycle",
"test:ingestion_oob_s3c": "MOCHA_FILE=_reports/ingestion.xml mocha --config mocha/.mocharc.js -t 180000 --reporter mocha-multi-reporters --reporter-options configFile=mocha/mocha-reporter.js --recursive mocha/backbeat/tests/ingestion",
"test:location_quota": "MOCHA_FILE=_reports/location-quota.xml mocha --config mocha/.mocharc.js -t 10000 --reporter mocha-multi-reporters --reporter-options configFile=mocha/mocha-reporter.js --recursive mocha/cloudserver/locationQuota/tests",
"test:bucket_get_v2": "MOCHA_FILE=_reports/bucket-get-v2.xml mocha --config mocha/.mocharc.js -t 10000 --reporter mocha-multi-reporters --reporter-options configFile=mocha/mocha-reporter.js --recursive mocha/cloudserver/bucketGetV2/tests",
"test:bucket_policy": "MOCHA_FILE=_reports/bucket-policy.xml mocha --config mocha/.mocharc.js -t 10000 --reporter mocha-multi-reporters --reporter-options configFile=mocha/mocha-reporter.js --recursive mocha/cloudserver/bucketPolicy/tests",
"test:operator": "MOCHA_FILE=_reports/operator.xml mocha --config mocha/.mocharc.js -t 10000 --reporter mocha-multi-reporters --reporter-options configFile=mocha/mocha-reporter.js mocha/init_test.js",
"test:smoke": "MOCHA_FILE=${MOCHA_FILE:-_reports/smoke.xml} mocha --config mocha/.mocharc.js -t 10000 --reporter mocha-multi-reporters --reporter-options configFile=mocha/mocha-reporter.js --recursive mocha/smoke_tests",
"test:iam_policies": "MOCHA_FILE=_reports/iam-policies.xml mocha --config mocha/.mocharc.js -t 15000 --reporter mocha-multi-reporters --reporter-options configFile=mocha/mocha-reporter.js --recursive mocha/iam_policies",
"test:all_extensions": "run-p --aggregate-output test:crr test:aws_crr test:expiration test:transition test:ingestion_oob_s3c",
"test:object_api": "mocha --config mocha/.mocharc.js -t 10000 --reporter-options configFile=mocha/mocha-reporter.json,cmrOutput=mocha-junit-reporter+testsuitesTitle+test_object_api --recursive mocha/cloudserver/keyFormatVersion/tests",
"test:object_api": "MOCHA_FILE=_reports/object-api.xml mocha --config mocha/.mocharc.js -t 10000 --reporter mocha-multi-reporters --reporter-options configFile=mocha/mocha-reporter.js --recursive mocha/cloudserver/keyFormatVersion/tests",
"lint": "eslint ."
},
"resolutions": {
Expand Down
Loading
Loading