test [M3-9256]: display cypress test results in html report

[dmcintyr-akamai]

Description 📝

Display the cypress test results in an html document. Added cypress-mochawesome-reporter plugin to enable this. The report looks nice and organized:

This task uncovered an existing, known issue w/ cypress where cypress does not allow multiple plugins to listen for the "before:run" and "after:run" hooks, the last plugin to register for the listeners wins. So i added another package as a workaround (cypress-on-fix) which allows an array of listeners for the same event.

Additionally, cypress has a limitation where only a single reporter is allowed, so i used the 'cypress-multi-reporters' package as a workaround to enable multiple reporters. If I don't add this package, then the html reporter would replace the junit reporter.

Adding the additional reporter involved touching the "component" report as well, so it was easier to refactor this. I also removed the configure-test-suite.ts plugin bc it was no longer being used.

Changes 🔄

When running the "cypress:e2e" command, it creates an html report that displays the test results. This content is not added to source control.

Reproduction steps

The junit reporter requires the CY_TEST_JUNIT_REPORT environmental variable to set set to a truthy value:
CY_TEST_JUNIT_REPORT='1'
The html reporter requires the CY_TEST_HTML_REPORT environmental variable to set set to a truthy value:
CY_TEST_HTML_REPORT='1'

For the html report to be enabled in the Jenkins CI pipeline, the CY_TEST_HTML_REPORT will need to be set in the jenkins-ui-library repo (in vars/testManager.groovy and perhaps other groovy files).

To run the e2e report:
pnpm run cy:e2e

To run the component report:
pnpm run cy:component:run

Both commands will result in the html report being created at packages/manager/cypress/reports/html/index.html. The junit report will be at packages/manager/cypress/results/test-results-.xml.

Author Checklists

As an Author, to speed up the review process, I considered 🤔

👀 Doing a self review
❔ Our contribution guidelines
🤏 Splitting feature into small PRs
➕ Adding a changeset
🧪 Providing/improving test coverage
🔐 Removing all sensitive information from the code and PR description
🚩 Using a feature flag to protect the release
👣 Providing comprehensive reproduction steps
📑 Providing or updating our documentation
🕛 Scheduling a pair reviewing session
📱 Providing mobile support
♿ Providing accessibility support

• I have read and considered all applicable items listed above.

As an Author, before moving this PR from Draft to Open, I confirmed ✅

• All unit tests are passing
• TypeScript compilation succeeded without errors
• Code passes all linting rules

[github-actions]

Coverage Report: ✅
Base Coverage: 79.96%
Current Coverage: 79.96%

[mjac0bs]

Thanks for doing this @dmcintyr-akamai. Will let Joe comment on anything specific to the Cypress set up.

Confirmed the HTML report was output in the specified spot when I ran:

pnpm cy:e2e -s "cypress/e2e/core/kubernetes/lke-create.spec.ts"

✅	Purposely triggered a failure to see what it'd look like

I'm not sure that cy:e2e as the command is the most intuitive name to know that it generates the HTML report. Some brief documentation of this in our developer docs End-to-End tests section may help?

[dmcintyr-akamai]

reverting back to draft so i can insert code for demo

[jdamore-linode]

Nicely done David! I know this is still in draft, happy to take a look if you make any other changes but this looks great

[linode-gh-bot]

Cloud Manager UI test results

🔺 1 failing test on test run #52 ↗︎

❌ Failing	✅ Passing	↪️ Skipped	🕐 Duration
1 Failing	533 Passing	4 Skipped	106m 59s

Details

Failing Tests
	Spec	Test
❌	account-logout.spec.ts	Cloud Manager Cypress Tests→Logout Test » Logout Test

Troubleshooting

Use this command to re-run the failing tests:

pnpm cy:run -s "cypress/e2e/core/account/account-logout.spec.ts"