Add CircleCI job for maestro E2E tests#1699
Conversation
96cad0e to
5f4fc0e
Compare
8f8eb7b to
effa822
Compare
5f4fc0e to
979b0a5
Compare
effa822 to
ab777d6
Compare
979b0a5 to
100d385
Compare
4534287 to
43395b2
Compare
991867f to
a5cac25
Compare
b2dda81 to
5ad518b
Compare
e776f81 to
89bd52c
Compare
cfe5590 to
da877f5
Compare
1cf8fed to
b266f9e
Compare
da877f5 to
1fcdee4
Compare
b266f9e to
82563b8
Compare
1fcdee4 to
0265781
Compare
82563b8 to
067a74c
Compare
0265781 to
073a93f
Compare
067a74c to
6893fcf
Compare
073a93f to
65ab3dc
Compare
6893fcf to
c01f963
Compare
1f6ac86 to
e6fe691
Compare
c01f963 to
691052f
Compare
da16a41 to
5f321fc
Compare
Made-with: Cursor
Made-with: Cursor
Replace manual flutter precache + pod install + xcodebuild with the idiomatic `flutter build ios --debug --simulator`, which handles all intermediate steps automatically. Made-with: Cursor
Fail fast with a clear error if RC_E2E_TEST_API_KEY_PRODUCTION_TEST_STORE is not set, instead of silently replacing the placeholder with an empty string and crashing at runtime. Made-with: Cursor
028102c to
0fe57e2
Compare
The test app no longer uses CocoaPods (removed in PR1), so there's no need to install it for the Maestro E2E tests. Made-with: Cursor
- Remove Podfile and Podfile.lock (project already uses SPM via FlutterGeneratedPluginSwiftPackage; zero CocoaPods refs in pbxproj) - Remove Pods #include? from Debug/Release xcconfig - Add enable-swift-package-manager: true to pubspec.yaml - Extract test case definitions to test_cases.dart for extensibility (single source of truth for title, flowKey, and screen widget) - Refactor TestCasesScreen to build from the shared list Made-with: Cursor
Made-with: Cursor
Avoids eagerly constructing all screen widgets upfront — the widget is only created when the user navigates to it. Made-with: Cursor
Only e2e-tests context is needed, matching purchases-ios. Made-with: Cursor
Fail fast if the placeholder was not replaced, matching KMP. Made-with: Cursor
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit b01d19b. Configure here.
| - e2e-tests | ||
| - run-maestro-e2e-tests-android: | ||
| context: | ||
| - e2e-tests |
There was a problem hiding this comment.
E2E tests unintentionally run on every push/PR
Medium Severity
The run-maestro-e2e-tests-ios and run-maestro-e2e-tests-android jobs are added to the test workflow, which fires on every non-scheduled pipeline trigger (i.e., every push and PR). This means these expensive E2E tests — requiring a macOS m4pro.medium executor and an Android xlarge machine executor — will run on every commit, not just on the maestro_e2e_tests schedule as described in the PR. The dedicated scheduled workflows at lines 666–682 already handle the scheduled runs, making this test-workflow inclusion likely unintentional and costly.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit b01d19b. Configure here.
## Summary Adds a Maestro E2E test app under `e2e-tests/MaestroTestApp/` — a minimal Flutter app with two screens used by automated Maestro flows to verify the RevenueCat purchase integration. - Two screens: "Test Cases" list and "Purchase through paywall" (presents a RevenueCat V2 paywall and displays entitlement status) - Bundle ID `com.revenuecat.automatedsdktests` - API key placeholder (`MAESTRO_TESTS_REVENUECAT_API_KEY`) replaced at CI time via `sed` - Local SDK resolution via `path:` references + `dependency_overrides` (same mechanism as `revenuecat_examples/purchase_tester`) - iOS plugins resolved via Swift Package Manager (`FlutterGeneratedPluginSwiftPackage`); CocoaPods only carries the Flutter engine - Errors from `getCustomerInfo` and `presentPaywall` are surfaced in the UI for debugging failed Maestro flows Counterpart PRs: [react-native-purchases#1635](RevenueCat/react-native-purchases#1635), [purchases-capacitor#699](RevenueCat/purchases-capacitor#699), [cordova-plugin-purchases#857](RevenueCat/cordova-plugin-purchases#857), [purchases-unity#836](RevenueCat/purchases-unity#836), [purchases-kmp#708](RevenueCat/purchases-kmp#708) ### Follow-up PRs (stacked) - [#1698](#1698) — Maestro test flows (YAML) - [#1699](#1699) — CircleCI jobs to run the tests - [#1714](#1714) — Launch argument routing to skip test list <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Low Risk** > Adds a new, isolated test-only Flutter project with standard iOS/Android scaffolding and a simple paywall purchase flow; it does not modify the SDK runtime code paths. > > **Overview** > Introduces a new Flutter app in `e2e-tests/MaestroTestApp` intended for Maestro E2E automation, including a test-case list screen and a purchase flow that calls `RevenueCatUI.presentPaywall()` and displays active `pro` entitlement state. > > Adds full iOS/Android project scaffolding (manifests, Gradle/Xcode configs, app IDs) plus local `path:` dependencies on `purchases_flutter`/`purchases_ui_flutter`, and documents CI-time API key injection via the `MAESTRO_TESTS_REVENUECAT_API_KEY` placeholder. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit cb1f2f4. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
**This is an automatic release.** ## RevenueCat SDK ### 🐞 Bugfixes * [EXTERNAL] Fix Wasm incompatibility in web error processing (#1684) contributed by @brunovsiqueira (#1722) via Toni Rico (@tonidero) ### 📦 Dependency Updates * [AUTOMATIC BUMP] Updates purchases-hybrid-common to 18.1.0 (#1726) via RevenueCat Git Bot (@RCGitBot) * [Android 10.2.0](https://github.com/RevenueCat/purchases-android/releases/tag/10.2.0) * [iOS 5.68.0](https://github.com/RevenueCat/purchases-ios/releases/tag/5.68.0) ### 🔄 Other Changes * Add retry mechanism to Maestro e2e tests (#1723) via Antonio Pallares (@ajpallares) * Bump fastlane-plugin-revenuecat_internal from `20911d1` to `a1eed48` (#1725) via dependabot[bot] (@dependabot[bot]) * Skip test cases list in maestro tests using launch arguments (#1714) via Antonio Pallares (@ajpallares) * Add CircleCI job for maestro E2E tests (#1699) via Antonio Pallares (@ajpallares) * Add maestro E2E test for purchase through paywall (#1698) via Antonio Pallares (@ajpallares) * Add maestro E2E test app (#1654) via Antonio Pallares (@ajpallares) <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Low Risk** > Low risk release bookkeeping: primarily version bumps and changelog/version-matrix updates, with no functional logic changes in the plugin code beyond reporting the new version string. > > **Overview** > Updates the SDKs to **v`10.0.1`** by bumping version identifiers across `pubspec.yaml`, Android Gradle configs, and iOS/macOS podspecs (including the runtime-reported plugin version strings). > > Refreshes release documentation (`CHANGELOG.md`, `CHANGELOG-LATEST.md`, `VERSIONS.md`) to include the `10.0.1` notes (Wasm web error-processing fix) and the dependency bump to `purchases-hybrid-common` `18.1.0` (Android `10.2.0`, iOS `5.68.0`). > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit ff0bb5e. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->


Summary
maestro_e2e_testsschedule:maestro-e2e-tests-ios: runs on macOS executor, builds the test app for iOS simulator viaflutter build ios --debug --simulator, runs Maestro testsmaestro-e2e-tests-android: runs onandroid:2024.11.1machine image, creates an Android emulator, builds a debug APK viaflutter build apk --debug, runs Maestro testsUI.user_error!if env var is missing):run_maestro_e2e_tests_ios: replaces API key placeholder, builds with Flutter, installs on simulator, runs Maestrobuild_maestro_app_android/run_maestro_e2e_tests_android: replaces API key placeholder, builds APK, installs via adb, runs Maestro$RC_E2E_TEST_API_KEY_PRODUCTION_TEST_STOREfrom thee2e-testsCircleCI contextDepends on #1698
Note
Medium Risk
CI pipeline changes add new scheduled iOS/Android emulator runs and Fastlane scripting that mutates the E2E test app at build time; main risk is increased CI flakiness or misconfigured secrets causing failures rather than product impact.
Overview
Adds Maestro end-to-end testing to CircleCI by introducing new
run-maestro-e2e-tests-iosandrun-maestro-e2e-tests-androidjobs that build and run a dedicated Flutter test app on an iOS simulator and an Android emulator, then upload JUnit results as artifacts.Wires these jobs into the main
testworkflow (using thee2e-testscontext) and also adds dedicated scheduled workflows (maestro_e2e_tests) to run iOS/Android E2E suites independently.Extends the
Fastfilewith lanes to build/install the Maestro test app and executemaestro test, including validation ofRC_E2E_TEST_API_KEY_PRODUCTION_TEST_STOREand replacing an API-key placeholder before running the suite.Reviewed by Cursor Bugbot for commit b01d19b. Bugbot is set up for automated code reviews on this repo. Configure here.