test(package): wire mobile package tests — packer + RN Android leg (#387)#398
Open
goosewobbler wants to merge 13 commits into
Open
test(package): wire mobile package tests — packer + RN Android leg (#387)#398goosewobbler wants to merge 13 commits into
goosewobbler wants to merge 13 commits into
Conversation
) Completes the mobile package-test packer and runs the React Native Android leg in CI via the cheap "reuse the e2e device" approach (option 1): the packed-tarball smoke runs as a final step inside the existing RN Android e2e job, against the already-booted emulator + Metro + APK — no second emulator. test-package.ts: - Add the `flutter` service (union, build filter, pack + skip-build blocks, override/ install, fixture resolution → flutter-app), mirroring react-native. - Fix a latent react-native gap: the packer never packed/overrode @wdio/native-mobile-core, a workspace dep of BOTH mobile services — so an isolated install of the packed @wdio/react-native-service could never resolve it. This was masked because the RN package path had never run in CI. Now packed + file:-overridden for RN and Flutter. - Add test:package:react-native / test:package:flutter root scripts. CI: append a new-arch-gated package smoke to _ci-e2e-react-native (Android). The uiautomator2 driver (already in APPIUM_HOME) and RN_APP_PATH are reused. Scope: this lands the packer (both services) + the one cleanly-wireable leg. The remaining three legs have per-leg integration obstacles documented for follow-up: - Flutter Android: appium-flutter-driver is a built-from-source fork (getVMServiceUrl) overwritten into e2e/node_modules; the isolated fixture's Appium needs it in APPIUM_HOME to drive the Dart VM. - RN/Flutter iOS: need the e2e's simulator-UDID pin + prebuilt-WDA derivedDataPath reuse threaded into the package fixture config. Validated locally: typecheck:scripts + actionlint clean. The Android leg itself is CI-validated (no local emulator). Refs #387 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
|
Standing release PR: #400 · 11 packages queued · open 22h 59m · ✅ ready to merge Release Preview — 11 packages
These changes will be added to the release PR (#400) when merged: ChangelogProject-wide changesChanged
Fixed
@wdio/electron-cdp-bridge wdio-electron-cdp-bridge@v10.0.0 → 10.0.1Changed
@wdio/electron-service wdio-electron-service@v10.0.0 → 10.0.1Changed
@wdio/flutter-service v1.0.0-next.0 → 1.0.1Changed
@wdio/native-cdp-bridge v0.1.0-next.0 → 0.1.1Changed
@wdio/native-mobile-core v1.0.0 → 1.0.1Changed
@wdio/native-spy wdio-native-spy@v1.1.0 → 1.1.1Changed
@wdio/native-types wdio-native-types@v2.3.1 → 2.3.2Changed
@wdio/native-utils wdio-native-utils@v2.4.0 → 2.4.1Changed
@wdio/react-native-service v1.0.0-next.0 → 1.0.1Changed
dioxus-package-test-app v0.1.0 → 0.1.1Changed
wdio-dioxus-embedded-driver N/A → 1.0.1Changed
Fixed
After merge — predicted release
Updated automatically by ReleaseKit |
Contributor
|
| Filename | Overview |
|---|---|
| scripts/test-package.ts | Adds full flutter service support and fixes the RN mobile-core packing bug; flutter is now correctly included in the CLI validation guard, build filters, pack blocks, override maps, and fixture routing. |
| .github/workflows/_ci-e2e-react-native.reusable.yml | Appends a single-line RN package smoke gated on new-arch inside the android-emulator-runner script; RN_APP_PATH is already in GITHUB_ENV and available to the emulator runner step. |
| .github/workflows/_ci-e2e-react-native-ios.reusable.yml | Appends iOS RN package smoke (new-arch gated), migrates WDA strategy to usePreinstalledWDA, bumps cache key to v2; RN_IOS_UDID and RN_WDA_DD are both available when the smoke runs. |
| .github/workflows/_ci-e2e-flutter.reusable.yml | Appends Android Flutter package smoke with AFD_FORK_BUILD_DIR override; FLUTTER_APP_PATH and RUNNER_TEMP paths are correctly available from the preceding steps. |
| .github/workflows/_ci-e2e-flutter-ios.reusable.yml | Adds logd warm-up step to prevent the syslog cold-start failure on Flutter iOS, migrates WDA to usePreinstalledWDA (v2 cache key), and appends the iOS Flutter package smoke; FLUTTER_IOS_DEVICE is correctly available via step-level env. |
| e2e/wdio.react-native.conf.ts | Migrates iOS WDA from derivedDataPath+usePrebuiltWDA to prebuiltWDAPath+usePreinstalledWDA, adds wdaStartupRetries, bumps specFileRetries to 2, and tightens wdaLaunchTimeout to 120s for the prebuilt path. |
| e2e/wdio.flutter.conf.ts | Same WDA migration and retry tuning as the RN conf; consistent changes across both mobile e2e configurations. |
| fixtures/package-tests/react-native-app/wdio.conf.ts | Adds iOS WDA reuse (usePreinstalledWDA + prebuiltWDAPath), UDID pinning, wdaStartupRetries, and specFileRetries for the package smoke fixture; Android path unchanged. |
| fixtures/package-tests/flutter-app/wdio.conf.ts | New flutter package fixture config with iOS WDA reuse, platform-aware timeouts, and specFileRetry; mirrors the RN fixture pattern. |
| fixtures/package-tests/flutter-app/test/smoke.spec.ts | Updates switchWindow→switchContext (API rename) and replaces the arbitrary Dart eval with a named handler invocation (bindingReady), which is the correct Tier-2 flutter:execute contract. |
| fixtures/package-tests/react-native-app/package.json | Adds appium, appium-uiautomator2-driver, and appium-xcuitest-driver as explicit devDeps so the isolated fixture provides its own Appium drivers without relying on APPIUM_HOME. |
| fixtures/package-tests/flutter-app/package.json | Adds appium and appium-flutter-driver; notably omits appium-xcuitest-driver (relied on transitively via appium-flutter-driver's own deps for iOS). |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[CI e2e job starts] --> B[Boot emulator / simulator]
B --> C[Build & install app]
C --> D[Start Metro / Dart VM]
D --> E[Run e2e tests]
E --> F{new-arch leg?}
F -- No --> Z[Done]
F -- Yes --> G[pnpm test:package:react-native or test:package:flutter]
G --> H[buildAndPackService packs service + deps as .tgz]
H --> I[Copy fixture to temp dir, write .pnpmrc + overrides]
I --> J[pnpm install + pnpm add tarballs]
J --> K{flutter + AFD_FORK_BUILD_DIR?}
K -- Yes --> L[Overwrite appium-flutter-driver build/ with fork]
K -- No --> M[Run wdio smoke spec]
L --> M
M --> N{pass?}
N -- Yes --> Z
N -- No, specFileRetries left --> M
N -- No, retries exhausted --> X[CI failure]
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
A[CI e2e job starts] --> B[Boot emulator / simulator]
B --> C[Build & install app]
C --> D[Start Metro / Dart VM]
D --> E[Run e2e tests]
E --> F{new-arch leg?}
F -- No --> Z[Done]
F -- Yes --> G[pnpm test:package:react-native or test:package:flutter]
G --> H[buildAndPackService packs service + deps as .tgz]
H --> I[Copy fixture to temp dir, write .pnpmrc + overrides]
I --> J[pnpm install + pnpm add tarballs]
J --> K{flutter + AFD_FORK_BUILD_DIR?}
K -- Yes --> L[Overwrite appium-flutter-driver build/ with fork]
K -- No --> M[Run wdio smoke spec]
L --> M
M --> N{pass?}
N -- Yes --> Z
N -- No, specFileRetries left --> M
N -- No, retries exhausted --> X[CI failure]
Reviews (12): Last reviewed commit: "fix(ci): keep WDA XCTest frameworks for ..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
- _ci-e2e-react-native: the new-arch gate was a multi-line if/then/fi, but android-emulator-runner runs the script line-by-line (each its own `sh -c`), so the `if` line executed without its `fi` → "Syntax error: end of file unexpected" (exit 2) AFTER the e2e passed. Collapse to a single-line `if …; then …; fi`. - test-package.ts (Greptile P1): add `flutter` to the runtime --service arg guard in main() + the error message — without it `test:package:flutter` threw "Invalid service value: flutter". - test-package.ts (Greptile P2): list @wdio/native-mobile-core explicitly in the react-native build filter (matches flutter; guards against a stale pack if the direct dep is ever dropped). - biome formatting of the above. The RN/Flutter iOS reds on the prior run are in the untouched "Run E2E on iOS simulator" step — pre-existing sim/WDA flakes, not from this PR. Refs #387 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The RN Android package smoke failed with "Appium is not installed locally" — the isolated fixture install (node-linker=isolated) only had @wdio/appium-service, not appium itself, and Appium detects drivers from node_modules (not APPIUM_HOME; the e2e log shows "uiautomator2 driver already installed" → install skipped because the devDep is present), so the driver wasn't reachable either. These fixtures are NOT workspace members — they're installed only in isolation by test-package.ts (which overrides workspace:* → file: tgz), so the fixture package.json is authoritative for the isolated install and no root lockfile change is needed. - react-native-app: add appium ^3.5.0, appium-uiautomator2-driver ^7.6.0, appium-xcuitest-driver ^11.9.0 (versions matched to e2e; xcuitest readies the iOS leg). - flutter-app: add appium ^3.5.0, appium-flutter-driver ^3.7.0 (readies the Flutter legs; the getVMServiceUrl fork is a separate Android-leg concern). - Correct the RN reusable comment: the driver comes from the fixture's node_modules, not APPIUM_HOME. Refs #387 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Append the packed-tarball smoke to the existing RN iOS e2e step (after the e2e run, where Metro is still backgrounded), gated to the new-arch leg so it runs once per service. A normal bash step → multi-line if/fi is fine (unlike the Android emulator-runner's line-by-line script). It reuses the e2e's already-booted simulator + prebuilt WebDriverAgent rather than standing up its own: the fixture wdio.conf now reads RN_IOS_UDID (pin the sim) and RN_WDA_DD (appium:derivedDataPath + usePrebuiltWDA + a tight wdaLaunchTimeout) when set — both unset locally, where Appium resolves by name and builds WDA itself. iOS connectionRetryTimeout bumped to 420s (XCUITest/WDA attach is slower than Android). appium + appium-xcuitest-driver were added to the fixture in the prior commit, so the isolated install already provides them. Refs #387 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Wire the packed-tarball smoke into both Flutter e2e jobs (reuse-the-e2e-device, matching RN): Android in the emulator-runner script (single line, runs once — no arch split), iOS appended to the sim run step reusing the prebuilt WDA. The Flutter smoke drives the Dart VM via @wdio/flutter-service → appium-flutter-driver's getVMServiceUrl, which exists only in the goosewobbler fork (the published driver can't reach the VM). The e2e jobs already build that fork; both legs pass AFD_FORK_BUILD_DIR=$RUNNER_TEMP/afd/driver/build and test-package.ts overwrites the isolated fixture's appium-flutter-driver build/ with it — mirroring what the flutter e2e does to e2e/node_modules. Gated on the env, so it's a no-op elsewhere; drop once getVMServiceUrl is published/upstreamed. - test-package.ts: AFD fork-build overwrite for the flutter service after install. - flutter-app fixture: iOS reuses the booted sim's prebuilt WDA (FLUTTER_WDA_DD → derivedDataPath + usePrebuiltWDA + wdaLaunchTimeout + simulatorStartupTimeout); iOS connectionRetryTimeout bumped to 420s. appium + appium-flutter-driver were added to the fixture earlier. Validated locally: typecheck:scripts, format:check, biome lint, actionlint clean. Refs #387 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Flutter iOS smoke threw "Electron service packages not available": main() re-derives the service per fixture from servicePrefixes (separate from the fixture-resolution filter), which lacked a flutter- entry, so flutter-app fell through to the `?? 'electron'` default. The `?? 'electron'` fallback masked the missing case from the typecheck. Add ['flutter-', 'flutter'] + widen the detectedService type. (Flutter Android failed earlier in its e2e on a Dart Observatory connection flake, so its smoke never reached this — it'll exercise the fix on the next run.) Refs #387 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
Author
Flutter iOS dependency: appium-flutter-driver fork fixThe Flutter iOS legs here (e2e and the new package smoke) depend on a fix in the Symptom: intermittent Root cause: appium-xcuitest's Fix: goosewobbler/appium-flutter-driver@ No change is needed in this repo for it (the service already consumes whatever the driver discovers via |
Two Flutter-iOS fixes surfaced by the first full run of both flutter legs:
1. Warm up logd before Appium's first session. The first `simctl … log stream` on
a freshly-booted sim routinely exceeds appium-xcuitest's 10s start window
("process did not start within 10000ms"), which it swallows — disabling the
syslog the Flutter driver reads the random, auth-coded VM-service URL from, so
the first session fails with "mandatory syslog service must be running" (warmed-up
later sessions pass). Running one stream after boot absorbs that cold start. The
fork's startLogCapture retry (goosewobbler/appium-flutter-driver@258dd26) is the
backstop for residual transients.
2. The Flutter package smoke used `flutter.execute('WidgetsBinding.instance != null')`
— but Flutter execute is the cooperative Tier-2 contract (the app registers named
handlers), not arbitrary Dart eval, so it threw "no handler … is registered".
Call the fixture app's registered 'bindingReady' handler instead (returns true once
WidgetsBinding is up), matching e2e/test/flutter/execute.spec.ts.
Refs #387
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…387) The Flutter package smoke's API-surface check asserted `browser.flutter.switchWindow` (undefined → "Expected function, Received undefined"). The service exposes `switchContext` (packages/flutter-service/src/service.ts:192), like the RN smoke. The registered-handler test (bindingReady) and the iOS logd warm-up both passed on the prior run; this was the last real failure in the Flutter iOS smoke. Refs #387 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Research on the residual iOS failures (WDA "ECONNREFUSED 127.0.0.1:8100" + session- create timeout) points to a well-known GitHub-Actions iOS-sim issue: WDA frequently fails to come up on the first attempt, and appium's default is only **2** startup retries — exactly what the failing logs showed. The widely-cited GHA fix is to bump wdaStartupRetries (+ interval); we never set them. - Add appium:wdaStartupRetries=5 + appium:wdaStartupRetryInterval=20000 to all four iOS cap blocks (e2e Flutter, e2e RN, and the Flutter/RN package fixtures), extending the e2e confs' custom cap types accordingly. - Bump specFileRetries 1→2 on the e2e confs and add specFileRetries=1 to the package fixtures: each retry is a fresh session which, with the in-session WDA retries, clears transient WDA/boot/attach flakes. The VM-discovery (syslog) fix from the prior commits + the goosewobbler fork retry is holding (Flutter iOS e2e now 5/5). The sticky iOS "unknown to FrontBoard" app-launch race still needs a leg re-run (same sim; #359) — documented in the confs. Refs: appium/appium-flutter-driver#674, appium/appium#20601, appium/appium#17434 Refs #387 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…387) After adding appium:wdaStartupRetries=5, the cold iOS session-create started failing with "operation aborted due to timeout": 5 retries × wdaLaunchTimeout(180s) ≈ 1000s overran the WDIO connectionRetryTimeout (420s), so the client aborted POST /session mid-retry — the retries helped warm sessions (RN iOS e2e went 6/6) but couldn't save the cold one. Align the budget so the retries actually run: - Prebuilt-WDA wdaLaunchTimeout 180000 → 120000 (prebuilt WDA only launches, no compile, so a tighter per-attempt ceiling is fine and lets the retries fit). - iOS connectionRetryTimeout (prebuilt) 420000 → 780000 (> 5×(120000+20000)=700s). - Package fixtures: connectionRetryCount 3 → 0 so a failed cold session-create doesn't retry the full 13-min timeout 3× — the spec retry (fresh session) covers it, matching the e2e confs. Applied to all four iOS configs (e2e Flutter/RN + both package fixtures). The sticky "unknown to FrontBoard" app-launch race still needs a leg re-run (#359). Refs #387 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…lock (#387) The RN package fixture's iOS WDA spread was missing appium:simulatorStartupTimeout, which the Flutter fixture and both e2e confs set (240000). Without it, a cold/slow runner's sim-boot monitor uses appium's default and can slow first-session attach (not a hard failure, but a needless inconsistency). Match the others. Refs #387 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The cold iOS session-create flake (UND_ERR_SOCKET on POST /session) came from usePrebuiltWDA still shelling out to `xcodebuild test`, whose multi-minute first launch overran undici's ~300s socket cap — lengthening connectionRetryTimeout to fit the WDA startup-retry budget could not help, since undici caps the request regardless. Switch all four iOS configs (RN + Flutter, e2e + package fixtures) from usePrebuiltWDA/derivedDataPath to usePreinstalledWDA/prebuiltWDAPath: appium simctl-installs and launches the CI-prebuilt Runner.app with no in-session xcodebuild, so POST /session lands well inside the socket cap. Both iOS reusables gain an unconditional, idempotent step that strips the runner's embedded XCTest frameworks (required for usePreinstalledWDA on iOS 17+ so WDA resolves the simulator's local ones). Revert the inflated connectionRetryTimeout (780000 -> 420000) now that the compile budget is gone. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ath (#387) The previous commit's strip step removed Frameworks/XC* from the prebuilt WebDriverAgentRunner-Runner.app. That is a *real-device* re-signing requirement (per the appium-xcuitest run-preinstalled-wda guide); on a simulator appium launches the runner standalone via `simctl launch`, which needs those embedded frameworks to load. Stripping them made the runner crash on launch — all three iOS legs failed with "The process did launch, but has since exited or crashed" / SBMainWorkspace denial, and the launch retries then re-blew the undici socket cap. Drop the strip step from both iOS reusables and bump the WDA DerivedData cache key v1 -> v2 so the next run rebuilds an unstripped runner instead of restoring the broken stripped one from cache. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Addresses #387 (run the mobile package tests in CI). Lands the shared packer work + the first cleanly-wireable leg; the remaining three legs are teed up with their specific obstacles.
Approach (option 1 — reuse the e2e device)
The packed-tarball smoke runs as a final step inside the existing mobile e2e job, against the already-booted emulator/sim + Metro + built app. Same
test-package.tsisolation as the desktop package tests (temp dir +node-linker=isolated+file:tgz overrides) — it just skips paying for a second device. No new jobs, noci-status/detect-changeschanges (the e2e jobs are already gated + required).What's in this PR
test-package.ts(both services):flutterservice end-to-end (union, build filter, both pack blocks, override/install,flutter-*fixture resolution).@wdio/native-mobile-core— a workspace dep of both mobile services — so an isolated install of the packed@wdio/react-native-servicecould never resolve it. Masked because the RN package path had never run in CI. Now packed +file:-overridden for RN and Flutter. (Closure verified:mobile-core → core/types/utils, all packed.)test:package:react-native/test:package:flutterroot scripts.CI: new-arch-gated package smoke appended to
_ci-e2e-react-native(Android), reusing theAPPIUM_HOMEuiautomator2 driver +RN_APP_PATH.Deliberately deferred (documented obstacles)
appium-flutter-driveris a built-from-source fork (getVMServiceUrl) overwritten intoe2e/node_modules; the isolated fixture's Appium needs it inAPPIUM_HOMEto reach the Dart VM.derivedDataPathreuse threaded into the package fixture config (else they re-hit the #359 boot flake / rebuild WDA).These are all cheaper in-job (reuse the e2e's already-solved driver/UDID/WDA) than as standalone jobs, but each needs a CI iteration cycle. Happy to do them as follow-ups once this leg is green.
Validation
pnpm typecheck:scripts+actionlint1.7.12 — clean.🤖 Generated with Claude Code