Write specs

A spec is a *.e2e.ts file that drives the app on a simulator or emulator with @delacour/warden-e2e; you write one per user flow. warden e2e leases the devices, installs the build, starts Metro and runs the specs against them. The package drives iOS simulators and Android emulators directly (see iOS driver helper), so there is no daemon and no other test runner to install.

bun add -d @delacour/warden-e2e

A spec

import { expect, test } from "@delacour/warden-e2e";

test("add a todo", async ({ device, screen }) => {
  await screen.getByTestId("todo-input").fill("Write e2e specs");
  await screen.getByTestId("todo-add").tap();
  await expect(screen.getByTestId("todos-remaining")).toHaveText("2 left");
});

test(title, options?, body) receives four fixtures: app, device, screen and platform ("ios" | "android"). Before every test the runner opens the app fresh (app.open()), so a test starts on the app's first screen. Everything a fixture does honours the test's deadline: when it passes, the running driver call is aborted and the test fails with a timeout.

test.skip, test.only, test.skipIf(condition) and test.todo(title) work as in bun:test (a test.todo fails when bun runs with --todo). A test without a body throws INVALID_ARGUMENT.

Find elements

Queries live on screen. They do not touch the device until you act or assert, and the same queries back warden ui, so a selector you proved by hand works in a spec.

screen.getBy… returns a lazy Locator. Nothing is read until you act on it or assert on it, so a locator can be created before the element exists.

A string matches the whole text after whitespace is collapsed (case-sensitive); { exact: false } makes it a case-insensitive substring; a RegExp is tested as written (its g and y flags are ignored). getByTestId takes a RegExp too. Refine a locator with .filter({ hasText, has }) (at least one of the two; has is another locator), .first(), .last() and .nth(i) (a non-negative integer); a bad refinement is INVALID_LOCATOR.

Role queries and visible: true drop hidden elements. Actions and reads need exactly one match: none gives LOCATOR_NOT_FOUND, several give LOCATOR_AMBIGUOUS (narrow the query, or use .first()).

Whole-screen gestures live on screen: tapAt({ x, y }), swipe({ direction, momentum? }) or swipe({ from, to, durationMs? }) (not a mix), and scrollUntilVisible(locator, { direction?, momentum?, maxSwipes?, timeout? }). direction is the way the content moves ("down" reveals what is below), momentum is "none" (default, a plain drag), "slow" or "fast" (a fling). scrollUntilVisible defaults to direction: "down" and maxSwipes: 10, and fails when the target is still off screen after that many swipes.

Multi-touch lives on screen.gesture:

A scale or degrees out of range, a paths count other than 1-2, or a path with no point or a non-finite coordinate throws E2eError INVALID_ARGUMENT, as does a gesture the device cannot do: pinch and rotate need capabilities.pinch, a path needs as many capabilities.maxPointers as it has fingers (Android without warden-uiad: no pinch or rotate, 1-finger paths only on API 26+). Skip such a test with test({ platforms: ["ios"] }).

Act on elements

Actions wait for the element (up to actionTimeout) and then act. Every action takes timeout (ms) to override actionTimeout for that call, and an option an action does not know is INVALID_ARGUMENT:

Reads never wait; they answer from the screen as it is now:

text() · value() · count() · all() · allTexts() · isVisible() · isHidden() · isEnabled() · isChecked() · boundingBox() · waitFor({ state?: "visible" | "hidden" | "attached" | "detached", timeout? })

waitFor is the one read that polls: state defaults to "visible" and timeout to actionTimeout, and it fails with TIMEOUT carrying the screen as text.

text() falls back to the text of the descendants when iOS leaves a container's own label empty (a Badge, a Chip), so asserting on the container works.

read() answers several of those from one look at the screen: { visible, text?, value?, checked? }. Nothing matching gives { visible: false }; text and value are there when the element has them, checked only for a switch or checkbox. It is cheaper than text() plus value() plus isChecked(), which each look at the screen. Like the other reads it fails with LOCATOR_AMBIGUOUS for several matches.

Tap until something shows

A tap can land before the screen is ready to take it. Give tap the thing the tap should lead to and it checks, and taps again when that is not there:

await screen.getByTestId("tab-settings").tap({ until: screen.getByTestId("settings-email") });
await screen.getByTestId("sync").tap({ until: async () => (await screen.getByTestId("sync").read()).text === "Synced" });

until is a locator (visible) or an async function returning a boolean. After the tap it waits up to windowMs (default 4000), checking every pollMs (default 250), and taps again when the window passes without the check holding, at most retries times (default 2, so three taps in all). The last tap is not waited on: when it did not work either, tap returns and the next assertion reports what is missing. A locator until is checked on the same look at the screen as the poll, so a check costs one snapshot, and that look is also compared with the screen the tap aimed from: once the screen changed (a spinner, a disabled button, the target gone) the tap landed and is never sent again, however late until is, and tap returns at the end of the window; a screen that has not changed at all is tapped again from 750 ms on instead of after the whole window. A function until always gets the whole window. retries, windowMs and pollMs are INVALID_ARGUMENT without until. Use it for taps that are known to be lost sometimes (a tab bar still animating in); a plain tap() is one tap.

Fewer snapshots

Every read, action and expect looks at the screen. The runner keeps the last look for a moment and reuses it when nothing you did since can have changed the screen: a tap, a key, a launch, a settle, any call it does not know all drop it, and it is dropped when it is older than 150 ms on iOS or 1 s on Android. Only the first look of a read or an expect may reuse it (later polls are real looks). Anything that aims at an element does not reuse it as it is: tap, longPress, fill, type, clear, press, scrollTo and boundingBox take a new snapshot, because a snapshot taken while a screen was still sliding in has the element somewhere else, and the tap would land on its neighbour (the one exception, a plain tap on iOS that checks the kept look with a hit-test, is below). The confirmation reads that check a fill took are real as well, and so are device.snapshot() and device.screenshot(). So isVisible() then text() then expect(...) is one snapshot, and expect(a).toBeVisible() followed by a.tap() is two. On iOS, where a hit-test costs a fraction of a snapshot, a plain tap() within 150 ms of the look that saw its element, with no input since, takes no second snapshot: it hit-tests its tap point and taps when the hit is the same element in the same place (same role, identifier, label, enabled state and frame), and otherwise looks again as usual. A tap({ until }) and a test with systemAlerts set always take the fresh look. Changes the runner cannot see (the app's own timers, devtools.evaluate) show up within that age at most.

Assert

expect(locator) polls every 100 ms for the first second, then every 250 ms, until the check holds or assertionTimeout (default 5 s) runs out. Pass { timeout } to override one check. Every matcher takes timeout, and an option it does not know is INVALID_ARGUMENT. All but toHaveCount need exactly one match: none keeps polling, several fail at once with ASSERTION.

.not inverts any matcher: await expect(row).not.toBeVisible(). A negated matcher passes once its condition has held for 1 s without a break (or the whole timeout, if shorter). Where the driver can tell the app went quiet (iOS with the accessibility bridge, Android with warden-uiad), it asks once after the first poll that holds, provided no action reached the driver in the last second (any call that is not a read counts: a tap, typing, a key press, a settle, an orientation or appearance change); if the screen is quiet (250 ms without activity), one more fresh look that still holds passes, usually within about 300 ms. Right after an action the full second is held (the matcher's own settle counts, so a second .not straight after one that ended early holds the full second too): an app waiting on the network is quiet until the reply comes, so submit.tap() then expect(error).not.toBeVisible() still sees an error that arrives within that second (a slower reply needs an explicit wait either way). A screen that is quiet but about to change by itself long after the last input (a toast on a timer) can pass before the change; wait for that change explicitly. device.devtools.evaluate and reloadJs go to Metro, not the driver, so they do not count as an action: a .not right after one may end early, so tap, wait or assert something positive first. expect.soft(locator) records a failure and carries on; the test fails at the end if any soft check failed. For anything that is not a locator, expect is bun's own expect.

Organize specs

describe, hooks and settings

import { beforeEach, describe, expect, test } from "@delacour/warden-e2e";

describe("settings", () => {
  test.use({ assertionTimeout: 10_000 });

  beforeEach(async ({ screen }) => {
    await screen.getByTestId("tab-settings").tap();
  });

  test("shows the account", async ({ screen }) => {
    await expect(screen.getByTestId("settings-email")).toHaveText("ada@example.com");
  });
});

• describe(title, body) groups tests; the group title is part of every test title. describe(title, { serial }, body) takes serial only for parity: bun runs a file's tests one after another unless --concurrent.
• beforeEach / afterEach hooks run outermost scope first / innermost first, and are typed with the base fixtures (app, device, screen, platform); afterEach runs even when the test failed. To get the fixtures test.extend added, register the hook on the extended test (it.beforeEach(({ account }) => …), it.afterEach): it is typed with that test's fixtures and runs only for tests made with it (or extended from it). beforeAll / afterAll run once per file or describe with the same base fixtures and never see the ones added by test.extend.
• test.use({ timeout, actionTimeout, assertionTimeout, launch, systemAlerts }) applies to every test in the current file or describe. launch: "none" skips the fresh app.open() before each test. Put test.use({ timeout }) above the tests it is for: after a test in the same scope bun has already fixed that test's timeout, and warden prints a warning. An unknown systemAlerts value is INVALID_ARGUMENT.
• actionTimeout (default 30 s) bounds each tap, fill and wait for an element; assertionTimeout (default 5 s) bounds each expect(locator).
• systemAlerts: "accept" | "dismiss" | "off" (default "off") answers an alert that is in the way. With "accept" or "dismiss", a locator read, action or expect that sees an alert on top answers it (device.alert("accept") or "dismiss") and looks again on a fresh screen, three answers at most per look. It answers every alert, the app's own as well as the system's, so leave it "off" for a test that asserts on an alert. iOS's "Open in …?" sheet is the exception: "dismiss" answers it with Open (Cancel would only throw away a link the test opened), and while device.openLink is waiting on a link no read answers any alert, because openLink answers the sheet itself.

Your own fixtures

test.extend adds fixtures. A fixture sets up, calls use(value) and cleans up afterwards, even when the test failed:

import { test as base } from "@delacour/warden-e2e";

export const test = base.extend<{ email: string }>({
  email: async (_fixtures, use) => {
    await use(`user-${Date.now()}@example.com`);
  },
});

App and device

app is the app under test (spec.bundleId):

open({ url?, args?, env?, wait? }) · close() · restart() · reinstall() · clearData() · isRunning()

open always starts the app fresh: it terminates a running instance first (iOS: simctl launch --terminate-running-process; Android: am start -W -S on the launcher activity, which also waits for the first frame, or am force-stop then monkey when the launcher cannot be resolved), so a device an aborted run left mid-spec does not leak into the next test. It launches with the suite's launch arguments (--initialUrl <metro url> for a dev client) plus args, and env adds environment variables to the launched process; url is opened once the app is launched. Android cannot take launch arguments: the suite's are dropped with one log line, while an explicit args still fails the launch, and env is unsupported there. wait: "none" (the default) returns when the launch call does, before the app is on screen. wait: "quiet" then also waits for the first screen and for it to stop moving. A launch screen is idle while the app loads, so open reads the screen after 100 ms, 200 ms, then every 400 ms (fewer full reads while the app starts) until it shows content (a visible control, or any element with a testID; a storyboard or splash has neither), for at most 3 s, then settles (device.settle() with 250 ms of quiet, at most 3 s). It is best effort: a first screen with no control and no testID waits out the 3 s, an app that is not quiet in time is returned as it is, and a driver that cannot settle skips the settle. wait: { until: locator, timeout? } waits for that locator to be visible (a TIMEOUT after timeout, default the assertion timeout, like toBeVisible()), then settles once, so the next tap lands on a mounted screen; it replaces open() followed by toBeVisible(): await app.open({ wait: { until: screen.getByTestId("welcome.screen") } }). restart() is the same relaunch (one stop, not two); close() only stops. launch: "none" skips the runner's own open() and leaves the app as it is. clearData wipes the app's data and the keychain; reinstall uninstalls the app when it is installed and installs the build again from WARDEN_APP_PATH (INVALID_ARGUMENT when that is not set).

device is the simulator or emulator:

openLink does not wait out a fixed window. It reads the system alert every 100 ms for the first second and every 250 ms after, presses "Open" once the sheet has been up for half a second, since a sheet that is still animating in ignores touches (again if it stays up, three times at most, 600 ms apart) and returns once the sheet is gone. With no sheet it returns after about 1.2 s, as soon as the app is in front: a link that shows no sheet costs about 1.4 s, and later links with the same scheme in the same run return as soon as the app is in front, without reading alerts. While another app (SpringBoard, Safari) is in front it keeps looking for the sheet until waitMs (default 8000, the longest wait), then returns without an error. until takes a locator and returns as soon as it is visible and no sheet is up, which is the fastest way to wait for the screen the link leads to:

await device.openLink("myapp://todo/42", { until: screen.getByTestId("todo-detail") });

until is for a screen the app is not already showing; if waitMs passes first, the next assertion reports the missing screen. Android has no sheet, so openLink returns at once unless until is given. accept: false skips the sheet.

alert() asks the system for an alert once and returns false straight away when there is none. When a sheet is up but its buttons are not readable yet it looks again a few times, 200 ms apart.

Imports and errors

@delacour/warden-e2e exports test, describe, beforeAll, beforeEach, afterEach, afterAll, expect, E2eError, WARDEN_E2E_VERSION and the types App, OpenOptions, OpenWaitUntil, Device, Screen, Locator, LocatorReading, TapOptions, TapUntil, PinchOptions, RotateOptions, ScreenGestures, ScreenPinchOptions, ScreenRotateOptions, ScreenPathOptions, SystemAlerts and TestFixtures. Everything else is internal.

Every failure a spec sees is an E2eError (instanceof works) with a code: LOCATOR_NOT_FOUND, LOCATOR_AMBIGUOUS, INVALID_LOCATOR, ACTION_FAILED, NOT_ACTIONABLE, TIMEOUT, INVALID_ARGUMENT, ASSERTION, SESSION or DRIVER. A locator failure carries the screen as text (screen); an ambiguous one the number of matches. WARDEN_E2E_VERSION is the package version: the preload exits 2 in a spec process whose @delacour/warden-e2e version differs from the warden that started it (WARDEN_VERSION; a by-hand run compares nothing).

Failure artifacts

Every test attempt writes trace.jsonl; when a test fails, the runner adds the rest of its evidence next to the results:

<out>/<spec>/<test-title>-<hash>/
  failure.png            the device at the moment of failure
  screen-at-failure.txt  the full accessibility tree, with testIDs, labels and values
  trace.jsonl            every step and every device / driver / locator-read call (`span` lines), with timings (every test, passed or not)
  console.jsonl          dev client: JS console
  network.jsonl          dev client: network requests
  screenshots/           device.screenshot() images (any test)

Read screen-at-failure.txt first: a query that found nothing is almost always an element that is missing or not accessible there. <spec> is the spec's path under the suite's specsDir without .e2e.ts; <test-title> is a slug of the describe and test titles. Under warden e2e, <out> is <batchDir>/e2e/<worker>-<seq>/ (<worker>-<seq>-p<pass> on a later pass) and every test (skipped ones too; with retry, only the final attempt) also appends one line to results.jsonl there with file, title, status (passed, failed, skipped or timed-out), durationMs, startedAt, error, artifacts and device; see Debugging.

Run a spec by hand

Under warden e2e the environment is set for you. By hand, install the build on a simulator you have claimed, then point the spec at it:

warden claim ios --json                                     # -> udid
WARDEN_UDID=<udid> WARDEN_BUNDLE_ID=com.example.app \
  bun test --preload @delacour/warden-e2e/preload e2e/todos.e2e.ts

WARDEN_UDID switches the runner on: without it the preload does nothing, and with it WARDEN_BUNDLE_ID is required (an invalid environment exits 2). WARDEN_E2E_OUT sets where artifacts go (default .warden-e2e/<timestamp> under the current directory). Add WARDEN_PLATFORM=android for an emulator, WARDEN_METRO_URL or WARDEN_LAUNCH_ARGS (a JSON array) for a dev client, and WARDEN_E2E_DEBUG=1 to log every helper call with its duration. The full list is in environment variables. Release the device with warden release <leaseId> when you are done.

Without --preload, bun test fails with no driver factory configured. To drop it, add preload = ["@delacour/warden-e2e/preload"] under [test] in bunfig.toml; warden e2e passes the preload itself.