e2e framework benchmark — ios 2026-10-25

Run eas-ios-argent, started 2026-10-25T12:30:40.328Z

Host
VirtualMac2,1, Apple M4 Pro (Virtual), 10 cores, 40.0 GB RAM, macOS 26.5.2
Toolchain
Xcode 26.6, Java 17.0.19, Node 22.23.1, bun 1.3.14, warden 0.1.1
Host load at start
5.09
Device
warden-iphone-17-1 (ios, iOS 26.5), udid 9AECFBA5-3281-4058-B4FA-9E80053A1546
App
nz.co.delacour.warden.example, hash import:2d35643eda49, sha256 2d35643eda49, 96.4 MB
Options
runs 5, retries 0, reboot between no, caffeinated yes
Frameworks
argent 0.30.1

Fairness rules

Summary

FrameworkStatusVersionSuite median (warm)p95RatioCold rep 1Cold-start costPrepareTeardownOverheadPass rateFlakeResidueFootprint (LOC / deps)
argent✓ ok0.30.1131.81 s132.14 s1.00×132.43 s2.35 s1.73 s511 ms168 ms100%0%0 proc / 0 app / 1 path161 LOC / 0 B

Warm suite wall time

Bar = warm median. Whisker = p95. Dot = cold rep 1. Lower is faster; ratios are against the fastest framework.

Warm suite median per framework with p95 whisker and cold rep 1 markerargent 0.30.1: argent 131.81 s · 1.00×050 s100 s150 sargent 0.30.1argent: 131.81 s · 1.00× (p95 132.14 s, cold rep 1 132.43 s)131.81 s · 1.00×

Per-test medians (ms)

Testargent
onboarding steps through to sign-in8220 (1.00×)
invalid email and short password show field errors17962 (1.00×)
a locked account shows the form alert14040 (1.00×)
valid credentials land on todos14982 (1.00×)
add, complete and filter todos20867 (1.00×)
open a todo, mark it done, then delete it18881 (1.00×)
deep link opens a todo by id17367 (1.00×)
settings shows the account, toggles switches and signs out19116 (1.00×)

Median per test

Warm median per test and framework, in millisecondsonboarding steps through to sign-in: argent 8220 ms · 1.00×. invalid email and short password show field errors: argent 17962 ms · 1.00×. a locked account shows the form alert: argent 14040 ms · 1.00×. valid credentials land on todos: argent 14982 ms · 1.00×. add, complete and filter todos: argent 20867 ms · 1.00×. open a todo, mark it done, then delete it: argent 18881 ms · 1.00×. deep link opens a todo by id: argent 17367 ms · 1.00×. settings shows the account, toggles switches and signs out: argent 19116 ms · 1.00×010 s20 s30 sonboarding steps through to sign-inargent: 8220 ms · 1.00×8220 ms · 1.00×invalid email and short password show field errorsargent: 17962 ms · 1.00×17962 ms · 1.00×a locked account shows the form alertargent: 14040 ms · 1.00×14040 ms · 1.00×valid credentials land on todosargent: 14982 ms · 1.00×14982 ms · 1.00×add, complete and filter todosargent: 20867 ms · 1.00×20867 ms · 1.00×open a todo, mark it done, then delete itargent: 18881 ms · 1.00×18881 ms · 1.00×deep link opens a todo by idargent: 17367 ms · 1.00×17367 ms · 1.00×settings shows the account, toggles switches and signs outargent: 19116 ms · 1.00×19116 ms · 1.00×

Spread across warm reps

Whisker = min to max. Shaded box = min to p95. Tick = median. A distribution carries no quartiles, so the box is not an interquartile range.

Warm rep spread per test: min to max whisker, min to p95 box, median tickonboarding steps through to sign-in: argent argent 8.22 s. invalid email and short password show field errors: argent argent 17.96 s. a locked account shows the form alert: argent argent 14.04 s. valid credentials land on todos: argent argent 14.98 s. add, complete and filter todos: argent argent 20.87 s. open a todo, mark it done, then delete it: argent argent 18.88 s. deep link opens a todo by id: argent argent 17.37 s. settings shows the account, toggles switches and signs out: argent argent 19.12 s010 s20 s30 sonboarding steps through to sign-inargent: argent 8.22 s (p95 8.29 s, min 8096, max 8293, n 4)argent 8.22 sinvalid email and short password show field errorsargent: argent 17.96 s (p95 18.14 s, min 17869, max 18142, n 4)argent 17.96 sa locked account shows the form alertargent: argent 14.04 s (p95 14.14 s, min 13968, max 14136, n 4)argent 14.04 svalid credentials land on todosargent: argent 14.98 s (p95 15.20 s, min 14942, max 15203, n 4)argent 14.98 sadd, complete and filter todosargent: argent 20.87 s (p95 21.11 s, min 20517, max 21113, n 4)argent 20.87 sopen a todo, mark it done, then delete itargent: argent 18.88 s (p95 19.07 s, min 18699, max 19066, n 4)argent 18.88 sdeep link opens a todo by idargent: argent 17.37 s (p95 17.55 s, min 17142, max 17554, n 4)argent 17.37 ssettings shows the account, toggles switches and signs outargent: argent 19.12 s (p95 19.41 s, min 19050, max 19414, n 4)argent 19.12 s

Per-step latency

Whisker = min to max. Shaded box = min to p95. Tick = median. Only frameworks that report steps appear.

Step latency per frameworkargent (1192 steps): argent 318 ms · p95 1.82 s01 s2 s3 sargent (1192 steps)argent: 318 ms · p95 1.82 s318 ms · p95 1.82 s
FrameworkStepsMedianp95Max
argent1192318 ms1.82 s2.82 s

Resources

CPU

CPU use per frameworkHost CPU average: argent 1.7%. Host CPU peak: argent 39.7%. Simulator CPU average: argent 131.3%. Simulator CPU peak: argent 788.2%0%200%400%600%800%Host CPU averageargent: 1.7%1.7%Host CPU peakargent: 39.7%39.7%Simulator CPU averageargent: 131.3%131.3%Simulator CPU peakargent: 788.2%788.2%

Memory

Peak resident memory per frameworkHost RSS peak: argent 261.8 MB. Simulator RSS peak: argent 36.5 GB0 B9.3 GB18.6 GB27.9 GB37.3 GBHost RSS peakargent: 261.8 MB261.8 MBSimulator RSS peakargent: 36.5 GB36.5 GB
FrameworkScopeCPU-s / repCPU avgCPU peakRSS avgRSS peak
argenthost2.79 s1.7%39.7%182.0 MB261.8 MB
argentsim—131.3%788.2%36.3 GB36.5 GB

CPU-s / rep is the host CPU time of the timed run (warm median) and the headline. CPU avg and peak are ps %cpu, a decaying average on macOS: they move with suite length, do not judge on them.

Where the CPU goes

One table per framework, from the warm-median rep's timed run (the rep whose host CPU-seconds is the lower median). Rows are processes grouped by label (top 12 per scope, the rest in other); host is the framework's own process tree, sim the simulator's launchd_sim tree (iOS only). Share is of the scope's CPU-seconds. RSS is the label's summed RSS per 500 ms tick (0 where it had no process). Spawns counts distinct pids a tick saw: a process that lives between two ticks is only seen through the (exit charge) rows (CPU of a child the bench ran, booked at its exit), so many short-lived xcrun simctl calls read low in Spawns and in CPU.

argent

ProcessScopeCPU-sShareRSS avgRSS peakSpawns
simulator-serverhost2.50 s92.0%96.6 MB96.7 MB1
node argenthost0.21 s7.8%85.7 MB87.9 MB1
node argent (exit charge)host0.01 s0.2%0 B0 B0
(node)host0.00 s0.0%0 B0 B0
node tool-server.cjshost0.00 s0.0%0 B0 B0
simctlhost0.00 s0.0%0 B0 B0
system_profilerhost0.00 s0.0%0 B0 B0
iOSsim98.77 s74.0%34.8 GB34.9 GB287
wardenexamplesim29.50 s22.1%375.5 MB435.1 MB8
bun cli.tssim2.78 s2.1%298.6 MB308.0 MB1
simctlsim0.41 s0.3%363.8 KB20.5 MB7
CoreSimulatorBridgesim0.40 s0.3%117.8 MB118.2 MB1
launchd_simsim0.37 s0.3%52.2 MB52.2 MB1
pssim0.29 s0.2%1.9 MB2.0 MB260
(mlhostd)sim0.26 s0.2%222.5 KB40.5 MB2
trialdsim0.20 s0.1%96.7 MB96.7 MB1
storekitdsim0.19 s0.1%128.4 MB128.4 MB1
(managedappdistri)sim0.14 s0.1%214.5 KB54.5 MB1
mlhostdsim0.13 s0.1%119.3 KB15.7 MB2
othersim0.05 s0.0%444.5 MB453.2 MB9

Residue

Unsupported / failed

None.