tester-army/e2e writes browser tests as goals and replays the clicks

tester-army/e2e is TesterArmy’s open-source browser testing framework, called e2e below (a product name, not the generic term). In place of a list of clicks you write a goal in one sentence, and a model works out the clicks on your real app. Each working step is recorded: the control’s role, name and surroundings, plus what the step changed on the page. The next run replays the recording without the model, and if it stops matching the page, the model takes over. e2e drives the browser through Playwright, but its test API is its own, so existing Playwright tests would need rewriting.

There are two ways to run it. By default a stale recording heals itself and the build stays green, at the cost of model calls and of sometimes reaching the goal through a different button when the intended one is broken; our headline results are from that mode. In strict mode (the --strict-cache flag) a stale recording fails the test, and the fix is a re-recorded run instead of a code edit.

Tested: commit cae1028a7a5c820a3543eee823dfa77bc3cdc086 on October 13, 2026, with e2e 0.19.0, @e2e-dev/web 0.14.0 and OpenAI’s low-cost gpt-6-luna model. Workflow: run the same ten regression tests for a demo shop in e2e and in plain Playwright, then change the shop five ways and plant six bugs, one at a time. Result: in default mode e2e passed all ten tests through a rename and a menu with no edits, where Playwright needed two-line fixes; the tools lost the same tests on a new required field and on changed confirmation text, and moving two buttons, which Playwright ignored, cost e2e a re-record. Each caught five of five bugs that had no workaround; e2e missed a sixth by using a different button, unless run in strict mode. Replay made no model calls and took two to three times as long as Playwright. Verdict: try it on a suite that finds controls by visible name and whose buttons keep being renamed; not worth it with test ids, rare renames, or an agent that already patches your selectors.

Set up e2e with your coding agent

Paste this into Claude Code, or another coding agent that can run commands, to install e2e from npm and add one test to your own project.

Set up e2e (https://github.com/tester-army/e2e) for one browser test in this project. Go one step at a time.
1. Check Node is 24.8 or newer (on Node 22, 22.22.3 or newer). Export E2E_TELEMETRY_DISABLED=1 for the whole session, then run: npm install --save-dev e2e @e2e-dev/web ai@^7
   Then install the browser: npx @e2e-dev/web install chromium --with-deps (ask me first if it wants sudo).
2. Run `npm ls e2e`. It should show e2e 0.19.0 or newer. Stop if it shows nothing.
3. Read the setup guide that ships in the package, then show me the config and file changes before you make them. I will give the model provider key myself; never read or print keys.
4. Write one test for a flow I name: one goal sentence, then exact checks of what the page should show afterwards. Set maxModelCalls [the most model calls one goal may use] to a number I choose.
5. Run the test once with no recordings, then run it again to replay the recording. Explain the default and --strict-cache modes for CI and ask which I want.
6. Tell me what you installed, what you changed, how many model calls each run made, and anything that failed. Do not retry a failure more than once without asking.

A test in e2e is a goal sentence plus the checks you write

A coding agent wrote both suites for a small shop with sign-in, a plan upgrade, a cart and a checkout. The shop has no test ids, so both find controls by role and visible text.

The demo shop, signed in, after a declined test payment. Upgrade to Pro and Remove Perch mug are the two buttons we later moved and renamed; Confirm upgrade appears once Upgrade to Pro is pressed.

Here is one of the ten tests as written for e2e. The goal is the agent.act line; signIn, preview and billing are helpers built on e2e’s own locators:

test('03 the Pro invoice preview opens without confirming', async ({ app, agent, screen }) => {
  await signIn(app, screen);
  await agent.act('open the invoice preview for upgrading to Pro, without confirming the upgrade');
  await expect(preview(screen)).toBeVisible();
  await expect(preview(screen).getByRole('heading', 'Invoice preview')).toBeVisible();
  await expect(preview(screen).getByText('Prorated amount: $10.00')).toBeVisible();
  await expect(billing(screen).getByText('Current plan: Free')).toBeVisible();
});

The Playwright version has the same checks; in place of the goal line it calls getByRole('button', { name: 'Upgrade to Pro' }).click(). The e2e version never names the button, so the rename did not break the action. Both tests name the exact text they expect afterwards, and that breaks in either tool when the text changes.

Replay made no model calls and took two to three times as long as Playwright

A cold run has no recordings, so the model chooses every action. We ran the unchanged shop cold, then replayed it, three times. The first pair (output trimmed; the 46% is the model provider’s prompt cache, not e2e’s recordings):

 Test Files  1 passed (1)
      Tests  10 passed (10)
         AI  248.3k tokens · 46% cached · 56 model calls · openai.responses/gpt-6-luna
      Cache  12 missed
   Duration  151.57s (2m 32s, startup 96ms)

and its replay:

 Test Files  1 passed (1)
      Tests  10 passed (10)
      Cache  12 replayed
   Duration  17.55s (startup 115ms)

The suite has 12 goals across its ten tests, and each goal gets one recording, a JSON file under .e2e/cache/. All three cold runs passed ten of ten, with 46 to 56 model calls in 97 to 152 seconds and under three cents each at gpt-6-luna’s list price. All three replays passed with no model calls in 13 to 18 seconds; Playwright ran the same ten tests in 6.0 seconds.

e2e passed the rename and menu unedited; a new field stopped both tools

We changed the shop five ways, one at a time, each starting from the unchanged shop and its recordings, with e2e in default mode. e2e ran at least three times per change, with the same pass count every time; Playwright ran once.

Change to the shop	e2e passed	Playwright passed	What e2e needed	What Playwright needed
Upgrade to Pro and Remove Perch mug buttons moved within their panels	10/10	10/10	no edit; re-recorded in a 68–78 s run	nothing
The same two buttons renamed, to Review Pro upgrade and Delete Perch mug	10/10	6/10	no edit; re-recorded in a 107–117 s run	2 lines, 1 try, 25 s
Upgrade to Pro moved into a menu	10/10	7/10	no edit; re-recorded in a 92–103 s run	2 lines, 3 tries, 203 s
Required postcode field added to checkout	8/10	8/10	2 lines, 3 tries, 132 s	1 line, 1 try, 25 s
Confirmation text changed	9/10	9/10	2 lines, 1 try, 45 s	2 lines, 1 try, 30 s

A coding agent made each edit from the failing suite and its errors. Cells with lines and tries give the agent’s whole session; re-recorded cells give one suite run. Every edited suite then passed ten of ten.

On the rename, e2e’s log for the upgrade test shows the recording failing to replay:

cache: no replay: a recorded target is not on the screen; at action 1 of 1, tap button "Upgrade to Pro" in "Billing"; recording saved

The model then found the button under its new name and saved a new recording in the same run; the next run replayed all ten tests with no model calls. The menu went the same way. These re-record runs made 10 to 15 model calls, under a cent at list price, yet took 92 to 117 seconds against 13 to 18 for a replay (we did not isolate why), so commit the new recordings or every CI run pays that. The move broke nothing in Playwright and still cost e2e a re-record: moving the remove button within its cart row made three recordings in that row stale.

So a rename costs Playwright an edit and a commit, and costs e2e a slower run and a commit of refreshed recordings. The saving is the edit, not the run time, and only where tests find controls by visible name, which is what Playwright’s docs recommend. Our shop had no test ids; in a one-test check on a copy that had them, a Playwright test using test ids survived the rename untouched.

On the renamed shop, --strict-cache failed five tests with no model calls; the fix is a local re-record with a normal npx e2e run (cache docs). That is one more than Playwright, because the recording for one test’s sign-in goal also expected to see the Upgrade to Pro button. Replaying committed recordings needed no model access: with the model unreachable, all ten tests still passed.

A new required field needs its value in the test

The added postcode field is where the model’s takeover did not help. The checkout page said which postcode to use, and with the existing recordings both payment tests still failed in every one of six runs of the suite. One failure:

agent.act failed: The card number and name are present, but Checkout also requires a delivery postcode (the screen says to use A1A1A1). Tapping Pay now produced no change, so the payment was not shown as submitted.

The model took over after the recorded Pay now tap, quoted the postcode from the page and still did not type it. Started cold, with no recording at all, the model passed both tests in one run and failed one of two in another, so with this low-cost model the value belongs in the test, as e2e’s authoring guide says. The fix added the postcode to the params of both payment goals, and took the agent three tries and 132 seconds against one line and 25 seconds for Playwright. The goal sentence did not change:

await agent.act('pay for the cart in Checkout with card number {card} and name on card {name}', {
  params: { card: '4242', name: 'Robin', deliveryPostcode: 'A1A1A1' },
});

The changed confirmation text failed one exact-text check in each tool, as it should.

Both tools caught five planted bugs, and e2e missed a sixth by using another button

We planted six bugs in the unchanged shop, one at a time, three runs per bug in each tool: five with no way around them (wrong charges, totals and counts, and a declined card shown as paid) and a broken Confirm upgrade button. For the first five, the covering test failed in every run in both tools. Of those 30 failing runs, 29 failed on a hand-written check and one, in e2e’s suite, on the goal itself. One such check, after a test raises the mug quantity:

expect.toBeVisible failed
locator: getByRole("region", name: "Your cart") >> getByText("Cart total: $40.00")
expected: visible
observed: no node (match count 0)

The sixth bug had a way around it: we broke Confirm upgrade, which now only showed a message pointing to a second button, Complete upgrade. Playwright clicked the broken button and failed. e2e’s goal was “confirm the upgrade shown in the invoice preview”; in all three default-mode runs the recorded tap did nothing, the model took over and tapped the other button, and the test passed. If the test exists to prove that button works, that is a missed bug: a goal checks that an outcome is reachable, not that one control works. In three strict-mode runs the same test failed with REPLAY_STALE and no model calls. Where one specific control must work, you can also tap it by name in place of a goal (we did not test this): screen.getByRole('button', 'Confirm upgrade').tap().

e2e’s explore mode hunts for bugs with no written tests. Four runs on a shop with all six bugs planted each found three of the six, four between them. Of 21 reported findings (eight repeat another), two were false and about a quarter had a wrong detail, so reproduce a finding before you act on it.

e2e is newly open-sourced, pre-1.0, and sends telemetry by default

e2e is Apache-2.0 and maintained by TesterArmy, a testing company that open-sourced it on October 1, 2026, 12 days before this test. The README warns that minor releases can change APIs before 1.0, so pin your versions. Telemetry is on by default; turn it off with E2E_TELEMETRY_DISABLED=1 or npx e2e telemetry disable.

The README also covers iOS and Android, hosted browsers and a GitHub pull-request reporter; we tested the web engine on Chromium, with one model, on one small shop whose controls all have roles and names.

Try e2e where buttons keep being renamed

Try e2e on a suite that finds controls by visible name and whose buttons get renamed or tucked into menus often. A trial is one new test beside your existing suite and costs cents, and what you are trying is what plain Playwright did not do here: in default mode a rename heals in the run itself, with nobody editing a test. Go in knowing the costs: e2e is pre-1.0, existing Playwright tests must be rewritten, replays took two to three times as long, and in default mode a goal can pass through a different button when the intended one is broken. --strict-cache catches that, but then a rename fails the build as it does in Playwright and the fix is a local re-record, so choose per suite. If your suite uses test ids, renames are rare, or you are content to let an agent patch selectors, skip it.