Ask
87
@overlap_owen ·

playwright suite passes on my laptop, 7 specs time out only on ubuntu-latest CI

60 specs, webServer runs a production build and serves it. Green locally in 42 seconds. On ubuntu-latest I get seven failures, always Test timeout of 30000ms exceeded on a .click(), and never the same seven twice.

The traces show the page rendered, the button is visibly on screen, and the click just sits there until the test dies. workers is 4 in both places.

Is the answer genuinely "buy bigger runners"?

9 answers Share
Report

Answering anonymously — a moderator will review it first.

  • @lunchboxrotation · 3h ago · 3 replies

    No. The answer is workers: process.env.CI ? 2 : undefined.

    That runner is 2 vCPU shared. Four chromium contexts plus your server plus node means everything is starved, and Playwright's actionability checks are the first thing to notice - before it clicks, it waits for the element to be stable across two animation frames, visible, and not covered. On a machine where a frame takes 400ms, "stable for two frames" stops being instant and starts being a race with your own animations.

    That is exactly your symptom: the button is there in the screenshot, and the click is waiting, because the thing it is waiting for is not visibility.

    Expand the failed action in the trace and you will find the stability line rather than a "not found" line. Two workers on a 2-core box is usually faster in wall clock than four, because you stop paying for retries.

    134
    Share
    Reply

    Answering anonymously — a moderator will review it first.

    Report
    • @row_level_sam · 3h ago

      Worth spelling out because it catches everyone: people look at the trace screenshot, see the button plainly rendered, and conclude the tool is lying to them. The action log underneath is where the actual reason lives.

      46
      Share
      Reply

      Answering anonymously — a moderator will review it first.

      Report
    • @fg_stall · 3h ago

      We went 4 workers to 2 and total suite time went down 90 seconds. Fewer workers, no retries, less time. Very unintuitive until you see it once.

      19
      Share
      Reply

      Answering anonymously — a moderator will review it first.

      Report
  • @nightbus_nina · 3h ago

    Also check what webServer is actually waiting for. If url points at / and / is a static shell, the server is "ready" before the database pool has connected. Point it at a health route that touches the db, and set reuseExistingServer: !process.env.CI.

    On a cold runner the first real request can spend 8-10 seconds connecting and warming, which comes straight out of the 30 second budget of whichever spec happens to go first. That also explains "never the same seven twice".

    71
    Share
    Reply

    Answering anonymously — a moderator will review it first.

    Report
  • @r2_bucketeer · 3h ago

    Bigger runners are real and cost roughly double per minute for double the cores. If the suite is on the critical path for every deploy that can be the cheapest fix you will ever buy. Do the workers change first though, otherwise you are paying more money for the same starvation with a longer wick.

    29
    Share
    Reply

    Answering anonymously — a moderator will review it first.

    Report
  • @kombucha_kai · 3h ago

    Turn on trace: 'on-first-retry' and upload test-results/ as an artifact. Ten minutes of setup and it pays for itself the first time this happens while you are asleep.

    If you want to reproduce it locally, run the suite pinned to two cores. Same starvation, on a machine where you can attach a debugger.

    44
    Share
    Reply

    Answering anonymously — a moderator will review it first.

    Report
  • @blown_cap · 3h ago · 2 replies

    Set timeout: 60000 in the config and move on. CI is slower than your laptop. That is the whole story.

    26
    Share
    Reply

    Answering anonymously — a moderator will review it first.

    Report
    • @hollis_pike · 3h ago

      That turns seven failures into seven tests that take a full minute each and then fail again in three weeks when the runner is busier. Raising the timeout is correct when the operation is genuinely slow - a large upload, a real redirect to a payment page. It is the wrong tool when the machine is oversubscribed, because the thing you extended is the queueing, not the work.

      19
      Share
      Reply

      Answering anonymously — a moderator will review it first.

      Report
  • @thriftedtweed · 3h ago

    --shard=1/3 across three jobs gets you three 2-core machines in parallel for the same per-minute price. Combined with two workers each you end up with six contexts spread over six real cores instead of four fighting over two.

    10
    Share
    Reply

    Answering anonymously — a moderator will review it first.

    Report