Nuxt•9mo ago

Playwright 🎭 Tests parallelizations's execution times issue

Hi guys, I'm using playwright to test my app, but I can use parallelism without exploding test timing if I use :

import { expect, test } from '@nuxt/test-utils/playwright'
import { fileURLToPath } from 'node:url'

test.use({
  trace: 'on-first-retry',
  nuxt: {
    rootDir: fileURLToPath(new URL('../playground/', import.meta.url))
  }
})

test('Test1', async ({ page, goto }) => {
  await goto('/', { waitUntil: 'hydration' })
  await expect(page.getByRole('heading').first()).toHaveText('Welcome')
})

import { expect, test } from '@nuxt/test-utils/playwright'
import { fileURLToPath } from 'node:url'

test.use({
  trace: 'on-first-retry',
  nuxt: {
    rootDir: fileURLToPath(new URL('../playground/', import.meta.url))
  }
})

test('Test1', async ({ page, goto }) => {
  await goto('/', { waitUntil: 'hydration' })
  await expect(page.getByRole('heading').first()).toHaveText('Welcome')
})

My tests are working nicely, and I have the playwright --ui working great too. But the tests suite is SO LONG if I have the option fullyParallel: true. Of course, it have to build the app, and serve it before doing the first test, but it's like it is doing this for every test("..", async({page, goto})){ ... } So I tried to start the server myself and use baseURL: "http://localhost:3000" to force using the same server, like:

import { expect, test } from '@nuxt/test-utils/playwright'
import { fileURLToPath } from 'node:url';

test.use({
  baseURL: "http://localhost:3000",
  trace: 'on-first-retry',
  nuxt: {
    // rootDir: fileURLToPath(new URL('../playground/', import.meta.url))
  }
})

test('Test1', async ({ page, goto }) => {
  await goto('/', { waitUntil: 'hydration' })
  await expect(page.getByRole('heading').first()).toHaveText('Welcome')
})

import { expect, test } from '@nuxt/test-utils/playwright'
import { fileURLToPath } from 'node:url';

test.use({
  baseURL: "http://localhost:3000",
  trace: 'on-first-retry',
  nuxt: {
    // rootDir: fileURLToPath(new URL('../playground/', import.meta.url))
  }
})

test('Test1', async ({ page, goto }) => {
  await goto('/', { waitUntil: 'hydration' })
  await expect(page.getByRole('heading').first()).toHaveText('Welcome')
})

My tests seems to work correclty, and with a fast execution time, nice ! But then the playwright --ui is almost unusable at all because it doesn't show anything in its right part (timeline etc) So I guess using rootDir option is the correct way to go as the doc mention, but is possible to make parallel test using the same nuxt server ? Or did I miss something in the configuration ? Reproduction link: https://github.com/lateek35/nuxt-playwright-issue-reproduction I added 5 identical tests: - With 5 tests: 73sec to complete on my machine - With 1 test (comment the 4 others): 20sec

GitHub

GitHub - lateek35/nuxt-playwright-issue-reproduction

Contribute to lateek35/nuxt-playwright-issue-reproduction development by creating an account on GitHub.

7 Replies

LATEEKOP•9mo ago

Anything else I can add to help? Maybe should I ask playwright's team instead ?

Bjorn van der Neut•9mo ago

I guess that will make more sense. They also have an Discord channel.

LATEEKOP•9mo ago

After more investigation it really seems to be linked the way nuxt handle things in https://github.com/nuxt/test-utils/blob/main/src/playwright.ts, probably in hooks.setup(). I don't directly use the expect, test of playwright but from @nuxt/test-utils/playwright which is what the documentation mention to make use of the Playwright first-class support.

dwol•9mo ago

Did you manage to find a solution to this? I'm kinda running into a similar issue. Seems like when I use test/expect from the @playwright/test package it runs quite fast but when using test/expect from nuxt test utils it is extremely slow.

LATEEKOP•9mo ago

Not really, for the moment I'm telling playwright to use my own application instance using "baseURL", but as mentioned, it makes the playwright UI empty, so we're blind during the test run. I'm still not sure if its intended that multiples workers run its own nuxt application build but didn't get any answer on that part. However, if it's intended, it will make workers option unusable for me as it increase the testing time sooo much (it also rebuild the app during a retry). So at least, I think we should be able to run test against our own application instance AND keep the playwright --ui working. For this option, I've good hope in this PR: https://github.com/nuxt/test-utils/pull/803

dwol•9mo ago

Thank you, yes that pr is exactly what I'm looking for haha

Maloooo•6mo ago

Hey, jumping into this interesting discussion! Thanks for the test.use, it stops building!! But I am surprised, it isn't possible to globally configure that into the playwright.config.ts ? 😕

Gaming

Programming

Playwright 🎭 Tests parallelizations's execution times issue