Nuxt•3mo ago

Vitest slow tests compared to jest

Hey, I can't really share more than the config i got right after this message, but we switched from Jest to Vitest for our tests (vitests with nuxt environment ofc) and the speed for the tests really increased, i wanted to know if it a slower execution was to be expected or not ? (it more that quadrupled so i guess we have other problems, and running with --pool=forks didn't change anything)

// vitest.config.ts
import { defineVitestConfig } from '@nuxt/test-utils/config'
import { configDefaults } from 'vitest/config'

export default defineVitestConfig({
    test: {
        fakeTimers: {
            ...configDefaults.fakeTimers,
            toFake: [...configDefaults.fakeTimers.toFake, 'performance']
        },

        reporters: process.env.GITHUB_ACTIONS
            ? ['basic', 'github-actions']
            : ['default'],
        environment: 'nuxt',
        environmentOptions: {
            nuxt: {
                overrides: {
                    imports: {
                        dirs: ['~/composables/**']
                    }
                }
            }
        },
        include: ['./tests/**/*.spec.ts'],
        globals: true,
        globalSetup: './tests/vitest.global.setup.ts',
        setupFiles: ['./tests/vitest.setup.ts'],
        server: {
            deps: {
                inline: ['vitest-canvas-mock']
            }
        },
        coverage: {
            reporter: ['lcov', 'text-summary'],
            include: [
                'composables/**/*.ts',
                'components/**/*.vue',
                'pages/**/*.vue'
            ],
            reportsDirectory: './coverage'
        }
    }
})

// vitest.config.ts
import { defineVitestConfig } from '@nuxt/test-utils/config'
import { configDefaults } from 'vitest/config'

export default defineVitestConfig({
    test: {
        fakeTimers: {
            ...configDefaults.fakeTimers,
            toFake: [...configDefaults.fakeTimers.toFake, 'performance']
        },

        reporters: process.env.GITHUB_ACTIONS
            ? ['basic', 'github-actions']
            : ['default'],
        environment: 'nuxt',
        environmentOptions: {
            nuxt: {
                overrides: {
                    imports: {
                        dirs: ['~/composables/**']
                    }
                }
            }
        },
        include: ['./tests/**/*.spec.ts'],
        globals: true,
        globalSetup: './tests/vitest.global.setup.ts',
        setupFiles: ['./tests/vitest.setup.ts'],
        server: {
            deps: {
                inline: ['vitest-canvas-mock']
            }
        },
        coverage: {
            reporter: ['lcov', 'text-summary'],
            include: [
                'composables/**/*.ts',
                'components/**/*.vue',
                'pages/**/*.vue'
            ],
            reportsDirectory: './coverage'
        }
    }
})

I'll try to provide a repro really soon

9 Replies

kapa.ai•3mo ago

Hi @Numil. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave: ---------------------- Based on the information provided in the knowledge sources, Vitest is generally expected to be faster than Jest, not slower. The significant slowdown you're experiencing (more than quadrupling the execution time) is not typical and suggests there may be other issues at play. Here are some points to consider: 1. Nuxt Environment: You're using the Nuxt environment for Vitest, which is correct for a Nuxt project. The Nuxt documentation mentions that they currently only have support for Vitest for unit testing code that needs a Nuxt runtime environment. 2. Configuration: Your Vitest configuration looks reasonable and follows the structure suggested in the Nuxt documentation. However, some of your specific settings might be impacting performance: - The fakeTimers configuration might affect performance, especially if you're faking a large number of timers. - The globalSetup and setupFiles could potentially introduce slowdowns if they're performing time-consuming operations. - The coverage settings might add some overhead, especially if you're running coverage on a large codebase. 3. Pool Option: You mentioned that using --pool=forks didn't change anything. This suggests that the issue might not be related to the threading model. 4. Possible Issues: The Nuxt documentation mentions that when running tests within the Nuxt environment, they will be running in a happy-dom or jsdom environment, and a global Nuxt app will be initialized. This initialization could potentially cause slowdowns if there are issues with your Nuxt configuration or plugins. To troubleshoot this issue: 1. Try running a minimal set of tests without the Nuxt environment to see if the problem persists. 2. Gradually add back configuration options and test files to identify what might be causing the slowdown. 3. Check for any heavy operations in your globalSetup or setupFiles. 4. Consider using Vitest's built-in profiling tools to identify bottlenecks. Without more specific information about your tests and codebase, it's difficult to pinpoint the exact cause of the slowdown. Creating a minimal reproduction of the issue, as you mentioned you're planning to do, will be very helpful in diagnosing the problem. ---------------------- Note: you can ask me a follow up question by @ mentioning me again :speech_balloon: ----------------------

kapa.ai•3mo ago

<@108292831583440896> Kapa.ai is still learning and improving, please let me know how I did by reacting below.

NumilOP•3mo ago

@kapa.ai i answered the mesasge with the missing config files

kapa.ai•3mo ago

Let me check the knowledge base for you again. One moment! :books: ---------------------- I apologize, but I don't have access to any additional configuration files or messages that you might have sent. The knowledge sources provided to me don't contain any new information beyond what was in your original question. To provide you with an accurate and helpful response, I would need to see the specific configuration files or additional information you're referring to. Without this context, I can't make any assumptions about what might be causing the performance issues with your Vitest setup. If you'd like to share the missing configuration files or any other relevant information, please provide them in your next message. This will allow me to give you a more informed and tailored response to your Vitest performance concerns. In the meantime, if you have any other questions about Nuxt or its testing capabilities that I can answer based on the information available to me, please feel free to ask. ---------------------- Note: you can ask me a follow up question by @ mentioning me again :speech_balloon: ----------------------

kapa.ai•3mo ago

<@108292831583440896> Kapa.ai is still learning and improving, please let me know how I did by reacting below.

NumilOP•3mo ago

@kapa.ai Would those config files affect performance for the tests: vitest.setup.ts

import { config } from '@vue/test-utils'
import { createI18n } from 'vue-i18n'
import 'vitest-canvas-mock'

const i18n = createI18n({
    locale: 'en',
    legacy: false,
    missing: (_, key) => key,

    numberFormats: {
        en: {
            currency: {
                style: 'currency',
                currency: 'EUR',
                minimumFractionDigits: 2,
                maximumFractionDigits: 4
            },
            decimal: {
                style: 'decimal',
                maximumFractionDigits: 1
            },
            rating: {
                minimumFractionDigits: 1,
                maximumFractionDigits: 1
            }
        },
        de: {
            currency: {
                style: 'currency',
                currency: 'EUR',
                minimumFractionDigits: 2,
                maximumFractionDigits: 4
            },
            decimal: {
                style: 'decimal',
                maximumFractionDigits: 1
            },
            rating: {
                minimumFractionDigits: 1,
                maximumFractionDigits: 1
            }
        },
        fr: {
            currency: {
                style: 'currency',
                currency: 'EUR',
                minimumFractionDigits: 2,
                maximumFractionDigits: 4
            },
            decimal: {
                style: 'decimal',
                maximumFractionDigits: 1
            },
            rating: {
                minimumFractionDigits: 1,
                maximumFractionDigits: 1
            }
        }
    }
})

config.global.plugins.push(i18n)

beforeEach(() => {
    vi.useFakeTimers()
})

afterEach(() => {
    vi.useRealTimers()
})

import { config } from '@vue/test-utils'
import { createI18n } from 'vue-i18n'
import 'vitest-canvas-mock'

const i18n = createI18n({
    locale: 'en',
    legacy: false,
    missing: (_, key) => key,

    numberFormats: {
        en: {
            currency: {
                style: 'currency',
                currency: 'EUR',
                minimumFractionDigits: 2,
                maximumFractionDigits: 4
            },
            decimal: {
                style: 'decimal',
                maximumFractionDigits: 1
            },
            rating: {
                minimumFractionDigits: 1,
                maximumFractionDigits: 1
            }
        },
        de: {
            currency: {
                style: 'currency',
                currency: 'EUR',
                minimumFractionDigits: 2,
                maximumFractionDigits: 4
            },
            decimal: {
                style: 'decimal',
                maximumFractionDigits: 1
            },
            rating: {
                minimumFractionDigits: 1,
                maximumFractionDigits: 1
            }
        },
        fr: {
            currency: {
                style: 'currency',
                currency: 'EUR',
                minimumFractionDigits: 2,
                maximumFractionDigits: 4
            },
            decimal: {
                style: 'decimal',
                maximumFractionDigits: 1
            },
            rating: {
                minimumFractionDigits: 1,
                maximumFractionDigits: 1
            }
        }
    }
})

config.global.plugins.push(i18n)

beforeEach(() => {
    vi.useFakeTimers()
})

afterEach(() => {
    vi.useRealTimers()
})

vitest.global.setup.ts

export const setup = () => {
    process.env.TZ = 'UTC'
}

export const setup = () => {
    process.env.TZ = 'UTC'
}

kapa.ai•3mo ago

Let me check the knowledge base for you again. One moment! :books: ---------------------- Based on the configuration files you've shared, there are a few aspects that could potentially affect the performance of your tests, although the impact might not be as significant as the slowdown you're experiencing. Let's analyze each file: 1. vitest.setup.ts: This file contains setup code that runs before each test, which could introduce some overhead: a) Creating an i18n instance: The creation of the i18n instance with multiple locales and number formats adds some initialization overhead. However, this should be a one-time cost per test file, not per individual test. b) Fake timers:

   beforeEach(() => {
       vi.useFakeTimers()
   })

   afterEach(() => {
       vi.useRealTimers()
   })

   beforeEach(() => {
       vi.useFakeTimers()
   })

   afterEach(() => {
       vi.useRealTimers()
   })

Using fake timers can impact performance, especially if you have many tests or if your tests involve a lot of timer-related operations. The constant switching between fake and real timers for each test might introduce some overhead. 2. vitest.global.setup.ts: This file seems to only set the timezone, which is unlikely to cause significant performance issues:

export const setup = () => {
    process.env.TZ = 'UTC'
}

export const setup = () => {
    process.env.TZ = 'UTC'
}

While these configurations might introduce some overhead, they are unlikely to be the sole cause of the significant slowdown you're experiencing (more than quadrupling the execution time). To investigate further: 1. Try commenting out parts of the vitest.setup.ts file, especially the i18n setup and the fake timers, to see if it improves performance. 2. Check if you have a large number of tests that might be affected by the setup code. 3. Consider using Vitest's built-in profiling tools to identify bottlenecks. You can run your tests with the --profile flag to generate a .cpuprofile file, which you can then analyze in Chrome DevTools. 4. Ensure that your tests are not inadvertently making real network requests or performing heavy I/O operations. 5. Check if there are any Nuxt-specific configurations or plugins that might be causing the slowdown when running in the Nuxt environment. If the issue persists after investigating these areas, it would be helpful to create a minimal reproduction of the problem, as you mentioned earlier. This would allow for a more thorough analysis of the performance issues you're encountering. Nuxt Documentation on Testing provides more information on testing in Nuxt, which might be helpful in optimizing your test setup. ---------------------- Note: you can ask me a follow up question by @ mentioning me again :speech_balloon: ----------------------

kapa.ai•3mo ago

<@108292831583440896> Kapa.ai is still learning and improving, please let me know how I did by reacting below.

NumilOP•3mo ago

After the answers of the bot I can say that the tests themselves are really fast the in between part is really slow

Gaming

Programming

Vitest slow tests compared to jest

Did you find this page helpful?