I need help in debugging resource exhausted (Too many open files)
I can't start my WASP project unless I do a
ulimit -n 10000 . To wasp db start. I need to do this in every terminal session I need to run wasp in.
I know if it's working and it's stupid it's not stupid, but still I don't feel comfortable doing it, I feel I'm hacking into the solution instead of tackling the problem and solving it the right way.
Any idea why I keep getting this error?
For context, below is the full error example:
❯ wasp db start
wasp-bin: /Users/x/Projects/WASP/x/app/src/client/fonts/Satoshi/Satoshi-LightItalic.eot: openFile: resource exhausted (Too many open files)10 Replies
Hi @abiroot. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave:
----------------------
This issue you're experiencing with "Too many open files" in your Wasp project is a known problem that has been reported by other users as well. It's related to how Wasp watches files in your project, and it seems to be more prevalent in newer versions of Wasp. Let me explain the issue and provide some potential solutions based on the information available.
The error occurs because Wasp is watching too many files simultaneously, hitting the default limit of open files set by your operating system. This became more noticeable with Wasp 0.12, which changed how file watching works Github Issue 1919 (https://github.com/wasp-lang/wasp/issues/1919).
Here are some insights and potential solutions:
1. Your current workaround of using
ulimit -n 10000 is actually a valid solution. As mentioned by a Wasp team member, "Right now the easiest solution is probably just to increase your limit of files that can be open at the same moment" Github Issue 1919 (https://github.com/wasp-lang/wasp/issues/1919).
2. The Wasp team is aware of this issue and plans to work on fixing it. They've identified that there might be some redundant file watching happening, causing the limit to be hit sooner than before Github Issue 1919 (https://github.com/wasp-lang/wasp/issues/1919).
3. To avoid setting the ulimit in every terminal session, you could add the command to your shell configuration file (e.g., .bashrc or .zshrc).
4. Another approach is to investigate which files are being watched. The Wasp team mentioned that they watch all files at the top level (but not recursively), and then watch recursively the src/ and public/ directories Discord support thread (https://discord.com/channels/686873244791210014/1212533517360177253).
5. If you have a large number of files in your assets directory or node_modules, these might be contributing to the issue. Ensure that you're not accidentally watching node_modules Discord support thread (https://discord.com/channels/686873244791210014/1212533517360177253).
6. The Wasp team is investigating how their file watching works, and they're looking into commands that can track how many files are being watched by which process Github Issue 1844 (https://github.com/wasp-lang/wasp/issues/1844).
While increasing the file limit is a valid solution, you're right to feel that it's not addressing the root cause. The Wasp team is aware of this issue and is working on a more permanent fix. In the meantime, you can continue using the ulimit solution or try to reduce the number of files being watched in your project if possible.
If you want to help the Wasp team debug this issue, you could provide more information about your project structure, particularly the contents of your assets directory and any large number of files you might have in your project.
----------------------
Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:
----------------------
Sorry @abiroot, I seem to be having a bit of trouble. Don't worry though, I've pinged my support and they will try to fix me up! :technologist:
Perhaps try again in 15mins?@martinsos @Filip I guess this is related to us watching too many files + Vite watching files + Nodemon watching files?
https://github.com/wasp-lang/wasp/issues/1844
GitHub
Investigate how our file watching works · Issue #1844 · wasp-lang/w...
With Wasp 0.12.0 and the restructuring (#1688), we've removed the constraint of separating server files from client files. We should investigate the impact this had on our watching procedures a...
@abiroot Do you mind sharing your project with us? It would be helpful to see if we can reproduce this behavior and see what the problem is, since this bug has been around for quite some time
Hey @Filip , apologies for the delay response and thank you for the awesome support.
I can definetly share the code, but would rather privately?
Wohooo @abiroot, you just became a Waspeteer level 3!
Sure, you can DM me! I sent a friend request 🙂
Done.
ulimit -n 200000
will work
and no problem in prod
It also happens to me with wasp build if I don't
ulimit -n 200000Hey @Joaquim, sorry for the holdup.
@abiroot shared his project so I'll try to take a stab at this as soon as possible (hopefully tomorrow).
If you want, you can send me your project too. It can't hurt 🙂