FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
I am using flowise with railway. I use pinecode to store data.
I want to upload text files and my website pages. I am using the Text Node and Puppeter web crawler Node.
I faced a first issue :
FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
So after investigation, I changed the chank size to 1000 and the bug was fixed for a text file. Then I added puppeteer web scrapper. I used the sitemap method, I get the urls in puppeteer web scraper. and the issue came back. Again a problem of memory limit.
So I tried with another web scraper node and the same issue.
I have 30 text files that represent 30 books. It is about 50 Mo in total.
I have some screenshots to see my configuration if necessary. I don't know what I am doing wrong. I am a newbie in Flowise and Pinecone.
here is railway logs :
2024-07-08 11:52:17 [INFO]: ⬆ POST /api/v1/vector/internal-upsert/ceee0b68-64a4-4f7b-b684-445d5ffb3c19
Jul 08 13:54:11
Jul 08 13:54:11
<--- Last few GCs --->
Jul 08 13:54:11
Jul 08 13:54:11
[36:0x7fa3236226c0] 483060 ms: Mark-Compact 245.4 (258.7) -> 244.7 (258.7) MB, 210.03 / 0.00 ms (average mu = 0.597, current mu = 0.017) allocation failure; scavenge might not succeed
Jul 08 13:54:11
[36:0x7fa3236226c0] 483211 ms: Mark-Compact 245.7 (258.7) -> 245.0 (259.2) MB, 147.53 / 0.00 ms (average mu = 0.435, current mu = 0.025) allocation failure; scavenge might not succeed
Jul 08 13:54:11
Jul 08 13:54:11
Jul 08 13:54:11
<--- JS stacktrace --->
Jul 08 13:54:11
Jul 08 13:54:11
FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
Jul 08 13:54:11
----- Native stack trace -----
Jul 08 13:54:11
Jul 08 13:54:31
container event container died
Can you help me to find a solution, please?
Solution:Jump to solution
you used a template to deploy flowise, if there is something wrong with the template, you need to take that up with the creator
17 Replies
Project ID:
ceee0b68-64a4-4f7b-b684-445d5ffb3c19you more than likely have tried to exceed the trial plans resource limits, please try again after upgrading to the hobby plan
ok. Thanks. I'll do that.
I get same issue when upsert
have you since redeployed the service?
oups. No
sorry
I previously deployed with custom : "node --max-old-space-size=4096 packages/server/bin/run start ". I found this while investigating. I have aroun 50Mo of text book splitted in 30 text files + 60 urls from sitemap.Should I keep that.
?
yeah thats fine
Thanks
Same issue It didn't work
then you would need to take this up with the template creator
What do you mean?
what was left unclear?
Who is and what is a template creator? You mean the flowise setup of my flow? It works with text file. Not web scraper.
Solution
you used a template to deploy flowise, if there is something wrong with the template, you need to take that up with the creator
Thank you for your help.