centredeformationfrance
FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
I am using flowise with railway. I use pinecode to store data.
I want to upload text files and my website pages. I am using the Text Node and Puppeter web crawler Node.
I faced a first issue :
FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
So after investigation, I changed the chank size to 1000 and the bug was fixed for a text file. Then I added puppeteer web scrapper. I used the sitemap method, I get the urls in puppeteer web scraper. and the issue came back. Again a problem of memory limit.
So I tried with another web scraper node and the same issue.
I have 30 text files that represent 30 books. It is about 50 Mo in total.
I have some screenshots to see my configuration if necessary. I don't know what I am doing wrong. I am a newbie in Flowise and Pinecone.
here is railway logs :
2024-07-08 11:52:17 [INFO]: ⬆ POST /api/v1/vector/internal-upsert/ceee0b68-64a4-4f7b-b684-445d5ffb3c19
Jul 08 13:54:11
Jul 08 13:54:11
<--- Last few GCs --->
Jul 08 13:54:11
Jul 08 13:54:11
[36:0x7fa3236226c0] 483060 ms: Mark-Compact 245.4 (258.7) -> 244.7 (258.7) MB, 210.03 / 0.00 ms (average mu = 0.597, current mu = 0.017) allocation failure; scavenge might not succeed
Jul 08 13:54:11
[36:0x7fa3236226c0] 483211 ms: Mark-Compact 245.7 (258.7) -> 245.0 (259.2) MB, 147.53 / 0.00 ms (average mu = 0.435, current mu = 0.025) allocation failure; scavenge might not succeed
Jul 08 13:54:11
Jul 08 13:54:11
Jul 08 13:54:11
<--- JS stacktrace --->
Jul 08 13:54:11
Jul 08 13:54:11
FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
Jul 08 13:54:11
----- Native stack trace -----
Jul 08 13:54:11
Jul 08 13:54:31
container event container died
Can you help me to find a solution, please?
22 replies