Bug : EAGAIN when use `plasmo package`

EAGAIN when using plasmo package

% yarn package --verbose
yarn run v1.22.22
warning package.json: No license field
$ plasmo package --verbose
🟡 0      | Running command: package
🟣 Plasmo v0.85.2
🔴 The Browser Extension Framework
🔵 INFO   | Prepare to package the extension bundle...
🟡 1      | Creating Manifest Factory...
🟡 2      | Ensure exists: /Users/dp/work/extension/.plasmo
🟡 3      | /Users/dp/work/extension/assets/icon.png found, creating resized icons
🟡 4      | Creating static templates for popup
🟡 5      | Creating static templates for options
🟡 6      | Creating static templates for newtab
🟡 7      | Creating static templates for devtools
🟡 8      | Creating static templates for sidepanel

(...)
🟡 18     | { messageHandlerList: [], portHandlerList: [] }
🟡 19     | Creating BGSW entry
🟡 20     | Hash changed, updating manifest


🔴 ERROR  | EAGAIN
🟡 21     | Error: EAGAIN
    at new Worker (node:internal/worker:291:19)
    at wk (file:///Users/dp/work/extension/node_modules/fflate/esm/index.mjs:22:13)
    at wrkr (file:///Users/dp/work/extension/node_modules/fflate/esm/index.mjs:893:12)
    at astrmify (file:///Users/dp/work/extension/node_modules/fflate/esm/index.mjs:930:13)
    at new AsyncDeflate (file:///Users/dp/work/extension/node_modules/fflate/esm/index.mjs:1086:9)
    at new AsyncZipDeflate (file:///Users/dp/work/extension/node_modules/fflate/esm/index.mjs:1949:18)
    at file:///Users/dp/work/extension/node_modules/plasmo/dist/index.js:166:10563
    at new Promise (<anonymous>)
    at Qr (file:///Users/dp/work/extension/node_modules/plasmo/dist/index.js:166:10341)
    at async Zh (file:///Users/dp/work/extension/node_modules/plasmo/dist/index.js:166:11510)
🔴 EXIT   | This program will terminate in 3 seconds...

% yarn package --verbose
yarn run v1.22.22
warning package.json: No license field
$ plasmo package --verbose
🟡 0      | Running command: package
🟣 Plasmo v0.85.2
🔴 The Browser Extension Framework
🔵 INFO   | Prepare to package the extension bundle...
🟡 1      | Creating Manifest Factory...
🟡 2      | Ensure exists: /Users/dp/work/extension/.plasmo
🟡 3      | /Users/dp/work/extension/assets/icon.png found, creating resized icons
🟡 4      | Creating static templates for popup
🟡 5      | Creating static templates for options
🟡 6      | Creating static templates for newtab
🟡 7      | Creating static templates for devtools
🟡 8      | Creating static templates for sidepanel

(...)
🟡 18     | { messageHandlerList: [], portHandlerList: [] }
🟡 19     | Creating BGSW entry
🟡 20     | Hash changed, updating manifest


🔴 ERROR  | EAGAIN
🟡 21     | Error: EAGAIN
    at new Worker (node:internal/worker:291:19)
    at wk (file:///Users/dp/work/extension/node_modules/fflate/esm/index.mjs:22:13)
    at wrkr (file:///Users/dp/work/extension/node_modules/fflate/esm/index.mjs:893:12)
    at astrmify (file:///Users/dp/work/extension/node_modules/fflate/esm/index.mjs:930:13)
    at new AsyncDeflate (file:///Users/dp/work/extension/node_modules/fflate/esm/index.mjs:1086:9)
    at new AsyncZipDeflate (file:///Users/dp/work/extension/node_modules/fflate/esm/index.mjs:1949:18)
    at file:///Users/dp/work/extension/node_modules/plasmo/dist/index.js:166:10563
    at new Promise (<anonymous>)
    at Qr (file:///Users/dp/work/extension/node_modules/plasmo/dist/index.js:166:10341)
    at async Zh (file:///Users/dp/work/extension/node_modules/plasmo/dist/index.js:166:11510)
🔴 EXIT   | This program will terminate in 3 seconds...

It then starts to use a lot of memory

52 Replies

dpOP•10mo ago

@lab @LoXatoR fyi

lab•10mo ago

https://stackoverflow.com/questions/4058368/what-does-eagain-mean

Stack Overflow

What does EAGAIN mean?

As in the title what does EAGAIN mean?

lab•10mo ago

Some resource related issues?

dpOP•10mo ago

What is strange it is throwing there -- at new Worker (node:internal/worker:291:19) at wk (file:///Users/dp/work/extension/node_modules/fflate/esm/index.mjs:22:13) at wrkr (file:///Users/dp/work/extension/node_modules/fflate/esm/index.mjs:893:12) at astrmify (file:///Users/dp/work/extension/node_modules/fflate/esm/index.mjs:930:13) at new AsyncDeflate (file:///Users/dp/work/extension/node_modules/fflate/esm/index.mjs:1086:9) at new AsyncZipDeflate (file:///Users/dp/work/extension/node_modules/fflate/esm/index.mjs:1949:18) at file:///Users/dp/work/extension/node_modules/plasmo/dist/index.js:166:10563 at new Promise (<anonymous>) at Qr (file:///Users/dp/work/extension/node_modules/plasmo/dist/index.js:166:10341) at async Zh (file:///Users/dp/work/extension/node_modules/plasmo/dist/index.js:166:11510) AsyncZipDeflate

Arcane•10mo ago

@dp has reached level 1. GG!

dpOP•10mo ago

What would you advise to debug @lab what makes this resource issue? I tried --verbose but nothing; We tried with a VM with 64GB or 128GB and it fails

lab•10mo ago

Seems like simultaneous read/write at the same place or maybe resource unavailable? Actually yeah try running plasmo build then plasmo package Separately

dpOP•10mo ago

we already did this; I will add flag verbose; we're running this on github actions;

dpOP•10mo ago

cf.

    "build": "plasmo build",
    "dev": "plasmo dev",
    "format:write": "prettier . --write",
    "lint": "eslint . --ext .ts,.tsx",
    "lint:fix": "eslint . --ext .ts,.tsx --fix",
    "type:check": "tsc --noEmit",
    "package": "plasmo package",

    "build": "plasmo build",
    "dev": "plasmo dev",
    "format:write": "prettier . --write",
    "lint": "eslint . --ext .ts,.tsx",
    "lint:fix": "eslint . --ext .ts,.tsx --fix",
    "type:check": "tsc --noEmit",
    "package": "plasmo package",

lab•10mo ago

Try running them in 2 steps

dpOP•10mo ago

Ok will try after this attempt Ok will retry with two different steps --

# plasmo build
🟢 DONE   | Finished in 36524ms!
Done in 44.66s.
yarn run v1.22.22
warning package.json: No license field
# plasmo package
$ plasmo package --verbose
🟡 0      | Running command: package
🟣 Plasmo v0.84.1
🔴 The Browser Extension Framework
🟠 WARN   | A new version of plasmo is available: v0.85.2
          | Run "yarn add [email protected]" to update
🔵 INFO   | Prepare to package the extension bundle...
🟡 1      | Creating Manifest Factory...
🟡 2      | Ensure exists: /home/runner/work/extension/extension/.plasmo
🟡 3      | /home/runner/work/extension/extension/assets/icon.png found, creating resized icons
🟡 4      | Creating static templates for popup
(...)
🟡 14     | Parsed config: { matches: [ '<all_urls>' ], world: 'MAIN', run_at: 'document_start' }
🟡 15     | Adding content script:  /home/runner/work/extension/extension/src/contents/inpage.ts
🟡 16     | Parsed config: { matches: [ '<all_urls>' ], all_frames: true }
🟡 17     | Adding content script:  /home/runner/work/extension/extension/src/contents/worker.ts
🟡 18     | { messageHandlerList: [], portHandlerList: [] }
🟡 19     | Creating BGSW entry
🟡 20     | Hash changed, updating manifest
<--- Last few GCs --->
<--- JS stacktrace --->
FATAL ERROR: Committing semi space failed. Allocation failed - JavaScript heap out of memory
----- Native stack trace -----
<--- Last few GCs --->
<--- JS stacktrace --->
FATAL ERROR: Committing semi space failed. Allocation failed - JavaScript heap out of memory
<--- Last few GCs --->
<--- JS stacktrace --->
#
# Fatal javascript OOM in MemoryChunk allocation failed during deserialization.
#
<--- Last few GCs --->
<--- JS stacktrace --->
FATAL ERROR: Committing semi space failed. Allocation failed - JavaScript heap out of memory
<--- Last few GCs --->
<--- JS stacktrace --->
#
# Fatal javascript OOM in MemoryChunk allocation failed during deserialization.
#

# plasmo build
🟢 DONE   | Finished in 36524ms!
Done in 44.66s.
yarn run v1.22.22
warning package.json: No license field
# plasmo package
$ plasmo package --verbose
🟡 0      | Running command: package
🟣 Plasmo v0.84.1
🔴 The Browser Extension Framework
🟠 WARN   | A new version of plasmo is available: v0.85.2
          | Run "yarn add [email protected]" to update
🔵 INFO   | Prepare to package the extension bundle...
🟡 1      | Creating Manifest Factory...
🟡 2      | Ensure exists: /home/runner/work/extension/extension/.plasmo
🟡 3      | /home/runner/work/extension/extension/assets/icon.png found, creating resized icons
🟡 4      | Creating static templates for popup
(...)
🟡 14     | Parsed config: { matches: [ '<all_urls>' ], world: 'MAIN', run_at: 'document_start' }
🟡 15     | Adding content script:  /home/runner/work/extension/extension/src/contents/inpage.ts
🟡 16     | Parsed config: { matches: [ '<all_urls>' ], all_frames: true }
🟡 17     | Adding content script:  /home/runner/work/extension/extension/src/contents/worker.ts
🟡 18     | { messageHandlerList: [], portHandlerList: [] }
🟡 19     | Creating BGSW entry
🟡 20     | Hash changed, updating manifest
<--- Last few GCs --->
<--- JS stacktrace --->
FATAL ERROR: Committing semi space failed. Allocation failed - JavaScript heap out of memory
----- Native stack trace -----
<--- Last few GCs --->
<--- JS stacktrace --->
FATAL ERROR: Committing semi space failed. Allocation failed - JavaScript heap out of memory
<--- Last few GCs --->
<--- JS stacktrace --->
#
# Fatal javascript OOM in MemoryChunk allocation failed during deserialization.
#
<--- Last few GCs --->
<--- JS stacktrace --->
FATAL ERROR: Committing semi space failed. Allocation failed - JavaScript heap out of memory
<--- Last few GCs --->
<--- JS stacktrace --->
#
# Fatal javascript OOM in MemoryChunk allocation failed during deserialization.
#

as it failed here

lab•10mo ago

That's an entirely diff error now :-? And it seems package is using v0.84.1 (which might contain some err btw, iirc, I downgraded some parcel stuffs bc it wasn't working correctly :d )

dpOP•10mo ago

I will bump to latest; (this is github actions runners; First log from my machine but it crashes/freezes) Yes it's not exiting and keep going on Github Runner -- Need to stop the job manually

🟡 19     | Creating BGSW entry
🟡 20     | Hash changed, updating manifest
#
# Fatal error in , line 0
# unreachable code
#
#
#
#FailureMessage Object: 0x7c4f096fdcc0
----- Native stack trace -----
<--- Last few GCs --->
<--- JS stacktrace --->
#
# Fatal javascript OOM in MemoryChunk allocation failed during deserialization.
#
<--- Last few GCs --->
<--- JS stacktrace --->
FATAL ERROR: Committing semi space failed. Allocation failed - JavaScript heap out of memory
----- Native stack trace -----
<--- Last few GCs --->
<--- JS stacktrace --->
#
# Fatal javascript OOM in MemoryChunk allocation failed during deserialization.
#

🟡 19     | Creating BGSW entry
🟡 20     | Hash changed, updating manifest
#
# Fatal error in , line 0
# unreachable code
#
#
#
#FailureMessage Object: 0x7c4f096fdcc0
----- Native stack trace -----
<--- Last few GCs --->
<--- JS stacktrace --->
#
# Fatal javascript OOM in MemoryChunk allocation failed during deserialization.
#
<--- Last few GCs --->
<--- JS stacktrace --->
FATAL ERROR: Committing semi space failed. Allocation failed - JavaScript heap out of memory
----- Native stack trace -----
<--- Last few GCs --->
<--- JS stacktrace --->
#
# Fatal javascript OOM in MemoryChunk allocation failed during deserialization.
#

I am running a new build with plasmo 0.85.2 now

lab•10mo ago

hmm, fyi the package command is basically a simple zip implementation that stream assets/files into the right place

lab•10mo ago

https://github.com/PlasmoHQ/plasmo/blob/main/cli/plasmo/src/commands/package.ts

GitHub

plasmo/cli/plasmo/src/commands/package.ts at main · PlasmoHQ/plasmo

🧩 The Browser Extension Framework. Contribute to PlasmoHQ/plasmo development by creating an account on GitHub.

lab•10mo ago

You can also just try using the zip command over the build directory

dpOP•10mo ago

Ha nice; let me see

Arcane•10mo ago

@dp has reached level 2. GG!

lab•10mo ago

I've seen it works with a 30MB bundle so not sure what's going on there, esp when build worked if you can debug and take a look at the package source, feel free to PR!

dpOP•10mo ago

Ok it seems our build dir is 402mb; compressed manually with zip makes it 114mb. I will look why this is aas big as that It seems the build node_modules directory include typescript, parcel... and lot of things (sass, react-dev-tools) ; probably we need to move to dev dependencies

dpOP•10mo ago

I see the plasmo build CLI just call parcel -- but we can't pass much options to parcel such as yarn build --reporter @parcel/reporter-bundle-analyzer https://github.com/PlasmoHQ/plasmo/blob/main/cli/plasmo/src/commands/build.ts#L38 WOuld it be possible to pass parcel functions?

GitHub

plasmo/cli/plasmo/src/commands/build.ts at main · PlasmoHQ/plasmo

🧩 The Browser Extension Framework. Contribute to PlasmoHQ/plasmo development by creating an account on GitHub.

dpOP•10mo ago

If you'd like to analyze your bundle, you can use the --bundle-buddy flag, combined with --source-maps to generate a Bundle Buddy Just read it ; nice Ok we just zip it using zip and it worked

abhishekjha•10mo ago

We are also facing issue using manual api to upload the zip & sign it. We are able to successfully upload the zip but during sign call it's timing out. Getting below error. An error occurred with your deployment FUNCTION_INVOCATION_TIMEOUT

dpOP•10mo ago

@lab

lab•10mo ago

@abhishekjha how big is your extension BTW? the zipped bundle Extended the timeout so you should be able to try again for longer

dpOP•10mo ago

100mb I believe But we'l try to reduce it

abhishekjha•10mo ago

@lab Still getting timeout

lab•10mo ago

I suggest you reduce the size of your extension - 100mb is too high and I don't think we should support that. Yeah it seems your extension was too large for us to generate a signed bundle for

dpOP•9mo ago

Yes it seems our build was keeping node_modules and creating some nested node_modules that was enlarging the directory before zipping.

lab•9mo ago

I think you should try to exclude them and bundle those assets extension doesn't play well with dynamic scripts loading

LoXatoR•9mo ago

any idea about this thread - https://discord.com/channels/946290204443025438/1219788890353827860 @lab ?

LoXatoR•9mo ago

Also, it takes around 30 - 45s on average for plasmo dev to recompile and if I stop the dev server and run it again, it keeps re-creating the build and placing it in the existing build folder, thus creating multiple levels of nesting as seen in the pic

lab•9mo ago

That should not be the case, unless there's some crazy introspection/recursion dependency I'd strip out and see which import is messing it up for you

abhishekjha•6mo ago

@lab We started getting 500 error for sign api.

Arcane•6mo ago

@abhishekjha has reached level 1. GG!

lab•6mo ago

I saw a couple of 500 reported - it seems the key were incorrect (or that it was not able to resolve to your extension ID)

abhishekjha•6mo ago

I downloaded a new private key & tried to sign it But got same error

lab•6mo ago

did you change anything in your code?

abhishekjha•6mo ago

Nope. Same pipeline were working fine so far but since few days we are getting 500 error

lab•6mo ago

What's the extension ID?

abhishekjha•6mo ago

Just now I retried for eanafdjekpnbkamplmoinkeojkjinakn I got 500 error

lab•6mo ago

Hmm, seems like your zip was too large it's over 100MB

abhishekjha•6mo ago

It works from UI but not from api Can we increase the limit ? @lab Can we increaes the limit please ?

lab•6mo ago

Appear it's not the file size that's the issue, but rather the signing operation that our backend is doing took way longer than the allowed time for the serverless function to execute... Tho I think it's really both

abhishekjha•6mo ago

Any ETA of the fix ?

lab•6mo ago

Just pushed a limit bump to the memory limit for the signing function, should be up soon

abhishekjha•6mo ago

Okay

Arcane•6mo ago

@abhishekjha has reached level 2. GG!

lab•6mo ago

if the signing operation cannot fit within the 3GB box, you will need to upload manually

abhishekjha•6mo ago

@lab Do you think we should also increase the timeout of lambda function ? I suspect lambda function might be timing out