Cloudflare Developers•14mo ago

Workers zlib failure when trying to gunzip data

Hi all, I'm trying to develop a worker that stores and fetches gzipped data in R2. Data is a ~300kb json file but when gzipped is only ~20kb. I've enabled polyfills (setting node_compat = true in wrangler.toml) to be able to use zlib. Compressing the json works fine. However, when trying to uncompress it I'm getting the following 1. "TypeError: Not a string or buffer" when trying to use an ArrrayBuffer 2. "TypeError: Cannot read properties of undefined (reading '-5')" when converting data from R2 object to a Buffer. Is there any supported way to uncompress a file in a worker? I believe the issue might be in https://github.com/ionic-team/rollup-plugin-node-polyfills that you're using - but the library seems to be no longer maintained.

GitHub

GitHub - ionic-team/rollup-plugin-node-polyfills

Contribute to ionic-team/rollup-plugin-node-polyfills development by creating an account on GitHub.

3 Replies

TiagoOP•14mo ago

Example code for 1. above is:

const zlib = require('zlib');
...
const object = await env.BUCKET.get(key);
const buffer = await object.arrayBuffer();
const data = zlib.gunzipSync(buffer);

const zlib = require('zlib');
...
const object = await env.BUCKET.get(key);
const buffer = await object.arrayBuffer();
const data = zlib.gunzipSync(buffer);

Ends up with:

✘ [ERROR] Uncaught (in promise) TypeError: Not a string or buffer

      at zlibBufferSync
  (file:///[REDACTED]/.wrangler/tmp/dev-YsNPmS/node-modules-polyfills:zlib:224:8)
      at gunzipSync
  (file:///[REDACTED]/.wrangler/tmp/dev-YsNPmS/node-modules-polyfills:zlib:172:8)
      at fetch
  (file:///[REDACTED]/src/index.js:66:28)

✘ [ERROR] Uncaught (in promise) TypeError: Not a string or buffer

      at zlibBufferSync
  (file:///[REDACTED]/.wrangler/tmp/dev-YsNPmS/node-modules-polyfills:zlib:224:8)
      at gunzipSync
  (file:///[REDACTED]/.wrangler/tmp/dev-YsNPmS/node-modules-polyfills:zlib:172:8)
      at fetch
  (file:///[REDACTED]/src/index.js:66:28)

kian•14mo ago

I’d recommend just using DecompressionStream https://developer.mozilla.org/en-US/docs/Web/API/DecompressionStream

TiagoOP•14mo ago

Example code for 2. is:

const object = await env.BUCKET.get(key);
const blob = await object.blob();
const tmp_array = new Uint8Array(await blob.arrayBuffer());
const data = zlib.gunzipSync(Buffer.from(tmp_array));

const object = await env.BUCKET.get(key);
const blob = await object.blob();
const tmp_array = new Uint8Array(await blob.arrayBuffer());
const data = zlib.gunzipSync(Buffer.from(tmp_array));

Ends up with:

✘ [ERROR] Uncaught (in promise) TypeError: Cannot read properties of undefined (reading '-5')

      at Zlib2._binding.onerror
  (file:///[REDACTED]/.wrangler/tmp/dev-YsNPmS/node-modules-polyfills:zlib:356:19)
      at status
  (file:///[REDACTED]/node_modules/rollup-plugin-node-polyfills/polyfills/zlib-lib/binding.js:263:6)
      at Zlib._write
  (file:///[REDACTED]/node_modules/rollup-plugin-node-polyfills/polyfills/zlib-lib/binding.js:221:7)
      at Zlib.writeSync
  (file:///[REDACTED]/node_modules/rollup-plugin-node-polyfills/polyfills/zlib-lib/binding.js:170:8)
      at Zlib2._processChunk
  (file:///[REDACTED]/.wrangler/tmp/dev-YsNPmS/node-modules-polyfills:zlib:509:8)
      at zlibBufferSync
  (file:///[REDACTED]/.wrangler/tmp/dev-YsNPmS/node-modules-polyfills:zlib:227:7)
      at gunzipSync
  (file:///[REDACTED]/.wrangler/tmp/dev-YsNPmS/node-modules-polyfills:zlib:172:8)
      at fetch
  (file:///[REDACTED]/src/index.js:66:28)

✘ [ERROR] Uncaught (in promise) TypeError: Cannot read properties of undefined (reading '-5')

      at Zlib2._binding.onerror
  (file:///[REDACTED]/.wrangler/tmp/dev-YsNPmS/node-modules-polyfills:zlib:356:19)
      at status
  (file:///[REDACTED]/node_modules/rollup-plugin-node-polyfills/polyfills/zlib-lib/binding.js:263:6)
      at Zlib._write
  (file:///[REDACTED]/node_modules/rollup-plugin-node-polyfills/polyfills/zlib-lib/binding.js:221:7)
      at Zlib.writeSync
  (file:///[REDACTED]/node_modules/rollup-plugin-node-polyfills/polyfills/zlib-lib/binding.js:170:8)
      at Zlib2._processChunk
  (file:///[REDACTED]/.wrangler/tmp/dev-YsNPmS/node-modules-polyfills:zlib:509:8)
      at zlibBufferSync
  (file:///[REDACTED]/.wrangler/tmp/dev-YsNPmS/node-modules-polyfills:zlib:227:7)
      at gunzipSync
  (file:///[REDACTED]/.wrangler/tmp/dev-YsNPmS/node-modules-polyfills:zlib:172:8)
      at fetch
  (file:///[REDACTED]/src/index.js:66:28)

Thanks for que quick reply, let me try that! 🙂 @kian it works, thank you!!

Gaming

Programming

Workers zlib failure when trying to gunzip data