Cloudflare Developers•2y ago

Instantiating wasm built with emscripten

I am trying to convert some code to a worker that uses 4 different webassembly modules (compiled from C/C++ libraries using emscripten) depending on the data that is posted. We had been using the .js file generated by emscripten to load and instantiate the module, but his fails on the Worker when it tries to do a require() to detect the environment. I am unclear how to correctly instantiate the module in a Worker, trying to import the .wasm files and using WebAssembly.instantiate() complains about not having the correct imports object. Is there a clear guide for how to do this correctly?

6 Replies

Jason HostetterOP•2y ago

When trying to use the emscripten-generated module.js file with the module factory, the worker errors with EvalError: Code generation from strings disallowed for this context, which is thrown by the emscripten .js file function createNamedFunction, which is trying to dynamically create a named function using the Function constructor. So I am not sure if using these emscripten generated wasm files is just not supported yet, or if there is a specific set of compilation flags or specific module loading method that will allow it to work.

Unknown User•2y ago

Message Not Public

Jason HostetterOP•2y ago

Thanks for that info- I’ll have to explore wasi. It’s not so much that I want to use C, I’m not a C developer, it’s that I want to use existing image compression libraries that others have compiled to wasm for use in the browser with emscripten. These work wonderfully in the browser and web workers but I have not been able to get it working in cloudflare workers, I think because of some limitations in the workers runtime that conflicts with the JS glue code that emscripten emits. The emscripten example in the templates repo doesn't work unfortunately on a clean install, maybe it hasn't kept up with api changes. I think I'll pause this project for now until the ecosystem here matures a little. Documentation is all over the place and the environment differences between the browser and Workers seems like too big a gap at this point. But thanks for the help!

Jason HostetterOP•2y ago

My god I actually got it working. For anyone else with issues, here is what I found in case it helps. This may be all specific to my scenario, but I don't know enough about emscripten to tell: Emscripten link flags that I needed:

-s MODULARIZE=1
-s ENVIRONMENT=web  // This I think prevents the glue code from checking for Node/server environment globals
-s DYNAMIC_EXECUTION=0  // Gets rid of `new Function()` constructor call in glue code, which fails on Workers because of unsafe code generation

-s MODULARIZE=1
-s ENVIRONMENT=web  // This I think prevents the glue code from checking for Node/server environment globals
-s DYNAMIC_EXECUTION=0  // Gets rid of `new Function()` constructor call in glue code, which fails on Workers because of unsafe code generation

On the web, we had been loading the module like this (This does NOT WORK on CF Workers):

import OpenJPEGModule from './wasm-codecs/openjpegwasm_decode.js';
import OpenJPEGWASM from './wasm-codecs/openjpegwasm_decode.wasm';
const m = OpenJPEGModule({
    locateFile: (f) => {
      if (f.endsWith('.wasm')) {
        return OpenJPEGWASM;
      }

      return f;
    },
  });

import OpenJPEGModule from './wasm-codecs/openjpegwasm_decode.js';
import OpenJPEGWASM from './wasm-codecs/openjpegwasm_decode.wasm';
const m = OpenJPEGModule({
    locateFile: (f) => {
      if (f.endsWith('.wasm')) {
        return OpenJPEGWASM;
      }

      return f;
    },
  });

I don't actually understand what is going on here, but it works (on the web). I believe the locateFile call is issuing an XHR request for the wasm? I'm not sure why it's imported directly then, but it seems to work. This does not work on CF Workers. For Workers, I used the following module instantiation code:

const openJpegModule = OpenJPEGModule({
  instantiateWasm: (imports, callback) => {
    const instance = new WebAssembly.Instance(OpenJPEGWASM, imports);
    callback(instance);
    return instance.exports;
  }
});
openJpegModule.then((instance) => {
  // Here is where we have a functioning instance we can use
}

const openJpegModule = OpenJPEGModule({
  instantiateWasm: (imports, callback) => {
    const instance = new WebAssembly.Instance(OpenJPEGWASM, imports);
    callback(instance);
    return instance.exports;
  }
});
openJpegModule.then((instance) => {
  // Here is where we have a functioning instance we can use
}

Here is where I dug up this instantiation method: https://github.com/robertaboukhalil/cf-workers-emscripten/blob/1f5ca765da38f3bf0c9482c6498ae65a909d1011/index.js

GitHub

cf-workers-emscripten/index.js at 1f5ca765da38f3bf0c9482c6498ae65a9...

Template for using Cloudflare Workers with WebAssembly and Emscripten - cf-workers-emscripten/index.js at 1f5ca765da38f3bf0c9482c6498ae65a909d1011 · robertaboukhalil/cf-workers-emscripten

Jason HostetterOP•2y ago

There are a bunch of issues with the wasm on Workers documentation I ran into. It's very confusing. For instance: - Apparently you no longer can upload the WASM directly into the cloud interface as mentioned in the announcement blog post - Apparently you cannot define [wasm_modules] in wrangler.toml as suggested in some forums/posts unless it's an old style service type worker? - Apparently you no longer have the .wasm file preloaded into the environment as a global as stated on multiple past docs/blog posts, you are supposed to just import it into the script directly - The examples are old/unmaintained it seems? For example, This: https://github.com/cloudflare/templates/tree/main/worker-emscripten is different than this: https://github.com/cloudflare/worker-emscripten-template

GitHub

templates/worker-emscripten at main · cloudflare/templates

A collection of starter templates and examples for Cloudflare Workers and Pages - templates/worker-emscripten at main · cloudflare/templates

GitHub

GitHub - cloudflare/worker-emscripten-template

Contribute to cloudflare/worker-emscripten-template development by creating an account on GitHub.

Unknown User•2y ago

Message Not Public

Gaming

Programming

Instantiating wasm built with emscripten