Create a bundle for browsers that includes the Wasm code?

[rauschma]

Suggestion: source-map would be easier to use if it completely hid the fact that it internally uses Wasm.

• For browsers, you can do that by distributing a bundle, as shown below.
• For Node.js, this should be even simpler.

webpack.config.js:

const path = require('path');

module.exports = {
  // ···
  module: {
    rules: [
      {
        test: /\.wasm$/,
        type: 'javascript/auto',
        loader: 'arraybuffer-loader', // installed via npm
      },    
    ],
  },
};

Setup code (when used outside the library):

import arrayBuffer from 'source-map/lib/mappings.wasm';
SourceMapConsumer.initialize({
  'lib/mappings.wasm': arrayBuffer,
});

AFAICT, .initialize() is not in the .d.ts file. Maybe add it?

[loganfsmyth]

Hey there!

source-map would be easier to use if it completely hid the fact that it internally uses Wasm

I 100% agree with you there.

Before I potentially go the wrong direction, would the module also be "completely hide" the bundle's existance? When you say "For browsers" and "For Node.js", is what you have in mind

1. two separate entrypoints, e.g. require("source-map") and require("source-map/browser")
2. one entrypoint with

   main: "lib/index.js",
   browser: "browser-bundle.js"
   

   so that require("source-map") would load one or the other based on context, and the bundle would be an implementation detail?

[rauschma]

Ah, good point! Tricky to get this right. Ideas:

• Separate entry points may make things simpler, because you don’t have to dynamically check for the current environment(?) Another possibility:
  • source-map: same as now – for people using Node.js and bundlers other than webpack
  • source-map/webpack: uses arraybuffer-loader
• It may make sense to do (then the entry in module.exports.module.rules may not be necessary):

  import arrayBuffer from 'arraybuffer-loader!!source-map/lib/mappings.wasm';
  

[loganfsmyth]

I think for me, since this is published on npm primarily, it should be targeted at users consuming the library via the most common tooling first. To me, that means that it would ideally work in Node and Webpack without users needing to know that the library is in use, which would preclude needing to install arraybuffer-loader. That seems extra important for file extensions like .wasm that have potentially conflicting definitions of what it means to import them.

That all makes me lean toward the default being a base64-encoded WASM included in the npm package by default to avoid the loader.

We could certainly try to expose ways to configure it manually, but I think that gets complicated quickly, especially once it is used as a subdependency instead of a direct dep.

[rauschma]

Base64, stored in a CommonJS module? Sounds good! The more transparent, the better, IMO. If it can be overridden, then it doesn’t matter if that’s a little slower. Must people will just appreciate that it’s easier to use. Bonus: it’s easier to migrate to something else in the future (to a first-class Wasm module, away from Wasm, etc.).

[kwonoj]

it's somewhat coincidence but #369 takes same approach for b64 encoded binary, and makes isomorphic module publishing easier.

[adjenks]

Could encode it more densely with ascii85:
https://en.wikipedia.org/wiki/Ascii85
https://www.npmjs.com/package/base85

Or z85:
https://rfc.zeromq.org/spec/32/
https://www.npmjs.com/package/z85

[puzrin]

From my experience, if 25% size increase is not deadly critical, loading "small" wasm from base64 string is optimal (simple & fast enough).

[gruns]

@rauschma i just accidentally, independently reinvented your

                test: /\.wasm$/,
                type: 'javascript/auto',
                use: [
                    {
                        loader: 'arraybuffer-loader',
                    },
                ],

technique to bundle and include mappings.wasm into a bundled .js output file. but i then hit the roadblock where the latest npm version, v0.7.4, does not support providing an ArrayBuffer to SourceMapConsumer.Initialize(), while the latest trunk does. then i Googled and found this issue

so, in short: as a first step to support people -- like myself and @rauschma -- bundling mappings.wasm in output .js bundles, please push trunk to npm as a new version, e.g. v0.7.5. the last release, v0.7.4, was published 4 Jun 2022, over a year ago

then, thereafter, i too wholly support bundling mappings.wasm into source-map itself, hiding its internals that wasm is involved at all. that's an implementation detail. but that's more effort than just publishing trunk to npm, so starting with the latter immediately is best 🙂

zooming out, I'm curious: why is mappings.wasm wasm instead of js to begin with? speed? compiled from other, native code?

cc @loganfsmyth

[JasonGore]

I'm hitting this now with a Node application that's using webpack and it's introducing an incredible amount of friction. This is very painful for what should be encapsulated internally. I'm still not sure how to best tackle this with the most recent version of source-map on NPM which is version 0.7.4. CopyWebpackPlugin doesn't work since source-map is using fs.readFile with an absolute path and completely sidestepping wepback. Not having any apparent access to initialize() to try and control this is adding significantly to the pain.

[gruns]

@JasonGore +1

i just emailed some of the devs a gentle nudge to see who at mozilla is in charge of npm releases nowadays 🙂. hopefully we can get a new release soon! 👏

[bomsy]

Hi, i work on devtools at mozilla. Apologies this has been a blocker. I’ll look into getting a new npm release out. thanks