[v4] added wasm cache #1471

nico-martin · 2025-12-01T14:30:45Z

This adds caching of the wasm Binary.
I also added an env.cacheKey so developers can modify the cacheKey. By default it will be transformers-cache but they are free to set something related to their app.

Note: this will only cache the wasm file. So there will still be a request to https://cdn.jsdelivr.net/ for the mjs file. In my opinion, this should be cached via the service-worker if the applications requires full offline support.

xenova · 2025-12-01T21:34:59Z

So there will still be a request to https://cdn.jsdelivr.net/ for the mjs file. In my opinion, this should be cached via the service-worker if the applications requires full offline support.

Indeed, caching the mjs file would be necessary to ensure full offline support, and imo is essential before adding a caching feature like this PR proposes. Any ideas for how we could do this? Perhaps bundling ort-wasm-simd-threaded.jsep.mjs with transformers.js file?

xenova

Very nice + clean PR! Now we just need to figure out how to completely remove the ort-wasm-simd-threaded.jsep.mjs dependency.

nico-martin · 2025-12-04T10:40:44Z

So after another deep-dive into onnxruntime I figured out that it's actually no problem at all to load the wasm factory (.mjs) as a blob, which allows us to load it from cache.
The only problem is that inside the factory there are URLs that try to resolve relative with "import.meta.url". If we just replace this with the actual baseURL ist works just fine.
https://github.com/huggingface/transformers.js/blob/v4-cache-wasm-file/src/backends/utils/cacheWasm.js#L76

On to of that I also did some refactoring of the hub.js. My goal is to keep large files that only have a handfull of exported methods as clean as possible by extracting some heloper functions and constants into their separate files.

I also wanted to improve the caching (which is now used not only in the hub.js but also in the backends/onnx.js) so I created a helper function also also an interface "CacheInterface" that any given cache has to implement.

src/backends/onnx.js

src/env.js

xenova

Very worthwhile refactor -- thanks!

I wonder if you think the following feature could be useful: design some form of CacheRegistry class which a user could import from the library... like

import { cache } from '@huggingface/transformers';

/// check if model is cached
// cache.match('org/model') or something -- API should be well-designed, returning a list/map of files that are cached for this model maybe?
// cache.delete('org/model') -- remove all files cached for this model

I think we can draw inspiration from hf cache cli tool

hf cache --help
Usage: hf cache [OPTIONS] COMMAND [ARGS]...

  Manage local cache directory.

Options:
  --help  Show this message and exit.

Commands:
  ls      List cached repositories or revisions.
  prune   Remove detached revisions from the cache.
  rm      Remove cached repositories or revisions.
  verify  Verify checksums for a single repo revision from cache or a...

may not need to be added in this PR, but maybe something to discuss here.

src/backends/utils/cacheWasm.js

src/utils/hub.js

nico-martin · 2025-12-09T12:23:28Z

I like the CacheRegistry! The only problem is that we normally don't know upfront all the files a model will load/expect (altough I think it would be great to add that as well).
I think that opens up a completely new topic. I will keep this on my radar but not implement it here in the PR.

xenova · 2025-12-16T05:19:17Z

src/backends/utils/cacheWasm.js

+        if (cache) {
+            try {
+                return await cache.match(url);
+            } catch (e) {
+                console.warn(`Error reading ${fileName} from cache:`, e);


if await cache.match(url) returns undefined (i.e., it is not in cache), then we return undefined from this function... and the fetch below is never called (meaning, it is never cached).

xenova · 2025-12-16T05:20:41Z

src/backends/utils/cacheWasm.js

+        if (cache) {
+            try {
+                return await cache.match(url);
+            } catch (e) {
+                console.warn(`Error reading ${fileName} from cache:`, e);


Suggested change

if (cache) {

try {

return await cache.match(url);

} catch (e) {

console.warn(`Error reading ${fileName} from cache:`, e);

if (cache) {

try {

const result = await cache.match(url);

if (result) {

return result;

}

} catch (e) {

console.warn(`Error reading ${fileName} from cache:`, e);

seems to fix it

made this change.

…ransformers.js into v4-cache-wasm-file

Don't throw error if we can't open cache or load file from cache, but we are able to make the request.

xenova

Thanks! 🚀 Had time today to finish the review, and it works well! I had to make one small adjustment (only return when await cache.match(url) matches... not when undefined), and it's good to go :)

I also like the refactor out of the monolithic hub.js file.

I tested it on the recent chatterbox webgpu demo, and it now runs fully offline thanks to this PR! 🔥

xmcp · 2025-12-17T06:02:13Z

I encountered the below exception after this PR. It was fine at the previous commit (aab2326).

The inner (caught) exception is:

TypeError: Invalid base URL
    at blob:https://localhost:5173/49689b73-721b-4544-9d07-d0402577925e:100:240
    at ortWasmThreaded (blob:https://localhost:5173/49689b73-721b-4544-9d07-d0402577925e:100:478)
    at https://localhost:5173/@fs/.../transformers.js/dist/transformers.web.js:2628:28592
    at new Promise (<anonymous>)
    at Xt (https://localhost:5173/@fs/.../transformers.js/dist/transformers.web.js:2628:28380)
    at async Ns (https://localhost:5173/@fs/.../transformers.js/dist/transformers.web.js:2630:28936)
    at async yr.init (https://localhost:5173/@fs/.../transformers.js/dist/transformers.web.js:2630:33169)
    at async ec (https://localhost:5173/@fs/.../transformers.js/dist/transformers.web.js:2624:1454)
    at async La (https://localhost:5173/@fs/.../transformers.js/dist/transformers.web.js:2624:1733)
    at async a.create (https://localhost:5173/@fs/.../transformers.js/dist/transformers.web.js:2624:19895)

The blob:...:100:240 (the first line of the stack trace) points to this statement:

Va ??= g.locateFile ? g.locateFile ? g.locateFile("ort-wasm-simd-threaded.asyncify.wasm", ja) : ja + "ort-wasm-simd-threaded.asyncify.wasm" : (new URL("/@fs/.../transformers.js/dist/ort-wasm-simd-threaded.asyncify.wasm",import.meta.url)).href;

Seems that we still have remaining import.meta.urls in the imported mjs file.

xmcp · 2025-12-17T06:09:11Z

Btw I managed to implement mjs&wasm cache in this way, which works fine with previous versions (both aab2326 and v3 from npm). I'm not sure why it does not encounter the above exception.

import {pipeline, env} from "@huggingface/transformers";

import onnx_wasm from "../node_modules/@huggingface/transformers/dist/ort-wasm-simd-threaded.asyncify.wasm?url";
import onnx_mjs from "../node_modules/@huggingface/transformers/dist/ort-wasm-simd-threaded.asyncify.mjs?url";

async function init() {
    let cache = await caches.open('transformers-cache');

    let wasm_file = await cache.match(onnx_wasm);
    if(!wasm_file) {
        await cache.add(onnx_wasm);
        wasm_file = await cache.match(onnx_wasm);
    }

    let mjs_file = await cache.match(onnx_mjs);
    if(!mjs_file) {
        await cache.add(onnx_mjs);
        mjs_file = await cache.match(onnx_mjs);
    }

    env.localModelPath = '/models';
    env.allowRemoteModels = false;
    env.allowLocalModels = true;
    env.backends.onnx.wasm.wasmPaths = {
        wasm: URL.createObjectURL(await wasm_file.blob()),
        mjs: URL.createObjectURL(await mjs_file.blob()),
    };

    return await pipeline(
        ...
    );
}

nico-martin · 2025-12-17T13:24:43Z

Hi @xmcp
Thanks for reaching out!

Btw I managed to implement mjs&wasm cache in this way, which works fine with previous versions (both aab2326 and v3 from npm). I'm not sure why it does not encounter the above exception.

In your set up that makes total sense. The import statement makes sure the .wasm and the .mjs files are copied to your dist/ folder and onnx_wasm and onnx_mjs are then links to those files.
You then use the caches api to cache the files and pass them as an internal reference URL to env.backends.onnx.wasm.wasmPaths. So far so good.

Hardcoding the wasmPaths is ok, but there could be one issue. We only use the asyncify version for non-iOS devices because we experienced some issues in the past (maybe @xenova could elaborate on this).
https://github.com/huggingface/transformers.js/blob/v4/src/backends/onnx.js#L285-L293
In your example, all clients will us the asyncify version.

But back to your main issue: I think the problem is that now your code tries to cache the request, but then our code tries to cache it again. And now the caches API tries to cache a blob URL and I am not sure if that works :D.
Could you try to set env.useWasmCache = false right where you set all the other env properties?

nico-martin · 2025-12-17T13:33:01Z

I also created a little check for that case. @xmcp could you verify if that solves your problem?
https://github.com/huggingface/transformers.js/compare/v4...v4-cache-wasm-file-blob-fix?expand=1

xmcp · 2025-12-17T14:59:29Z

I can confirm that setting env.useWasmCache = false; solves the exception. The patch at v4-cache-wasm-file-blob-fix does not fix the exception if I pass normal URLs (not the blob one) to env.backends.onnx.wasm.wasmPaths. I will try to diagnose this.

xmcp · 2025-12-17T15:30:36Z

I think I know the problem. In my code snippet, onnx_wasm and onnx_mjs are relative URLs (/assets/.../xxx.mjs in prod or /@fs/.../xxx.mjs in dev). Therefore, the loadWasmFactory function replaced all import.meta.urls to relative path, which does not work for the second argument of a URL constructor.

Making the baseUrl absolute solves the problem. Here is a patch for that:

--- a/src/backends/utils/cacheWasm.js
+++ b/src/backends/utils/cacheWasm.js
@@ -74,7 +74,7 @@ export async function loadWasmFactory(libURL) {
     try {
         let code = await response.text();
         // Fix relative paths when loading factory from blob, overwrite import.meta.url with actual baseURL
-        const baseUrl = libURL.split('/').slice(0, -1).join('/');
+        const baseUrl = new URL(libURL.split('/').slice(0, -1).join('/'), location.href).href;
         code = code.replace(/import\.meta\.url/g, `"${baseUrl}"`);
         const blob = new Blob([code], { type: 'text/javascript' });
         return URL.createObjectURL(blob);

(As a bonus, it also urlencodes the baseUrl correctly, so it will no longer crash the program if libURL contains double quotes.)

nico-martin · 2025-12-18T09:29:42Z

Thanks for taking the time to investigate!
But I think that will break on the server (NodeJS, Bun, Deno). I'll dig a bit deeper and come back to you.

nico-martin · 2025-12-18T12:48:44Z

@xmcp
could you try this?
https://github.com/huggingface/transformers.js/compare/v4...v4-cache-wasm-file-blob-fix?expand=1

xmcp · 2025-12-18T14:26:36Z

Yes this version should work. Btw the isBlobURL(url) || isRemoteURL(url) check seems redundant because the URL class works fine for absolute URLs.

nico-martin · 2025-12-18T21:22:50Z

Good catch! I removed it.PR is open :)
#1489

xenova · 2025-12-21T22:57:37Z

PR is merged ✅

added wasm cache

1a758a0

nico-martin requested a review from xenova December 1, 2025 14:30

nico-martin changed the title ~~added wasm cache~~ [v4] added wasm cache Dec 2, 2025

xenova reviewed Dec 3, 2025

View reviewed changes

xenova mentioned this pull request Dec 3, 2025

[v4] Switch build system to use esbuild #1466

Merged

some refactoring of the hub.js and caching of the wasm factory

1ce23bc

xenova reviewed Dec 4, 2025

View reviewed changes

src/backends/onnx.js Outdated Show resolved Hide resolved

xenova reviewed Dec 4, 2025

View reviewed changes

src/env.js Show resolved Hide resolved

xenova reviewed Dec 4, 2025

View reviewed changes

src/env.js Show resolved Hide resolved

nico-martin added 2 commits December 4, 2025 17:37

fixed comment

e480fc3

added string as cache return

24596d9

xenova requested changes Dec 9, 2025

View reviewed changes

src/backends/utils/cacheWasm.js Show resolved Hide resolved

src/utils/hub.js Outdated Show resolved Hide resolved

fixes after review

3adc971

resolved conflicts

afef880

nico-martin requested a review from xenova December 15, 2025 09:13

Merge branch 'v4' into v4-cache-wasm-file

ddad7cb

xenova reviewed Dec 16, 2025

View reviewed changes

xenova and others added 3 commits December 16, 2025 00:21

Only return if match is found

01ba6ae

Merge branch 'v4-cache-wasm-file' of https://github.com/huggingface/t…

154b261

…ransformers.js into v4-cache-wasm-file

Return response even if cache doesn't exist

29c4010

Don't throw error if we can't open cache or load file from cache, but we are able to make the request.

xenova approved these changes Dec 16, 2025

View reviewed changes

xenova merged commit 0082c20 into v4 Dec 16, 2025
1 check failed

xenova deleted the v4-cache-wasm-file branch December 16, 2025 05:38

nico-martin mentioned this pull request Dec 18, 2025

V4 cache wasm file blob fix #1489

Merged

[v4] added wasm cache #1471

[v4] added wasm cache #1471

Uh oh!

Conversation

nico-martin commented Dec 1, 2025

Uh oh!

xenova commented Dec 1, 2025

Uh oh!

xenova left a comment

Choose a reason for hiding this comment

Uh oh!

nico-martin commented Dec 4, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

xenova left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

nico-martin commented Dec 9, 2025

Uh oh!

xenova Dec 16, 2025

Choose a reason for hiding this comment

Uh oh!

xenova Dec 16, 2025

Choose a reason for hiding this comment

Uh oh!

xenova Dec 16, 2025

Choose a reason for hiding this comment

Uh oh!

xenova left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

xmcp commented Dec 17, 2025

Uh oh!

xmcp commented Dec 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nico-martin commented Dec 17, 2025

Uh oh!

nico-martin commented Dec 17, 2025

Uh oh!

xmcp commented Dec 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

xmcp commented Dec 17, 2025

Uh oh!

nico-martin commented Dec 18, 2025

Uh oh!

nico-martin commented Dec 18, 2025

Uh oh!

xmcp commented Dec 18, 2025

Uh oh!

nico-martin commented Dec 18, 2025

Uh oh!

xenova commented Dec 21, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

xenova left a comment •

edited

Loading

xmcp commented Dec 17, 2025 •

edited

Loading

xmcp commented Dec 17, 2025 •

edited

Loading