Skip to content

fix(module-runner): handle non-ascii characters in base64 sourcemaps#21985

Merged
sapphi-red merged 10 commits intovitejs:mainfrom
WhiteKiwi:fix/utf8-decoding-issue
Mar 25, 2026
Merged

fix(module-runner): handle non-ascii characters in base64 sourcemaps#21985
sapphi-red merged 10 commits intovitejs:mainfrom
WhiteKiwi:fix/utf8-decoding-issue

Conversation

@WhiteKiwi
Copy link
Copy Markdown
Contributor

Fix UTF-8 decoding issue caused by using atob for base64 decoding.
Use Buffer.from(..., 'base64').toString('utf-8') when available to ensure correct decoding.

Closes #21984

@WhiteKiwi WhiteKiwi changed the title fix: Fix utf8 decoding issue fix: fix utf8 decoding issue Mar 23, 2026
@hi-ogawa
Copy link
Copy Markdown
Contributor

Is this somehow related to this? vitest-dev/vitest#9640

@WhiteKiwi
Copy link
Copy Markdown
Contributor Author

Is this somehow related to this? vitest-dev/vitest#9640

Yes, this seems to be the same issue.

@WhiteKiwi
Copy link
Copy Markdown
Contributor Author

I encountered this when updating snapshots after running tests. It seems to be the same issue.

@WhiteKiwi
Copy link
Copy Markdown
Contributor Author

Added browser support for base64 UTF-8 decoding
image

@WhiteKiwi
Copy link
Copy Markdown
Contributor Author

The bundle size limit was exceeded, so it was increased from 54 kB to 55 kB (53.87 kB → 54.10 kB).

@WhiteKiwi WhiteKiwi requested a review from themavik March 23, 2026 07:50
return new TextDecoder().decode(bytes)
}

return decodeURIComponent(escape(binary))
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it’s deprecated. I used TextDecoder for modern environments, and escape as a fallback for older ones.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It will be a bit slower in my experience. I think there’s more possible here.

Saw your change. Would hoist the textdecoder outside of the function to not reinstantiate all the time.

don’t know if this this function is used a lot in vite. If it’s the case I would probably try to optimize via conditional exports for each runtimes

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've applied the feedback here
9e7202b

@WhiteKiwi WhiteKiwi requested a review from belgattitude March 23, 2026 10:37
@afurm
Copy link
Copy Markdown

afurm commented Mar 23, 2026

Good iterative approach. On the escape/decodeURIComponent fallback: Vite's minimum supported Node version is 18, and TextDecoder has been available in Node since v11 (global since v16) and in all modern browsers for years. The fallback to the deprecated escape is effectively dead code for any environment Vite realistically runs in.

Removing it entirely would simplify the function and avoid shipping a deprecated API call. If there's a specific runtime that matters where TextDecoder is absent, it's worth calling that out explicitly in a comment rather than silently falling back to escape.

@WhiteKiwi
Copy link
Copy Markdown
Contributor Author

Good iterative approach. On the escape/decodeURIComponent fallback: Vite's minimum supported Node version is 18, and TextDecoder has been available in Node since v11 (global since v16) and in all modern browsers for years. The fallback to the deprecated escape is effectively dead code for any environment Vite realistically runs in.

Removing it entirely would simplify the function and avoid shipping a deprecated API call. If there's a specific runtime that matters where TextDecoder is absent, it's worth calling that out explicitly in a comment rather than silently falling back to escape.

I've applied the feedback here
c35679d

Copy link
Copy Markdown
Member

@bluwy bluwy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense to me

@sapphi-red sapphi-red added feat: ssr p2-edge-case Bug, but has workaround or limited in scope (priority) labels Mar 25, 2026
@sapphi-red sapphi-red changed the title fix: fix utf8 decoding issue fix(module-runner): handle non-ascii characters in base64 sourcemaps Mar 25, 2026
@sapphi-red sapphi-red merged commit 77c95bf into vitejs:main Mar 25, 2026
18 of 19 checks passed
@hi-ogawa
Copy link
Copy Markdown
Contributor

Vitest CI got ReferenceError: TextDecoder is not defined https://github.com/vitest-dev/vitest/actions/runs/23629687860/job/68826224687?pr=9986#step:8:1908 Probably due to this change and Vitest side needs to do something.

@sapphi-red
Copy link
Copy Markdown
Member

Hmm, I've checked that TextDecoder is available in Node 11+ so I thought it won't cause problems.

@hi-ogawa
Copy link
Copy Markdown
Contributor

I think this is some node vm or vitest vm quirks. I couldn't reproduce with simple test case, so it's possible that Vitest CI quirk.

renovate bot added a commit to andrei-picus-tink/auto-renovate that referenced this pull request Mar 31, 2026
| datasource | package | from  | to    |
| ---------- | ------- | ----- | ----- |
| npm        | vite    | 7.3.1 | 8.0.3 |


## [v8.0.3](https://github.com/vitejs/vite/blob/HEAD/packages/vite/CHANGELOG.md#small-803-2026-03-26-small)

##### Features

- update rolldown to 1.0.0-rc.12 ([#22024](vitejs/vite#22024)) ([84164ef](vitejs/vite@84164ef))

##### Bug Fixes

- **html:** cache unfiltered CSS list to prevent missing styles across entries ([#22017](vitejs/vite#22017)) ([5464190](vitejs/vite@5464190))
- **module-runner:** handle non-ascii characters in base64 sourcemaps ([#21985](vitejs/vite#21985)) ([77c95bf](vitejs/vite@77c95bf))
- **module-runner:** skip re-import if the runner is closed ([#22020](vitejs/vite#22020)) ([ee2c2cd](vitejs/vite@ee2c2cd))
- **optimizer:** scan is not resolving sub path import if used in a glob import ([#22018](vitejs/vite#22018)) ([ddfe20d](vitejs/vite@ddfe20d))
- **ssr:** ssrTransform incorrectly rewrites `meta` identifier inside `import.meta` when a binding named `meta` exists ([#22019](vitejs/vite#22019)) ([cff5f0c](vitejs/vite@cff5f0c))

##### Miscellaneous Chores

- **deps:** bump picomatch from 4.0.3 to 4.0.4 ([#22027](vitejs/vite#22027)) ([7e56003](vitejs/vite@7e56003))

##### Tests

- **html:** add tests for `getCssFilesForChunk` ([#22016](vitejs/vite#22016)) ([43fbbf9](vitejs/vite@43fbbf9))


## [v8.0.2](https://github.com/vitejs/vite/blob/HEAD/packages/vite/CHANGELOG.md#small-802-2026-03-23-small)

##### Features

- update rolldown to 1.0.0-rc.11 ([#21998](vitejs/vite#21998)) ([ff91c31](vitejs/vite@ff91c31))

##### Bug Fixes

- **deps:** update all non-major dependencies ([#21988](vitejs/vite#21988)) ([9b7d150](vitejs/vite@9b7d150))

##### Miscellaneous Chores

- **deps:** update dependency [@vitejs/devtools](https://github.com/vitejs/devtools) to ^0.1.5 ([#21992](vitejs/vite#21992)) ([b2dd65b](vitejs/vite@b2dd65b))


## [v8.0.1](https://github.com/vitejs/vite/blob/HEAD/packages/vite/CHANGELOG.md#small-801-2026-03-19-small)

##### Features

- update rolldown to 1.0.0-rc.10 ([#21932](vitejs/vite#21932)) ([b3c067d](vitejs/vite@b3c067d))

##### Bug Fixes

- **bundled-dev:** properly disable `inlineConst` optimization ([#21865](vitejs/vite#21865)) ([6d97142](vitejs/vite@6d97142))
- **css:** lightningcss minify failed when `build.target: 'es6'` ([#21933](vitejs/vite#21933)) ([5fcce46](vitejs/vite@5fcce46))
- **deps:** update all non-major dependencies ([#21878](vitejs/vite#21878)) ([6dbbd7f](vitejs/vite@6dbbd7f))
- **dev:** always use ESM Oxc runtime ([#21829](vitejs/vite#21829)) ([d323ed7](vitejs/vite@d323ed7))
- **dev:** handle concurrent restarts in `_createServer` ([#21810](vitejs/vite#21810)) ([40bc729](vitejs/vite@40bc729))
- handle `+` symbol in package subpath exports during dep optimization ([#21886](vitejs/vite#21886)) ([86db93d](vitejs/vite@86db93d))
- improve `no-cors` request block error ([#21902](vitejs/vite#21902)) ([5ba688b](vitejs/vite@5ba688b))
- use precise regexes for transform filter to avoid backtracking ([#21800](vitejs/vite#21800)) ([dbe41bd](vitejs/vite@dbe41bd))
- **worker:** `require(json)` result should not be wrapped ([#21847](vitejs/vite#21847)) ([0672fd2](vitejs/vite@0672fd2))
- **worker:** make worker output consistent with client and SSR ([#21871](vitejs/vite#21871)) ([69454d7](vitejs/vite@69454d7))

##### Miscellaneous Chores

- add changelog rearrange script ([#21835](vitejs/vite#21835)) ([efef073](vitejs/vite@efef073))
- **deps:** bump required `@vitejs/devtools` version to 0.1+ ([#21925](vitejs/vite#21925)) ([12932f5](vitejs/vite@12932f5))
- **deps:** update rolldown-related dependencies ([#21787](vitejs/vite#21787)) ([1af1d3a](vitejs/vite@1af1d3a))
- rearrange 8.0 changelog ([8e05b61](vitejs/vite@8e05b61))
- rearrange 8.0 changelog ([#21834](vitejs/vite#21834)) ([86edeee](vitejs/vite@86edeee))


## [v8.0.0](https://github.com/vitejs/vite/blob/HEAD/packages/vite/CHANGELOG.md#800-2026-03-12)

##### Features

- update rolldown to 1.0.0-rc.9 ([#21813](vitejs/vite#21813)) ([f05be0e](vitejs/vite@f05be0e))
- warn when `vite-tsconfig-paths` plugin is detected ([#21781](vitejs/vite#21781)) ([ada493e](vitejs/vite@ada493e))

##### Bug Fixes

- **deps:** update all non-major dependencies ([#21786](vitejs/vite#21786)) ([eaa4352](vitejs/vite@eaa4352))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feat: ssr p2-edge-case Bug, but has workaround or limited in scope (priority)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Incorrect UTF-8 decoding when using atob in Node.js

6 participants