Skip to content

feat: add retainBody option to the glob() loader#15022

Merged
Princesseuh merged 3 commits intomainfrom
strip-entry-body
Jan 28, 2026
Merged

feat: add retainBody option to the glob() loader#15022
Princesseuh merged 3 commits intomainfrom
strip-entry-body

Conversation

@ascorbic
Copy link
Contributor

@ascorbic ascorbic commented Dec 15, 2025

Changes

Adds a new retainBody option to the glob() loader to allow reducing the size of the data store.

Currently, the glob() loader stores the raw body of each content file in entry.body, in addition to the rendered HTML in entry.rendered.html.

The retainBody option defaults to true, but you can set it to false to prevent the raw body of content files from being stored in the data store. This significantly reduces the deployed size of the data store and helps avoid hitting size limits for sites with very large collections. This has caused issues for some very large sites, which have hit the maximum JSON string size.

The rendered body will still be available in the entry.rendered.html property for markdown files, and the entry.filePath property will still point to the original file. For MDX collections this will dramatically reduce the size of the collection, as there will no longer be any body retained in the store.

import { defineCollection } from 'astro:content';
import { glob } from 'astro/loaders';

const blog = defineCollection({
  loader: glob({
    pattern: '**/*.md',
    base: './src/content/blog',
    retainBody: false
  }),
});

When retainBody is false, entry.body will be undefined instead of containing the raw file contents.

Testing

Docs

withastro/docs#13150

@changeset-bot
Copy link

changeset-bot bot commented Dec 15, 2025

🦋 Changeset detected

Latest commit: be0976d

The changes in this PR will be included in the next version bump.

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@github-actions github-actions bot added the pkg: astro Related to the core `astro` package (scope) label Dec 15, 2025
@codspeed-hq
Copy link

codspeed-hq bot commented Dec 15, 2025

CodSpeed Performance Report

Merging this PR will not alter performance

Comparing strip-entry-body (be0976d) with main (0aafc83)1

Summary

✅ 9 untouched benchmarks

Footnotes

  1. No successful run was found on main (8ca97e1) during the generation of this report, so 0aafc83 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report.

@matthewp
Copy link
Contributor

Would it make sense to have this enabled by default in Astro 6?

Copy link
Contributor

@matthewp matthewp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocked as a minor change.

@ascorbic
Copy link
Contributor Author

@matthewp yeah, I was going to suggest that

@delucis
Copy link
Member

delucis commented Dec 15, 2025

While thinking about content store size, it would be good to get #14315 over the line too. Even with an option like this, at some point things can get too big, so chunking feels like a good solution to that general problem.

@ascorbic
Copy link
Contributor Author

@delucis yeah, it needs some manual testing and benchmarking though. I would like to be confident it doesn't cause performance regressions.

@ascorbic ascorbic added the pr preview Apply this label to a PR to generate a preview release label Dec 15, 2025
@github-actions github-actions bot removed the pr preview Apply this label to a PR to generate a preview release label Dec 15, 2025
@pkg-pr-new
Copy link

pkg-pr-new bot commented Dec 15, 2025

npm i https://pkg.pr.new/astro@15022
npm i https://pkg.pr.new/@astrojs/svelte@15022

commit: fe1513f

@delucis
Copy link
Member

delucis commented Dec 15, 2025

@delucis yeah, it needs some manual testing and benchmarking though. I would like to be confident it doesn't cause performance regressions.

Thanks. Hopefully it works! While this option is a nice one for advanced optimizations, the “Hit limit => discover obscure option => double check you’re not actually using body anywhere” flow is not the nicest DX, so if Astro can make it rarer to hit those limits, that might be nice.

(Although we could also potentially one day consider defaulting to not storing the body and require people set retainBody: true in order to use it if we think it’s a rarer use case.)

@ascorbic
Copy link
Contributor Author

Yes, as @matthewp said, I would suggest we make this the default in Astro 6

@matthewp matthewp added this to the 5.17.0 milestone Dec 16, 2025
@matthewp
Copy link
Contributor

Added this to the 5.17 milestone. Not yet sure when that will be.

@sarah11918
Copy link
Member

(Just adding myself here, and nothing that we'll need docs when it gets closer to time!)

@Princesseuh Princesseuh self-assigned this Jan 21, 2026
Copy link
Member

@sarah11918 sarah11918 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving the changeset for docs, and I have a docs PR ready to go (suggestions welcome if you got 'em)!

@Princesseuh Princesseuh merged commit f1fce0e into main Jan 28, 2026
26 checks passed
@Princesseuh Princesseuh deleted the strip-entry-body branch January 28, 2026 14:43
@astrobot-houston astrobot-houston mentioned this pull request Jan 28, 2026
dadezzz pushed a commit to dadezzz/ice-notes that referenced this pull request Feb 7, 2026
This PR contains the following updates:

| Package | Change | [Age](https://docs.renovatebot.com/merge-confidence/) | [Confidence](https://docs.renovatebot.com/merge-confidence/) |
|---|---|---|---|
| [astro](https://astro.build) ([source](https://github.com/withastro/astro/tree/HEAD/packages/astro)) | [`5.16.15` → `5.17.1`](https://renovatebot.com/diffs/npm/astro/5.16.15/5.17.1) | ![age](https://developer.mend.io/api/mc/badges/age/npm/astro/5.17.1?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/npm/astro/5.16.15/5.17.1?slim=true) |

---

### Release Notes

<details>
<summary>withastro/astro (astro)</summary>

### [`v5.17.1`](https://github.com/withastro/astro/blob/HEAD/packages/astro/CHANGELOG.md#5171)

[Compare Source](https://github.com/withastro/astro/compare/[email protected]@5.17.1)

##### Patch Changes

- [#&#8203;15334](withastro/astro#15334) [`d715f1f`](withastro/astro@d715f1f) Thanks [@&#8203;florian-lefebvre](https://github.com/florian-lefebvre)! - **BREAKING CHANGE to the experimental Fonts API only**

  Removes the `getFontBuffer()` helper function exported from `astro:assets` when using the experimental Fonts API

  This experimental feature introduced in v15.6.13 ended up causing significant memory usage during build. This feature has been removed and will be reintroduced after further exploration and testing.

  If you were relying on this function, you can replicate the previous behavior manually:

  - On prerendered routes, read the file using `node:fs`
  - On server rendered routes, fetch files using URLs from `fontData` and `context.url`

### [`v5.17.0`](https://github.com/withastro/astro/blob/HEAD/packages/astro/CHANGELOG.md#5170)

[Compare Source](https://github.com/withastro/astro/compare/[email protected]@5.17.0)

##### Minor Changes

- [#&#8203;14932](withastro/astro#14932) [`b19d816`](withastro/astro@b19d816) Thanks [@&#8203;patrickarlt](https://github.com/patrickarlt)! - Adds support for returning a Promise from the `parser()` option of the `file()` loader

  This enables you to run asynchronous code such as fetching remote data or using async parsers when loading files with the Content Layer API.

  For example:

  ```js
  import { defineCollection } from 'astro:content';
  import { file } from 'astro/loaders';

  const blog = defineCollection({
    loader: file('src/data/blog.json', {
      parser: async (text) => {
        const data = JSON.parse(text);

        // Perform async operations like fetching additional data
        const enrichedData = await fetch(`https://api.example.com/enrich`, {
          method: 'POST',
          body: JSON.stringify(data),
        }).then((res) => res.json());

        return enrichedData;
      },
    }),
  });

  export const collections = { blog };
  ```

  See [the `parser()` reference documentation](https://docs.astro.build/en/reference/content-loader-reference/#parser) for more information.

- [#&#8203;15171](withastro/astro#15171) [`f220726`](withastro/astro@f220726) Thanks [@&#8203;mark-ignacio](https://github.com/mark-ignacio)! - Adds a new, optional `kernel` configuration option to select a resize algorithm in the Sharp image service

  By default, Sharp resizes images with the `lanczos3` kernel. This new config option allows you to set the default resizing algorithm to any resizing option supported by [Sharp](https://sharp.pixelplumbing.com/api-resize/#resize) (e.g. `linear`, `mks2021`).

  Kernel selection can produce quite noticeable differences depending on various characteristics of the source image - especially drawn art - so changing the kernel gives you more control over the appearance of images on your site:

  ```js
  export default defineConfig({
    image: {
      service: {
        entrypoint: 'astro/assets/services/sharp',
        config: {
          kernel: "mks2021"
        }
    }
  })
  ```

  This selection will apply to all images on your site, and is not yet configurable on a per-image basis. For more information, see [Sharps documentation on resizing images](https://sharp.pixelplumbing.com/api-resize/#resize).

- [#&#8203;15063](withastro/astro#15063) [`08e0fd7`](withastro/astro@08e0fd7) Thanks [@&#8203;jmortlock](https://github.com/jmortlock)! - Adds a new `partitioned` option when setting a cookie to allow creating partitioned cookies.

  [Partitioned cookies](https://developer.mozilla.org/en-US/docs/Web/Privacy/Guides/Privacy_sandbox/Partitioned_cookies) can only be read within the context of the top-level site on which they were set. This allows cross-site tracking to be blocked, while still enabling legitimate uses of third-party cookies.

  You can create a partitioned cookie by passing `partitioned: true` when setting a cookie. Note that partitioned cookies must also be set with `secure: true`:

  ```js
  Astro.cookies.set('my-cookie', 'value', {
    partitioned: true,
    secure: true,
  });
  ```

  For more information, see the [`AstroCookieSetOptions` API reference](https://docs.astro.build/en/reference/api-reference/#astrocookiesetoptions).

- [#&#8203;15022](withastro/astro#15022) [`f1fce0e`](withastro/astro@f1fce0e) Thanks [@&#8203;ascorbic](https://github.com/ascorbic)! - Adds a new `retainBody` option to the `glob()` loader to allow reducing the size of the data store.

  Currently, the `glob()` loader stores the raw body of each content file in the entry, in addition to the rendered HTML.

  The `retainBody` option defaults to `true`, but you can set it to `false` to prevent the raw body of content files from being stored in the data store. This significantly reduces the deployed size of the data store and helps avoid hitting size limits for sites with very large collections.

  The rendered body will still be available in the `entry.rendered.html` property for markdown files, and the `entry.filePath` property will still point to the original file.

  ```js
  import { defineCollection } from 'astro:content';
  import { glob } from 'astro/loaders';

  const blog = defineCollection({
    loader: glob({
      pattern: '**/*.md',
      base: './src/content/blog',
      retainBody: false,
    }),
  });
  ```

  When `retainBody` is `false`, `entry.body` will be `undefined` instead of containing the raw file contents.

- [#&#8203;15153](withastro/astro#15153) [`928529f`](withastro/astro@928529f) Thanks [@&#8203;jcayzac](https://github.com/jcayzac)! - Adds a new `background` property to the `<Image />` component.

  This optional property lets you pass a background color to flatten the image with. By default, Sharp uses a black background when flattening an image that is being converted to a format that does not support transparency (e.g. `jpeg`). Providing a value for `background` on an `<Image />` component, or passing it to the `getImage()` helper, will flatten images using that color instead.

  This is especially useful when the requested output format doesn't support an alpha channel (e.g. `jpeg`) and can't support transparent backgrounds.

  ```astro
  ---
  import { Image } from 'astro:assets';
  ---

  <Image
    src="/transparent.png"
    alt="A JPEG with a white background!"
    format="jpeg"
    background="#ffffff"
  />
  ```

  See more about this new property in [the image reference docs](https://docs.astro.build/en/reference/modules/astro-assets/#background)

- [#&#8203;15015](withastro/astro#15015) [`54f6006`](withastro/astro@54f6006) Thanks [@&#8203;tony](https://github.com/tony)! - Adds optional `placement` config option for the dev toolbar.

  You can now configure the default toolbar position (`'bottom-left'`, `'bottom-center'`, or `'bottom-right'`) via `devToolbar.placement` in your Astro config. This option is helpful for sites with UI elements (chat widgets, cookie banners) that are consistently obscured by the toolbar in the dev environment.

  You can set a project default that is consistent across environments (e.g. dev machines, browser instances, team members):

  ```js
  // astro.config.mjs
  export default defineConfig({
    devToolbar: {
      placement: 'bottom-left',
    },
  });
  ```

  User preferences from the toolbar UI (stored in `localStorage`) still take priority, so this setting can be overridden in individual situations as necessary.

### [`v5.16.16`](https://github.com/withastro/astro/blob/HEAD/packages/astro/CHANGELOG.md#51616)

[Compare Source](https://github.com/withastro/astro/compare/[email protected]@5.16.16)

##### Patch Changes

- [#&#8203;15281](withastro/astro#15281) [`a1b80c6`](withastro/astro@a1b80c6) Thanks [@&#8203;matthewp](https://github.com/matthewp)! - Ensures server island requests carry an encrypted component export identifier so they do not accidentally resolve to the wrong component.

- [#&#8203;15304](withastro/astro#15304) [`02ee3c7`](withastro/astro@02ee3c7) Thanks [@&#8203;cameronapak](https://github.com/cameronapak)! - Fix: Remove await from getActionResult example

- [#&#8203;15324](withastro/astro#15324) [`ab41c3e`](withastro/astro@ab41c3e) Thanks [@&#8203;Princesseuh](https://github.com/Princesseuh)! - Fixes an issue where certain unauthorized links could be rendered as clickable in the error overlay

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0Mi45Mi4xMiIsInVwZGF0ZWRJblZlciI6IjQyLjkyLjEyIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6W119-->

Co-authored-by: Renovate Bot <[email protected]>
Co-committed-by: Renovate Bot <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pkg: astro Related to the core `astro` package (scope)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants