[charts] Add MapImagePlot for raster base maps#22977
Conversation
Add a `MapImagePlot` component to `@mui/x-charts-premium/Map`. It renders an `<image>` stretched to the bounding box of the displayed `geoData`, so a base map raster (e.g. a satellite mosaic) can sit under the series. It accepts `href` and any other SVG image attribute; the computed `x`/`y`/`width`/`height` can be overridden through props. Also add a Map demo rendering Mars: the 30 USGS quadrangles colored by mean elevation over a Viking surface mosaic (now drawn with `MapImagePlot`) plus labeled landmarks and mission sites, and a `geoIdentity` floor-plan example showing the provider is not tied to geographic data.
JCQuintas
added
type: new feature
Deploy previewBundle size
PerformanceTotal duration: 2,130.96 ms +268.85 ms(+14.4%) | Renders: 63 (+0)
…and 8 more (+13 within noise) — details Metric alarms
Check out the code infra dashboard for more information about this PR. |
Drop the geoIdentity floor-plan example and fix prettier formatting in the Mars demo.
Add a documented MapImagePlotProps interface (href, preserveAspectRatio with its 'none' default) so the API page is populated, and a dedicated section on the Map docs page.
Instead of stretching a rectangle to the geoData bounds, reproject the source image per-pixel (projection.invert) onto a canvas so it follows the chart projection. Add an imageBounds prop for sources that do not cover the whole globe.
Reproject only pixels that round-trip through the projection so areas outside the visible footprint stay transparent. Update the Mars demo to an orthographic globe with an orthographic/equirectangular toggle, and hide feature markers on the back hemisphere.
The regression harness blocks image downloads and does not serve docs/public, so the Viking mosaic never loaded. Inline it as a base64 data URI (downsized to 768x384) like other test-visible demo images, and drop the unused public asset.
Move the per-pixel inverse-resampling logic out of the effect into reprojectEquirectangularImage, with a thorough explanation of the inverse-mapping approach, the round-trip footprint test, and the failure modes.
Clear the previous data URL when inputs change so the stale (wrong-projection) raster is not shown during the async recompute.
Render the reprojected raster in every named d3-geo projection and wait for a sentinel (revealed once all canvases finish) before screenshotting.
| // Skip pixels hidden by the projection: if the coordinate does not | ||
| // project back to this pixel, it lies outside the visible footprint. | ||
| const reprojected = projection([lon, lat]); | ||
| if ( | ||
| !reprojected || | ||
| Math.abs(reprojected[0] - deviceX) > 0.5 || | ||
| Math.abs(reprojected[1] - deviceY) > 0.5 | ||
| ) { | ||
| continue; | ||
| } |
There was a problem hiding this comment.
Is this a thing?
I get that for spherical mars you need to test that geographic coord are visible. But here you start from pixels, so I doubt the project.invert will return an hidden coordinate
There was a problem hiding this comment.
If I remove this part, the image goes a bit wild 😆
Claude confirmed
the streaks radiating outside the disk are exactly the pixels that should be transparent. So the round-trip WAS necessary. Reason: d3.geoOrthographic.invert clamps points outside the disk to the limb rather than returning null, so those pixels pass the bounds check and smear edge colors. The round-trip (project(invert(pixel)) !== pixel) is what catches them. Reverting commit 1 — restoring the round-trip.
I tried a different approach by clipping the circle in the canvas instead, which also gives it a less jagged edge
There was a problem hiding this comment.
That explain why the drag feature work on this projection :)
This + zoom should give "interesting" result 😄
Split the effect: load/decode the image only on href change, then reproject synchronously on projection/size/bounds changes without reloading. Add an onReady callback fired after each reprojection, and use it for deterministic readiness in the all-projections regression fixture (the SVG <image> load event was racing the fast data-URI decode).
…ract isHidden Move the Mars demo into the MapImagePlot docs section and drop the duplicate 'Mapping any geography' section. Extract the projection round-trip visibility check into a createIsHidden helper. MapPointPlot tracked in mui#22991.
Starting from destination pixels, projection.invert already returns null outside the visible footprint, so the forward round-trip never culled anything. Remove it (one projection per pixel instead of two).
Move the Viking mosaic back to docs/public/static and reference it via /static. Serve docs/public in the regression build and allow the image past the image-block so the canvas reprojection still works in screenshots.
Reverts the previous removal: d3 azimuthal projections (e.g. orthographic) clamp invert outside the visible disk to the limb instead of returning null, so without the round-trip test those pixels smear edge colors around the globe. Confirmed visually.
…line
Replace the per-pixel round-trip cull with a single destination-in mask of the projected { type: 'Sphere' }. One inverse projection per pixel instead of two, anti-aliased footprint edge. Projections with unbounded outlines (gnomonic, stereographic, albersUsa) may need a fallback — the all-projections regression fixture will show.
The sphere mask hangs on projections with an unbounded outline (gnomonic, stereographic): their {type:'Sphere'} projects to infinite coordinates, freezing the canvas fill/toDataURL and timing out the regression. Restore the round-trip cull (handles all projections). Also count only non-null onReady in the fixture so the sentinel waits for real rasters instead of the null-on-mount call.
Playwright waitForSelector defaults to state: 'visible', so the display:none sentinel never resolved and the test timed out. Render it 1x1 instead. Verified the regression test passes locally (3s vs 20s timeout).
| const edge = (from: number, to: number) => { | ||
| const steps = Math.max(1, Math.ceil(Math.abs(to - from) / STEP)); | ||
| return Array.from({ length: steps }, (_, i) => from + ((to - from) * i) / steps); | ||
| }; |
There was a problem hiding this comment.
@alexfauquette The edge function is mainly to make the line between from -> to to have more points from -> p1 -> p2 ... -> to so that when projecting it can actually move the points on the line to the correct projection position.
If we remove the function, the orthographic projection becomes distorted:
There was a problem hiding this comment.
we can probably solve this by changing the data shape itself, but since it is in the dataset part, it shouldn't matter much.
| 'conicConformal', | ||
| 'conicEqualArea', | ||
| 'conicEquidistant', | ||
| 'albers', |
There was a problem hiding this comment.
@alexfauquette It identified this issue, is this what you were mentioning regarding zoom? Or should we create a new issue?
1. Why conic + albers cells are blank
Not the reprojection — it's the chart provider's conic fitting. Verified by replicating its fit logic on the globe data:
| projection | translate | map bounds (cell is 200×130) | |
|---|---|---|---|
| orthographic | [104, 65] |
[[39,0],[161,130]] |
VISIBLE |
| equirectangular | [100, 65] |
[[0,15],[200,115]] |
VISIBLE |
| conicConformal | [480, 250] |
[[350,250],[480,380]] |
OFF-CANVAS |
| conicEqualArea | [480, 250] |
[[372,190],[572,298]] |
OFF-CANVAS |
| albers | [480, 250] |
[[383,171],[577,301]] |
OFF-CANVAS |
The provider's projection selector has a separate branch for conic projections that sets only scale, never translate (or fitExtent). So conic maps stay at d3's default center [480, 250] — far outside the small cell → blank. geoAlbers is conic-based, so it hits the same branch.
albersUsa is different: it's a USA-only composite, so Mars coordinates fall outside its inset regions → also blank (expected; it's Earth/USA-specific).
So the fixture surfaced a real gap: ChartsGeoDataProviderPremium doesn't position conic projections into the drawing area. That's a provider bug, separate from this PR.
There was a problem hiding this comment.
Connic projections are weird. I updated to way to handle them in the zoom PR. I would not worry about them before merging the zoom PR
For example the conicConformal with antartica is buggy because bounding box with antartica have absurdes values
onReady can fire truthy more than once per cell as fitExtent settles; counting increments could reach the target before every cell rendered, racing the screenshot. Track a Set keyed by projection name and gate on size === count.
When the source image's extent has west > east it wraps across the antimeridian. Handle the wrapping longitude range in the inside-bounds test and the nearest-neighbor sampling instead of assuming west < east.
Forwarded SVG attributes were spread after x/y/width/height/href, letting a consumer override them and break the alignment with the projection. Spread first so the computed geometry always wins.
Render MapImagePlot with a synthetic banded image under both default and antimeridian-wrapping (west > east) imageBounds. Verified the wrapping case paints only the antimeridian half and leaves the rest transparent.
Render the from/to longitude-band mapping as a colored table next to the maps so the wrapping result is self-documenting. Corrected the last two 'to' bands (red, green) to match the actual reprojection.
|
About identifying when we are outside the projected map, I looked for a quick win. (which ended up eating my entire morning to understand what the hell stream are in D3-geo 😭) But I got something that work :) The idea is to add a 
function getIsVisible(projection: GeoProjection) {
let visible, x, y;
const postClipStream = projection.postclip?.();
// @ts-ignore d3 types require all three methods, but we only use point
const stream = projection.stream({
point(xArg, yArg) {
visible = true;
x = xArg;
y = yArg;
},
});
const visiblityTest = ([lon, lat]) => {
visible = false;
stream.point(lon, lat);
return visible && [x, y];
};
// @ts-ignore d3 types require all three methods, but we only use point
const postStream = postClipStream?.({
point() {
visible = true;
},
});
const visiblityTestCartesian = ([x, y]) => {
if (!postStream) {
return true;
}
visible = false;
postStream?.point(x, y);
return visible;
};
return ([deviceX, deviceY]: [number, number]): [number, number] | null => {
visible = false;
// Test if pixels are clipped post projection
if (!visiblityTestCartesian([deviceX, deviceY])) {
return null;
}
const coordinates = projection.invert?.([deviceX, deviceY]);
if (!coordinates) {
return null;
}
const [lon, lat] = coordinates;
const reverted = visiblityTest([lon, lat]);
if (
!reverted ||
Math.abs(reverted[0] - deviceX) > 0.5 ||
Math.abs(reverted[1] - deviceY) > 0.5
) {
return null;
}
return [lon, lat] as [number, number];
};
} |
This feels like more work rather than less. 🤔 Outputs are better though |
The provider does not translate conic projections into the drawing area, so conicConformal/conicEqualArea/conicEquidistant/albers rendered blank, and albersUsa is USA-only. Per review, conic fitting is reworked in the zoom PR; omit them here rather than bake blank baselines that will churn later.
Adopt alexfauquette's approach: probe the projection's post-clip (cartesian) stream and stream the inverted coordinate back through the projection, instead of only calling projection() for the round-trip. This also honors the cartesian clip (clipExtent) for future pan/zoom, and reads the visible-area boundary from d3 directly. Typed PointSink sink, no ts-ignore. Verified all projections + the antimeridian fixture still render correctly (no smear, no hang).
It the chart has a The remaining part is exactly what you did before: do projection.invert, do the projection back and look at the difference. Not sure if using stream for those help or not |
2 participants
Summary
Adds
MapImagePlotto@mui/x-charts-premium/Map: a raster base-map layer (satellite mosaic, scanned plan, …) that draws under the map series.Unlike a plain
<image>, it reprojects the raster to match the chart'sprojection, so the base map follows the geography instead of staying a flat rectangle—and pixels outside the projection's visible footprint stay transparent.How it works
useGeoPath().projection()and the drawing area fromuseDrawingArea.projection(invert(pixel)) === pixel) discards pixels outside the visible footprint, leaving them transparent—so projections likeorthographicrender a clean globe.Props
href— image URL. Must be same-origin or CORS-enabled (the raster is read off a canvas).imageBounds—[[west, south], [east, north]]geographic extent of the source, for images that don't cover the whole globe. Defaults to the full globe.Demo
The Map docs page gains:
MapImagePlot" section.The Mars texture is inlined as a data URI so it renders in the regression tests (which block image downloads).
Naming
Used
MapImagePlotto match the*Plotconvention (GeoDataPlot,MapShapePlot). Open toGeoImagePlot,MapRasterPlot, orBasemapImageif preferred.🤖 Generated with Claude Code