Repository navigation
Commit 105817b
authored
Hoist camera model dispatch per row in image warps (colmap#4935)
Supersedes colmap#4930.
## Problem
`WarpImageBetweenCameras` and `WarpImageWithHomographyBetweenCameras`
call `Camera::CamFromImg` and `Camera::ImgFromCam` on every pixel. Both
go through the 19-way runtime `switch (model_id)` in
`sensor/models/runtime.h` (~24M dispatches for a 12 MP image), which
prevents the compiler from inlining the concrete camera model across a
scanline, hoisting row-invariant terms, constant-folding unit depth (`w
= 1`), and vectorizing the inner loop.
## Fix
Hoist the `CAMERA_MODEL_SWITCH_CASES` dispatch out of the inner `x` loop
into two scanline helpers (`CamFromImgRow` and `ImgFromCamRow`) in
`src/colmap/image/warp.cc`, and unify both warp functions into a single
`WarpBetweenCamerasImpl` (+30 net lines in `warp.cc`):
- Optimizes both `WarpImageBetweenCameras` and
`WarpImageWithHomographyBetweenCameras`, while deduplicating their
setup, scanline loop, interpolation dispatch, and rescaling.
- Accelerates both `CamFromImg` (target) and `ImgFromCam` (source) for
**all** camera models without leaking camera math or parameter indices
outside `sensor/models/`.
- Kept local to `warp.cc` rather than `sensor/models/runtime.h`: other
`CamFromImg` / `ImgFromCam` loops in the codebase
(`P3PEstimator::Residuals`, `HomographyMatrixRayEstimator::Residuals`,
`UndistortCamera`, `UndistortReconstruction`) operate on 100–10k points
(<0.5–3 ms) or 3D rays with `w != 1`, whereas scanline generation (`x +
0.5` with scalar `y`, unit depth `w = 1`, and separate input/output row
buffers) is significantly faster than generic in-place point-array
batching.
### Comparison with colmap#4930
colmap#4930 special-cases `PinholeCameraModel` in `WarpImageBetweenCameras` by
indexing `params[0..3]` directly. Compared to colmap#4930, this PR:
- Keeps all camera parameter layout and projection math inside
`sensor/models/`.
- Works for all 19 camera models on both source and target cameras (not
only `PINHOLE` targets).
- Covers `WarpImageWithHomographyBetweenCameras` (stereo rectification)
as well.
- Delivers larger speedups even on `PINHOLE` targets by also eliminating
per-pixel dispatch and `w = 1` division in `ImgFromCam`.
## Benchmark
4032×3024 RGB image (12.2 MP), Release build (`g++ 15.2 -O3 -DNDEBUG`)
on Intel Xeon @ 2.20 GHz, best of 5 runs after warmup; all outputs are
bit-identical to `main`:
| Function | Cameras (`source -> target`) | Interpolation | `main` |
colmap#4930 | **This PR** |
|---|---|---|---|---|---|
| `WarpImageBetweenCameras` | `OPENCV -> PINHOLE` | Bilinear | 1175.9 ms
| 908.7 ms (−22.7%) | **723.9 ms (−38.4%)** |
| `WarpImageBetweenCameras` | `OPENCV -> PINHOLE` | Nearest | 726.3 ms |
447.4 ms (−38.4%) | **274.4 ms (−62.2%)** |
| `WarpImageBetweenCameras` | `SIMPLE_RADIAL -> PINHOLE` | Bilinear |
1109.1 ms | 844.7 ms (−23.8%) | **660.3 ms (−40.5%)** |
| `WarpImageBetweenCameras` | `OPENCV -> SIMPLE_PINHOLE` | Bilinear |
1217.4 ms | 1180.9 ms (−3.0%) | **669.5 ms (−45.0%)** |
| `WarpImageWithHomographyBetweenCameras` | `OPENCV -> PINHOLE` |
Bilinear | 1159.3 ms | 1159.3 ms (0.0%) | **767.6 ms (−33.8%)** |
## Tests
- `Warp.CameraModelsMatchPerPixelProjection`
(`src/colmap/image/warp_test.cc`) checks bit-exact output against
per-pixel `Camera::CamFromImg` and `Camera::ImgFromCam` across `OPENCV
-> PINHOLE`, `SIMPLE_DIVISION -> FOV` (partial `ImgFromCam` failure),
and `SIMPLE_RADIAL -> EQUIRECTANGULAR` (partial `CamFromImg` failure)
for both interpolation modes, both direct and intermediate-resolution
warp paths, and both `WarpImageBetweenCameras` and
`WarpImageWithHomographyBetweenCameras`.
- `image/warp_test`, `image/undistortion_test`, `image/line_test`, and
`controllers/undistorters_test` pass.1 parent d0a411f commit 105817b
2 files changed
Lines changed: 295 additions & 124 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2 | 2 | | |
3 | 3 | | |
4 | 4 | | |
| 5 | + | |
5 | 6 | | |
6 | 7 | | |
7 | 8 | | |
8 | 9 | | |
9 | 10 | | |
10 | 11 | | |
| 12 | + | |
| 13 | + | |
11 | 14 | | |
12 | 15 | | |
13 | 16 | | |
| |||
61 | 64 | | |
62 | 65 | | |
63 | 66 | | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
64 | 133 | | |
65 | | - | |
66 | | - | |
67 | | - | |
68 | | - | |
69 | | - | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
70 | 140 | | |
71 | 141 | | |
72 | 142 | | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
73 | 147 | | |
74 | 148 | | |
75 | 149 | | |
76 | 150 | | |
77 | | - | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
78 | 166 | | |
79 | 167 | | |
| 168 | + | |
80 | 169 | | |
81 | 170 | | |
82 | 171 | | |
83 | 172 | | |
84 | 173 | | |
85 | 174 | | |
86 | | - | |
87 | | - | |
88 | | - | |
89 | | - | |
90 | | - | |
91 | | - | |
92 | | - | |
93 | | - | |
94 | | - | |
95 | | - | |
96 | | - | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
97 | 206 | | |
98 | 207 | | |
99 | | - | |
100 | | - | |
101 | | - | |
102 | | - | |
103 | | - | |
104 | | - | |
105 | | - | |
106 | | - | |
| 208 | + | |
| 209 | + | |
107 | 210 | | |
108 | | - | |
109 | | - | |
110 | 211 | | |
111 | 212 | | |
112 | 213 | | |
| |||
116 | 217 | | |
117 | 218 | | |
118 | 219 | | |
119 | | - | |
120 | | - | |
121 | | - | |
122 | | - | |
123 | | - | |
124 | | - | |
125 | | - | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
126 | 226 | | |
127 | 227 | | |
128 | | - | |
| 228 | + | |
129 | 229 | | |
130 | | - | |
| 230 | + | |
131 | 231 | | |
132 | | - | |
133 | | - | |
134 | | - | |
| 232 | + | |
| 233 | + | |
135 | 234 | | |
136 | 235 | | |
137 | 236 | | |
138 | 237 | | |
139 | 238 | | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
| 250 | + | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
140 | 254 | | |
141 | 255 | | |
142 | 256 | | |
| |||
163 | 277 | | |
164 | 278 | | |
165 | 279 | | |
166 | | - | |
167 | | - | |
168 | | - | |
169 | | - | |
170 | | - | |
171 | | - | |
172 | | - | |
173 | | - | |
174 | | - | |
175 | | - | |
176 | | - | |
177 | | - | |
178 | | - | |
179 | | - | |
180 | | - | |
181 | | - | |
182 | | - | |
183 | | - | |
184 | | - | |
185 | | - | |
186 | | - | |
187 | | - | |
188 | | - | |
189 | | - | |
190 | | - | |
191 | | - | |
192 | | - | |
193 | | - | |
194 | | - | |
195 | | - | |
196 | | - | |
197 | | - | |
198 | | - | |
199 | | - | |
200 | | - | |
201 | | - | |
202 | | - | |
203 | | - | |
204 | | - | |
205 | | - | |
206 | | - | |
207 | | - | |
208 | | - | |
209 | | - | |
210 | | - | |
211 | | - | |
212 | | - | |
213 | | - | |
214 | | - | |
215 | | - | |
216 | | - | |
217 | | - | |
218 | | - | |
219 | | - | |
220 | | - | |
221 | | - | |
222 | | - | |
223 | | - | |
224 | | - | |
225 | | - | |
226 | | - | |
227 | | - | |
228 | | - | |
229 | | - | |
230 | | - | |
231 | | - | |
232 | | - | |
233 | | - | |
234 | | - | |
235 | | - | |
236 | | - | |
237 | | - | |
238 | | - | |
239 | 280 | | |
240 | 281 | | |
241 | 282 | | |
242 | 283 | | |
243 | 284 | | |
244 | 285 | | |
245 | | - | |
246 | | - | |
247 | | - | |
248 | | - | |
249 | | - | |
250 | | - | |
251 | | - | |
252 | | - | |
253 | | - | |
254 | | - | |
255 | | - | |
256 | | - | |
| 286 | + | |
| 287 | + | |
257 | 288 | | |
258 | 289 | | |
259 | 290 | | |
| |||
0 commit comments