An Introduction to

FFmpeg, DaVinci Resolve, Timelapse and Fulldome Video Production,

Special Effects, Color Grading, Streaming, Audio Processing,

Canon 6D, Canon 5D-MK4, Panasonic LUMIX GH5S,

Kodak PIXPRO SP360 4K, Ricoh Theta V,

Synthesizers, Image Processing and Astronomy Software

by

Michael Koch, astroelectronic@[Link]

Version from January 27, 2024

1
Hint: The table of contents is quite long and it may be difficult to find what you are looking for. Try the index at the end of the book!

Contents
1 Introduction to FFmpeg............................................................................ 12 2.29 Color grading with color look-up tables, simplified workflow ....66
1.1 What can be done with FFmpeg?.................................................... 15 2.30 Size of Haldclut tables.................................................................... 67
1.2 If FFmpeg has no graphical user interface, how do we use it?....16 2.31 1D- and 3D *.cube LUT's................................................................. 68
1.3 The first example.............................................................................. 18 2.32 Convert a *.cube LUT to a Haldclut file......................................... 73
1.4 Using variables................................................................................. 19 2.33 Colorchart source........................................................................... 74
2 FFmpeg in detail....................................................................................... 20 2.34 Colormap filter................................................................................ 76
2.1 Convert from one video format to another video format...............20 2.35 Colorhold, chromahold and hsvhold filters.................................. 79
2.2 Change the container format........................................................... 20 2.36 Atmospheric dispersion correction............................................... 81
2.3 Fit timelapse length to music length............................................... 21 2.37 Amplify filter.................................................................................... 82
2.4 Timelapse or slideshow from many images, with crossfading.....22 2.38 Sharpen or blur images.................................................................. 82
2.5 Slideshow with different durations ................................................ 24 2.39 FFT Filtering.................................................................................... 85
2.6 Slideshow with scrolling images..................................................... 25 2.40 Extract a time segment from a video............................................. 92
2.7 Extract many images from a video.................................................. 28 2.41 Remove a segment from a video................................................... 93
2.8 Extract the n_th frame from a video................................................ 29 2.42 Trim filter......................................................................................... 94
2.9 Extract the first and last frame from a video.................................. 29 2.43 Tpad filter, add a few seconds black at the beginning or end....95
2.10 Modify brightness, contrast, saturation, gamma and hue...........30 2.44 Extract the last 30 seconds of a video.......................................... 96
2.11 Strong contrast enhancement....................................................... 31 2.45 Fade-in and fade-out....................................................................... 97
2.12 Inverting a video or image (make a negative)............................... 34 2.46 Crossfading..................................................................................... 98
2.13 Colorchannelmixer filter................................................................. 36 2.47 Crop a video.................................................................................... 99
2.14 Shuffleplanes filter.......................................................................... 38 2.48 Add borders to a video................................................................. 100
2.15 Day-for-Night................................................................................... 39 2.49 Zoompan........................................................................................ 100
2.16 Colorcorrect filter............................................................................ 40 2.50 Changing the speed: slow motion and timelapse...................... 101
2.17 Colortemperature filter................................................................... 42 2.51 Slow motion or timelapse only for a segment of the video.......102
2.18 Hue filter.......................................................................................... 44 2.52 Time Remapping........................................................................... 103
2.19 Huesaturation filter......................................................................... 46 2.53 Insert a text which is visible for the whole duration.................. 105
2.20 Selectivecolor filter......................................................................... 48 2.54 Slowly fade a text in and out........................................................ 105
2.21 Colorbalance filter.......................................................................... 50 2.55 Vertical alignment of text.............................................................. 107
2.22 Colorcontrast filter.......................................................................... 52 2.56 Show a running clock in the video.............................................. 108
2.23 Monochrome filter........................................................................... 54 2.57 Generation of curved text for fulldome projection .................... 110
2.24 Colorize a monochrome image...................................................... 57 2.58 Write text on a transparent layer.................................................. 113
2.25 Vibrance filter.................................................................................. 58 2.59 Combine multiple videos with concat demuxer......................... 115
2.26 Swapuv filter................................................................................... 61 2.60 Combine multiple videos with concat filter................................ 117
2.27 Gradation curves............................................................................ 62 2.61 The "fps" filter............................................................................... 117
2.28 Color grading with color look-up tables, full workflow ...............64 2.62 Split a video in multiple segments............................................... 118

2
2.63 Switch between two inputs at a certain time.............................. 119 2.101 Vertical jitter effect...................................................................... 177
2.64 Switch between two cameras, using audio from camera1........121 2.102 CRT Screen Effect....................................................................... 178
2.65 Stack videos side by side (or on top of each other) .................122 2.103 Macroblock motion vectors....................................................... 178
2.66 Horizontal and vertical flipping.................................................... 122 2.104 Deblock filter............................................................................... 178
2.67 Stereo3d filter................................................................................ 122 2.105 Gradfun filter............................................................................... 179
2.68 Stack four videos to a 2x2 mosaic .............................................. 123 2.106 Dilation filter................................................................................ 179
2.69 Blink comparator.......................................................................... 124 2.107 Morphological transforms.......................................................... 180
2.70 Creating animated GIF or PNG.................................................... 126 2.108 Correct the radial distortion of (fisheye-) lenses...................... 183
2.71 Overlay an animated GIF over a video........................................ 127 2.109 Curve fitting with GeoGebra...................................................... 187
2.72 Changing the size of an animated GIF........................................ 127 2.110 Lensfun........................................................................................ 188
2.73 Replace one frame in a video by another................................... 128 2.111 Replace the lensfun filter by the remap filter............................ 191
2.74 Blend filter..................................................................................... 129 2.112 Deep-Zoom-In.............................................................................. 194
2.75 Circular mask (View through eyepiece)...................................... 131 2.113 V360 filter for rotation of equirectangular 360° videos............195
2.76 Radial attenuation (for overhead sky in a planetarium).............133 2.114 Using "v360" as a workaround for "rotate"............................... 199
2.77 Edge Attenuation.......................................................................... 136 2.115 Equirectangular images of the night sky.................................. 199
2.78 Binocular view simulation............................................................ 137 2.116 Equirectangular images of the surface of earth and planets. .200
2.79 Vignetting...................................................................................... 138 2.117 Undo spherical rotations............................................................ 201
2.80 Subtracting a darkframe............................................................... 139 2.118 Remap a fisheye video to an equirectangular video................202
2.81 Flatfield Correction....................................................................... 140 2.119 Remap an equirectangular video to a fisheye video................206
2.82 Histogram...................................................................................... 143 2.120 Realtime remapping from equirectangular to fisheye .............208
2.83 Lagfun filter................................................................................... 144 2.121 How to replace the v360 filter by the remap filter..................... 212
2.84 Deflicker a video........................................................................... 146 2.122 Optional Spherical Effects......................................................... 224
2.85 Star trails....................................................................................... 149 2.123 Off-center fisheye projection..................................................... 244
2.86 Bird trails....................................................................................... 150 2.124 DLP Beamer output..................................................................... 249
2.87 Rainbow-trail effect....................................................................... 153 2.125 Remap from equirectangular to double-fisheye....................... 250
2.88 Temporal slice-stacking effect..................................................... 154 2.126 Remap an equirectangular video to a "Little planet" video.....251
2.89 Extract and merge planes, split planes....................................... 155 2.127 Remap an equirectangular video to a "Mirror sphere" video..253
2.90 Extract the alpha channel............................................................. 156 2.128 Shifting the viewing direction in a fisheye image or video......257
2.91 Backgroundkey............................................................................. 156 2.129 How the "drawbox" filter works................................................. 259
2.92 Colorkey......................................................................................... 157 2.130 Stitching together double-fisheye videos................................. 260
2.93 Chromakey.................................................................................... 158 2.131 Remove stitching artefacts........................................................ 262
2.94 HSVkey........................................................................................... 159 2.132 Stitch double-fisheye images with alignment errors...............266
2.95 Lumakey........................................................................................ 160 2.133 Stitching multiple flat images to a 360° image......................... 270
2.96 Bluescreen / greenscreen............................................................ 161 2.134 Preprocessing a flat video for fulldome projection.................. 272
2.97 Real-time bluescreening .............................................................. 172 2.135 Rotating the earth, moon or planet............................................ 275
2.98 Extract moving objects from a video.......................................... 174 2.136 Live rotation of the moon........................................................... 281
2.99 Datascope...................................................................................... 175 2.137 Insert a crosshair in a live video................................................ 283
2.100 Video line jitter effect.................................................................. 176 2.138 Enlarge the corners of a live video............................................ 284

3
2.139 Tunnel effect................................................................................ 286 2.177 Create two-dimensional gradients............................................. 387
2.140 Black hole simulation with remap filter..................................... 290 2.178 Create random two-dimensional wave functions..................... 388
2.141 Wormhole simulation.................................................................. 296 2.179 Make spectrum images............................................................... 389
2.142 Realtime Wormhole in Planetarium (Hackathon 2022).............301 2.180 Radial Gradients......................................................................... 400
2.143 Spherical Transformations with two Inputs.............................. 314 2.181 Zoneplate..................................................................................... 401
2.144 Simulation of a moving wormhole............................................. 320 2.182 Make an additive color mixing image........................................ 402
2.145 Dynamically change commands with "sendcmd" or "zmq"....324 2.183 Make a test image with shifted RGB channels......................... 403
2.146 Sendcmd and commands........................................................... 325 2.184 Make a 8-color test image.......................................................... 404
2.147 Sending commands with ZMQ................................................... 331 2.185 Make a 125-color test image with hex or decimal numbers....405
2.148 Changing an input image while FFmpeg is running................333 2.186 Make a 729-color test image with hex numbers....................... 407
2.149 FFmpeg and C# Programming................................................... 334 2.187 Polar Bear Equation.................................................................... 408
2.150 FFmpeg wrapper libraries for C#............................................... 337 2.188 Redshift....................................................................................... 409
2.151 Video stabilization....................................................................... 338 2.189 SMPTE Color Bars...................................................................... 411
2.152 Remove linear drift..................................................................... 340 2.190 Make many JPG test images...................................................... 411
2.153 Stabilization of 360° Videos....................................................... 341 2.191 Make a grid video........................................................................ 412
2.154 Stabilization of 360° Videos, improved..................................... 346 2.192 Make a chessboard video........................................................... 412
2.155 Remap Video-in-Video with perspective filter.......................... 348 2.193 Make a test video with audio...................................................... 412
2.156 Image warping with displace filter............................................. 350 2.194 Make a noise video..................................................................... 413
2.157 Noise reduction........................................................................... 355 2.195 Make a grayscale test image...................................................... 413
2.158 -filter_complex_script................................................................. 356 2.196 Make a grayscale test image with decimal numbers...............414
2.159 Time delay within a filter chain.................................................. 356 2.197 Make a 10-bit grayscale test image with hex numbers............415
2.160 Recording one-time events with pre-trigger ............................ 357 2.198 Pseudocolor filter........................................................................ 416
2.161 Chroma subsampling, pixel formats of images or videos.......358 2.199 Test a filtergraph for 10-bit compatibility.................................. 417
2.162 Automatic format conversions.................................................. 367 2.200 Make a 10-bit test video with audio........................................... 421
2.163 RGB Color Cube.......................................................................... 369 2.201 Find an object in a video and hide it......................................... 421
2.164 Convert RGB to HSV or HSL...................................................... 373 2.202 Feedback filter............................................................................. 422
2.165 Convert HSV to RGB................................................................... 374 2.203 Find the coordinates of a moving object in a video ................423
2.166 Video Codecs.............................................................................. 375 2.204 Detect black frames and replace them by previous frame......424
2.167 Bitrate.......................................................................................... 378 2.205 Image formats............................................................................. 425
2.168 Keyframes.................................................................................... 378 2.206 Image size limits......................................................................... 426
2.169 Open GOP and closed GOP....................................................... 378 2.207 CR2 and DNG Images................................................................. 427
2.170 Video codecs with alpha channel.............................................. 379 2.208 Colorspace and zscale filter....................................................... 431
2.171 Working with Rawvideo.............................................................. 380 2.209 Video sizes.................................................................................. 434
2.172 Change the byte order (LE <--> BE)........................................... 382 2.210 Editing videos from PanoView XDV360 fulldome camera.......437
2.173 Metadata...................................................................................... 383 2.211 Editing videos from the Kodak SP360_4K camera................... 438
2.174 Video filters "copy" and "null"................................................... 384 2.212 Postprocessing of real time videos of the night sky...............439
2.175 Re-numbering images................................................................ 384 2.213 Workflow for night sky videos with GH5S................................ 442
2.176 Filenames for images................................................................. 386 2.214 Combine many options and filters............................................ 443

4
2.215 Timelapse example with masking, deflicker, rotation, fading. 444 2.253 FFmpeg console output............................................................. 505
2.216 Slow motion with Panasonic GH5S at 240fps.......................... 445 2.254 FFmpeg source code.................................................................. 506
2.217 Create a light curve of a star occultation.................................. 447 2.255 FFmpeg: Suggestions for improvement................................... 507
2.218 Create a light curve of the crab pulsar in M1............................ 453 3 Audio processing with FFmpeg............................................................. 529
2.219 Oscilloscope................................................................................ 455 3.1 Audio codecs................................................................................... 529
2.220 Vectorscope................................................................................. 455 3.2 Concat two or more audio files...................................................... 529
2.221 Capture a video or image from a webcam................................ 456 3.3 Combine multiple audio files with crossfadings.......................... 530
2.222 Capture from ASI178MM Camera............................................... 458 3.4 Change audio volume..................................................................... 530
2.223 Capture video from the desktop or from a window .................460 3.5 Change audio volume for a segment, with start / stop ramps....531
2.224 Capture video and audio from a HDMI to USB converter........462 3.6 Change audio sample rate and number of audio channels.........533
2.225 Adding *.srt subtitles to a video................................................. 466 3.7 Change audio length, sample rate and/or pitch .......................... 534
2.226 Adding *.ass subtitles or credit files to a video........................ 468 3.8 Add one or more sound effects to an audio track........................ 535
2.227 Remove Subtitles........................................................................ 469 3.9 Replace a segment of the audio stream by silence..................... 536
2.228 Remove Data Tracks................................................................... 469 3.10 Replace a segment of the audio stream by another stream......537
2.229 frei0r............................................................................................. 470 3.11 Add an audio stream to a video, if no audio stream exists........538
2.230 Mapping streams to the output (-map)...................................... 470 3.12 Stereo --> mix into one mono channel........................................ 539
2.231 Creating multiple outputs........................................................... 470 3.13 Check if both stereo channels are equal.................................... 540
2.232 Attaching a preview image to a video....................................... 471 3.14 Check if two mono inputs are equal............................................ 540
2.233 Attach Cover Art to a Video........................................................ 472 3.15 Extract one mono channel from stereo....................................... 540
2.234 Timeline support......................................................................... 473 3.16 Stereo --> two mono channels..................................................... 541
2.235 Expression evaluation................................................................ 484 3.17 Use only one channel of a stereo signal..................................... 541
2.236 Evaluation of mathematical functions....................................... 488 3.18 Mono --> stereo............................................................................. 542
2.237 Error handling in expressions................................................... 489 3.19 Two mono channels --> stereo..................................................... 542
2.238 Debugging of expressions......................................................... 490 3.20 Mix two stereo channels to one stereo channel......................... 543
2.239 List the R,G,B values as ASCII text file...................................... 492 3.21 Create a file with multiple audio streams.................................... 543
2.240 Downloading videos from Facebook........................................ 494 3.22 How to choose the correct audio volume level.......................... 544
2.241 Uploading spherical 360 images to Facebook.......................... 494 3.23 Remove low frequencies (wind noise) from an audio track......544
2.242 Uploading videos to Facebook.................................................. 495 3.24 Remove silence from an audio stream ....................................... 545
2.243 Downloading videos from Youtube........................................... 495 3.25 Make a video from an audio file................................................... 545
2.244 Youtube recommended settings................................................ 496 3.26 Convert ultrasound to the audible range, e.g. to hear bats.......546
2.245 Streaming from FFmpeg to YouTube Live................................ 497 3.27 Record sound with the computer's built-in microphone...........549
2.246 Limiting FFmpeg's encoding speed to realtime....................... 497 3.28 Record a "Voice-Over" audio track.............................................. 550
2.247 Point-to-point streaming............................................................ 498 3.29 Passing the FFmpeg output to FFplay........................................ 552
2.248 Streaming a video to a multicast address................................. 500 3.30 Record sound and pass the output to FFplay ........................... 552
2.249 RTMP Streaming......................................................................... 502 3.31 Record, process and play sound with FFplay............................ 553
2.250 Hardware acceleration................................................................ 503 3.32 Live ultrasound conversion......................................................... 554
2.251 OpenCL Hardware Acceleration................................................. 504 3.33 Extract the audio from a video..................................................... 556
2.252 Benchmarks................................................................................ 505 3.34 Split a video into audio-only and video-only.............................. 556

5
 3.35 Synchronize audio with video...................................................... 556 6.11 The "Edit" page in DaVinci Resolve............................................ 614
3.36 Sources for royalty-free music.................................................... 557 6.12 Speed Change............................................................................... 618
3.37 Sound effects, from frequency domain to time domain............558 6.13 Change the framerate of a clip..................................................... 620
3.38 Direct Digital Synthesis (DDS)..................................................... 564 6.14 GUI Controls in the "Edit" page................................................... 621
3.39 Doppler Effect............................................................................... 569 6.15 Markers.......................................................................................... 621
3.40 Create the continuously rising tone............................................ 571 6.16 Edit the audio volume of a clip.................................................... 622
3.41 Create the continuously falling tone........................................... 572 6.17 Normalize Audio............................................................................ 622
3.42 Create a Quadrature Signal.......................................................... 573 6.18 The "Fusion" page in DaVinci Resolve....................................... 623
3.43 Gravitational waves from binary black hole mergers................574 6.19 Add a new "MediaIn" node........................................................... 624
3.44 Create band-limited noise............................................................ 578 6.20 Fusion Clips.................................................................................. 625
3.45 Show the frequency response of a filter..................................... 579 6.21 The "Color" page in DaVinci Resolve.......................................... 626
3.46 Make an audio file with a short test tone.................................... 580 6.22 Corrector nodes in "Color" page................................................. 628
3.47 Measuring the audio volume........................................................ 580 6.23 Secondary Adjustments............................................................... 629
3.48 Convert an audio waveform to a picture..................................... 581 6.24 Save Color Gradings.................................................................... 629
3.49 a3dscope filter............................................................................... 587 6.25 Wipes............................................................................................. 630
3.50 Optical sound effect..................................................................... 588 6.26 The "Fairlight" page in DaVinci Resolve..................................... 630
3.51 Which equipment is useful for making sound records?...........589 6.27 The "Deliver" page in DaVinci Resolve....................................... 631
3.52 Create an alternating left/right stereo sound.............................. 590 6.28 Make a quadratic output video.................................................... 632
3.53 The "avsynctest" source.............................................................. 590 6.29 Day-for-Night................................................................................. 632
3.54 Comparison of Rode NT1 and NTG2 microphones.................... 591 6.30 Dynamic Zoom.............................................................................. 632
3.55 Mathematical properties of sample rates 44100 and 48000.......592 6.31 Synchronize audio with video...................................................... 633
3.56 Speed of sound............................................................................. 594 6.32 Cutting on the beat of the music................................................. 633
4 FFprobe................................................................................................... 595 6.33 Video stabilization........................................................................ 634
4.1 Count the number of frames.......................................................... 596 6.34 Processing of drone videos......................................................... 635
4.2 Find the keyframe timestamps...................................................... 596 6.35 Track an object.............................................................................. 636
5 FFplay...................................................................................................... 597 6.36 Track and blur (or pixelize) an object.......................................... 639
5.1 Wrong colors in FFplay.................................................................. 600 6.37 Track an object and cover it by a colored rectangle.................. 640
6 DaVinci Resolve 15 / 16 / 17 / 18............................................................ 601 6.38 Track an object and cover it by an image................................... 641
6.1 Tutorials on Youtube....................................................................... 602 6.39 Track an object and corner pin an image.................................... 642
6.2 Mouse buttons and keyboard shortcuts....................................... 604 6.40 Isolate Humans with "Magic Mask"............................................. 642
6.3 GUI Controls.................................................................................... 606 6.41 Overlay text................................................................................... 643
6.4 Preferences..................................................................................... 607 6.42 Color matching.............................................................................. 644
6.5 Project settings............................................................................... 608 6.43 Generate a LUT............................................................................. 645
6.6 Timeline Proxy Mode and Timeline Resolution............................ 609 6.44 Add a LUT...................................................................................... 645
6.7 The "Media" page in DaVinci Resolve........................................... 610 6.45 Apply a look-up-table to a clip..................................................... 646
6.8 Replace Pictures............................................................................. 611 6.46 Spline Editor.................................................................................. 646
6.9 Archive projects.............................................................................. 612 6.47 Special effects............................................................................... 647
6.10 The "Cut" page in DaVinci Resolve............................................. 613 6.48 Alpha masking............................................................................... 651

6
6.49 Qualifier......................................................................................... 652 6.87 Metadata........................................................................................ 696
6.50 Exclude two regions from a mask............................................... 653 6.88 Sound library................................................................................. 696
6.51 Snowfall......................................................................................... 654 6.89 Blackmagic RAW speed test........................................................ 697
6.52 Polyline toolbar............................................................................. 655 6.90 Recommended Hardware for DaVinci Resolve........................... 697
6.53 Artificial lightning.......................................................................... 656 6.91 DaVinci Resolve, Problems and Solutions................................. 697
6.54 Adjustment clips........................................................................... 658 6.92 Miscellaneous unsorted things................................................... 699
6.55 Trails.............................................................................................. 659 7 Shotcut.................................................................................................... 699
6.56 Fog Effect...................................................................................... 660 8 Exiftool..................................................................................................... 700
6.57 Explosion effect............................................................................ 661 9 IrfanView.................................................................................................. 702
6.58 Bluescreen / greenscreen............................................................ 663 9.1 Create an image with a transparent color..................................... 702
6.59 3D Effects...................................................................................... 664 10 Gimp....................................................................................................... 703
6.60 3D Water Effect.............................................................................. 665 11 Faststone Image Viewer....................................................................... 703
6.61 3D Camera Tracking...................................................................... 666 12 Adobe DNG converter........................................................................... 703
6.62 3D Plant Models............................................................................ 670 13 Batch files (DOS, Windows 7, 10 and 11)............................................ 704
6.63 Resolve Live.................................................................................. 670 13.1 Wildcards in filenames................................................................. 704
6.64 Time remapping............................................................................ 671 13.2 Create beeps in a batch file.......................................................... 705
6.65 Slow motion and timelapse.......................................................... 671 13.3 Loop over all files in a directory.................................................. 706
6.66 Reducing the framerate (Super 8 simulation)............................. 671 13.4 Create short text files or append text to a file............................ 706
6.67 Noise reduction............................................................................. 672 13.5 Calculate variables in a batch file................................................ 707
6.68 Reverse a video............................................................................. 673 13.6 if conditions................................................................................... 707
6.69 Recording a Voice-Over audio track ........................................... 674 13.7 Start a new process...................................................................... 707
6.70 ADR (Automated Dialog Replacement)....................................... 676 13.8 Redirection and Piping in Batch Files......................................... 708
6.71 VR360............................................................................................. 677 13.9 Command line length................................................................... 708
6.72 Reframing 360° videos with KartaVR.......................................... 678 14 Batch files (Unix, Linux)....................................................................... 708
6.73 Little Planet.................................................................................... 680 15 Regular Expressions............................................................................ 709
6.74 Spherical stabilizer....................................................................... 682 16 VLC Player............................................................................................. 710
6.75 Ambisonics.................................................................................... 683 16.1 Wrong colors in VLC Player......................................................... 711
6.76 Mirror sphere................................................................................. 684 16.2 How fullscreen video on the extended desktop......................... 711
6.77 Black hole effect........................................................................... 686 16.3 Show live video from a webcam (and make screenshots)........712
6.78 Wormhole effect............................................................................ 688 17 MPV........................................................................................................ 713
6.79 Correcting Lens Distortion........................................................... 688 18 CD Rippers............................................................................................ 714
6.80 Programming fuses...................................................................... 689 19 DVD Rippers.......................................................................................... 714
6.81 Multicam Editing........................................................................... 689 20 SpyderX Elite, Monitor calibration....................................................... 714
6.82 Subtracting a darkframe............................................................... 689 21 Color grading with 3D LUT Creator..................................................... 715
6.83 Subtitles......................................................................................... 690 21.1 Known problems and solutions................................................... 718
6.84 Supported Codecs........................................................................ 691 21.2 Beginner tutorials for 3D LUT Creator........................................ 720
6.85 Convert *.mkv videos for DaVinci Resolve................................. 691 21.3 Advanced tutorials for 3D LUT Creator....................................... 721
6.86 Convert 10-bit videos from GH5S for DaVinci Resolve.............693 21.4 Working with Color Targets in for 3D LUT Creator.................... 722

7
 21.5 Working with video in for 3D LUT Creator.................................. 722 26.11 Height of optical Axis.................................................................. 776
22 OBS - Open Broadcaster Software...................................................... 724 26.12 Test patterns and videos for fulldome projection.................... 777
22.1 OBS - Problems and solutions.................................................... 725 26.13 Fulldome Videos......................................................................... 780
22.2 CPU Load....................................................................................... 726 26.14 Other interesting Videos............................................................ 781
22.3 Facebook Live............................................................................... 727 26.15 Ideas for new Fulldome Videos.................................................. 782
22.4 Facebook Live - Mode "Camera"................................................. 727 26.16 Domes and projection systems................................................. 783
22.5 Facebook Live - Mode "Connect"................................................ 728 27 Canon 6D............................................................................................... 786
22.6 YouTube Live................................................................................. 729 27.1 Remote Control............................................................................. 786
22.7 Desktop for live broadcasting...................................................... 730 27.2 Problems and Solutions............................................................... 786
22.8 Live broadcasting, or recording in advance?............................. 730 28 Canon 5D-Mark4.................................................................................... 787
22.9 Using the same source multiple times........................................ 731 28.1 All Canon 5D-Mark4 video modes for PAL video system...........787
22.10 Studio Mode................................................................................ 731 28.2 All Canon 5D-Mark4 video modes for NTSC video system........788
22.11 Virtual Camera............................................................................. 731 28.3 Remote Control............................................................................. 789
22.12 OBS with ZWO ASI178MM Camera............................................ 732 28.4 Canon 5D-Mark4 Field of view..................................................... 789
22.13 Taking Screenshots.................................................................... 733 28.5 Active Canon EF / M42 Adapters.................................................. 791
23 Tips and tricks for video....................................................................... 734 28.6 Video tutorials for Canon 5D-Mark4............................................ 792
23.1 Neutral Density Filters.................................................................. 735 29 Panasonic LUMIX GH5S....................................................................... 796
23.2 Shutter Angle................................................................................. 736 29.1 GH5S Autofocus............................................................................ 796
23.3 Panning Speed ............................................................................. 737 29.2 GH5S Record formats................................................................... 797
23.5 Camera Movements and Rotations............................................. 738 29.3 GH5S Exposing for VLog-L.......................................................... 798
23.6 Camera Lens Comparison Tools................................................. 738 29.4 GH5S HLG (Hybrid Log Gamma)................................................. 803
24 Screenwriting........................................................................................ 739 29.5 GH5S Metering Modes.................................................................. 805
25 Unix (Ubuntu), compiling FFmpeg ..................................................... 740 29.6 GH5S Manual Focus..................................................................... 805
25.1 GIT.................................................................................................. 742 29.7 Focusing for Still Images of the Night Sky ................................ 805
25.2 Using the FFmpeg libraries in own programs............................ 744 29.8 GH5S Recommended settings..................................................... 806
25.3 Compiling FFmpeg under Windows............................................ 745 29.9 GH5S Custom settings C1, C2, C3-1, C3-2, C3-2........................ 807
26 Cameras and lenses for fulldome video production.......................... 746 29.10 GH5S Luminance level .............................................................. 808
26.1 Pixel Size in Fulldome Planetariums........................................... 747 29.11 GH5S Master pedestal level....................................................... 808
26.2 Read-out chip size of cameras at different video modes..........748 29.12 GH5S Video Size......................................................................... 809
26.3 Overview of available fisheye lenses.......................................... 749 29.13 GH5S Photo Size......................................................................... 810
26.4 Useful Formulas for Fisheye Lenses........................................... 765 29.14 GH5S Mechanical / electronic shutter ...................................... 811
26.5 Favorable camera / fisheye combinations (for video, not for still 29.15 GH5S Variable frame rate........................................................... 812
images).................................................................................................... 767 29.16 GH5S Longer exposure time than framerate allows................813
26.6 Fisheye projection lenses............................................................ 769 29.17 Recording duration on SD cards............................................... 814
26.7 Non-fisheye projection lenses..................................................... 771 29.18 GH5S Cable remote trigger........................................................ 814
26.8 Beamers for fulldome projection................................................. 773 29.19 GH5S Cheap chinese battery adapters..................................... 815
26.9 Flange distances........................................................................... 774 29.20 GH5S Telescopic effect.............................................................. 815
26.10 Aperture numbers, rounded and exact...................................... 776 29.21 GH5S External HDMI................................................................... 815

8
 29.22 GH5S Synchro Scan................................................................... 815 40 Analog Video Standards....................................................................... 884
29.23 GH5S Field of view with and without 0.64x SpeedBooster.....816 41 Color temperature test with video lights............................................. 885
29.24 Image Scale in Arcseconds per Pixel........................................ 819 42 TASCAM DR-70D................................................................................... 887
29.25 GH5S, all 77 video modes.......................................................... 821 43 TASCAM DR-701D................................................................................. 889
29.26 GH5S, all C4K 8 bit modes......................................................... 825 43.1 Matching the DR-701D's output level to the GH5S' input level. 891
29.27 GH5S, all C4K 10 bit modes....................................................... 825 44 The Apprehension Engine: Sound effects for horror films...............893
29.28 GH5S, all 4K 8 bit modes............................................................ 826 45 Synthesizers and Midi.......................................................................... 897
29.29 GH5S, all 4K 10 bit modes.......................................................... 827 45.1 The keyboard................................................................................. 897
29.30 GH5S, all anamorphic 8 bit modes............................................ 827 45.2 Frequencies and control voltages of the keys........................... 898
29.31 GH5S, all anamorphic 10 bit modes.......................................... 828 45.3 Envelope Generator...................................................................... 900
29.32 GH5S, all FHD 8 bit modes ........................................................ 829 45.4 Frequency Shifters........................................................................ 901
29.33 GH5S, all FHD 10 bit modes....................................................... 830 45.5 Midi................................................................................................. 902
29.34 GH5S, Problems and Solutions................................................. 830 45.6 3.5mm Midi Connectors............................................................... 903
30 ZWO ASI178MM Camera....................................................................... 831 45.7 USB/MIDI Adapters....................................................................... 903
31 Immersive 360° videos.......................................................................... 833 45.8 MIDI Editors, Sequencers and DAWs.......................................... 904
32 Pinhole Photography............................................................................ 833 45.9 MIDI Monitors................................................................................ 905
33 PanoView XDV360 camera................................................................... 834 45.10 Voice allocation modes.............................................................. 905
34 Kodak PIXPRO SP360 4K camera........................................................ 835 45.11 Analog Computer Music............................................................. 906
34.1 Remote control.............................................................................. 836 45.12 Useful Links................................................................................. 907
34.2 Using the PIXPRO SP360 4K Camera as a Webcam.................. 837 46 Arturia MatrixBrute................................................................................ 908
34.3 Convert 235° fisheye image to equirectangular panorama.......840 46.1 Presets / Patches.......................................................................... 908
35 Chinese 360° Panoramic camera......................................................... 841 46.2 Potentiometer behaviour.............................................................. 908
36 Ricoh Theta V........................................................................................ 842 46.3 Oscillator tuning........................................................................... 909
36.1 Processing equirectangular / ambisonic videos with FFmpeg. 843 46.4 Power saving mode...................................................................... 909
36.2 Plugins........................................................................................... 844 46.5 AUDIO MOD Section..................................................................... 909
36.3 Bluetooth remote control............................................................. 844 46.6 Control Voltages I/O...................................................................... 910
36.4 Change the viewing direction...................................................... 845 46.7 Visualizing or decoding MatrixBrute project files...................... 910
36.5 How to hide something from a 360° image................................. 845 46.8 Which Modules are unused?........................................................ 911
36.6 Make an equirectangular 360° image or video........................... 848 46.9 Ring Modulation............................................................................ 912
36.7 Make a "Little-Planet" picture or video....................................... 850 46.10 Show Modulation Sources on an Oscilloscope....................... 912
36.8 Live Streaming with Ricoh Theta V............................................. 854 46.11 Make a modulation source unipolar.......................................... 912
36.9 First Order Ambisonics................................................................ 859 47 Arturia MicroFreak................................................................................ 914
36.10 Making 360° Test Videos with First Order Ambisonic Sound..861 48 Arturia DrumBrute................................................................................. 919
36.11 Third-Order Ambisonics............................................................. 864 49 Moog Subsequent 37............................................................................ 922
36.12 Play First-Order Ambisonic Sound with 4 Speakers................872 49.1 Presets / Patches.......................................................................... 922
37 Insta360 GO 2........................................................................................ 879 49.2 Potentiometer behaviour.............................................................. 922
38 JJRC X17 Drone.................................................................................... 882 50 Moog Subharmonicon.......................................................................... 923
39 Lightning............................................................................................... 883 51 MidiEditor.............................................................................................. 924

9
52 Timelapse duration table...................................................................... 926 61 SharpCap............................................................................................... 973
53 Zhiyun Crane 3S.................................................................................... 927 61.1 Background Subtraction.............................................................. 973
54 Timelapse+ View................................................................................... 929 62 AutoHotKey........................................................................................... 975
55 Timelapse_Control_V2......................................................................... 930 63 Autostakkert!......................................................................................... 978
56 Guide 9.1................................................................................................ 934 64 Tycho Tracker........................................................................................ 978
56.1 Install Guide 9.1............................................................................ 934 65 C# Programming Project / Digital maps and Elevation Data ............979
56.2 Control a LX200-compatible telescope....................................... 934 65.1 Where is the best Place for recording Nature Sounds?............979
56.3 How to find out the COM number of a USB/RS232 adapter? . . .935 65.2 Digital topographic maps............................................................. 980
56.4 How to reset all COM numbers?.................................................. 935 65.3 Digital elevation data.................................................................... 981
56.5 Add new comets to Guide 9.1...................................................... 936 65.4 Noise map (red = traffic noise, blue = running water noise).....983
56.6 Add ephemerides to Guide 9.1..................................................... 937 66 Astronomy............................................................................................. 984
56.7 Add an overlay with comments .................................................. 938 66.1 Moon observing............................................................................ 984
56.8 Update the position of Jupiter's great red spot.......................... 939 66.2 Moon videos.................................................................................. 985
56.9 Add a user-definede location....................................................... 940 66.3 Interactive Maps of Moons and Planets:..................................... 986
56.10 Add a user-defined horizon........................................................ 940 66.4 Limiting magnitude for video astronomy.................................... 987
56.11 Switch between several user-defined horizon files.................. 942 66.5 Limiting magnitude for single or stacked exposures................987
56.12 Install the Gaia2 catalog for Guide 9.1...................................... 942 66.6 Useful calculations for Magnitudes............................................. 988
56.13 Set up Guide 9.1 to show only the Gaia2 catalog..................... 942 66.7 Artificial Stars................................................................................ 989
56.14 Find objects from Tycho calalog............................................... 943 66.8 Viewing below the horizon........................................................... 989
56.15 Moon libration............................................................................. 943 66.9 Crab Pulsar.................................................................................... 990
56.16 Switch Meteor Radiants on / off................................................. 944 66.10 How to visualize the size of the universe.................................. 991
57 Stellarium............................................................................................... 945 66.11 Solar System Model.................................................................... 991
58 Space Engine........................................................................................ 947 66.12 Selfmade Planetariums............................................................... 992
58.1 Keyboard shortcuts...................................................................... 949 66.13 Gravitational force of the moon................................................. 992
58.2 Planet classification...................................................................... 960 66.14 Distance units............................................................................. 992
58.3 Climate Model................................................................................ 963 66.15 Digital Terrain Maps.................................................................... 992
58.4 Interesting objects........................................................................ 963 66.16 N-Body-Problem.......................................................................... 993
58.5 Export Textures............................................................................. 963 66.17 Live All Sky Images..................................................................... 993
58.6 Export Skybox............................................................................... 964 66.18 Beteigeuze................................................................................... 993
58.7 Scripts............................................................................................ 965 67 DCF77 Decoding................................................................................... 994
58.8 Useful links.................................................................................... 966 68 The invisible Lattice Mast.................................................................... 996
58.9 Wormholes.................................................................................... 967 69 Fireflies, glowworms, lightning bugs ................................................. 997
58.10 Abbreviations.............................................................................. 967 70 Night-for-Day Pictures.......................................................................... 998
58.11 Suggestions for improvement................................................... 967 71 Spherical Trigonometry and Rotation Matrices................................ 1000
59 Fitswork................................................................................................. 969 72 List of Abbreviations........................................................................... 1001
60 DeepSkyStacker 4.2.3........................................................................... 970 73 Special characters.............................................................................. 1005
60.1 How to align images without stacking them............................... 970 74 Acknowledgements............................................................................ 1006
60.2 How to stack on comets with known motion ............................ 971 75 My To Do List....................................................................................... 1007

10
76 Index.................................................................................................... 1008

11
1 Introduction to FFmpeg

FFmpeg is an open-source software and available for Linux, Windows and OS-X. It's a very powerful command line tool and has no graphic user
interface.
Main project website: [Link]
Download site for the Windows builds: [Link]
Alternative download site for Windows builds: [Link]

How to install the Windows build:

Download the file "ffmpeg-git-essentials.7z", open the ZIP file, open the "bin" folder and copy [Link], [Link] and [Link] to a new folder, for
example to c:\ffmpeg\ That's all.
In rare cases, if you need some special libraries (for example lensfun), you might need to download the "ffmpeg-git-full" version instead. But you won't
need it for most examples in this book.
An alternative download site is here: [Link]

[Link] is the very powerful software for manipulating videos.

[Link] is a program for viewing the properties of videos, pictures or audio files. It's useful for troubleshooting.
[Link] is a video player. In most cases we don't need it if we use the VLC player instead.
It's also a good idea to save the file doc\[Link] somewhere on your own computer. This file contains (almost) the full documentation for
FFmpeg. The most important chapters are "Audio Filters" and "Video Filters".

Additional to the official documentation, there are also two wikis available:
• [Link] This wiki contains many useful informations. All available pages are listed here:
[Link]
• [Link] This wiki is very incomplete and outdated.

12
How to search in the FFmpeg-user archives:
The archives are located at [Link]
You can use a search engine and search for example for: setpts site:[Link]

Other websites with FFmpeg examples:

[Link]

Most examples in this document were tested with Windows 7, and beginning in March 2020 I also used Windows 10.

What are the pros and cons of FFmpeg, compared to other programs for video manipulation?
• Very powerful capabilities.
• It's an active project, updates come almost daily.
• Conversion from almost all formats to almost all other formats.
• In most cases, there are no restrictions for video size (width * height), except for extremely large sizes.
• There is a mailing list where you can ask questions in english. Before you ask, make sure that the problem is reproducible in the latest FFmpeg
version. Always include the complete FFmpeg console output, because it contains many useful informations.
• FFmpeg is a command line program and has no graphical user interface. At first glimpse this sounds like a big drawback. But it's a nice idea to
have all commands in a batch file, because later you can easily make modifications at all arbitrary steps in the workflow. Just modify the batch file
and execute it again.
• You will need some time for learning FFmpeg.
• Unfortunately the documentation is the weak point of the project, and many times I wished that the documentation contained more informations
and especially more examples.1
• It's always a good idea to begin with a working example, and then modify it step by step. I hope that the examples in this book are a good starting
point for you.

1 Why is FFmpeg's official documentation so incomplete? I think documentation has the lowest possible priority for the developers, and most of those
users who could write better documentation (including me) are unable or inwilling to work with GIT, which is the only way to make any changes to the
documentation.

13
• Unfortunately it's very complicated to compile FFmpeg for Windows, making it almost impossible to make your own changes to the code. I
haven't yet figured out how that works, after years of using FFmpeg. Compiling for Linux is easier.
• FFmpeg and DaVinci Resolve complement each other. It's best if you know and use both of them.

14
1.1 What can be done with FFmpeg?

• Convert a video, picture or sound from one format to another.

• Make a (timelapse) video from many pictures.
• Make many pictures from a video.
• Cut segments from a video, for example remove the beginning or the end.
• Add or remove audio, or modify the audio volume.
• Change the video size (width x height).
• Enlarge parts of the video or cut away the borders, for example make a rectangular video square.
• Change the speed, timelapse or slow motion.
• Rotate, mirror or flip.
• Add texts to a video.
• Correct brightness, contrast, gamma, saturation, color temperature, also with look-up-tables.
• Masking, for example superimpose a circular mask over a video.
• Fade-in, fade-out and crossfade for video and audio.
• Morphing, for example curved texts for fulldome projection, or simulation of curved spacetime near block holes.
• Stabilizing of shaky videos
• Deflicker, for reducing brightness steps in timelapse.
• Change the video compression, to make the video smaller.
• and many, many more interesting things...

15
1.2 If FFmpeg has no graphical user interface, how do we use it?

There are three possible ways:

1. Open a console window All_Programs / Accessories / Command_Promt (german) Alle_Programme / Zubehör / Eingabeaufforderung
Another way to open the console window is to press WINDOW R and then enter "cmd".
2. In the Windows File Explorer, in the address bar, you can type cmd and press enter to get a command prompt in the directory you are currently
examining.
3. But the above ways aren't recommended, because in many cases the command lines are quite long and you don't want to type the same
command line over and over again. The recommended way is to write a batch file which contains the FFmpeg command line:
• A batch file has the extension *.bat and can be created and modified with any text editor. When you save a batch file with Notepad, make sure that
you choose "all files" and save it as *.bat and don't choose "text files", because then the extension would be *.[Link] (Hint: Configure the explorer
so that all file extensions are visible!)
• You can edit a batch file by right clicking on it, and then choose "Edit".
• You can execute a batch file by double clicking on the icon or filename.
• Once you've created a batch file, you can place either it, or a short to it, on your Windows desktop. Then you can drag-and-drop one or more
(depending on how you've designed it) media files onto the icon for processing by the batch file.
• It's recommended to begin with a working example, and then modify it step by step. Make small steps and always make a test run. If it fails, go
back to the last working version.
• The % character has a special meaning inside a batch file. If you need a one % character in the FFmpeg command line, you must replace it in the
batch file by two %% characters.
• It's recommended to insert the command "pause" at the end of the batch file. This means the batch file waits for a keypress. Without this
command, the console window would close immediately when FFmpeg has finished, and you wouldn't see if there were any error messages.
• With the command "set" you can define variables in the batch file.
• With the command "rem" you can insert comments, so that you later understand how the batch file works. Comments can also begin with :: in the
same line as a command. Everything from :: to the end of the line is a comment.
• If the command line becomes too long, you can insert a ^ character at the end of the line and continue in the next line.
• How to copy and paste the content of the console window: Right click in the title of the Command_Prompt window, Edit -> Select_All, then Edit ->

16
 Copy, then paste the content with ctrl-v somewhere else.
• (german) Wenn man den Inhalt des CMD-Fensters kopieren möchte, geht man so vor: Rechtsklick auf die Titelleiste des Fensters, Bearbeiten -->
Alles auswählen, dann Bearbeiten -> Kopieren, dann mit Control-V irgendwo anders einfügen.
• If you don't want to write to full path to FFmpeg in each batch file, then you should add the path to the PATH system variable. In this article is
described how to do this: [Link]

The following was copied from the above link:

For Windows 7:

1. From the desktop, right-click the Computer icon and select Properties. If you don't have a Computer icon on your desktop, click Start, right-click
the Computer option in the Start menu, and select Properties.
2. Click the Advanced System Settings link in the left column.
3. In the System Properties window, click the Advanced tab, then click the Environment Variables button near the bottom of that tab.
4. In the Environment Variables window (pictured below), highlight the Path variable in the System variables section and click the Edit button. Add
or modify the path lines with the paths you want the computer to access. Each different directory is separated with a semicolon.

For Windows 10:

1. From the desktop, right-click the very bottom-left corner of the screen to get the "Power User Task Menu".
2. From the Power User Task Menu, click System.
3. In the Settings window, scroll down to the Related settings section and click the System info link.
4. In the System window, click the Advanced system settings link in the left navigation pane.
5. In the System Properties window, click the Advanced tab, then click the Environment Variables button near the bottom of that tab.
6. In the Environment Variables window (pictured below), highlight the Path variable in the System variables section and click the Edit button. Add
or modify the path lines with the paths you want the computer to access. Each different directory is separated with a semicolon [...].

17
1.3 The first example
This is a simple batch file:
rem A simple batch file for making a video from many pictures

c:\ffmpeg\ffmpeg -framerate 5 -start_number 3551 -i IMG_%%[Link] -i birds.mp3 ^

-shortest -codec:v mpeg4 -q:v 3 out.mp4

pause :: wait for a keypress

What's the meaning of the parts?

rem A simple ... This is a comment
c:\ffmpeg\ffmpeg This is the path to [Link]
-framerate 5 This defines how fast the pictures are read in, in this case 5 pictures per second.
-start_number 3551 This is the number of the first picture, in this case 3551
-i IMG_%%[Link] This is the filename of the input pictures. The term %%4d stands for a 4-digit number. The filename of the first picture is
IMG_3551.jpg and the number will be increased by 1, until no more picture is found. For 3-digit numbers you would write %%3d
instead. The double %% characters are only required in batch files, because the % character must be escaped. If you type the
command line directly in the console window, then you must use a single % character instead.
-i birds.mp3 This is the second input file, in this case an audio file.
^ If the command line becomes too long in the batch file, you can break it with the ^ character and continue in the next line.
FFmpeg will get the whole line without the ^ character.
-shortest This option means that the length of the output video is determined by the shortest of the two input files.
-codec:v mpeg4 This option means that a MPEG4 video will be produced.
-q:v 2 This is an option for the quality of the output video. 1 is best quality, 3 is normal, 31 is strongest compression.
out.mp4 Filename of the output video
pause This command waits for a keypress, so that you have a chance to see any error messages before the console window closes.
:: wait for ... Everything right of :: is a comment until the end of the line.
Important: Options are always written before the file they refer to.
The options "-framerate 5" and "-start_number 3551" refer to the first input file "IMG_%%[Link]". Use "IMG_%%[Link]" for 0-padded numbers.
The second input file "birds.mp3" doesn't have any options in this case.
The options "-shortest -codec:v mpeg4 -q:v 3" refer to the output video "out.mp4".

18
1.4 Using variables
Using variables is much better programming style. This batch file has exactly the same function as the first example:

rem A simple batch file for making a video from many pictures

set "FF=c:\ffmpeg\ffmpeg" :: Path to [Link]

set "FR=5" :: Framerate for reaing in the pictures (Frames per second)
set "SN=3551" :: Number of the first picture
set "IN=IMG_%%[Link]" :: Filename of the pictures
set "AUDIO=birds.mp3" :: Audio filename
set "QU=3" :: MP4 Quality, 1 ist best quality, 3 is normal, 31 is strongest compression
set "OUT=out.mp4" :: Output filemane

%FF% -framerate %FR% -start_number %SN% -i %IN% -i %AUDIO% -shortest -codec:v mpeg4 -q:v %QU% %OUT%

pause :: wait for a keypress

This is much clearer, because each variable is written in a new line and has its own comment.
It's recommended to use capital letters for the variables, so that you can easily distinguish them from command line options.
All variable names are allowed, but don't use special characters like ÄÖÜ.
You can copy a batch file and save it under a new name for a new project. Then you must only set the variables, so that they fit to the new project. There
is no need to modify (or even understand) the command line.
Why are the variable definitions written in " " quotation marks? This is only necessary if you want to add a comment in the same line. Without comments,
the quotation marks are unnecessary.

19
2 FFmpeg in detail

2.1 Convert from one video format to another video format

Some examples for format conversion:

rem Convert any input format to any output format

ffmpeg -i [Link] [Link]

rem Convert MP4 to mov

ffmpeg -i in.mp4 -acodec copy -vcodec copy -f mov [Link]

rem Convert mov to MP4

ffmpeg -i [Link] -acodec copy -vcodec copy out.mp4

rem Convert mov to MP4 using h265 compression, default preset is medium, default crf is 28
ffmpeg -i [Link] -c:v libx265 -preset slow -crf 25 -acodec copy out.mp4

2.2 Change the container format

If want to change only the container format from mkv to mp4, it's not necessary to re-encode the video and audio streams. These commands are very
fast:
ffmpeg -i [Link] -vcodec copy -acodec copy out.mp4
or
ffmpeg -i [Link] -c:v copy -c:a copy out.mp4

20
2.3 Fit timelapse length to music length

How to give a timelapse video exactly the same length as the music?
We don't want to cut off the end of the music, and we don't want to hear silence at the end of the timelapse video.
The solution is to adjust the framerate, so that the length of the timelapse becomes equal to the music length.
Framerate = Number_of_images / Time_in_seconds
In this example we have 30 images and the music is 20 seconds long, so that the framerate must be 1.5.
rem A simple batch file for combining many images to a video

set "FR=1.5" :: Framerate for reading in the images (frames per second)
set "RATE=30" :: Output framerate
set "SN=3551" :: Number of the first image
set "IN=IMG_%%[Link]" :: Filename of the images
set "AUDIO=birds.mp3" :: Audio filename
set "QU=3" :: MP4 Quality, 1 is best Quality, 3 is normal, 31 is strongest compression
set "OUT=out.mp4" :: Output file

ffmpeg -framerate %FR% -start_number %SN% -i %IN% -i %AUDIO% -r %RATE% -shortest -codec:v mpeg4 -q:v %QU% %OUT%

pause :: Wait for a keypress

In this example we have two different framerates, which have different purpose:
• -framerate %FR% this is the framerate for reading in the images
• -r %RATE% this is the framerate of the output video.
These two framerates are totally independent from each other, and can be different. If the images are read in slower than the output framerate, FFmpeg
will automatically duplicate images. If the images are read in faster, then FFmpeg will automatically skip images.

21
2.4 Timelapse or slideshow from many images, with crossfading

rem Make a timelapse or slideshow from many images, with crossfading

set "RATE=30" :: Output framerate

set "SN=3551" :: Number of first image
set "IN=IMG_%%[Link]" :: Filename of the images
set "W=2000" :: Image width
set "H=1500" :: Image height
set "QU=3" :: MP4 Quality, 1 is best Quality, 3 is normal, 31 is strongest compression
set "OUT=out.mp4" :: Output file
:: A is the duration how long each image is shown (without crossfading), here 1.0 sec
:: B is the duration of the crossfade, here 0.5 sec
set "C=3" :: set C = (A+B)/B (you must calculate this integer manually)
set "D=2" :: set D = 1/B (you must calculate this floating point value manually)

ffmpeg -start_number %SN% -i %IN% ^

-vf zoompan=d=%C%:fps=%D%:s=%W%x%H%,framerate=fps=%RATE%:interp_start=0:interp_end=255:scene=100 ^
-codec:v mpeg4 -q:v %QU% %OUT%

pause :: Wait for a keypress

Inside the video filter (beginning with -vf) we have in this example two filters, which are applied one after the other. The first is "zoompan" and the
second is "framerate".
You must calculate the variables C and D manually, because there are no expressions allowed inside the "zoompan" filter.

Detailed explanations for this filter chain:

-vf zoompan=d=%C%:fps=%D%:s=%W%x%H%,framerate=%RATE%:interp_start=0:interp_end=255:scene=100

In this filter chain two video filters are applied consevitively, separated by a (,) comma.
1. "zoompan", with the parameters "d" , "fps" and "s"
2. "framerate", with the parameters "fps", "interp_start", "interp_end", and "scene"

22
[Link]
The zoompan filter is here not used for zooming in, but for duplicating the frames and passing them to the next filter with a certain framerate.
"d" specifies how often each frame is repeated.
"fps" is the output framerate of this filter.
"s" is the size of the output frames. It must be specified in most cases, because the default is 1280x720.

[Link]
The framerate filter can calculate intermediate images between consecutive images. This is not a motion interpolation but a crossfade.
"fps" is the output framerate. It's not required to explicitely write this parameter; you could also write framerate=fps=%RATE%:...
The remaining three parameters "interp_start", "interp_end", and "scene" specify, when interpolation is active and when not. With those values that I
used (0, 255, 100), interpolation is always active.

These two filters together produce a video in which each image is shown for a certain duration, followed by a crossfade to the next image which also has
a certain duration. Both durations can be choosen freely, these are the values A and B in the comments. From these values you must manually calculate
the variables C and D, which are used in the command line. I haven't yet found a way to make this calculation automatically. It's possible to make
calculations in the batch file, but this works only with integer precision.
If you omit the zoompan filter and use only the framerate filter, the next crossfade would immediately follow when the previous has ended. With other
words: You always have a crossfade and there is no time where the image is shown without crossfade. That's why we use the trick with the zoompan
filter. Now it's still the case that one crossfade follows immediately on the prevoius one, but now we have crosssfades between identical images,
because the images were duplicated by the zoompan filter. A crossfade between identical images isn't visible, of course.

How to repeat the frames, so that the length fits to the music length:
Add -loop 1 and -shortest

23
2.5 Slideshow with different durations

ffmpeg -i img%[Link] -vf zoompan=d=25+'50*eq(in,3)'+'100*eq(in,5)' out.mp4

pause

In this example each frame is shown one second (25 frames), except the 4th image which is shown 3 seconds (25+50 frames) and the 6th image which is
shown 5 seconds (25+100 frames). Please note that the image numbering starts with 0, if not specififed differently with "-start_number".
Please note that it might also be useful to specify the size of the output frames with the "s" option, because the default size is 1280x720.

It's also possible to do the same thing with the concat demuxer. Make a text file with this content:
file '/path/to/[Link]'
duration 5
file '/path/to/[Link]'
duration 1
file '/path/to/[Link]'
duration 3
file '/path/to/[Link]'
duration 2
file '/path/to/[Link]'
Note: The last image has to be specified twice, the second one without any duration.
Then use this command line:
ffmpeg -f concat -i [Link] -vsync vfr -pix_fmt yuv420p output.mp4

pause

See also: [Link]

24
2.6 Slideshow with scrolling images

Images scrolling from left to right:

set "IN=test%%[Link]" :: Input images
set "N=6" :: Number of images
set "SX=400" :: X Size
set "SY=300" :: Y Size
set "T=5" :: Time in seconds for scrolling from one image to the next image
set "FPS=30" :: Output framerate
set "OUT=out.mp4" :: Output filename

rem Make some test images

ffmpeg -f lavfi -i testsrc2=size=%SX%x%SY%:duration=%N%:rate=1 -start_number 0 -y test%%[Link]

rem Make a scrolling slideshow

ffmpeg -framerate 1/%T% -start_number 1 -i %IN% -framerate 1/%T% -start_number 0 -i %IN% -filter_complex [0]
[1]hstack,fps=%FPS%,crop=w=iw/2:x='iw/2*(1-mod(t,%T%)/%T%)' -y %OUT%

pause

Images scrolling from right to left:

ffmpeg -framerate 1/%T% -start_number 0 -i %IN% -framerate 1/%T% -start_number 1 -i %IN% -filter_complex [0]
[1]hstack,fps=%FPS%,crop=w=iw/2:x='iw/2*mod(t,%T%)/%T%' -y %OUT%

pause

25
Images scrolling from top to bottom:
ffmpeg -framerate 1/%T% -start_number 1 -i %IN% -framerate 1/%T% -start_number 0 -i %IN% -filter_complex [0]
[1]vstack,fps=%FPS%,crop=h=ih/2:y='ih/2*(1-mod(t,%T%)/%T%)' -y %OUT%

pause

Images scrolling from bottom to top:

ffmpeg -framerate 1/%T% -start_number 0 -i %IN% -framerate 1/%T% -start_number 1 -i %IN% -filter_complex [0]
[1]vstack,fps=%FPS%,crop=h=ih/2:y='ih/2*mod(t,%T%)/%T%' -y %OUT%

pause

This is similar, but now showing two images simultaneously side by side. The width of the output video is twice the width of the input images:
set "IN=test%%[Link]" :: Input images
set "N=6" :: Number of images
set "SX=400" :: X Size
set "SY=300" :: Y Size
set "T=5" :: Time in seconds for scrolling from one image to the next image
set /a "D=%T%*(%N%-2)" :: Total duration in seconds
set "FPS=30" :: Output framerate
set "OUT=out.mp4" :: Output filename

rem Make some test images

ffmpeg -f lavfi -i testsrc2=size=%SX%x%SY%:duration=%N%:rate=1 -start_number 0 -y test%%[Link]

rem Make a scrolling slideshow

ffmpeg -framerate 1/%T% -start_number 2 -i %IN% -framerate 1/%T% -start_number 1 -i %IN% -framerate 1/%T% -start_number
0 -i %IN% -filter_complex [0][1][2]hstack=inputs=3,fps=%FPS%,crop=w=2*iw/3:x='iw/3*(1-mod(t,%T%)/%T%)' -t %D% -y %OUT%

pause

Note: "set /a" is a Windows batch command and calculates a variable (in this case: the total duration of the output video). Only integer arithmetic is
possible, no floating point. This is necessary in this batch file, because the "-t" option doesn't accept expressions, and using the "trim" filter as a

26
workaround is also impossible, because it doen's accept expressions.

Same thing as before, but scrolling from right to left:

ffmpeg -framerate 1/%T% -start_number 0 -i %IN% -framerate 1/%T% -start_number 1 -i %IN% -framerate 1/%T% -start_number
2 -i %IN% -filter_complex [0][1][2]hstack=inputs=3,fps=%FPS%,crop=w=2*iw/3:x='iw/3*mod(t,%T%)/%T%' -t %D% -y %OUT%

pause

Same thing as before, but scrolling from top to bottom:

ffmpeg -framerate 1/%T% -start_number 2 -i %IN% -framerate 1/%T% -start_number 1 -i %IN% -framerate 1/%T% -start_number
0 -i %IN% -filter_complex [0][1][2]vstack=inputs=3,fps=%FPS%,crop=h=2*ih/3:y='ih/3*(1-mod(t,%T%)/%T%)' -t %D% -y %OUT%

pause

Same thing as before, but scrolling from bottom to top:

ffmpeg -framerate 1/%T% -start_number 0 -i %IN% -framerate 1/%T% -start_number 1 -i %IN% -framerate 1/%T% -start_number
2 -i %IN% -filter_complex [0][1][2]vstack=inputs=3,fps=%FPS%,crop=h=2*ih/3:y='ih/3*mod(t,%T%)/%T%' -t %D% -y %OUT%

pause

27
2.7 Extract many images from a video

rem Extract many images from a video

ffmpeg -i in.mp4 -vf fps=0.2 -y image%%[Link]

pause :: Wait for a keypress

This batch file reads the file in.mp4 and produces images with the filenames
[Link], [Link], [Link], and so on.
-vf fps=0.2 this specifies that images are extracted with a framerate of 0.2, which means one frame every 5 seconds.
Omit this option if you want to extract all images.
-y this option means that FFmpeg will overwrite any output files that already exist with the same filename, without asking. If you omit this option,
FFmpeg would ask before overwriting a file.

This example extracts each n_th frame from a video, beginning with the 0_th frame:
set "IN=video.mp4" :: Input video
set "STEP=10" :: Step width
set "OUT=image%%[Link]" :: Output images filename

ffmpeg -i %IN% -vf framestep=%STEP% -y %OUT%

pause

28
2.8 Extract the n_th frame from a video

rem Make a test video

ffmpeg -f lavfi -i testsrc2=size=vga -t 3 -y [Link]

rem Extract the N_th frame

set "N=10" :: Frame number to be extracted (assuming the first frame is number 1)

ffmpeg -i [Link] -vf select='eq(n,%N%-1)' -frames 1 -y [Link]

pause

Note: The frame numbers begin with 0, that's why you must compare n with 9, if you want the 10_th frame.

2.9 Extract the first and last frame from a video

rem Extract the first frame

ffmpeg -i in.mp4 -frames 1 -y first_frame.jpg

rem Extract the last frame

ffmpeg -sseof -0.2 -i in.mp4 -update 1 -y last_frame.jpg

pause

29
2.10 Modify brightness, contrast, saturation, gamma and hue

rem Modify brightness, contrast, saturation, gamma and hue

set "INPUT=PanoView.mp4" :: Input video

set "OUTPUT=out.mp4" :: Output video
set "CONTRAST=1.0" :: Contrast in range -1000 to 1000, normal is 1.0
set "BRIGHT=0.0" :: Brightness in range -1.0 bis 1.0, normal is 0.0
set "SATUR=1.2" :: Saturation in range 0.0 bis 3.0, normal is 1.0
set "GAMMA=1.0" :: Gamma in range 0.1 to 10.0, normal is 1.0
set "HUE=20" :: Color correction (hue), negative shifts towards red and positive towards blue, normal is 0
:: Typical values are in the -30...+30 range
set "QU=3" :: MP4 Quality, 1 is best Quality, 3 is normal, 31 is strongest compression

ffmpeg -i %INPUT% -vf hue=h=%HUE%,eq=contrast=%CONTRAST%:brightness=%BRIGHT%:saturation=%SATUR%:gamma=%GAMMA% ^

-q:v %QU% -codec:v mpeg4 %OUTPUT%

pause

-vf is the command for "Video Filter". There are many different filters, see chapter "Video Filter" in the FFmpeg documentation.
In this case we use two filters, which are separated by a (,) comma.
• The first filter is "hue" and makes a rotation of the color circle.
• The second filter is "eq" and adjusts contrast, brightness, saturation and gamma.
From a mathematically point of view these functions work as follows:
• Contrast is a multiplication by a constant. Please note that what contrast does is scale the distance of a pixel's value from the median value i.e.
128 for a 8-bit input. So, if a pixel channel has a value of 100, then a contrast of 3 results in a value of 128 + 3*(100-128) = 44.
• Brightness is the addition of a constant.
• Saturation is difficult to describe mathematically. Setting saturation to 0 would produce a black and white video.
Warning: De-saturating a video with eq=saturation=0 produces a greenish tint in the output video. It's better to use hue=s=0.
• Gamma is a nonlinear distortion of the transfer function. When you increase the gamma value, details in dark areas become better visible.
It doesn't care in which order you write the parameters in the command line. They are always executed in the order contrast, brightness, gamma.

30
2.11 Strong contrast enhancement

There are several filters that can be used for a strong contrast enhancement, In FFmpeg it's surprisingly difficult to amplify the contrast of a video,
especially if the bit depth is more than 8-bit:

Filter Example for strong contrast enhancement by a factor 5: Notes

Input range [0.1 ... 0.3], Output range [0.0 ... 1.0], Output = 5 * (Input - 0.1)
eq eq=brightness=0.3,eq=contrast=5 The pivot point for contrast is always at 0.5 which
means you have to adjust both brightness and
contrast. The eq filter must be invoked two times,
because we need first the brightness adjustment and
then the contrast adjustment.
Warning: The eq filter introduces dithering when used
with 10-bit data.
eq=brightness=0.3:contrast=5 This doesn't work as expected because the eq filter is
invoked only one time, which means the order is
contrast before brightness and that's the wrong order
in this case.
eq=brightness=1.5:contrast=5 This doesn't work because the brightness value isn't
in the allowed range [-1 ... +1]
geq For 8-bit data: The drawback of the geq filter is that it's slow, has no
geq=lum='5*(lum(X,Y)-25.6)':cr='cr(X,Y)':cb='cb(X,Y)' built-in limiter and the function must be different for
8-bit and 10-bit videos.
For 8-bit data with limiter: It should be used together with the clip function as
geq=lum='clip(5*(lum(X,Y)-25.6),0,255)':cr='cr(X,Y)':cb='cb(X,Y)' limiter.
For 10-bit data:
geq=lum='5*(lum(X,Y)-102.4)':cr='cr(X,Y)':cb='cb(X,Y)' geq does also support 16-bit and float.

For 10-bit data with limiter:

geq=lum='clip(5*(lum(X,Y)-102.4),0,1023)':cr='cr(X,Y)':cb='cb(X,Y)'

31
lutyuv Version 1: lutyuv=y='5*(val-0.1*maxval)' Both versions work fine with 8-bit data and the
Version 2: lutyuv=y='5*val-0.5*maxval' additional limiter seems to be unnecessary.
With limiter: lutyuv=y='clip(5*(val-0.1*maxval),minval,maxval)' Warning: This filter may fail with 10-bit data!
For 10-bit data:
lutyuv=y='32*val-16384' (This works, but I don't understand why)
lut1d lut1d=file=[Link] I'm not sure if this works with 10-bit data
with this content in the [Link] file:
LUT_1D_SIZE 11
LUT_1D_INPUT_RANGE 0.0 1.0
0.000 0.000 0.000
0.000 0.000 0.000
0.500 0.500 0.500
1.000 1.000 1.000
1.000 1.000 1.000
1.000 1.000 1.000
1.000 1.000 1.000
1.000 1.000 1.000
1.000 1.000 1.000
1.000 1.000 1.000
1.000 1.000 1.000

This LUT should also work, but due to a bug in FFmpeg it doesn't
yet work:
LUT_1D_SIZE 2
LUT_1D_INPUT_RANGE 0.1 0.3
0.000 0.000 0.000
1.000 1.000 1.000

colorlevels colorlevels=rimin=0.1:gimin=0.1:bimin=0.1:rimax=0.3:gimax=0.3:bimax Best method because you can directly set the black
=0.3 and white points. The only drawback is that you have
to write the same values three times, but that can be
done with variables in the batch file.
Supports also 16-bit and float.

32
curves curves=all='0/0 0.1/0 0.3/1 1/1':interp=pchip This is a nonlinear transfer function because it uses a
smooth curve through the specified points.
Warning: You should always use the "interp=pchip"
option. The default (interp=natural) may have large
under- or overshoots, which are clipped to black or
white.
exposure I can't recommend this filter at all. The order is first "black level", then "exposure". This
filter works internally with floats.
Example for demonstrating that the "exposure" filter doesn't work Because it has no built-in limiter, it's highly
as expected: recommended to use an additional limiter.
"exposure" is limited to the [-3..+3] range, which
rem Make a 10-bit test image:
possibly corresponds to a factor from 0.05 to 20.
ffmpeg -f lavfi -i color=black:s=16x64,format=gray10le -lavfi These are not photographic EV values!
geq=lum='X+16*Y',format=gray10le,datascope=s=640x768:mode=color2:fo "black" is limited to the [-1..+1] range. Increase the
rmat=hex -frames 1 -y [Link] value to make the image darker.
Warning: This filter gives wrong results in some
rem Apply the exposure filter: cases, for example the amplification becomes infinite
if exopsure=1, black=0.5 or the image becomes a
ffmpeg -i [Link] -vf negative if exposure=1, black>0.5).
format=rgb48,exposure=exposure=1:black=0.5,limiter -y [Link] Have a look at the source code and try to reverse
engineer it. The mathematics is a total mess.
pause

33
2.12 Inverting a video or image (make a negative)

There are several methods for inverting (which means black becomes white, and vice versa):

Filter Example Notes

eq eq=contrast=-1 This changes only bright to dark and vice versa, but keeps the colors as they are.
negate negate This negates all channels and changes the colors to their complementary colors.
geq geq=r='255-r(X,Y)':g='255-g(X,Y)':b='255-b(X,Y)' Same result as negate. Can also be used if only one or two channels are to be
inverted. For 10-bit videos you must replace 255 by 1023.

This is an example. The input image is a spectrum with white at the top and black at the bottom:
ffmpeg -f lavfi -i nullsrc=s=1536x512 -vf geq=r='st(0,clip(512-X,0,255)+clip(X-
1024,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)':g='st(0,lt(X,512)*clip(X,0,255)+gte(X,512)*clip(1024-
X,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)':b='st(0,lt(X,1024)*clip(X-
512,0,255)+gte(X,1024)*clip(1536-X,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)',scale=iw/4:ih/4 -frames
1 -y [Link]

ffmpeg -i [Link] -vf eq=contrast=-1 -y [Link]

ffmpeg -i [Link] -vf negate -y [Link]
ffmpeg -i [Link] -vf geq=r='255-r(X,Y)':g='g(X,Y)':b='b(X,Y)' -y geq_r.png
ffmpeg -i [Link] -vf geq=r='r(X,Y)':g='255-g(X,Y)':b='b(X,Y)' -y geq_g.png
ffmpeg -i [Link] -vf geq=r='r(X,Y)':g='g(X,Y)':b='255-b(X,Y)' -y geq_b.png
ffmpeg -i [Link] -vf geq=r='255-r(X,Y)':g='255-g(X,Y)':b='b(X,Y)' -y geq_rg.png
ffmpeg -i [Link] -vf geq=r='255-r(X,Y)':g='g(X,Y)':b='255-b(X,Y)' -y geq_rb.png
ffmpeg -i [Link] -vf geq=r='r(X,Y)':g='255-g(X,Y)':b='255-b(X,Y)' -y geq_gb.png
ffmpeg -i [Link] -vf geq=r='255-r(X,Y)':g='255-g(X,Y)':b='255-b(X,Y)' -y geq_rgb.png

pause

34
These are the input and output images:

Input Output eq=contrast=-1 Output negate (or geq all channels inverted)

Output geq (only red channel inverted) Output geq (only green channel inverted) Output geq (only blue channel inverted)

Output geq (green and blue channels inverted) Output geq (red and blue channels inverted) Output geq (red and green channels inverted)

35
2.13 Colorchannelmixer filter

Color corrections can be made with the "colorchannelmixer" filter. In this example the red channel is enhanced by a factor 1.2 and the blue channel is
attenuated by a factor 0.8. The values must be in the [-2 ... +2] range. The input image is a spectrum with white at the top and black at the bottom:
set "R=1.2" :: Factor for red channel
set "G=1.0" :: Factor for green channel
set "B=0.8" :: Factor for blue channel

ffmpeg -f lavfi -i nullsrc=s=1536x512 -vf geq=r='st(0,clip(512-X,0,255)+clip(X-

1024,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)':g='st(0,lt(X,512)*clip(X,0,255)+gte(X,512)*clip(1024-
X,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)':b='st(0,lt(X,1024)*clip(X-
512,0,255)+gte(X,1024)*clip(1536-X,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)',scale=iw/4:ih/4 -frames
1 -y [Link]

ffmpeg -i [Link] -lavfi colorchannelmixer=rr=%R%:gg=%G%:bb=%B% [Link]

pause

These are the input and output images:

Input Output

Cyclic swapping of the RGB channels in one or the other direction:

colorchannelmixer=[Link] :: red becomes green, green becomes blue, blue becomes red
colorchannelmixer=[Link] :: red becomes blue, green becomes red, blue becomes green

36
Example for exchanging the red and green components:
ffmpeg -f lavfi -i nullsrc=s=1536x512 -vf geq=r='st(0,clip(512-X,0,255)+clip(X-
1024,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)':g='st(0,lt(X,512)*clip(X,0,255)+gte(X,512)*clip(1024-
X,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)':b='st(0,lt(X,1024)*clip(X-
512,0,255)+gte(X,1024)*clip(1536-X,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)',scale=iw/4:ih/4 -frames
1 -y [Link]

ffmpeg -i [Link] -vf colorchannelmixer=[Link] -y [Link]

pause

These are the input and output images:

Input Output

The options of the "colorchannelmixer" filter are specified in this order:

rr, rg, rb, ra, red_out = red_in * rr + green_in * rg + blue_in * rb + alpha_in * ra
gr, gg, gb, ga, green_out = red_in * gr + green_in * gg + blue_in * gb + alpha_in * ga
br, bg, bb, ba, blue_out = red_in * br + green_in * bg + blue_in * bb + alpha_in * ba
ar, ag, ab, aa alpha_out = red_in * ar + green_in * ag + blue_in * ab + alpha_in * aa

Copy the red channel to the green and blue channels

colorchannelmixer=[Link] :: copy the red channel to the green and blue channels

37
2.14 Shuffleplanes filter

This filter can swap or copy planes. What's stored in a plane depends on the pixel format. It must be a planar pixel format like GBGP or YUV for example.
If the input pixel format isn't planar (like RGB24), it will be automatically converted to a planar format and you might get unexpected results. In the
following examples the pixel format is GBRP:
ffmpeg -f lavfi -i nullsrc=s=1536x512 -vf geq=r='st(0,clip(512-X,0,255)+clip(X-
1024,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)':g='st(0,lt(X,512)*clip(X,0,255)+gte(X,512)*clip(1024-
X,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)':b='st(0,lt(X,1024)*clip(X-
512,0,255)+gte(X,1024)*clip(1536-X,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)',scale=iw/4:ih/4 -frames
1 -y [Link]

ffmpeg -i [Link] -lavfi format=gbrp,shuffleplanes=[Link] -y [Link]

ffmpeg -i [Link] -lavfi format=gbrp,shuffleplanes=[Link] -y [Link]
ffmpeg -i [Link] -lavfi format=gbrp,shuffleplanes=[Link] -y [Link]
ffmpeg -i [Link] -lavfi format=gbrp,shuffleplanes=[Link] -y red_green_swapped.png

pause

These are the input and output images:

Input redshift blueshift

green2gray red_green_swapped

38
2.15 Day-for-Night

Day-for-Night is a technique for shooting night scenes in daytime. In times of analog filming, it was realized by using a red filter (which darkens the blue
sky) and underexposing by about 2 stops.
It can be simulated as follows. The input image is a spectrum with white at the top and black at the bottom:
set "BW=0.3" :: This parameter sets how much of the red channel is converted to black and white
set "RGB=0.1" :: This parameter sets how much of the original RGB channels is kept

ffmpeg -f lavfi -i nullsrc=s=1536x512 -vf geq=r='st(0,clip(512-X,0,255)+clip(X-

1024,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)':g='st(0,lt(X,512)*clip(X,0,255)+gte(X,512)*clip(1024-
X,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)':b='st(0,lt(X,1024)*clip(X-
512,0,255)+gte(X,1024)*clip(1536-X,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)',scale=iw/4:ih/4 -frames
1 -y [Link]

ffmpeg -i [Link] -lavfi colorchannelmixer=rr=%BW%+%RGB%:gr=%BW%:br=%BW%:gg=%RGB%:bb=%RGB% -y [Link]

pause

These are the input and output images:

See also [Link]

For example, day-for-night was used in Wim Wenders' film "Kings of the Road" (german: "Im Lauf der Zeit"):
[Link]

39
2.16 Colorcorrect filter
The "colorcorrect" filter has 4 parameters, which can be set to any value in the [-1 ... +1] range. The default values are 0:
rl Correction value for red shadows
bl Correction value for blue shadows
rh Correction value for red highlights
bh Correction value for blue highlights

This is an example. The input image is a spectrum with white at the top and black at the bottom:
ffmpeg -f lavfi -i nullsrc=s=1536x512 -vf geq=r='st(0,clip(512-X,0,255)+clip(X-
1024,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)':g='st(0,lt(X,512)*clip(X,0,255)+gte(X,512)*clip(1024-
X,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)':b='st(0,lt(X,1024)*clip(X-
512,0,255)+gte(X,1024)*clip(1536-X,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)',scale=iw/4:ih/4 -frames
1 -y [Link]

ffmpeg -i [Link] -lavfi colorcorrect=rl=-0.2 -y rl_-[Link]

ffmpeg -i [Link] -lavfi colorcorrect=rl=0.2 -y rl_+[Link]
ffmpeg -i [Link] -lavfi colorcorrect=rh=-0.2 -y rh_-[Link]
ffmpeg -i [Link] -lavfi colorcorrect=rh=0.2 -y rh_+[Link]

pause

40
These are the input and output images:

Input Input

Output rl = -0.2 Output rl = 0.2

Output rh = -0.2 Output rh = 0.2

The colorcorrect filter does also have a "saturation" option which is pretty clear.
There is also an "analyze" option, but I didn't fully understand it. If you know how it works, please let me know.

41
2.17 Colortemperature filter

This is an example for the "colortemperature" filter. The default temperature is 6500K. The input image is a spectrum with white at the top and black at
the bottom:
ffmpeg -f lavfi -i nullsrc=s=1536x512 -vf geq=r='st(0,clip(512-X,0,255)+clip(X-
1024,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)':g='st(0,lt(X,512)*clip(X,0,255)+gte(X,512)*clip(1024-
X,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)':b='st(0,lt(X,1024)*clip(X-
512,0,255)+gte(X,1024)*clip(1536-X,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)',scale=iw/4:ih/4 -frames
1 -y [Link]

ffmpeg -i [Link] -lavfi colortemperature=3000 -y [Link]

ffmpeg -i [Link] -lavfi colortemperature=9000 -y [Link]
ffmpeg -i [Link] -lavfi colortemperature=3000:pl=1 -y [Link]
ffmpeg -i [Link] -lavfi colortemperature=9000:pl=1 -y [Link]

pause

42
These are the input and output images:

Input Input

Output colortemperature = 3000 Output colortemperature = 9000

Output colortemperature = 3000, pl = 1 Output colortemperature = 9000, pl = 1

43
2.18 Hue filter

This filter changes hue, saturation and brightness:

ffmpeg -f lavfi -i nullsrc=s=1536x512 -vf geq=r='st(0,clip(512-X,0,255)+clip(X-
1024,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)':g='st(0,lt(X,512)*clip(X,0,255)+gte(X,512)*clip(1024-
X,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)':b='st(0,lt(X,1024)*clip(X-
512,0,255)+gte(X,1024)*clip(1536-X,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)',scale=iw/4:ih/4 -frames
1 -y [Link]

ffmpeg -i [Link] -lavfi hue=h=60 -y h_60.png

ffmpeg -i [Link] -lavfi hue=h=30 -y h_30.png
ffmpeg -i [Link] -lavfi hue=h=-60 -y h_-[Link]
ffmpeg -i [Link] -lavfi hue=h=-30 -y h_-[Link]
ffmpeg -i [Link] -lavfi hue=s=2 -y s_2.png
ffmpeg -i [Link] -lavfi hue=s=0 -y s_0.png
ffmpeg -i [Link] -lavfi hue=s=0.5 -y s_05.png
ffmpeg -i [Link] -lavfi hue=s=-1 -y s_-[Link]
ffmpeg -i [Link] -lavfi hue=b=2 -y b_2.png
ffmpeg -i [Link] -lavfi hue=b=-2 -y b_-[Link]

pause

44
These are the input and output images:

Input b=-2 b=2

h=-60 h=-30 h=30

h=60 s=2 s=0.5

s=0 s=-1

45
2.19 Huesaturation filter

This filter works in RGB color space and changes hue, saturation and intensity:
ffmpeg -f lavfi -i nullsrc=s=1536x512 -vf geq=r='st(0,clip(512-X,0,255)+clip(X-
1024,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)':g='st(0,lt(X,512)*clip(X,0,255)+gte(X,512)*clip(1024-
X,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)':b='st(0,lt(X,1024)*clip(X-
512,0,255)+gte(X,1024)*clip(1536-X,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)',scale=iw/4:ih/4 -frames
1 -y [Link]

ffmpeg -i [Link] -lavfi huesaturation=hue=60 -y h_60.png

ffmpeg -i [Link] -lavfi huesaturation=hue=30 -y h_30.png
ffmpeg -i [Link] -lavfi huesaturation=hue=-60 -y h_-[Link]
ffmpeg -i [Link] -lavfi huesaturation=hue=-30 -y h_-[Link]
ffmpeg -i [Link] -lavfi huesaturation=saturation=0.5 -y s_05.png
ffmpeg -i [Link] -lavfi huesaturation=saturation=-0.5 -y s_-[Link]
ffmpeg -i [Link] -lavfi huesaturation=intensity=0.5 -y i_05.png
ffmpeg -i [Link] -lavfi huesaturation=intensity=-0.5 -y i_-[Link]
ffmpeg -i [Link] -lavfi huesaturation=saturation=-1:colors=c -y cyan_s_-[Link]
ffmpeg -i [Link] -lavfi huesaturation=saturation=-1:colors=c:strength=3 -y cyan_s_-1_strength_3.png
ffmpeg -i [Link] -lavfi huesaturation=hue=120:colors=c -y cyan_h_120.png

pause

46
These are the input and output images:

Input intensity=-0.5 intensity=0.5

hue=-60 hue=-30 hue=30

hue=60 saturation=-0.5 saturation=0.5

saturation=-1:colors=c saturation=-1:colors=c:strength=3 hue=120:colors=c

47
2.20 Selectivecolor filter

How does it work? There are 9 ranges of colors:

reds Adjustments for red pixels (pixels where the red component is the maximum)
yellows Adjustments for yellow pixels (pixels where the blue component is the minimum)
greens Adjustments for green pixels (pixels where the green component is the maximum)
cyans Adjustments for cyan pixels (pixels where the red component is the minimum)
blues Adjustments for blue pixels (pixels where the blue component is the maximum)
magentas Adjustments for magenta pixels (pixels where the green component is the minimum)
whites Adjustments for white pixels (pixels where all components are greater than 128)
neutrals Adjustments for all pixels except pure black and pure white
blacks Adjustments for black pixels (pixels where all components are less than 128)

For each of these color ranges, four values in the [-1 .. 1] range can be specified for adjustment of cyan, magenta, yellow and black.
There are absolute and relative modes, however the difference between them isn't easy to understand.
That's why I calculated some example images. The input image is a spectrum with white at the top and black at the bottom:
ffmpeg -f lavfi -i nullsrc=s=1536x512 -vf geq=r='st(0,clip(512-X,0,255)+clip(X-
1024,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)':g='st(0,lt(X,512)*clip(X,0,255)+gte(X,512)*clip(1024-
X,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)':b='st(0,lt(X,1024)*clip(X-
512,0,255)+gte(X,1024)*clip(1536-X,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)',scale=iw/4:ih/4 -frames
1 -y [Link]

ffmpeg -i [Link] -lavfi selectivecolor=greens="0 1 0 0":correction_method=absolute -y abs_greens_0_1_0_0.png

ffmpeg -i [Link] -lavfi selectivecolor=greens="0 1 0 0":correction_method=relative -y rel_greens_0_1_0_0.png
ffmpeg -i [Link] -lavfi selectivecolor=greens="0 -1 0 0":correction_method=absolute -y abs_greens_0_-1_0_0.png
ffmpeg -i [Link] -lavfi selectivecolor=greens="0 -1 0 0":correction_method=relative -y rel_greens_0_-1_0_0.png
ffmpeg -i abs_greens_0_-1_0_0.png -i rel_greens_0_-1_0_0.png -lavfi blend=all_mode=grainextract -y [Link]

pause

48
These are the input and output images:

Absolute mode (this is the default) Relative mode

Input (same image for absolute and

relative mode)

Output for greens="0 1 0 0"

Output for greens="0 -1 0 0"

Obviously there is no difference between absolute and relative modes, if the correction value is negative (this was verified by calculating the difference
image in the last FFmpeg command).
I find it quite surprising that pure green with +1 correction for magenta (in absolute mode) gives black output (and not white).
The same filter is also in Photoshop (including the absolute and relative modes) and you can google how it is assumed to work.
In my opinion the behaviour of this filter is difficult to understand.

49
2.21 Colorbalance filter

This is an example for the colorbalance filter. The input image is a spectrum with white at the top and black at the bottom:
ffmpeg -f lavfi -i nullsrc=s=1536x512 -vf geq=r='st(0,clip(512-X,0,255)+clip(X-
1024,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)':g='st(0,lt(X,512)*clip(X,0,255)+gte(X,512)*clip(1024-
X,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)':b='st(0,lt(X,1024)*clip(X-
512,0,255)+gte(X,1024)*clip(1536-X,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)',scale=iw/4:ih/4 -frames
1 -y [Link]

ffmpeg -i [Link] -lavfi colorbalance=gs=0.5 -y gs_05.png

ffmpeg -i [Link] -lavfi colorbalance=gm=0.5 -y gm_05.png
ffmpeg -i [Link] -lavfi colorbalance=gh=0.5 -y gh_05.png

pause

Input Output gs = 0.5

Output gm = 0.5 Output gh = 0.5

50
There is also a "preserve lightness" option which can be enabled:
ffmpeg -f lavfi -i nullsrc=s=1536x512 -vf geq=r='st(0,clip(512-X,0,255)+clip(X-
1024,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)':g='st(0,lt(X,512)*clip(X,0,255)+gte(X,512)*clip(1024-
X,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)':b='st(0,lt(X,1024)*clip(X-
512,0,255)+gte(X,1024)*clip(1536-X,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)',scale=iw/4:ih/4 -frames
1 -y [Link]

ffmpeg -i [Link] -lavfi colorbalance=gs=0.5:pl=1 -y gs_05.png

ffmpeg -i [Link] -lavfi colorbalance=gm=0.5:pl=1 -y gm_05.png
ffmpeg -i [Link] -lavfi colorbalance=gh=0.5:pl=1 -y gh_05.png

pause

Input Output gs = 0.5, pl = 1

Output gm = 0.5, pl = 1 Output gh = 0.5, pl = 1

51
2.22 Colorcontrast filter

This is an example for the colorcontrast filter. The input image is a spectrum with white at the top and black at the bottom:
ffmpeg -f lavfi -i nullsrc=s=1536x512 -vf geq=r='st(0,clip(512-X,0,255)+clip(X-
1024,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)':g='st(0,lt(X,512)*clip(X,0,255)+gte(X,512)*clip(1024-
X,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)':b='st(0,lt(X,1024)*clip(X-
512,0,255)+gte(X,1024)*clip(1536-X,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)',scale=iw/4:ih/4 -frames
1 -y [Link]

ffmpeg -i [Link] -lavfi colorcontrast=gm=0.5:gmw=1 -y gm_05.png

ffmpeg -i [Link] -lavfi colorcontrast=gm=-0.5:gmw=1 -y gm_-[Link]

pause

Input Input

Output gm = -0.5 Output gm = 0.5

52
There is also a "preserve lightness" option which can be enabled:
ffmpeg -f lavfi -i nullsrc=s=1536x512 -vf geq=r='st(0,clip(512-X,0,255)+clip(X-
1024,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)':g='st(0,lt(X,512)*clip(X,0,255)+gte(X,512)*clip(1024-
X,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)':b='st(0,lt(X,1024)*clip(X-
512,0,255)+gte(X,1024)*clip(1536-X,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)',scale=iw/4:ih/4 -frames
1 -y [Link]

ffmpeg -i [Link] -lavfi colorcontrast=gm=0.5:gmw=1:pl=1 -y gm_05.png

ffmpeg -i [Link] -lavfi colorcontrast=gm=-0.5:gmw=1:pl=1 -y gm_-[Link]

pause

Input Input

Output gm = -0.5, pl = 1 Output gm = 0.5, pl = 1

53
2.23 Monochrome filter

The "monochrome" filter can be used to convert a colored video to monochrome.

This is an example. The input image is a spectrum with white at the top and black at the bottom:
ffmpeg -f lavfi -i nullsrc=s=1536x512 -vf geq=r='st(0,clip(512-X,0,255)+clip(X-
1024,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)':g='st(0,lt(X,512)*clip(X,0,255)+gte(X,512)*clip(1024-
X,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)':b='st(0,lt(X,1024)*clip(X-
512,0,255)+gte(X,1024)*clip(1536-X,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)',scale=iw/4:ih/4 -frames
1 -y [Link]

ffmpeg -i [Link] -lavfi monochrome -y cb_0_cr_0.png

ffmpeg -i [Link] -lavfi monochrome=cb=0.5:cr=0.5 -y cb_05_cr_05.png
ffmpeg -i [Link] -lavfi monochrome=cb=-0.5:cr=0.5 -y cb_-05_cr_05.png
ffmpeg -i [Link] -lavfi monochrome=cb=0.5:cr=-0.5 -y cb_05_cr_-[Link]
ffmpeg -i [Link] -lavfi monochrome=cb=-0.5:cr=-0.5 -y cb_-05_cr_-[Link]

pause

54
These are the input and output images:

Input Output cb = 0, cr = 0

Output cb = -0.5, cr = -0.5 Output cb = +0.5, cr = -0.5

Output cb = -0.5, cr = +0.5 Output cb = +0.5, cr = +0.5

55
This is an example for the "size" option, which can be set in the [0.1 ... 10] range:
ffmpeg -f lavfi -i nullsrc=s=1536x512 -vf geq=r='st(0,clip(512-X,0,255)+clip(X-
1024,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)':g='st(0,lt(X,512)*clip(X,0,255)+gte(X,512)*clip(1024-
X,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)':b='st(0,lt(X,1024)*clip(X-
512,0,255)+gte(X,1024)*clip(1536-X,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)',scale=iw/4:ih/4 -frames
1 -y [Link]

ffmpeg -i [Link] -lavfi monochrome=cr=0:cb=0:size=.1 -y [Link]

ffmpeg -i [Link] -lavfi monochrome=cr=0:cb=0:size=1 -y [Link]
ffmpeg -i [Link] -lavfi monochrome=cr=0:cb=0:size=10 -y [Link]

pause

These are the input and output images:

Input Output size = 0.1

Output size = 1 Output size = 10

56
It's also possible to make a video monochrome by setting the saturation to 0, for example with the "eq" filter.

ffmpeg -i in.mp4 -lavfi eq=saturation=0 -y out.mp4

pause

This is a very simple method for converting a YUV color video to a monochrome video. The Y plane is extracted:
ffmpeg -i in.mp4 -lavfi extractplanes=y -y out.mp4

pause

2.24 Colorize a monochrome image

rem Make a monochrome image:

ffmpeg -f lavfi -i testsrc2,format=gray -frames 1 -y [Link]

rem Colorize this image with orange color:

ffmpeg -i [Link] -vf geq=r='1.0*p(X,Y)':g='0.5*p(X,Y)':b='0.0*p(X,Y)' -frames 1 -y [Link]

pause

The same thing can also be done with the "colorchannelmixer" filter:
ffmpeg -i [Link] -vf colorchannelmixer=rr=1.0:gr=0.5:br=0:gg=0:bb=0 -frames 1 -y [Link]

pause

57
2.25 Vibrance filter

Vibrance is difficult to unerstand. It's similar to saturation, but it protects skintones.

See also: [Link]

This is an example. The input image is a spectrum with white at the top and black at the bottom:
ffmpeg -f lavfi -i nullsrc=s=1536x512 -vf geq=r='st(0,clip(512-X,0,255)+clip(X-
1024,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)':g='st(0,lt(X,512)*clip(X,0,255)+gte(X,512)*clip(1024-
X,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)':b='st(0,lt(X,1024)*clip(X-
512,0,255)+gte(X,1024)*clip(1536-X,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)',scale=iw/4:ih/4 -frames
1 -y [Link]

ffmpeg -i [Link] -lavfi vibrance=intensity=1 -y int_1.png

ffmpeg -i [Link] -lavfi vibrance=intensity=-1 -y int_-[Link]
ffmpeg -i [Link] -lavfi vibrance=intensity=-1:alternate=1 -y int_-1_alt.png

ffmpeg -i [Link] -lavfi vibrance=intensity=1:gbal=-3 -y int_1_gbal_-[Link]

ffmpeg -i [Link] -lavfi vibrance=intensity=-1:gbal=-3 -y int_-1_gbal_-[Link]
ffmpeg -i [Link] -lavfi vibrance=intensity=-1:alternate=1:gbal=-3 -y int_-1_alt_gbal_-[Link]

ffmpeg -i [Link] -lavfi vibrance=intensity=1:gbal=3 -y int_1_gbal_3.png

ffmpeg -i [Link] -lavfi vibrance=intensity=-1:gbal=3 -y int_-1_gbal_3.png
ffmpeg -i [Link] -lavfi vibrance=intensity=-1:alternate=1:gbal=3 -y int_-1_alt_gbal_3.png

ffmpeg -i [Link] -lavfi vibrance=intensity=1:glum=0 -y int_1_glum_0.png

ffmpeg -i [Link] -lavfi vibrance=intensity=-1:glum=0 -y int_-1_glum_0.png
ffmpeg -i [Link] -lavfi vibrance=intensity=-1:alternate=1:glum=0 -y int_-1_alt_glum_0.png

ffmpeg -i [Link] -lavfi vibrance=intensity=1:glum=1 -y int_1_glum_1.png

ffmpeg -i [Link] -lavfi vibrance=intensity=-1:glum=1 -y int_-1_glum_1.png
ffmpeg -i [Link] -lavfi vibrance=intensity=-1:alternate=1:glum=1 -y int_-1_alt_glum_1.png

pause

58
These are the input and output images:

Input Input Input

intensity=1 intensity=-1 intensity=-1:alternate=1

intensity=1:gbal=-3 intensity=-1:gbal=-3 intensity=-1:alternate=1:gbal=-3

intensity=1:gbal=3 intensity=-1:gbal=3 intensity=-1:alternate=1:gbal=3

59
 Input Input Input

intensity=1 intensity=-1 intensity=-1:alternate=1

intensity=1:glum=0 intensity=-1:glum=0 intensity=-1:alternate=1:glum=0

intensity=1:glum=1 intensity=-1:glum=1 intensity=-1:alternate=1:glum=1

Note: The default value of the "glum" value seems to be about 0.7.

60
2.26 Swapuv filter

This filter swaps the U and V planes. This is an example. The input image is a spectrum with white at the top and black at the bottom:
ffmpeg -f lavfi -i nullsrc=s=1536x512 -vf geq=r='st(0,clip(512-X,0,255)+clip(X-
1024,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)':g='st(0,lt(X,512)*clip(X,0,255)+gte(X,512)*clip(1024-
X,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)':b='st(0,lt(X,1024)*clip(X-
512,0,255)+gte(X,1024)*clip(1536-X,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)',scale=iw/4:ih/4 -frames
1 -y [Link]

ffmpeg -i [Link] -lavfi swapuv -y [Link]

pause

These are the input and output images:

Input Output

61
2.27 Gradation curves

Note: This is obsolete. It's better to do this with a color-look-up-table.

-- An image is opened in GIMP.

-- Colors::Values --> Select suitable points for black, white and gray in the image.
-- Click on "Edit these settings as curves".
-- make fine corrections on the curves
-- Set as many points as possible on the curves, because later they will be interpolated by straight lines.
-- Click on "Export settings as file".
-- Check the box "Use old file format for curves".
-- filename: [Link]
-- Save
-- Then call the GIMP2ACV converter (1). This converter reads the file [Link], converts it and saves it as [Link]. The file [Link] must be
located in the same folder where the converter is called.
-- In the batch file for the FFmpeg editing the corresponding video filter is called: -vf curves=psfile='[Link]'

Example:
ffmpeg -i [Link] -vf curves=psfile='[Link]' -y [Link]

pause

(1) [Link]

62
ffmpeg -f lavfi -i color=black:s=32x8,format=rgb24 -lavfi
geq=r='X+32*Y':g='X+32*Y':b='X+32*Y',format=rgb24,datascope=s=960x288:mode=color2:format=dec,scale='2*iw':'2*ih' -frames
1 -y test_gray.png

ffmpeg -i test_gray.png -vf "curves=m='0.0/0.0 0.0275/0.596 0.255/0.918 1.0/1.0':interp=pchip" [Link]

pause

Warning: The "curves" filter should always be used with the "interp=pchip" option. The default (interp=natural) may have large under- or overshoots,
which are clipped to black or white.

63
2.28 Color grading with color look-up tables, full workflow

A color look-up table (CLUT) is a mathematical rule according to which any color is replaced by another color.
There are different file formats for the CLUT:
The *.cube format normally has a color space of typically 25 * 25 * 25 or 33 * 33 * 33 entries, so that the table contains 25^3 = 15625 or 33^3 = 35937
different colors. Colors between the specified table entries are interpolated. You can also create tables with 64^3 entries, but for most applications 25^3
or 33^3 entries are sufficient.

It's also possible to save a CLUT in any uncompressed image format. The complete workflow is now described step by step for a 10-bit video. This
workflow can be simplified, see the next chapter.

Step 1: With this batch file a single image is extracted from the 10-bit video at a suitable location and saved lossless as 16-bit PNG:
set "IN=Video_62.mov" :: Input video
set "T=35" :: Time where image is extracted

ffmpeg -ss %T% -i %IN% -frames 1 -y [Link]

pause

Step 2: This batch file is used to create a CLUT ( = Color-look-up-Table). This is a PNG image with 512x512 pixels that contains exactly one pixel of each
possible color. I'm not yet sure if the image has to have 16 bit resolution at this point. At least it doesn't hurt. If 8 bits are enough, you would omit "-
pix_fmt rgb48be".
The LEVEL parameter determines how many different colors are contained in the CLUT. The height and width of the square image is
LEVEL*LEVEL*LEVEL, at LEVEL=8 there are 64*64*64=262144 colors and the image has 512*512=262144 pixels. It is important that the file is saved in an
uncompressed or lossless compressed format, so PNG is well suited.

64
set "LEVEL=8"

ffmpeg -f lavfi -i haldclutsrc=%LEVEL% -frames 1 -pix_fmt rgb48be [Link]

pause

Step 3: The extracted image is opened in GIMP.

Step 4: The color table will be opened in GIMP, selected with "Select all" and copied with ctrl-c.
Step 5: The first image is clicked and then the color table is inserted in the upper left corner with "Paste in Place". Since the first image is much greater
than the color table, the table does not interfere at this position.
Step 6: Right click on "Floating Selection" and select "To New Layer".
Step 7: Right click on the newly created "Pasted Layer" and select "Merge Down".
Step 8: Now the image is edited as it should look in the video. And of course the color table in the upper left corner will be edited as well. Color
corrections, color temperature, color saturation, gradation curve, brightness, contrast. The image may contain visible noise. Later in the video, the noise
doesn't stand out so much, because it is partly averted by the fast sequence of images. Operations that cannot be described by a color look-up table,
such as noise reduction, soft focus or sharpening, are not permitted.
Step 9: The finished image is trimmed to a size of 512x512 pixels so that only the color table in the upper left corner remains. Image > Canvas Size >
Width=512, Height=512, then click on "Resize".
Step 10: Export the image under the name [Link] as 16-bit PNG and select "16bpc RGB" as pixel format. GIMP can now be closed.
Step 11: This color look-up table is now applied to the whole video with FFmpeg. The color table is applied with 10 bit accuracy. Colors not included in
the table are interpolated. Only then is the color table converted to 8 bit accuracy and an MP4 generated:
set "IN=Video_62.mov" :: Input video

ffmpeg -i %IN% -i [Link] -filter_complex [0][1]haldclut out.mp4

pause

65
2.29 Color grading with color look-up tables, simplified workflow

The above workflow can be simplified as follows:

Step 1: In this batch file FFmpeg does immediately combine the CLUT with the extracted image:
set "IN=[Link]" :: Input video
set "T=5" :: Time where image is extracted

ffmpeg -ss %T% -i %IN% -f lavfi -i haldclutsrc=8 -filter_complex "[1]format=pix_fmts=rgb48be[a];[a]

[0]xstack=inputs=2:layout=0_0|w0_0" -frames 1 -y Image_with_CLUT.png

pause

Step 2: This image is now processed in GIMP (or any other suitable image processing software) and then exported with the same file name as 16-bit
PNG. You can edit brightness, contrast, gamma, saturation and hue. You can also adjust the curves. Of course, all modifications must be applied to the
whole image consisting of the video frame and the clut. Filters like noise reduction, sharpening or softening are not allowed.

Step 3: This batch file does first use the crop filter to remove the image so that only the CLUT remains. Why the small brightness correction is necessary
before applying the haldclut filter isn't yet fully understood. In the second FFmpeg run the CLUT is applied to the input video. If the CLUT isn't required
for correcting other videos, it can be deleted.
set "IN=[Link]" :: Input video
set "BR=0.06" :: Small brightness adjustment before applying the CLUT

ffmpeg -i Image_with_CLUT.png -vf crop=[Link] -y [Link]

ffmpeg -i %IN% -i [Link] -filter_complex [0]eq=brightness=%BR%[a];[a][1]haldclut -y out.mp4

del [Link]

pause

Note: The "haldclut" filter has an option "clut", which sets which frames are processed from second input stream, can be first or all. Default is all. If the
second stream is an image, then this option doesn't care.

66
2.30 Size of Haldclut tables

The size of the 3D Color-look-up table for the haldclut filter depends on the "Level" parameter as follows:
Level Size of CLUT Typical file Number of points Number of support Distance between Distance between Distance between
size of along the edge of points support points for 8-bit support points for 10-bit support points for 16-bit
n n^3 x n^3 CLUT as the RGB cube ( = number of pixels ) 255 / (n^2 - 1) 1023 / (n^2 - 1) 65535 / (n^2 - 1)
16-bit PNG n^2 n^6
2 8x8 Pixels 0.17 kB 4 64 85 341 21845
3 27x27 Pixels 1.2 kB 9 729 31.87 127.87 8191.87
4 64x64 Pixels 16.4 kB 16 4096 17 68.2 4369
5 125x125 Pixels 61.9 kB 25 15625 10.62 42.62 2730.62
6 216x216 Pixels 179 kB 36 46656 7.29 29.23 1872.43
7 343x343 Pixels 436 kB 49 117649 5.31 21.31 1365.31
8 512x512 Pixels 0.97 MB 64 262144 4.05 16.24 1040.24
9 729x729 Pixels 81 531441 3.19 12.79 819.19
10 1000x1000 Pixels 100 1000000 2.58 10.33 661.97
11 1331x1331 Pixels 121 1771561 2.12 8.52 546.12
12 1728x1728 Pixels 144 2985984 1.78 7.15 458.29
13 2197x2197 Pixels 169 4826809 1.52 6.09 390.09
14 2744x2744 Pixels 196 7529536 1.31 5.25 336.08
15 3375x3375 Pixels 225 11390625 1.14 4.57 292.57
16 4096x4096 Pixels 256 16777216 1 4.01 257

67
2.31 1D- and 3D *.cube LUT's

See also: [Link]

What's the difference between 1D- and 3D-LUTs?

red green blue

1D-LUT ROUT := f(RIN) GOUT := f(GIN) BOUT := f(BIN)

3D-LUT ROUT := f(RIN, GIN, BIN) GOUT := f(RIN, GIN, BIN) BOUT := f(RIN, GIN, BIN)

There exist two different syntax versions for *.cube files.

1. This version is supported by DaVinci Resolve and (partly) by FFmpeg. However there seems to exist no official specification for this syntax.
This *.cube file may contain the lines:
LUT_1D_INPUT_RANGE min max
LUT_3D_INPUT_RANGE min max
1D- and 3D-LUTs can be specified in the same *.cube file. If both LUTs are specified, the 1D-LUT is applied first to the input data, and the output
of this operation is then used as input for the 3D-LUT. I'm not sure if this feature is implemented in FFmpeg.
There is a bug in FFmpeg, which restricts the inverse of (max-min) to the [0...1] range. With other words: It works only if (max-min) > 1.
See this ticket: [Link]
2. This version is not supported by DaVinci Resolve and not yet supported by FFmpeg, but has an official documentation from Adobe:
[Link]
[Link]
It's not mentioned in the specification if it's possible to have 1D- and 3D-LUTs in the same file.
This *.cube file may contain the lines:
DOMAIN_MIN rl gl bl
DOMAIN_MAX rh gh bh

68
Example for 1D-LUT:

ffmpeg -f lavfi -i colorspectrum=type=all:s=1536x512 -frames 1 -y [Link]

ffmpeg -i [Link] -vf lut1d=1D_LUT.cube -frames 1 -y [Link]

pause

This is the content of the file 1D_LUT.cube (in this case it's a strong nonlinear curve):
LUT_1D_SIZE 5
LUT_1D_INPUT_RANGE 0.0 1.0
0.000 0.000 0.000
0.125 0.125 0.125
0.250 0.250 0.250
0.500 0.500 0.500
1.000 1.000 1.000
First column is red, second is green, third is blue. It seems the parameter LUT_1D_INPUT_RANGE is not used at all.

Example for a simple *.cube LUT with LUT_1D_INPUT_RANGE (this does work in DaVinci Resolve, but not in FFmpeg):
LUT_1D_SIZE 2
LUT_1D_INPUT_RANGE 0.0 1.0
0.000 0.000 0.000
1.000 1.000 1.000

Example for a simple *.cube LUT with DOMAIN_MIN and DOMAIN_MAX (this doesn't work in DaVinci Resolve and FFmpeg):
LUT_1D_SIZE 2
DOMAIN_MIN 0.1 0.1 0.1
DOMAIN_MAX 0.3 0.3 0.3
0 0 0
1 1 1

69
The size of 3D *.cube files is as follows:
LUT_3D_SIZE Number of points Typical file size Distance between support Distance between support
n n^3 points for 8-bit points for 16-bit
255 / (n-1) 65535 / (n-1)
2 8 255 65535
3 27 127.5 32767.5
5 125 63.75 16383.75
9 729 31.875 8191.875
17 4913 15.937 4095.937
25 15625 10.625 2730.625
33 35937 1 MB 7.969 2047.969
64 262144 7 MB 4.047 1040.238
65 274625 7 MB 3.984 1023.984

*.cube LUT files can be loaded into the FOTGA DP500/A70TLS monitor as follows:
Save the *.cube file in the root folder of a FAT32 USB stick. Insert the USB stick into the monitor. Press on the menu wheel, select "LUT Settings" and
"LUT Import". All LUTs from the USB stick will be imported (up to 8 of them), and all already existing LUTs will be deleted, except those that are in the
firmware (VLOG is one of them). Each single LUT file must be less than 7.9 MB.

It's also possible to apply LUTs to the HDMI output of the GH5S camera. Use "HDMI Rec Output" -> "apply LUT". The LUT must be saved on the SD card.

70
This is the identity *.cube 3D-LUT for 8 colors:
LUT_3D_SIZE 2
# black
0 0 0
# red
1 0 0
# green
0 1 0
# yellow
1 1 0
# blue
0 0 1
# magenta
1 0 1
# cyan
0 1 1
# white
1 1 1

This is the identity *.cube 3D-LUT for 27 colors:

LUT_3D_SIZE 3
# black
0.0 0.0 0.0
# dark red
0.5 0.0 0.0
# red
1.0 0.0 0.0
# dark green
0.0 0.5 0.0
# dark yellow
0.5 0.5 0.0
# red-green = orange
1.0 0.5 0.0
# green
0.0 1.0 0.0
# yellow-green
0.5 1.0 0.0
# yellow
1.0 1.0 0.0
# dark blue
0.0 0.0 0.5
# dark magenta
0.5 0.0 0.5
# red-magenta
1.0 0.0 0.5

71
# dark cyan
0.0 0.5 0.5
# gray
0.5 0.5 0.5
# red-white
1.0 0.5 0.5
# green-cyan
0.0 1.0 0.5
# green-white
0.5 1.0 0.5
# yellow-white
1.0 1.0 0.5
# blue
0.0 0.0 1.0
# blue-magenta = violet
0.5 0.0 1.0
# magenta
1.0 0.0 1.0
# cyan-blue
0.0 0.5 1.0
# blue-white
0.5 0.5 1.0
# magenta-white
1.0 0.5 1.0
# cyan
0.0 1.0 1.0
# cyan-white
0.5 1.0 1.0
# white
1.0 1.0 1.0

My C# source code for creating *.cube 3D-LUT files can be downloaded here: [Link]
(In this case it's a simple gamma function)

72
2.32 Convert a *.cube LUT to a Haldclut file

In this example it's shown how a haldclut file can be generated from a *.cube LUT. Applying either of these two LUTs to an image gives almost the same
results [Link] and [Link]. In the last command line the difference between the two results is calculated. If you try different values for the haldclut
level option, you see that in most cases level 5 or 6 is sufficient. This depends also on the size of the *.cube file.
rem Apply a *.cube lut to the image:

ffmpeg -i [Link] -vf lut3d="VLog_to_V709.cube" -y [Link]

rem Convert the *.cube lut to a haldclut image:

ffmpeg -f lavfi -i haldclutsrc=8 -vf lut3d="VLog_to_V709.cube" -frames 1 -y [Link]

rem Apply the haldclut file to the image:

ffmpeg -i [Link] -i [Link] -lavfi haldclut -y [Link]

rem Show the difference between the two results:

ffmpeg -i [Link] -i [Link] -lavfi blend=all_mode=grainextract -y [Link]

pause

Note: The "lut3d" filter accepts LUTs in the following file formats:
• *.3dl (AfterEffects)
• *.cube (Iridas)
• *.dat (DaVinci Resolve)
• *.m3d (Pandora)
• *.csp (cineSpace)

73
2.33 Colorchart source

The "colorchart" video source generates an image with 24 colors, similar to the X-Rite ColorChecker.
ffmpeg -f lavfi -i colorchart=preset=reference -frames 1 -y [Link]

pause

These are the colors and RGB values in the output image.
First row: RGB values from the "colorchart=preset=reference" source, which are identical to the values listed for the X-Rite ColorChecker.
Second row: RGB values printed on the back side of the cheap chinese ColorChecker, which has slightly different colors (Source:
[Link] ).
Third row: RGB values of the cheap chinese ColorChecker, as measured by comparing with a new "Calibrite ColorChecker Classic Mini".
dark skin light skin blue sky foliage blue flower bluish green
115, 82, 68 194, 150, 130 98, 122, 157 87, 108, 67 133, 128, 177 103, 189, 170
115, 82, 69 204, 161, 141 101, 134, 179 89, 109, 61 141, 137, 194 132, 228, 208
120, 95, 82 202, 159, 136 102, 130, 163 102, 117, 85 159, 168, 190 127, 206, 186
orange purple red moderate red purple yellow green orange yellow
214, 126, 44 80, 91, 166 193, 90, 99 94, 60, 108 157, 188, 64 224, 163, 46
249, 118, 35 80, 91, 182 222, 91, 125 91, 63, 123 173, 232, 91 255, 164, 26
247, 146, 83 86, 97, 174 232, 119, 131 108, 76, 130 181, 202, 106 252, 179, 60
blue green red yellow magenta cyan
56, 61, 150 70, 148, 73 175, 54, 60 231, 199, 31 187, 86, 149 8, 133, 161
44, 56, 142 74, 148, 81 179, 42, 50 250, 226, 21 191, 81, 160 6, 142, 172
67, 70, 94 69, 146, 81 187, 78, 84 244, 213, 35 194, 93, 148 7, 138, 168
white neutral 80 neutral 65 neutral 50 neutral 35 black
243, 243, 242 200, 200, 200 160, 160, 160 122, 122, 121 85, 85, 85 52, 52, 52
252, 252, 252 230, 230, 230 200, 200, 200 143, 143, 142 100, 100, 100 50, 50, 50
237, 241, 243 225, 229, 232 169, 171, 175 130, 132, 136 97, 98, 102 63, 64, 64

See also: [Link]

See also: [Link]

74
This batch file generates a test image with the RGB values from the back side of the cheap chinese ColorChecker:

rem Make a test image with RGB values for chinese colorchecker:

ffmpeg ^
-f lavfi -i color=0x735245:s=64x64 -f lavfi -i color=0xcca18d:s=64x64 -f lavfi -i color=0x6586b3:s=64x64 ^
-f lavfi -i color=0x596d3d:s=64x64 -f lavfi -i color=0x8d89c2:s=64x64 -f lavfi -i color=0x84e4d0:s=64x64 ^
-f lavfi -i color=0xf97623:s=64x64 -f lavfi -i color=0x505bb6:s=64x64 -f lavfi -i color=0xde5b7d:s=64x64 ^
-f lavfi -i color=0x5b3f7b:s=64x64 -f lavfi -i color=0xade85b:s=64x64 -f lavfi -i color=0xffa41a:s=64x64 ^
-f lavfi -i color=0x2c388e:s=64x64 -f lavfi -i color=0x4a9451:s=64x64 -f lavfi -i color=0xb32a32:s=64x64 ^
-f lavfi -i color=0xfae215:s=64x64 -f lavfi -i color=0xbf51a0:s=64x64 -f lavfi -i color=0x068eac:s=64x64 ^
-f lavfi -i color=0xfcfcfc:s=64x64 -f lavfi -i color=0xe6e6e6:s=64x64 -f lavfi -i color=0xc8c8c8:s=64x64 ^
-f lavfi -i color=0x8f8f8e:s=64x64 -f lavfi -i color=0x646464:s=64x64 -f lavfi -i color=0x323232:s=64x64 ^
-lavfi [0][1][2][3][4][5]hstack=6[a];[6][7][8][9][10][11]hstack=6[b];^
[12][13][14][15][16][17]hstack=6[c];[18][19][20][21][22][23]hstack=6[d];[a][b][c][d]vstack=4 -frames 1 -y [Link]

pause

This batch file generates a test image with the RGB values for the cheap chinese ColorChecker, as per my own calibration:

rem Make a test image with RGB values for chinese colorchecker:

ffmpeg ^
-f lavfi -i color=0x785f52:s=64x64 -f lavfi -i color=0xca9f88:s=64x64 -f lavfi -i color=0x6682a3:s=64x64 ^
-f lavfi -i color=0x667555:s=64x64 -f lavfi -i color=0x9fa8be:s=64x64 -f lavfi -i color=0x7fceba:s=64x64 ^
-f lavfi -i color=0xf79253:s=64x64 -f lavfi -i color=0x5661ae:s=64x64 -f lavfi -i color=0xe87783:s=64x64 ^
-f lavfi -i color=0x6c4c82:s=64x64 -f lavfi -i color=0xb5ca6a:s=64x64 -f lavfi -i color=0xfcb33c:s=64x64 ^
-f lavfi -i color=0x43465e:s=64x64 -f lavfi -i color=0x459251:s=64x64 -f lavfi -i color=0xbb4e54:s=64x64 ^
-f lavfi -i color=0xf4d523:s=64x64 -f lavfi -i color=0xc25d94:s=64x64 -f lavfi -i color=0x078aa8:s=64x64 ^
-f lavfi -i color=0xedf1f3:s=64x64 -f lavfi -i color=0xe1e5e8:s=64x64 -f lavfi -i color=0xa9abaf:s=64x64 ^
-f lavfi -i color=0x828488:s=64x64 -f lavfi -i color=0x616266:s=64x64 -f lavfi -i color=0x3f4040:s=64x64 ^
-lavfi [0][1][2][3][4][5]hstack=6[a];[6][7][8][9][10][11]hstack=6[b];^
[12][13][14][15][16][17]hstack=6[c];[18][19][20][21][22][23]hstack=6[d];[a][b][c][d]vstack=4 -frames 1 -y [Link]

pause

75
2.34 Colormap filter

The colormap filter can be used for correcting the colors, so that they fit to the colors of the ColorChecker:
rem Make the reference image:

ffmpeg -f lavfi -i colorchart=preset=reference -frames 1 -y [Link]

rem Now edit the reference image in image processing software. Change brightness,
rem contrast, gamma, hue and saturation.
rem Draw some additional lines in the image, but keep the centers of the patches unaffected.
rem Save the image as "[Link]"

rem Now try to reconstruct the original colors from the modified image:

ffmpeg -i [Link] -i [Link] -i [Link] -lavfi colormap=nb_patches=24 -frames 1 -y [Link]

rem Show the difference between the two images. It should be a uniform gray surface,
rem if all colors have been reconstructed successfully. The additional lines must be visible.

ffmpeg -i [Link] -i [Link] -lavfi blend=all_mode=grainextract -y [Link]

pause

Note: It's possible that "colormap" fails without error message, for example if two colors in the "source" image are identical, while the corresponding
colors in the target image are different. It's impossible to map one source color to two different target colors. In this case the output of the "colormap"
filter is a copy of the first input.

Note: The second (source) and third (target) input of the colormap filter must have the same size. The first input can have any size.

Note: The "colormap" filter picks only the central pixels from the patches. There is no averaging done in this filter. That means some preprocessing is
required for images of real-world colorcheckers, especially geometric transformation (for example with "perspective" filter) and averaging with a blurring
filter. See next example.

Note: The default value of the "patch_size" option is 64x64, which is also the default patch size in the colorchart source. So it's not necessary to specify

76
them.

Note: The option "nb_patches" is the number of patches that are used. By default it equals the number of available patches. If "nb_patches" is smaller
than the number of available patches, then the patches are used line-wise from left to right, beginning in the top left corner.

This is a real-world example for color mapping. Geometric transformation is applied to the image of the ColorChecker, and then the image is scaled to the
same size as the reference image. It is also blurred to improve color accuracy. Finally the colors of the input image are mapped to the correct output
colors.
rem Take a picture of the ColorChecker and measure the corner coordinates

set "X0=1656" :: Top left corner (dark skin)

set "Y0=691"
set "X1=4116" :: Top right corner (bluish green)
set "Y1=1226"
set "X2=1269" :: Bottom left corner (white)
set "Y2=2316"
set "X3=3796" :: Bottom right corner (black)
set "Y3=2870"

rem Apply geometric transformation, scale to 384x256 and use avgblur for averaging:

ffmpeg -i [Link] -lavfi format=argb,perspective=x0=%X0%:y0=%Y0%:x1=%X1%:y1=%Y1%:x2=%X2%:y2=%Y2%:x3=%X3%:y3=

%Y3%,scale=384x256,avgblur=10 -y source_colors.png

rem Make the reference image (this is the perfect ColorChecker image):

ffmpeg -f lavfi -i colorchart=preset=reference -frames 1 -y [Link]

rem Now reconstruct the true colors of the input image:

ffmpeg -i [Link] -i source_colors.png -i [Link] -lavfi colormap=nb_patches=24 -frames 1 -y [Link]

rem Calculate the difference between output and input:

ffmpeg -i [Link] -i [Link] -lavfi blend=all_mode=grainextract -y [Link]

pause

77
This is the input image with wrong colors because of fluorescent lamp. This is the extracted "source_colors.png" image.

This is the corrected output image. This is the "[Link]" image which contains the target colors.

Note: It's possible to speed up filtering by using the "haldclutsrc" source and "haldclut" filter. First apply the "colormap" filter to a "haldclutsrc" image,
and then filter the image or video with the "haldclut" filter.

78
2.35 Colorhold, chromahold and hsvhold filters

This video filter removes all color informations except for one certain color. It has three parameters:
"color" is the color to be preserved, can be specified by name or by RGB values, for example "orange" can be replaced by FFA500 (or 0xFFA500 or
#FFA500)
"similarity" is a fraction, 0.01 means only the specified color is preserved, 1.0 means all colors are preserved.
Note: The normalization of the "similarity" value was changed in May 2020. Old values must now be divided by sqrt(3) to get the same result as before.
"blend" is a fraction, 0.0 makes pixels fully gray, higher values result in more preserved color.
This example preserves only colors from yellow to orange to light brown:
ffmpeg -i [Link] -filter_complex split[1][2];[1]colorhold=color="orange":similarity=0.29:blend=0[3];[2][3]hstack
-y [Link]

pause

Output of this example:

79
There is also a "chromahold" filter which is similar to the "colorhold" filter but works in YUV range.
There is also a "hsvhold" filter, which is similar to the "colorhold" filter but works in the HSV (Hue-Saturation-Value) range:
ffmpeg -i [Link] -filter_complex split[1][2];[1]hsvhold=hue=45:sat=0.7:val=0.5:similarity=0.30:blend=0[3];[2]
[3]hstack -y [Link]

pause

Output of this example:

The problem with these filters is that it's difficult to adjust the parameters properly, without seeing the result in real time.
Hint: Use the "FastStone Image Viewer" for showing the result image. This viewer does automatically detect when the image is overwritten by a new
image, and shows the new image automatically.

80
2.36 Atmospheric dispersion correction

It's possible to shift the RGB channels with respect to each other:
set "IN=[Link]" :: Input video
set "OUT=out.mp4" :: Output video
set "RV=5" :: Vertical shift of red channel
set "BV=-5" :: Vertical shift of blue channel

ffmpeg -i %IN% -lavfi "rgbashift=rv=%RV%:bv=%BV%" -y %OUT%

pause

This example shows a white square before and after applying the correction:
set "RV=5" :: Vertical shift of red channel
set "BV=-5" :: Vertical shift of blue channel

ffmpeg -f lavfi -i color=black:s=100x100 -lavfi drawbox=color=white:x=40:y=40:w=20:h=20:t=fill,split[a][b];

[b]rgbashift=rv=%RV%:bv=%BV%[c];[a][c]hstack -frames 1 -y [Link]

pause

This is the output image:

The chromashift filter seems to be similar, but I haven't yet tested it.

81
2.37 Amplify filter

The "amplify" filter amplifies differences between adjacent frames. Good for motion detection, but it's also sensitive to noise.

2.38 Sharpen or blur images

Images or videos can be blurred or sharpened with the "unsharp" filter:

ffmpeg -i [Link] -vf unsharp=la=2 -y [Link]

pause

These are the parameters of the "unsharp" filter:

Parameter Default value Description

lx 5 Luma matrix horizontal size, it must be an odd integer between 3 and 23.
ly 5 Luma matrix vertical size, it must be an odd integer between 3 and 23.
la 1 Luma effect strength, it must be a floating point number, reasonable values are between -1.5 and 1.5.
Negative values will blur the input video, while positive values will sharpen it, a value of zero will disable the effect.
cx 5 Chroma matrix horizontal size, it must be an odd integer between 3 and 23.
cy 5 Chroma matrix vertical size, it must be an odd integer between 3 and 23.
ca 0 Chroma effect strength, it must be a floating point number, reasonable values are between -1.5 and 1.5.
Negative values will blur the input video, while positive values will sharpen it, a value of zero will disable the effect.

82
For blurring you can also use these filters:
• "dblur" for directional blur (any directions are possible)
• "gblur" for gaussian blur (circular, elliptical, horizontal or vertical)
• "avgblur" for average blur (horizontal and vertical box size can be set independently, but not smaller than 3x3)
• "convolution" filter (size from 3x3 to 7x7)
• "fftfilt" filter
• "removegrain" filter with mode=20 (only 3x3 averaging)
• "sab" filter (Shape Adaptive Blur)
• "smartblur" filter (This filter doesn't impact the outlines)
• "yaepblur" filter ("yet another edge preserving blur filter")

Brightness distributiuon (resulting from a single white pixel in the center):

Pixels from center: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

dblur=angle=0:radius=5 25 20 17 13 11 9 8 5 4 4 3 2 2 2 1 1 1 1 1 0
gblur=sigma=5:sigmaV=0 36 26 19 15 11 8 6 4 3 2 2 1 1 1 1 0 0 0 0 0
avgblur=sizeX=5:sizeY=1 6 6 6 6 6 6 0 0 0 0 0 0 0 0 0 0 0 0 0 0

Command line for testing with a single white pixel:

ffmpeg -f lavfi -i color=black:s=50x50 -vf drawbox=w=1:h=1:x=25:y=25:color=white,avgblur=sizeX=5:sizeY=1 -frames 1 -y
[Link]

pause

83
For variable blur you can use the "varblur" filter:
rem Create a circular mask:

ffmpeg -f lavfi -i nullsrc=size=vga -lavfi format=gray8,geq='25*lt(hypot(X-W/2,Y-H/2),200)' -frames 1 -y [Link]

rem Apply variable blurring:

ffmpeg -f lavfi -i testsrc2=size=vga -i [Link] -lavfi [0]format=gbrp[a];[a][1]varblur=max_r=25 -t 10 -y out.mp4

pause

84
2.39 FFT Filtering

First let's make a test image which contains different wavelengths, both horizontal and vertical:
ffmpeg -f lavfi -i color=black:s=300x50 -lavfi drawgrid=c=white:y=-1:w=2:h=51,split[a][b];^
[b]crop=iw/2:x=0,scale=2*iw:ih:flags=neighbor,split[b][c];^
[c]crop=iw/2:x=0,scale=2*iw:ih:flags=neighbor,split[c][d];^
[d]crop=iw/2:x=0,scale=2*iw:ih:flags=neighbor,split[d][e];^
[e]crop=iw/2:x=0,scale=2*iw:ih:flags=neighbor,split[e][f];^
[f]crop=iw/2:x=0,scale=2*iw:ih:flags=neighbor[f];^
[a][b][c][d][e][f]vstack=6,split[h][v];[v]transpose[v];[v][h]hstack -frames 1 -y [Link]

pause

This is the test image. The wavelengths are 2, 4, 8, 16, 32 and 64 pixels per linepair:

85
After some experimenting I found out that it's better to use sine waves:
ffmpeg -f lavfi -i color=black:s=300x300 -lavfi geq='r=127.5+127.5*cos(X*PI/
(pow(2,trunc(Y/50))))',colorchannelmixer=[Link],split[h][v];[v]transpose[v];[v][h]hstack -frames 1 -y
[Link]

This is the test image. The wavelengths are 2, 4, 8, 16, 32 and 64 pixels per linepair:

86
Now let's test the lowpass example from the official documentation:
ffmpeg -f lavfi -i color=black:s=300x300 -lavfi geq='r=127.5+127.5*cos(X*PI/
(pow(2,trunc(Y/50))))',colorchannelmixer=[Link],split[h][v];[v]transpose[v];[v][h]hstack -frames 1 -y
[Link]

ffmpeg -i [Link] -vf fftfilt=dc_Y=0:weight_Y='squish((Y+X)/100-1)' -y [Link]

pause

This is the output image:

It's clearly visible that the horizontal filtering frequency is different from the vertical filtering frequency. That's because the image has a 2:1 aspect ratio,
but in the expression the X and Y values are simply added with the same weight.
It's also visible that the highest frequency isn't fully filtered out. This problem can be solved by scaling the image to double size before filtering and
scaling down to the original size after filtering.
If the weight_U and weight_V expressions aren't set, they are by default copied from weight_Y. That's why the output image has a greenish tint in this
example. To solve this problem, you should always set weight_U and weight_V to 1.

87
This is an infinitely sharp lowpass filter which has the same cutoff frequencies in both directions:
ffmpeg -f lavfi -i color=black:s=300x300 -lavfi geq='r=127.5+127.5*cos(X*PI/
(pow(2,trunc(Y/50))))',colorchannelmixer=[Link],split[h][v];[v]transpose[v];[v][h]hstack -frames 1 -y
[Link]

ffmpeg -i [Link] -vf

scale=2*iw:2*ih,fftfilt=dc_Y=0:dc_U=0:dc_V=0:weight_Y='lt(hypot(Y/H,X/W),0.333)':weight_U=1:weight_V=1,scale=iw/2:ih/2
-y [Link]

pause

Note: In the above example, the "colorchannelmixer" copies the red channel to the green and blue channels.

This is the output image:

Note: The "fftfilt" filter works only in YUV pixel format.

88
It's better to use sine intensity profiles for testing, because square waves contain many harmonics. This command creates a test image where the
wavelength changes continuously from 4 to 8 (in the center) to 16 pixels per linepair. The size can be changed, but width and height of the "color" source
must be equal, because otherwise the hstack at the end filter would fail:
ffmpeg -f lavfi -i color=black:s=256x256 -lavfi geq='r=127.5+127.5*cos((X-W/2)*PI/(pow(2,
(1+2*Y/H))))',colorchannelmixer=[Link],split[h][v];[v]transpose[v];[v][h]hstack -frames 1 -y [Link]

pause

This is the test image:

89
Note: The internal FFT array size changes between image size=230 and image size=232, as can be shown by the following example. This is probably
because the FFT size is always a power of 2 and it's chosen at least about 10% bigger than the input size because a window function is used.
See also vf_fftfilt.c line 285: for (rdft_hbits = 1; 1 << rdft_hbits < w*10/9; rdft_hbits++);
set "P=8"

ffmpeg -f lavfi -i color=black:s=230x230 -lavfi geq='r=127.5+127.5*cos((X-W/2)*PI/(pow(2,

(1+2*Y/H))))',colorchannelmixer=[Link],split[h][v];[v]transpose[v];[v][h]hstack -frames 1 -y [Link]

ffmpeg -i [Link] -vf scale=2*iw:2*ih,fftfilt=dc_Y=128:dc_U=1:dc_V=1:weight_Y='between(hypot(Y/H,X/W),1.9/%P%,2.1/%P

%)':weight_U=1:weight_V=1,scale=iw/2:ih/2 -y [Link]

ffmpeg -f lavfi -i color=black:s=232x232 -lavfi geq='r=127.5+127.5*cos((X-W/2)*PI/(pow(2,

(1+2*Y/H))))',colorchannelmixer=[Link],split[h][v];[v]transpose[v];[v][h]hstack -frames 1 -y [Link]

ffmpeg -i [Link] -vf scale=2*iw:2*ih,fftfilt=dc_Y=128:dc_U=1:dc_V=1:weight_Y='between(hypot(Y/H,X/W),1.9/%P%,2.1/%P

%)':weight_U=1:weight_V=1,scale=iw/2:ih/2 -y [Link]

pause

The size of the FFT arrays in the "fftfilt" filter is accessible vis the "WS" and "HS" variables, or could be calculated as follows:
set "ARRAY_H=pow(2,ceil(log(ceil(W*10/9))/log(2)))" :: horizontal fft array size
set "ARRAY_V=pow(2,ceil(log(ceil(H*10/9))/log(2)))" :: vertical fft array size

90
Example for lowpass, highpass, bandpass and notch filters, where the filter wavelength is independent of the image size:
set "P=8" :: filter wavelength = pixels per linepair
set "SIZE=512" :: vertical size of test image

:: create test image, wavelength varies continuously from 4 to 8 (in the center) to 16 pixels per linepair:

ffmpeg -f lavfi -i color=black:s=%SIZE%x%SIZE% -lavfi geq='r=127.5+127.5*cos((X-W/2)*PI/(pow(2,

(1+2*Y/H))))',colorchannelmixer=[Link],split[h][v];[v]transpose[v];[v][h]hstack -frames 1 -y [Link]

:: lowpass, highpass, bandpass and notch filtering:

ffmpeg -i [Link] -vf scale=2*iw:2*ih,fftfilt=dc_Y=0:dc_U=0:dc_V=0:weight_Y='lte(hypot(X/WS,Y/HS),1.0/%P

%)':weight_U=1:weight_V=1,scale=iw/2:ih/2 -y [Link]

ffmpeg -i [Link] -vf scale=2*iw:2*ih,fftfilt=dc_Y=128:dc_U=1:dc_V=1:weight_Y='gte(hypot(X/WS,Y/HS),1.0/%P

%)':weight_U=1:weight_V=1,scale=iw/2:ih/2 -y [Link]

ffmpeg -i [Link] -vf scale=2*iw:2*ih,fftfilt=dc_Y=128:dc_U=1:dc_V=1:weight_Y='between(hypot(X/WS,Y/HS),0.8/%P%,1.2/%P

%)':weight_U=1:weight_V=1,scale=iw/2:ih/2 -y [Link]

ffmpeg -i [Link] -vf scale=2*iw:2*ih,fftfilt=dc_Y=0:dc_U=0:dc_V=0:weight_Y='1-between(hypot(X/WS,Y/HS),0.8/%P%,1.2/%P

%)':weight_U=1:weight_V=1,scale=iw/2:ih/2 -y [Link]

pause

Why is the image scaled up before filtering and scaled down after filtering? If the image contains the highest possible frequency (2 pixels per linepair),
this frequency wouldn't be filtered out with a lowpass filter. I think that's because of the YUV subsampling. As a workaround the image is scaled up
before filtering and scaled down after filtering.

91
2.40 Extract a time segment from a video

When you have a fisheye camera pointing upwards, it's unavoidable that you are visible in the video at the beginning and the end, because you must
start and stop the camera. That means we must cut off the beginning and the end.
rem Extract a time segment from a video

set "INPUT=PanoView.mp4" :: Input video

set "OUTPUT=out.mp4" :: Output video
set "START=2.0" :: Start time in seconds
set "LENGTH=3.0" :: Length of the segment in seconds

ffmpeg -ss %START% -t %LENGTH% -i %INPUT% -c copy %OUTPUT%

pause

The arguments for -ss and -t can also be specified in hours, minutes and seconds:
1:20 = 1 minute, 20 seconds
[Link] = 1 hour, 10 minutes, 30 seconds

Instead of the length it's also possible to specify the end time with the -to option.

If you want to save the output video with exactly the same quality as the input video (without re-encoding), then use the -c copy option. In this case it
makes no sense to specify the output video quality.
ffmpeg -ss 5 -i [Link] -t 10 -c copy [Link]

pause

The same thing can also be done with the "trim" filter.
For more informations about seeking, see also [Link]

92
Note: If -ss is written before the input file, the cut will be at the nearest keyframe but not at the accurate time. However if -ss is written after the input file,
the cut will be at the accurate time but there may be an empty part at the beginning of the file, until the next keyframe.

2.41 Remove a segment from a video

rem Make a 20s test video:

ffmpeg -f lavfi -i testsrc2 -t 20 -y test.mp4

rem Remove the segment from t>5 to t<10

ffmpeg -i test.mp4 -vf select='between(t,0,5)+between(t,10,20)',setpts=N/(FRAME_RATE*TB) -y out.mp4

rem This is (almost) the same, remove the segment from t=5 to t=10:
ffmpeg -i test.mp4 -vf select='not(between(t,5,10))',setpts=N/(FRAME_RATE*TB) -y out.mp4

pause

93
2.42 Trim filter

Drop everything except the second minute of input:

ffmpeg -i in.mp4 -vf trim=60:120 out.mp4

pause

Keep only the first second:

ffmpeg -i in.mp4 -vf trim=duration=1 out.mp4

pause

See also [Link]

94
2.43 Tpad filter, add a few seconds black at the beginning or end

Method 1, using the "tpad" filter:

set "IN=my_video.mp4" :: Input video
set "DUR=3" :: Duration in seconds
set "OUT=out.mp4" :: Output video

ffmpeg -i %IN% -vf tpad=start_duration=%DUR% %OUT%

pause

The "tpad" filter inserts frames at the beginning or at the end of a video. These frames contain either a uniform color or a copy of the first or last frame.
The default color is black.

Method 2, using the concat filter:

set "IN=my_video.mp4" :: Input video
set "DUR=3" :: Duration in seconds
set "OUT=out.mp4" :: Output video

ffmpeg -i %IN% -an -filter_complex 'color=black:duration=%DUR%[black];[black][0:0]concat=n=2:v=1:a=0[v]' -map [v] %OUT%

pause

95
2.44 Extract the last 30 seconds of a video

When I make real-time videos of meteors, I let the Panasonic LUMIX GH5S camera record continuously. When I see a meteor, I speak to the soundtrack in
which part of the sky I've seen it, and after about 10 seconds I press the REC button to stop the recording, and immediately start a new recording. That
means after downloading the videos to the computer, meteors are always at the end of the videos. There is no need to watch the videos in full length
(that would be boring). This batch file extracts the last 30 seconds of the video which is drag-and-dropped over it, and for the output filename the string
"P1" is replaced by "CUT" (e.g. [Link] becomes [Link]). It's lossless because the "-c copy" option is used.
set INPUT=%1
set OUTPUT=%INPUT:P1=CUT%

ffmpeg -sseof -30 -i %INPUT% -c copy %OUTPUT%

pause

This batch file (for Windows 7) does the same thing for all P1*.MOV files in the current folder:
for %%f in (P1*.MOV) do call :for_body %%f
goto :the_end

:for_body
set INPUT=%1
set OUTPUT=%INPUT:P1=CUT%
ffmpeg -sseof -30 -i %INPUT% -c copy -y %OUTPUT%
exit /b

:the_end

pause

96
2.45 Fade-in and fade-out

Fade-in and fade-out for a video of known length (only for video, not for audio). Here the times are expressed in frames:
ffmpeg -i input.mp4 -vf 'fade=in:0:30,fade=out:9650:30' output.mp4

pause

Fade-in and fade-out of a video of known length (both video and audio). Here the times are in seconds:
ffmpeg -i input.mp4 -vf 'fade=in:st=0:f=1,fade=out:st=32:d=1' -af 'afade=in:st=0:d=1,afade=out:st=32:d=1' output.mp4

pause

This is a workaround for fade in/out a video with unknown duration:

ffmpeg -i input.mp4 -sseof -1 -copyts -i input.mp4 -filter_complex
"[1]fade=out:0:30[t];[0][t]overlay,fade=in:0:30[v]; anullsrc,atrim=0:2[at];[0][at]acrossfade=d=1,afade=d=1[a]"
-map "[v]" -map "[a]" -c:v libx264 -crf 22 -preset veryfast -shortest output.mp4

pause

The trick is to feed the same input twice. From the second input only the last second is used. The timestamps are preserved. A fade-out is applied to the
short second input, and then both files are combined with overlay. For audio a 2 seconds dummy with silence is created, and then crossfaded with the
input audio. The -shortest option cuts the output to the same length as the input.

Another workaround for making fade-in and fade-out for audio of unknown length:
ffmpeg -i input.mp4 -filter_complex "afade=d=0.5, areverse, afade=d=0.5, areverse" output.mp4

pause

97
The same thing does also work for video, but keep in mind that you need a lot of memory for the reverse filter:
ffmpeg -i input.mp4 -filter_complex "fade=d=0.5, reverse, fade=d=0.5, reverse" output.mp4

pause

Another option is to use acrossfade with a silent track, but this works not for video because there is no crossfade filter for video:
ffmpeg -i input.mp4 -filter_complex "aevalsrc=0:d=0.6 [a_silence]; [Link] [a_silence] acrossfade=d=0.6" output.mp4

pause

Afade curves are shown on this wiki page: [Link]

2.46 Crossfading

The different types of xfade crossfadings are shown on this wiki page:
[Link]

Both inputs must be constant frame-rate and have the same resolution, pixel format, framerate and timebase.

98
2.47 Crop a video

Cropping means to cut off the borders, and in the next step you can also set the size (width * height) of the output video:

rem Crop and set the output size

set "INPUT=PanoView.mp4" :: Input video

set "OUTPUT=out.mp4" :: Output video
set "CROP=[Link]" :: Specify the visible part: Width, height, left edge, top edge
set "SIZE=800x800" :: Width and height of the output video (can be smaller or greater than the input video)
:: Keep the width/height ratio constant, otherwise the video looks distorted,
:: for example a circle would become an ellipse.
set "QU=3" :: MP4 Quality, 1 is best Quality, 3 is normal, 31 is strongest compression

ffmpeg -i %INPUT% -vf crop=%CROP% -s %SIZE% -q:v %QU% -codec:v mpeg4 %OUTPUT%

pause

In the crop filter you can use the variables "iw" and "ih", which are the width and height of the input video.
If the 3rd and 4th parameter (coordinates of top left corner) isn't specified, the crop will be automatically centered.

crop=ih:ih makes a centered square crop, useful for fulldome videos

crop=iw/2:ih:0 returns the left half of the input video
crop=iw/2:ih:iw/2 returns the right half of the input video
crop=iw/4:ih/4 strong enlargement by a factor 4 in the center of the video

The "pad" filter does the opposite thing, it adds paddings with a uniform color to the video. See next chapter.

99
2.48 Add borders to a video

Borders can be added with the "pad" filter. This example adds a black 40 pixel border at the bottom of the video, for example for writing text into it:
ffmpeg -i input.mp4 -vf pad=iw:ih+32 -y output.mp4

pause

2.49 Zoompan

This is a very powerful filter. It can also be used for making slideshows. The "d" option specifies how long each image in shown.

Parameters:
'out_time' or 'ot' Timestamp in seconds of each output frame produced by zoompan.
'in_time' or 'it' Timestamp in seconds of each input frame to the zoompan filter.

In most cases it's useful to specify the size of the output frames with the "s" option, because the defaulf is 1280x720.

See also "Use of 'geq' as 'zoompan' alternative":

[Link]

(I'm still working on this chapter...)

100
2.50 Changing the speed: slow motion and timelapse

rem Changing the speed (slow motion ot timelapse)

set "INPUT=PanoView.mp4" :: Input video

set "OUTPUT=out.mp4" :: Output video
set "RATE=30" :: Output framerate
set "SPEED=3.0" :: Speed factor, smaller than 1 = timelapse, 1 = real time, greater than 1 = slow motion
set "QU=3" :: MP4 Quality, 1 is best Quality, 3 is normal, 31 is strongest compression

ffmpeg -i %INPUT% -vf setpts=%SPEED%*PTS -r %RATE% -q:v %QU% -codec:v mpeg4 -an -y %OUTPUT%

pause

In this example the settings for "RATE" and "SPEED" are totally independent of each other. FFmpeg will automatically skip or duplicate frames, if
required.
Example: If both input and output frame rate are 30, and if SPEED = 3, then each frame will automatically duplicated 2 times, so that we see it 3 times in
the output video. If SPEED = 0.5, then each second frame is skipped.
In this example the slow motion or timelapse effect affects only video and not audio. It makes sense to disable the audio channel with the -an option.
The "setpts" filter is described in the "Multimedia Filters" section in the FFmpeg documentation.
The timebase (TB in setpts filter) is expressed in seconds [s].
The framerate (FR in setpts filter) is expressed in 1/seconds [s^-1]
In many cases the timebase is the reciprocal of the framerate, but this isn't always the case.

101
Some more examples:
setpts=0.5*PTS Double speed
setpts=2.0*PTS Half speed
setpts=PTS+x/(FR*TB) or tpad=x Delay by x frames (assuming the framerate is constant)
setpts=PTS+x/TB or tpad=x/framerate Delay by x seconds
setpts=PTS-STARTPTS Start counting PTS from zero
setpts=N/(FRAME_RATE*TB) Set new presentation timestamps, independant of input PTS
asetpts=N/(SR*TB)

See also these Wiki pages:

[Link]
[Link]

2.51 Slow motion or timelapse only for a segment of the video

See the comments for explanation.

set "IN=[Link]" :: Input Video
set "T1=5" :: Start time T1
set "T2=8.5" :: Time T2 when slow motion begins
set "T3=9.7" :: Time T3 when slow motion ends
set "T4=11" :: End time T4
set "SPEED=5" :: Speed factor, smaller than 1 = timelapse, greater than 1 = slow motion
set "FR=30" :: Output framerate
set "OUT=out.mp4" :: Output video

ffmpeg -i %IN% -filter_complex "[0:v]trim=%T1%:%T2%,setpts=PTS-STARTPTS[v1];[0:v]trim=%T2%:%T3%,setpts=%SPEED%*(PTS-

STARTPTS)[v2];[0:v]trim=%T3%:%T4%,setpts=PTS-STARTPTS[v3];[v1][v2][v3]concat=n=3:v=1" -an -r %FR% -q:v 2 -y out.mp4

pause

102
2.52 Time Remapping

This is an example for a gradual ramp into and out of slow motion:
ffmpeg -f lavfi -i testsrc2=size=vga:duration=10:rate=20 -lavfi "^
[0]trim=0.0:3.2,setpts=(PTS-STARTPTS)[1];^
[0]trim=3.2:3.6,setpts=(PTS-STARTPTS)/0.80[2];^
[0]trim=3.6:4.0,setpts=(PTS-STARTPTS)/0.60[3];^
[0]trim=4.0:6.0,setpts=(PTS-STARTPTS)/0.40[4];^
[0]trim=6.0:6.4,setpts=(PTS-STARTPTS)/0.60[5];^
[0]trim=6.4:6.8,setpts=(PTS-STARTPTS)/0.80[6];^
[0]trim=6.8:10.0,setpts=(PTS-STARTPTS)[7];^
[1][2][3][4][5][6][7]concat=n=7:v=1" -y out.mp4

pause

This is an example for a 10s input video where the framerate changes linearly from 20 to 10:
ffmpeg -f lavfi -i testsrc2=size=vga:duration=10:rate=20 -lavfi "
[0]trim=0:1,setpts=(PTS-STARTPTS)/0.975[1];
[0]trim=1:2,setpts=(PTS-STARTPTS)/0.925[2];
[0]trim=2:3,setpts=(PTS-STARTPTS)/0.875[3];
[0]trim=3:4,setpts=(PTS-STARTPTS)/0.825[4];
[0]trim=4:5,setpts=(PTS-STARTPTS)/0.775[5];
[0]trim=5:6,setpts=(PTS-STARTPTS)/0.725[6];
[0]trim=6:7,setpts=(PTS-STARTPTS)/0.675[7];
[0]trim=7:8,setpts=(PTS-STARTPTS)/0.625[8];
[0]trim=8:9,setpts=(PTS-STARTPTS)/0.575[9];
[0]trim=9:10,setpts=(PTS-STARTPTS)/0.525[10];[1][2][3][4][5][6][7][8][9][10]concat=n=10:v=1" -y out.mp4

pause
The length of the output video is 13.65s

103
Use the following example carefully, as I'm not 100% convinced that the approach is correct. This is based on an posting from Nicolas George in the
FFmpeg user mailing list, September 23, 2019. In the first equation it's unclear if t is the time in the input video or in the output video.
rem > So, to compute the timestamp of a frame with variable speed:
rem >
rem > * Express your frame rate as a complete formula: t → v
rem >
rem > * Integrate it: t → f.
rem >
rem > * Find the reciprocal: f → t.
rem
rem Let's assume we have a 10s video and the framerate changes linearly from 20 at the beginning to 10 at the end:
rem v = 20 - t v(0) = 20 v(10) = 10
rem
rem After integrating we get: f = 20 * t - 0.5 * t^2
rem
rem The inverse function is: t = 20 - sqrt(400 - 2 * f)

rem Creaste a test video with framerate=20 and length=10s:

ffmpeg -f lavfi -i testsrc2=size=vga:duration=10:rate=20 -y test.mp4

rem Apply the time remapping:

ffmpeg -i test.mp4 -lavfi setpts='(20-sqrt(400-2*N))/TB' -y out.mp4

pause

The resulting video gets slower towards the end (too slow, in fact), and the length is 18.95s and that seems to be wrong. With a constant framerate of 20
the length is 10s, with a constant framerate of 10 the length is 20s, and if the framerate changes from 20 to 10 the length should be about 15s. I don't fully
understand what's going on here.

Note: It's much easier to do time remapping in DaVinci Resolve.

Keywords for searching: "Time remapping", "Time ramp", "Slow motion ramp", "Speed ramp"

104
2.53 Insert a text which is visible for the whole duration

set "IN=[Link]" :: Input video

set "OUT=output.mp4" :: Output video
set "FONT=[Link]" :: Font
set "TEXT=Hello_World" :: Text (no space characters allowed, see next example)
set "COLOR=yellow" :: Text color
set "SIZE=20" :: Font size
set "POS_X=(w-tw)/2" :: X position of text, use (w-tw)/2 for centering
set "POS_Y=(h-th)/2" :: Y position of text, use (h-th)/2 for centering

ffmpeg -i %IN% -vf drawtext='fontfile=%FONT%:text=%TEXT%:fontcolor=%COLOR%:fontsize=%SIZE%:x=%POS_X%:y=%POS_Y%' -c:v

mpeg4 -q:v 1 -y %OUT%

pause

2.54 Slowly fade a text in and out

rem Slowly fade a text in and out

set "INPUT=PanoView.mp4" :: Input video

set "OUTPUT=out.mp4" :: Output video
set "QU=3" :: MP4 Quality, 1 is best Quality, 3 is normal, 31 is strongest compression

set "NAME=TEXT1" :: Unique name for this text

set "FONT=[Link]" :: Font
set "TEXT=[Link]" :: Text filename (must be UTF-8 coded, if the text contains non-ASCII characters like
:: ä, ö, ü. The text can be ASCII coded if no special characters are used.
set "COLOR=yellow" :: Text color
set "SIZE=250" :: Font size
set "POS_X=(w-tw)/2" :: X position of text, use (w-tw)/2 for centering
set "POS_Y=(h-th)/2" :: Y position of text, use (h-th)/2 for centering

105
set "DS=0.5" :: Start time
set "DE=4.5" :: End time
set "FID=1.0" :: Fade-in duration (may be small, but not zero)
set "FOD=1.0" :: Fade-out duration (may be small, but not zero)

set %NAME%=drawtext='fontfile=%FONT%:textfile=%TEXT%:fontcolor_expr=%COLOR%@%%{e\:clip(((t-%DS%)/%FID%)*^
between(t,%DS%,(%DS%+%DE%)/2)+(%DE%-t)/%FOD%*between(t,(%DS%+%DE%)/2,%DE%),0,1)}:fontsize=%SIZE%:x=%POS_X%:y=%POS_Y%'

set "NAME=TEXT2" :: Unique name for this text

set "FONT=[Link]" :: Font
set "TEXT=[Link]" :: Text filename (must be UTF-8 coded, if the text contains non-ASCII characters like
:: ä, ö, ü. The text can be ASCII coded if no special characters are used.
set "COLOR=red" :: Text color
set "SIZE=250" :: Font size
set "POS_X=(w-tw)/2" :: X position of text, use (w-tw)/2 for centering
set "POS_Y=(h-th)/3" :: Y position of text, use (h-th)/2 for centering
set "DS=0.0" :: Start time
set "DE=3.0" :: End time
set "FID=0.5" :: Fade-in duration (may be small, but not zero)
set "FOD=0.5" :: Fade-out duration (may be small, but not zero)

set %NAME%=drawtext='fontfile=%FONT%:textfile=%TEXT%:fontcolor_expr=%COLOR%@%%{e\:clip(((t-%DS%)/%FID%)*^
between(t,%DS%,(%DS%+%DE%)/2)+(%DE%-t)/%FOD%*between(t,(%DS%+%DE%)/2,%DE%),0,1)}:fontsize=%SIZE%:x=%POS_X%:y=%POS_Y%'

ffmpeg -i %INPUT% -vf "%TEXT1%,%TEXT2%" -q:v %QU% -codec:v mpeg4 -an -y %OUTPUT%

pause

The text must be saved as a *.txt file. If the text contains non-ASCII special characters like ä, ö, ü then the encoding must be UTF-8. If the text contains
only ASCII characters, then "ANSI" encoding is possible as well.
In some cases drawtext shows a non-printable character (for example an empty rectangle) at the beginning of the text. This is a BOM (Byte Order Mark)
that was automatically added to the text by some Windows programs at the beginning of the file. Older versions of Notepad (on Windows 7) show this
behaviour and you can't disable it. The BOM consists of three bytes EFhex BBhex BFhex.
See also here: [Link]
There are several solutions for this problem:
• Open the text file with a hex editor and remove the first three characters (EFhex BBhex BFhex). For example you can use Hex Editor MX:
[Link]

106
 • If you can't find a newer 32-bit Notepad version for Windows 7, you can use Notepad++ instead. Select "UTF-8" in the "Encoding" menue.
[Link]
Notepad++ has the unexpected behaviour that it always restores the last session when you start it. To disable this behaviour, go to Settings -->
Preferences --> Backup and untick "Remember current session for next launch". Then it just opens a new empty file when you start it by double-
clicking on the Notepad++ icon.
• Newer versions of Notepad (on Windows 10) have a selection between "UTF-8" and "UTF-8 with BOM". Using "UTF-8" will solve the problem.

Problem: You want to show the content of a credits file scrolling [Link] file contains many lines of different lengths.
drawtext=textfile=[Link]:x=(w-text_w)/2:y=h-100*t

The variable text_w is the width of the longest line in the text file. This line is center-aligned in the frame, and all other (shorter) lines are left-aligned to
the same X position as the longest line. But that's not what you want. Is it somehow possible that each line is center-aligned?
Solution: See the workaround with *.ass subtitles in this document.

2.55 Vertical alignment of text

Problem: In the "drawtext" filter, the content of the variable "text_h" depends on which characters are printed. For example, the characters "a", "A", "g",
"_" and "^" do all have different heights.

Vertical position What does it do?

y=100 The highest point of the string (the ascent) is used for vertical alignment.
Warning: The vertical alignment depends on the content of the string.
y=100-text_h The lowest point of the string (the descent) is used for vertical alignment.
Warning: The vertical alignment depends on the content of the string.
y=100-ascent The baseline of the string is used for alignment. The vertical alignment doesn't depend on the content of the string.
This is the recommended method when you want to print several strings in the same line.

107
2.56 Show a running clock in the video

In this example a running clock is inserted in each frame of the video, in the format "hours:minutes:[Link]"
set "IN=[Link]" :: Input video
set "OUT=sylvia.mp4" :: Output video
::
set "BP_R=0.015" :: Black point red, positive value makes background darker
set "BP_G=0.005" :: Black point green, positive value makes background darker
set "BP_B=0.015" :: Black point blue, positive value makes background darker
::
set "WP=0.26" :: White point
::
set "S=300" :: Start time
set "T=40" :: Duration
::
set "FONT=[Link]" :: Font
set "COLOR=white" :: Font color
set "BOXCOLOR=black" :: Background color
set "SIZE=30" :: Font size
set "POSITION_X=0" :: X position of clock
set "POSITION_Y=(h-th)" :: Y position of clock
set "OF=2340" :: Offset time in seconds, shown in the first frame
set "I=0.04" :: Time intervall from one frame to the next = 1/framerate

set CLOCK=drawtext='fontfile=%FONT%:text=%%{eif\:mod((%OF%+%I%*n)/3600,24)\:'d'\:2}"\:"%%{eif\:mod((%OF%+%I
%*n)/60,60)\:'d'\:2}"\:"%%{eif\:mod(%OF%+%I%*n,60)\:'d'\:2}"."%%{eif\:mod((%OF%+%I%*n)*1000,1000)\:'d'\:3}:fontcolor=
%COLOR%:boxcolor=%BOXCOLOR%:box=1:fontsize=%SIZE%:x=%POSITION_X%:y=%POSITION_Y%'

ffmpeg -ss %S% -i %IN% -vf "colorlevels=rimin=%BP_R%:gimin=%BP_G%:bimin=%BP_B%:rimax=%WP%:gimax=%WP%:bimax=%WP%,%CLOCK%"

-pix_fmt yuv420p -t %T% -y %OUT%

pause

108
This batch file does the same thing and is simpler:
set "IN=[Link]" :: Input video
set "OUT=sylvia.mp4" :: Output video
set "BP_R=0.015" :: Black point red, positive value makes background darker
set "BP_G=0.005" :: Black point green, positive value makes background darker
set "BP_B=0.015" :: Black point blue, positive value makes background darker
set "WP=0.26" :: White point
set "S=300" :: Start time
set "T=40" :: Duration
set "FONT=[Link]" :: Font
set "COLOR=white" :: Font color
set "BCOLOR=black" :: Background color
set "SIZE=30" :: Font size
set "POS_X=0" :: X position of clock
set "POS_Y=(h-th)" :: Y position of clock
set "OFFSET=2340" :: Offset time in seconds, added to the timestamp of the first frame

set CLOCK=drawtext='fontfile=%FONT%:text=%%{pts\:hms\:%OFFSET%}:fontcolor=%COLOR%:boxcolor=%BCOLOR%:box=1:fontsize=%SIZE
%:x=%POS_X%:y=%POS_Y%'

ffmpeg -ss %S% -i %IN% -vf "colorlevels=rimin=%BP_R%:gimin=%BP_G%:bimin=%BP_B%:rimax=%WP%:gimax=%WP%:bimax=%WP%,%CLOCK%"

-pix_fmt yuv420p -t %T% -y %OUT%

pause

This is another example, using the "timecode" option of the drawtext filter:
ffmpeg -f lavfi -i testsrc2=size=hd720:duration=10 -vf
drawtext=fontsize=60:fontcolor=Black:fontfile='[Link]':timecode='00\:00\:00\:00':r=25:x=20:y=40 -y out.mp4

pause

109
2.57 Generation of curved text for fulldome projection

rem Create a video with curved text fade-in fade-out, silent audio

set "SIZE=1200" :: Video size (square)

set "QU=3" :: MP4 quality level, 1 is best quality, 3 is normal, 31 is strong compression
set "FPS=30" :: Output Framerate
set "FONT=[Link]" :: font
set "FSIZE=60" :: font size
set "COLOR=white" :: text color
set "BACK=black" :: background color
set "DUR=10" :: duration of video
set "TEXT=[Link]" :: text file
set "POS_X=(w-tw)/2" :: X text position, for centered text: (w-tw)/2
set "POS_Y=h*0.9" :: Y text position
set "S=1" :: start time for text
set "E=9" :: end time for text
set "FI=2" :: fade-in duration (may be small, but not zero)
set "FO=2" :: fade-out duration (may be small, but not zero)
set "OUTPUT=text13.mp4" :: Output filename

ffmpeg -f lavfi -i color=c=%BACK% -i xmap_3648.pgm -i ymap_3648.pgm -f lavfi -i anullsrc -r %FPS% -t %DUR% -aspect "1:1"
-lavfi scale=3648:3648,drawtext='fontfile=%FONT%:textfile=%TEXT%:fontcolor_expr=%COLOR%@%%{e\:clip((t-%S%)/%FI
%*between(t,%S%,%S%+%FI%)+(%E%-t)/%FO%*between(t,%S%+%FI%,%E%),0,1)}:fontsize=%FSIZE%:x=%POS_X%:y=%POS_Y
%',format=pix_fmts=rgb24,remap -s %SIZE%x%SIZE% -c:v mpeg4 -c:a aac -shortest -q:v %QU% -y %OUTPUT%

pause

I have to admit that this is a complicated command line. The actual core is the "remap" filter, with which you can create arbitrary distortions. The
distortion is described in the two files xmap_3648.pgm and ymap_3648.pgm. In these files the pixel in the input video from which it is retrieved is
indicated for each pixel. You have to write a (C#) program that can create these files.
-i color=c=black creates a black image
-i anullsrc creates an empty audio track

This is the C# code for generating the xmap and ymap files:

110
int a = (int)[Link]; // get the size of the square map
double c = (double)[Link]; // this is the aspect ratio of the text, normal = 1
int b = a/2;
int xx, yy;

TextWriter xmap = [Link]("xmap_" + [Link]() + ".pgm");

[Link]("P2\n");
[Link]("# Xmap file for fulldome remap \n");
[Link]([Link]() + " " + [Link]() + " \n");
[Link]("65535\n");

TextWriter ymap = [Link]("ymap_" + [Link]() + ".pgm");

[Link]("P2\n");
[Link]("# Ymap file for fulldome remap \n");
[Link]([Link]() + " " + [Link]() + " \n");
[Link]("65535\n");

for (int y = 0; y < a; y++)

{
for (int x = 0; x < a; x++)
{
xx = x;
yy = y;
if (y > b)
{
xx = b + (int)(b / c * [Link]((double)(x - b) / (double)(y - b)));
yy = b + (int)[Link]((x - b) * (x - b) + (y - b) * (y - b));
if (xx < 0) xx = 0;
if (yy < 0) yy = 0;
if (xx > a - 1) xx = a - 1;
if (yy > a - 1) yy = a - 1;
}
[Link](xx + " ");
[Link](yy + " ");
}
[Link]("\n");
[Link]("\n");
}
[Link]("\n");
[Link]("\n");
[Link]();
[Link]();

111
This is a simpler example for generating curved text for fulldome projection, using the v360 filter:
set "UP=30" :: Up-looking angle in degrees (center of the rectangular video)
set "H=64" :: Horizontal field of view, this is for 16:9 aspect ratio
set "V=36" :: Vertical field of view, this is for 16:9 aspect ratio
set "SIZE=1200" :: Square size of the output video
set "FONT=[Link]" :: font
set "FSIZE=120" :: font size
set "COLOR=white" :: text color
set "BACK=black" :: background color
set "TEXT=[Link]" :: text file
set "POS_X=(w-tw)/2" :: X text position, for centered text: (w-tw)/2
set "POS_Y=(h-th)/2" :: Y text position, for centered text: (h-th)/2
set "S=1" :: start time for text
set "E=9" :: end time for text
set "FI=2" :: fade-in duration (may be small, but not zero)
set "FO=2" :: fade-out duration (may be small, but not zero)
set "DUR=10" :: duration of video
set "OUT=out.mp4" :: Output video

ffmpeg -f lavfi -i color=%BACK%:size=hd1080 -vf drawtext='fontfile=%FONT%:textfile=%TEXT%:fontcolor_expr=%COLOR%@%%

{e\:clip((t-%S%)/%FI%*between(t,%S%,%S%+%FI%)+(%E%-t)/%FO%*between(t,%S%+%FI%,%E
%),0,1)}:fontsize=%FSIZE%:x=%POS_X%:y=%POS_Y%',v360=input=flat:ih_fov=%H%:iv_fov=%V
%:output=fisheye:h_fov=180:v_fov=180:pitch='90-%UP%':w=%SIZE%:h=%SIZE% -t %DUR% -y %OUT%

pause

112
2.58 Write text on a transparent layer

In this example text is written on a transparent background (black@0). This video is scaled to the same size as the input video with the "scale2ref" filter.
Finally the text video is overlaid over the main video.
The advantage of this method is that you can modify the geometry of the text before overlaying it. For example you can use the "displace", "perspective",
"remap", "rotate" or "v360" filters for modifying the geometry.
set "IN=R0010008_er.mp4" :: Input video
set "OUT=out.mp4" :: Output video with overlaid test

ffmpeg -i %IN% -f lavfi -i color=black@0,format=rgba -lavfi [1][0]scale2ref[a][b],

[a]drawtext="fontsize=80:text='TEST':box=1:boxcolor=red:boxborderw=10:fontcolor=yellow:x=(w-text_w)/2:y=(h-
text_h)/2"[c];[b][c]overlay -t 5 -y %OUT%

pause

Note: It is required to add "format=rgba" after the "color" video source. Otherwise the format negotiation could fail and agree on a yuv420 format (which
doesn't have a transparency layer).

How the "scale2ref" filter works:

This filter has two inputs and two outputs. The first input is the video that shall be scaled, and the second input is the reference video from which the
size is used. The first output is the scaled video, and the second output is a copy of the second input. The filter has many options but none of them are
required for the basic function, as in this example.

113
In this example the white square has a dynamically changing size, and "scale2ref" is used to scale the red input to the same size:
ffmpeg -f lavfi -i color=black:size=400x400 ^
-f lavfi -i color=white:size=100x100 ^
-f lavfi -i color=red ^
-lavfi [1]scale=width='100+n':height='100+n':eval=frame[w];[2][w]scale2ref=eval=frame[r][w];[r]null[r];[w][r]overlay[w];
[0][w]overlay -t 4 -y gif.mp4

pause

It's unclear why the "null" filter is required in this example. If the "null" filter is omitted, then it seems there a one frame delay so that the red square is
one pixel too small, which is visible as a white border. See also [Link]

114
2.59 Combine multiple videos with concat demuxer

The concat demuxer combines several videos without re-encoding. It's very fast.
rem Final cut with concat demuxer

ffmpeg -f concat -i concat_list.txt -c copy -y MyVideo.mp4

pause

You simply write all existing scenes into a text file (here: concat_list.txt), which looks like this:
file text1.mp4 :: 10 Title: A year in the woods
file text2.mp4 :: 10 When and where
file Videos/scene20.mp4 :: 12 Live video in the wood
# This is a comment
file text22.mp4 :: 10 In 15 months...
file Videos/scene22.mp4 :: 52 Live video, camera
file text98.mp4 :: 10 the end

To the right of the double colons are optional comments (e.g. the length of the scenes and a short description). Comments can also begin with #.
This method, however, requires that all scenes have
• the same size (width and height)
• the same pixel format
• the same video codec
• the same framerate
• the same audio codec
• the same number of audio tracks (take care when you use a camera which writes only a mono soundtrack)
• the same audio sample rate

If one of these conditions isn't met, an error message is issued. You can then look at the properties of the files with FFprobe or Exiftool to find out where
the files differ.

115
How to create a concat_list file which contains all *.mp4 files from a folder:

if exist concat_list.txt del concat_list.txt

(for %%G in (*.mp4) do @echo file '%%G') >> concat_list.txt

pause

See also here: [Link]

116
2.60 Combine multiple videos with concat filter

In this example the concat filter is used for input videos of the same size and no audio.
Each of the -ss and -t specifies the start time and length of the next input file. You can remove these options if you want to use the full videos.
The value n=3 passed to the concat filter should match the number of input files.
This filter does re-encode the videos, so the process is slow but you can also specify the encoding quality.
set "I1=my_video1.mp4" :: Input video 1
set "S1=0" :: Set start time 1
set "L1=4" :: Set length 1
set "I2=my_video2.mp4" :: Input video 2
set "S2=3" :: Set start time 2
set "L2=3" :: Set length 2
set "I3=my_video3.mp4" :: Input video 3
set "S3=6" :: Set start time 3
set "L3=2" :: Set length 3
set "OUT=out.mp4" :: Output video

ffmpeg -ss %S1% -t %L1% -i %I1% -ss %S2% -t %L2% -i %I2% -ss %S3% -t %L3% -i %I3% -lavfi "concat=n=3:v=1:a=0" -an %OUT%

pause

See also here: [Link]

Note: Cutting the input videos to the required section can also be done with the "trim" filter.

The opposite of the "concat" filter is the "segment" filter, which splits a video into several streams.

2.61 The "fps" filter

This filter is described in detail on Jim DeLaHunt's website: [Link]

117
2.62 Split a video in multiple segments

A video can be split in multiple segments with the segment muxer. All segments will have the same length, except the last one.
set "IN=my_video.mov" :: Input video
set "L=10" :: Segment length in seconds
seu "OUT=out%%[Link]" :: Output filename

ffmpeg -i %IN% -f segment -segment_time %L% -c copy %OUT%

pause
Note: The duration of the segments can also be expressed as minutes:seconds

This batch fill extracts a segment with known start and end frame numbers:
set "start=100" :: First frame number
set "end=200" :: Last frame number

set /a startms=%start%*1001/30 :: This calculation is for framerate 30000/1001 = 29.97

set /a endms=(%end%+1)*1001/30 :: Note that in the batch file only integer arithmetic is possible!
:: It's important to do first the multiplication and then the division

ffmpeg -i in.mp4 -ss %startms%ms -to %endms%ms -c copy -y out.mp4

pause

Note: The above command line with "-c copy" works only for intraframe codecs, meaning that all frames are I-frames. For interframe codecs you must
remove "-c copy", but then then the video will be re-encoded and the process is much slower.

I found the following hint here: [Link]

Try this "template" to generate .ts segment files:
ffmpeg -i <source> -y -codec copy -bsf:v h264_mp4toannexb -f segment -segment_time <seconds> -segment_time_delta 0.05 %[Link]
MPEG-TS container is concatenable directly using "cat" from linux, after that you could use ffmpeg to change the container, .ts to .mov or .mp4 without
reencoding (-codec copy). May be this doesn't generate those extra keyframes, the option -segment_time_delta 0.05 has something to do with the
boundaries where ffmpeg will make a segment, i remember it was better this way, but i don't remember exactly why.

118
2.63 Switch between two inputs at a certain time

In this example the first input is used to the first 5 seconds, and then the second input is used.
ffmpeg -f lavfi -i color=red -f lavfi -i color=yellow -lavfi blend=all_expr='if(lt(T,5),A,B)' -y -t 10 test.mp4

pause

The same thing can lso be done with the "streamselect" filter.
Select the first stream for the first 5 seconds, and the second stream for the rest of time:
ffmpeg -re -f lavfi -i color=red:s=1280x720 -f lavfi -i testsrc2=s=1280x720 -lavfi sendcmd="5.0 streamselect map
1",streamselect=map=0 -f sdl2 -

pause

Note: The argument of the "sendcmd" filter must be encapsulated in double " " quotes. It doesn't work with single ' ' quotes. The example in the official
documentation is wrong, at least for Windows.
Note: The -re option is used here to slow down the input streams to realtime.
Note: For audio, use "astreamselect" instead of "streamselect".

This is the same as above, but with an image as first input and a live webcam as second input:
ffmpeg -loop 1 -framerate 10 -i [Link] -f dshow -video_size 1280x720 -framerate 10 -i video="BisonCam,NB Pro" -lavfi
streamselect=map=0,format=rgb24,sendcmd="5.0 streamselect map 1" -f sdl2 -

pause

Note: Make sure that the image has the same size as the webcam.

The same thing can also be done with the "overlay" filter. In this example an image is shown for 5 seconds, and then the output is switched to the live

119
webcam input:
ffmpeg -loop 1 -framerate 10 -i [Link] -f dshow -video_size 1280x720 -framerate 10 -i video="BisonCam,NB Pro" -lavfi
[1][0]overlay=enable='lt(t,5)':format=rgb -f sdl2 -

pause

120
2.64 Switch between two cameras, using audio from camera1

rem Create a 6 seconds red video with 400Hz tone

ffmpeg -f lavfi -i color=c=red:s=vga -f lavfi -i sine=frequency=400 -t 6 -y video1.mp4

rem Create a 6 seconds test video with 1200Hz tone

ffmpeg -f lavfi -i testsrc2=s=vga -f lavfi -i sine=frequency=1200 -t 6 -y video2.mp4

rem Switch to video2 from 2 to 4 seconds, but use always the audio from video1
ffmpeg -i video1.mp4 -i video2.mp4 -filter_complex blend=all_expr='if(between(T,2,4),B,A)' -y test.mp4

pause

Note: In this example both videos start at the same time. The video2 segment from 2 to 4 seconds is inserted in the output video from 2 to 4 seconds.
You get this output video:

0<t<2 2<t<4 4<t<6

Video from video1 (0...2) from video2 (2...4) from video1 (4...6)
Audio from video1 (0...2) from video1 (2...4) from video1 (4...6)

If you want to insert the video2 segment from 0 to 2 seconds in the output video from 2 to 4 seconds, use this command line instead:
ffmpeg -i video1.mp4 -i video2.mp4 -filter_complex [1]tpad=start_duration=2[2];[0][2]blend=all_expr='if(between
(T,2,4),B,A)' -y test.mp4

pause

In this case you get this output video:

0<t<2 2<t<4 4<t<6

Video from video1 (0...2) from video2 (0...2) from video1 (4...6)
Audio from video1 (0...2) from video1 (2...4) from video1 (4...6)

121
2.65 Stack videos side by side (or on top of each other)

set "IN1=left.mp4"
set "IN2=right.mp4"
set "OUT=out.mp4"
rem use "hstack" for horizontal stacking and "vstack" for vertical stacking

ffmpeg -i %IN1% -i %IN2% -filter_complex hstack -an -shortest -c:v mpeg4 -y %OUT%

pause

Note: If the videos have different width or height, use the "xstack" filter instead.

2.66 Horizontal and vertical flipping

This can be done with the "hflip" and "vflip" filters.

2.67 Stereo3d filter

The stereo3d filter can be used for splitting a square (fisheye) image in two halves (top and bottom) and stacking the halves side by side (left and right):
In this example a 4096x4096 fisheye image is splitted in two halves and the output size 8192x2048:
rem Test pattern from [Link]

ffmpeg -i [Link] -vf stereo3d=abl:sbsl -y [Link]

pause
Note: If left/right images are swapped, use "sbsr" instead of "sbsl".

122
2.68 Stack four videos to a 2x2 mosaic

set "IN1=topleft.mp4"
set "IN2=topright.mp4"
set "IN3=bottomleft.mp4"
set "IN4=bottomright.mp4"
set "OUT=mosaic.mp4"

ffmpeg -i %IN1% -i %IN2% -i %IN3% -i %IN4% -filter_complex [0:v][1:v]hstack[t];[2:v][3:v]hstack[b];[t][b]vstack -an

-shortest -c:v mpeg4 -q:v 1 -y %OUT%

pause

Other method using xstack:

set "IN1=topleft.mp4"
set "IN2=topright.mp4"
set "IN3=bottomleft.mp4"
set "IN4=bottomright.mp4"
set "OUT=mosaic.mp4"

ffmpeg -i %IN1% -i %IN2% -i %IN3% -i %IN4% -filter_complex "xstack=inputs=4:layout=0_0|0_h0|w0_0|w0_h0" -shortest %OUT%

pause

Display 4 inputs into a vertical 1x4 grid, note that the input videos may have different widths (vstack can't handle this case).

ffmpeg -i %IN1% -i %IN2% -i %IN3% -i %IN4% -filter_complex "xstack=inputs=4:layout=0_0|0_h0|0_h0+h1|0_h0+h1+h2" %OUT%

pause

123
2.69 Blink comparator

This is an example of a blink comparator. It creates an animated GIF that continuously toggles between two (or more) images.
rem Blink comparator, animated GIF

set "IN=pluto_%%[Link]" :: Filename of the images

set "FR=2.0" :: Frame rate
set "OUT=[Link]" :: Animated GIF output file

ffmpeg -framerate %FR% -i %IN% -q:v 1 -y %OUT%

pause

Please note that there is a known problem with FFmpeg's GIF encoder which may result in wrong colors in the output file. See the next chapter for a
workaround.

If you want to create an MP4 instead, then you have to specify how long it should be and the input and output framerates:
rem Blink comparator, MP4

set "IN=pluto_%%[Link]" :: Filename of the images

set "FI=2.0" :: Framerate for reading in the pictures
set "T=10" :: Lenght in seconds
set "FO=25" :: Output framerate
set "OUT=out.mp4" :: Output MP4 file

ffmpeg -loop 1 -framerate %FI% -i %IN% -t %T% -r %FO% -q:v 1 -y %OUT%

pause

The parameter "-loop 1" causes the same images to be read in again and again. If you do this, you have to limit the length of the video somehow, in this
case with "-t 10".

124
This is an example for toggling between two images or two video streams:
ffmpeg -f lavfi -i color=yellow -vf drawtext='text=1:fontcolor=red:fontsize=100:x=140:y=80' -frames 1 -y [Link]
ffmpeg -f lavfi -i color=yellow -vf drawtext='text=2:fontcolor=red:fontsize=100:x=140:y=80' -frames 1 -y [Link]

ffmpeg -loop 1 -i [Link] -loop 1 -i [Link] -lavfi blend=all_expr='if(lt(mod(T,2),1),A,B)' -t 10 -y out.mp4

pause
Note: "blend" is slow.

It's also possible to toggle between two images or video streams with "sendcmd" and "streamselect", but it's quite complicated to escape the commas:
ffmpeg -f lavfi -i color=yellow -vf drawtext='text=1:fontcolor=red:fontsize=100:x=140:y=80' -frames 1 -y [Link]
ffmpeg -f lavfi -i color=yellow -vf drawtext='text=2:fontcolor=red:fontsize=100:x=140:y=80' -frames 1 -y [Link]

ffmpeg -loop 1 -i [Link] -loop 1 -i [Link]

-lavfi "sendcmd=c='0 [expr] streamselect map '\''gte(mod(T\,2)\,1)'\''',streamselect=map=0" -t 10 -y out.mp4

pause
Note: I have inserted a line feed in the command line only for clarity. Of course it must all be written in one line.

125
2.70 Creating animated GIF or PNG

There is a known problem with FFmpeg's GIF encoder which may result in wrong colors in the animated GIF output file. If you encounter this problem,
you can use the following workaround which uses the palettegen and paletteuse filters. Thanks to Javier Infante Porro for posting this workaround in the
FFmpeg user mailing list on September 26, 2019.
set "IN=[Link]" :: Input video (animated GIF)
set "COL=8" :: Number of colors (including one transparent color)
set "OUT=[Link]" :: Output video (animated GIF)

ffmpeg -i %IN% -lavfi "split[s0][s1];[s0]palettegen=max_colors=%COL%[p];[s1][p]paletteuse" -y %OUT%

pause

Please note that one entry in the palette is reserved for the transparent color by default. So when you set the max_colors parameter to 8, you have only 7
different visible colors. If you don't want a transparent color, you must disable it with the reserve_transparent=0 option.

Much more about this subject can be found here:

[Link]

For animated PNG (APNG) see: [Link]

See also the "elbg" filter for mapping an input image to an output image with a distinct number of colors.

126
2.71 Overlay an animated GIF over a video

set "BACK=[Link]" :: Background video

set "OVL=[Link]" :: Overlay video
set "OUT=out.mp4" :: Output video
set "X=200" :: X position of overlay
set "Y=400" :: Y position of overlay
set "S=0.5" :: Size factor for overlay
set "T=10" :: Maximum length in seconds

ffmpeg -i %BACK% -ignore_loop 0 -i %OVL% -lavfi [1]scale=iw*%S%:ih*%S%[a];[0][a]overlay=x=%X%:y=%Y% -t %T% -y %OUT%

pause

Note: Use "-ignore_loop 0" if you want to loop the GIF, or remove this option if you want to play the GIF only one time.

2.72 Changing the size of an animated GIF

set "IN=[Link]" :: Input file

set "S=0.5" :: Size factor
set "OUT=[Link]" :: Output file

ffmpeg -i %IN% -lavfi scale=iw*%S%:-1,split[a][b];[a]palettegen=reserve_transparent=on:transparency_color=ffffff[p];[b]

[p]paletteuse -y %OUT%

pause

The trick with palettegen and paletteuse is required to keep the transparency color.

127
2.73 Replace one frame in a video by another

This example shows how to replace a single image in a video with another image.
You may have heard of a trick to insert a product image into a film for advertising purposes, only for the duration of a singe frame. For example, if the
frame rate is 25 frames per second, then a single frame will be shown for 40ms. That's too short to recognize the product clearly, but it's long enough to
make viewers feel that they want this product. If, for example, a bratwurst or popcorn is shown for 40ms in the film, the sales figures for exactly these
products increase after the end of the film. Although the viewer is not aware of why he has now gotten an appetite for a bratwurst or popcorn.

set "IN=scene8.mp4" :: Input video

set "BW=[Link]" :: Image of bratwurst
set "W=1920" :: Width of input video
set "H=1080" :: Height of input video
set "T=3.0" :: Time when the image shall be insert
set "OUT=out.mp4" :: Output video

ffmpeg -i %IN% -i %BW% -lavfi "[1]scale=w=%W%:h=%H%,setpts=%T%/TB[im];[0][im]overlay=eof_action=pass" -c:a copy -q:v 0

%OUT%

pause

The "scale" filter scales the image to the same size as the input video. If the image already has the correct size, you can omit this filter. The "setpts" filter
sets the time for the image. The "overlay" filter then combines the two sources. The audio track is taken unchanged from the input video.

The same thing can also be done with the freezeframes filter:
set "IN=scene8.mp4" :: Input video
set "IN2=test.mp4" :: Second input which contains the replacement frame
set "F=75" :: Number of the frame to be replaced
set "R=1" :: Number of the replacement frame from the second input

ffmpeg -i %IN% -i %IN2% -lavfi freezeframes=first=%F%:last=%F%:replace=%R% out.mp4

pause

128
2.74 Blend filter

Unfortunately the FFmpeg documentation doesn't explain what all the modes do. So you have to look it up in the source code (the filename is
"blend_modes.c"). The default mode is "normal".

Mode Function Notes

normal A * opacity + B * (1 - opacity) Output is a mix of A and B, the default opacity is 1
addition A+B Output is (A+B) with an upper limit at white level
average (A + B) / 2 Output is the arithmetic mean of A and B
subtract A- B Output is (A-B) with a lower limit at black level
multiply A* B Output is the product of A by B. Both inputs are normalized to the [0...1] range before multiplication.
divide A/ B Output is A divided by B with an upper limit at white level. The output is normalized to white level.
difference abs(A - B) Output is the absolute difference of A and B
grainextract 50%_gray_level + A - B Output is (A-B), shifted to 50% gray level, with limits at black and white levels
darken min(A,B) Output is the minimum of A and B
lighten max(A,B) Output is the maximum of A and B
and A&B Output is bitwise AND of A and B
or A| B Output is bitwise OR of A and B
xor A^B Output is bitwise XOR of A and B
screen 1 - (1 - A) * (1 - B) Output is inverse of product of inverted A by inverted B.
Might be usable for clips which have no alpha channel, black becomes transparent?
This table contains only a subset of the modes. There are more of them.
For a comparison of all blend modes, see also: [Link]
Note: When using one of the preset modes of the blend filter, don't forget to specify "all_mode", as it's not the default!
Note: The "opacity" options aren't used if a user-defined expression is used. They are only used used if one of the "mode" options is used. The default is
"all_opacity=1", which means the full blend effect is applied. A smaller opacity value means the output is mixed with the first input stream.
"all_opacity=0" means the effect is disabled and the first input stream is returned. But there is one exception from this rule: If the mode is "normal", then
"all_opacity=1" returns the first stream and "all_opacity=0" returns the second stream.

129
This batch file can be used for showing the different modes of the blend filter:
ffmpeg -f lavfi -i color=s=256x256,geq=r='H-1-Y':g='H-1-Y':b='H-1-Y' -frames 1 -y [Link]

ffmpeg -i [Link] -vf "split[a][b];[b]transpose[b];[a][b]blend=all_mode=harmonic,pseudocolor=preset=turbo" -y

[Link]

pause

See also: [Link]

Calculate the difference between two images. If the images are identical, the output is 50% gray:
ffmpeg -i [Link] -i [Link] -lavfi blend=all_mode=grainextract -y [Link]

pause

Apply a 5 second sine-shaped crossfade to two videos:

ffmpeg -f lavfi -i color=red -f lavfi -i color=yellow -lavfi blend=all_expr='A*(0.5+0.5*sin(T*2*PI/5))+B*(0.5-
0.5*sin(T*2*PI/5))' -t 30 -y out.mp4

pause

130
2.75 Circular mask (View through eyepiece)

This batch file simulates the view through an eyepiece of a telescope. The outside of the circular field of view is black.
set "IN=[Link]" :: Input video
set "SIZE=3840x2160" :: Video size
set "D=0.7" :: Circle diameter relative to image height
set "OUT=out.mp4" :: Output video

ffmpeg -f lavfi -i color=black:s=%SIZE% -lavfi format=argb,geq=a='255*gt(hypot(((2*X-W)/H),(2*Y/H)-1),%D%)':r=0:g=0:b=0

-frames 1 -y [Link]

ffmpeg -i %IN% -i [Link] -lavfi overlay=format=yuv422p10 -y %OUT%

pause
Note: format=yuv422p10 is only required for 10-bit videos. The default output format of the overlay filter is yuv420.

This batch file simulates the view through an eyepiece with an unsharp edge:
set "IN=[Link]" :: Input video
set "SIZE=3840x2160" :: Video size
set "D=0.95" :: Circle diameter, relative to image height
set "T=0.1" :: Width of smooth transition region, relative to image height
:: (can be made small, but not zero)
set "OUT=out.mp4" :: Output video

ffmpeg -f lavfi -i color=black:s=%SIZE% -lavfi format=argb,geq=a='clip(128+128/%T%*(hypot(((2*X-W)/H),(2*Y/H)-1)-%D

%),0,255)':r=0 -frames 1 -y [Link]

ffmpeg -i %IN% -i [Link] -lavfi overlay=format=yuv422p10 -y %OUT%

pause

How is the circle with the unsharp edge made?

hypot(X-x0,Y-y0) This is the distance from the center x0,y0 of the circle.

131
gt(hypot(X-x0,Y-y0),radius) If you compare with a radius (in pixels), then you get a circle with a sharp edge. This function is 0
inside and 1 outside of the circle. Multiply by 255 and you are done.
However if you want a smooth edge, you must get rid of the gt() function.
hypot(X-x0,Y-y0)-radius This is the distance of a point from the edge of the circle. This function is negative inside the
circle, it's 0 at the edge and it's positive outside of the circle.
256/width*(hypot(X-x0,Y-y0)-radius) Multiply by a factor 256 and divide by the width of the transition band (in pixels).
128+256/width*(hypot(X-x0,Y-y0)-radius) Add 128 (for middle gray). This means at the exact radius you have middle gray.
If you don't add 128, then you have black at the exact radius, and the transition is fully outside of
the circle. If you add 255, then you have white at the exact radius, and the transition is fully inside
of the circle.
clip(128+256/width*(hypot(X-x0,Y-y0)-radius),0,255) Finally you must clip to the 0...255 range (black to white). The result is a circular mask with a
smooth edge.

132
2.76 Radial attenuation (for overhead sky in a planetarium)

In a fulldome planetarium it's a good idea to avoid large white areas in videos, because the bright light is reflected from the dome and illuminates the
opposite side of the dome, where it reduces contrast. In this example is shown how the overhead sky can be attenuated with the "multiply" filter:
set "IN=[Link]" :: Input image
set "SIZE=2880" :: Image size
set "R1=500" :: Inner radius in pixels
set "R2=1000" :: Outer radius in pixels
set "DARK=90" :: Brightness level inside of R1, 0 is black, 255 is original brightness
set "BRIGHT=255" :: Brightness level outside of R2, 0 is black, 255 is original brightness

rem Create the mask file

ffmpeg -f lavfi -i nullsrc=size=%SIZE%x%SIZE% -lavfi format=gray8,geq='clip(lerp(%DARK%,%BRIGHT%,(hypot(X-%SIZE%/2,Y-

%SIZE%/2)-%R1%)/(%R2%-%R1%)),%DARK%,%BRIGHT%)',format=rgb24 -frames 1 -y [Link]

rem Apply the mask

ffmpeg -i %IN% -i [Link] -lavfi multiply=offset=0 -frames 1 -y [Link]

pause

Input image, mask and output image:

133
With the same technique it's also possible to attenuate the corners of a fisheye image:
set "IN=[Link]" :: Input image
set "SIZE=2880" :: Image size
set "R3=1300" :: Inner radius in pixels, full brightness inside of this radius
set "RAMP=200" :: Width of the ramp in pixels

rem Create the mask file

ffmpeg -f lavfi -i nullsrc=size=%SIZE%x%SIZE% -lavfi format=gray8,geq='clip(lerp(255,0,(hypot(X-%SIZE%/2,Y-%SIZE%/2)-

%R3%)/(%RAMP%)),0,255)',format=rgb24 -frames 1 -y [Link]

rem Apply the mask

ffmpeg -i %IN% -i [Link] -lavfi multiply=offset=0 -frames 1 -y [Link]

pause

Input image, mask and output image:

134
Both effects can be combined together in the same mask file:
set "IN=[Link]" :: Input image
set "SIZE=2880" :: Image size
set "R1=500" :: Inside of radius R1 the value is DARK
set "R2=1000" :: Between R1 and R2 the value is a ramp from DARK to 255
set "R3=1300" :: Between R2 and R3 the value is 255 (original brightness)
set "R4=1500" :: Between R3 and R4 the value is a ramp from 255 to 0
:: Outside of radius R4 the value is 0
set "DARK=90" :: Brightness level in the center, 0 is black, 255 is original brightness

rem Create the mask file

ffmpeg -f lavfi -i nullsrc=size=%SIZE%x%SIZE% -lavfi format=gray8,geq='st(0,hypot(X-%SIZE%/2,Y-%SIZE%/2));lte(ld(0),

%R1%)*%DARK%+bitand(gte(ld(0),%R1%),lt(ld(0),%R2%))*lerp(%DARK%,255,(ld(0)-%R1%)/(%R2%-%R1%))+bitand(gte(ld(0),
%R2%),lt(ld(0),%R3%))*255+bitand(gte(ld(0),%R3%),lt(ld(0),%R4%))*lerp(255,0,(ld(0)-%R3%)/(%R4%-%R3%))',format=rgb24
-frames 1 -y [Link]

rem Apply the mask

ffmpeg -i %IN% -i [Link] -lavfi multiply=offset=0 -frames 1 -y [Link]

pause

Input image, mask and output image:

135
2.77 Edge Attenuation

rem Create a mask file with one 50% gray line at the edges

ffmpeg -f lavfi -i color=color=white:size=1920x1080 -lavfi drawbox=color=gray:t=1 -frames 1 -y [Link]

rem Or alternatively create a mask file with two 33% and 67% gray lines at the edges

ffmpeg -f lavfi -i color=color=white:size=1920x1080 -lavfi

drawbox=color=0x555555:t=1,drawbox=color=0xAAAAAA:x=1:y=1:w=iw-2:h=ih-2:t=1 -frames 1 -y [Link]

rem Apply the mask

ffmpeg -i [Link] -i [Link] -lavfi multiply=offset=0 -frames 1 -y [Link]

pause

136
2.78 Binocular view simulation

This batch file simulates the view through a binocular:

set "IN=[Link]" :: Input video
set "SIZE=3840x2160" :: Video size
set "D=0.8" :: Circle diameter, relative to image height
set "P=0.5" :: Pupil distance, relative to image height
set "T=0.05" :: Width of smooth transition region, relative to image height
:: (can be made small, but not zero)
set "OUT=out.mp4" :: Output video

ffmpeg -f lavfi -i color=black:s=%SIZE% -lavfi format=argb,geq=a='clip(128+128/%T%*min((hypot(((2*X-W-%P%*H)/H),(2*Y/H)-

1)-%D%),(hypot(((2*X-W+%P%*H)/H),(2*Y/H)-1)-%D%)),0,255)':r=0 -frames 1 -y [Link]

ffmpeg -i %IN% -i [Link] -lavfi overlay=format=yuv422p10 -y %OUT%

pause

Note: format=yuv422p10 is only required for 10-bit videos. The default output format of the overlay filter is yuv420.
This is an output image:

137
2.79 Vignetting

Vignetting at the edge of the image can be compensated with the "vignette" filter. "mode=backward" makes the corners brighter and "mode=forward"
makes them darker. The value must be set so that the corners are neither too bright nor too dark.
Example:
ffmpeg -i [Link] -vf vignette=a=0.5:mode=backward:x0=(w-1)/2:y0=(h-1)/2 -y [Link]

pause

Note: The "a" value is clipped to the [0...pi/2] range.

Note: The default values for x0, y0 are w/2 and h/2. With these values, the vignetting effect isn't exactly centered in the frame. It's offset by 0.5 pixels.
That's why you should always use x0=(w-1)/2 and y0=(h-1)/2.

This is the formula for forward mode: output = input / cos^4(a * dist / dist_max)
This is the formula for backward mode: output = input * cos^4(a * dist / dist_max)
with a = "angle" option dist = distance from the pixel to x0,y0 dist_max = half of the image diagonal

angle Corner pixel in forward mode Corner pixel in backward mode

(input = 1) (input = 1)
PI/16 1.08 0.925
PI/8 1.37 0.723
PI/5 (default) 2.33 0.428
PI/4 4 0.25
PI/2 infinite 0

138
2.80 Subtracting a darkframe

Noise, hot pixels and amplifier glow in a low-light video can be reduced by subtracting a darkframe. Make a dark video with the same settings and at the
same temperature as your main video. The only difference is that you put the cap on the lens. Then you can average many (up to 1024) frames from the
dark video and save the darkframe lossless as 16-bit PNG:
set "DARKVID=[Link]" :: Dark video

ffmpeg -i %DARKVID% -vf "tmix=128,format=rgb48" -frames 1 -y [Link]

pause

Now you can subtract this darkframe from all frames of your video:
set "IN=[Link]" :: Input video
set "OUT=meteor-dark.mp4" :: Output video

ffmpeg -i %IN% -i [Link] -filter_complex "format=rgb48[a];[a][1]blend=all_mode=subtract" -y %OUT%

pause

139
2.81 Flatfield Correction

Most lenses have vignetting. Especially in astronomical images, vignetting must be corrected before any other image processing can be done. For this
purpose a flatfield image is taken with the same lens at the same aperture, but with a uniform white screen in front of the lens. Normally the flatfield
image is exposed at roughly 50% gray level, to avoid problems with nonlinearity near white level.

Make a flatfield video with the same lens at the same aperture as the video that you want to correct. Then you can average many (up to 1024) frames from
the flat video and save the flat lossless as 16-bit PNG:
set "FLATVID=[Link]" :: Flatfield video

ffmpeg -i %FLATVID% -vf "tmix=128,format=rgb48,eq=saturation=0" -frames 1 -y [Link]

pause

Vignetting in an astronomical image is corrected by dividing the image by the flatfield. That can be done with the blend filter. If the flatfied is used as-is
(with roughly 50% gray level), then the image is effectively multiplied by a factor 2. That's a problem because bright pixels might get clipped. To minimize
this problem, the flatfield image must be normalized as close as possible to white level.

Normalize the flatfield:

ffmpeg -i [Link] -vf drawbox=w=1:h=1:color=black,normalize -frames 1 -y normalized_flat.png

pause
Note: The "normalize" filter maps the darkest pixel to black and the brightest pixel to white. That's not what we need here. We want to map black to black
and the brightest pixel to white. That's why a trick is used: To make sure that the flat contains a black pixel, such a pixel is inserted in the top left corner
with the drawbox filter. The problem of the "drawbox" filter is that it's output is only 8-bit.

The flatfield can also be normalized by the "colorlevels" filter, which supports 16-bit output:
ffmpeg -i [Link] -vf format=rgb48,colorlevels=rimin=0:gimin=0:bimin=0:rimax=-1:gimax=-1:bimax=-1 -frames 1 -y
normalized_flat.png

pause
Note: Here an undocumented feature of the "colorlevels" filter is used. If rimax, gimax and bimax are set to -1, the brightest values from the image are
used.

140
Now you can divide the input video by the flatfield:
set "IN=[Link]" :: Input video
set "OUT=comet_corrected.mp4" :: Output video

ffmpeg -i %IN% -i normalized_flat.png -filter_complex "format=rgb48[a];[a][1]blend=all_mode=divide" -y %OUT%

pause

Same as before, but with additional contrast enhancement by a factor 2:

set "IN=[Link]" :: Input video
set "OUT=comet_corrected.mp4" :: Output video

ffmpeg -i %IN% -i normalized_flat.png -filter_complex "format=rgb48[a];[a]

[1]blend=all_mode=divide,geq=lum='clip(2*(lum(X,Y)-60),0,1023)':cr='cr(X,Y)':cb='cb(X,Y)'" -y %OUT%

pause

141
Video processing with float, work in progress:
set "IN=[Link]" :: Input video
set "FLATVID=flat_85mm_1T3_SB.mov" :: Flatfield video
set "OUT=[Link]" :: Output video
set "BP=0.23" :: Black point
set "WP=0.50" :: White point
set "GAMMA=0.5" :: Gamma
set "T=5" :: Duration

rem Average 128 frames and extract the flatfield image

ffmpeg -i %FLATVID% -vf "format=gbrpf32,tmix=128" -frames 1 -y [Link]

rem Normalize the flatfield

ffmpeg -i [Link] -vf format=gbrpf32,colorlevels=rimin=0:gimin=0:bimin=0:rimax=-1:gimax=-1:bimax=-1 -frames 1 -y
normalized_flat.png

rem Apply the flatfield correction to the video

ffmpeg -i %IN% -i normalized_flat.png -filter_complex "[0]format=gbrpf32[a];[1]format=gbrpf32[b];[a]
[b]blend=all_mode=divide,scale=1920:1080,colorlevels=rimin=%BP%:gimin=%BP%:bimin=%BP%:rimax=%WP%:gimax=%WP%:bimax=%WP
%,geq=r='pow(r(X,Y),%GAMMA%)':g='pow(g(X,Y),%GAMMA%)':b='pow(b(X,Y),%GAMMA%)'" -an -t %T% -y %OUT%

pause

Which filters support float (2023 February 7)?

blend yes
colorlevels yes
eq no
geq yes
scale yes
tmix yes

142
2.82 Histogram

This batch file generates a histogram for the R,G,B components from a video:
set "IN=MVI_2562.mov" :: Input video

ffmpeg -i %IN% -vf format=pix_fmts=rgb24,histogram=levels_mode=logarithmic -y out.mp4

pause

143
2.83 Lagfun filter

The lagfun filter makes short pulses of light appear longer, with an exponential decay curve. It's good for meteors in the night sky.
It works as follows:
The previous output frame is multiplied by the decay constant, which is in the range [0 ... 1] and a typical value is 0.95. This image is used as the next
output frame. But if a pixel in the next input frame is brighter, then the brighter value is used. So all pixels have a fast rise time constant and a slow decay
time constant. Like an oscilloscope screen with a long persistance time.
Time_constant_in_seconds = 1 / ((1 - decay) * framerate)
The time constant is the duration during which a signal drops from level 1.0 to 1 / e ≈ 0.368

rem Example for lagfun, left side of output video is without lagfun and right side is with lagfun

set "SN=1400" :: Start number

set "CONTRAST=2.0" :: Contrast im range [-1000 ... 1000], normal is 1.0
set "BRIGHT=0.22" :: Brightness in range [-1.0 ... 1.0], normal is 0.0
set "GAMMA=2.5" :: Gamma in range [0.1 ... 10.0], normal is 1.0
set "DEF=10" :: Deflicker frames
set "DECAY=0.95" :: Decay factor
set "QU=3" :: MP4 quality level, 1 is best quality, 3 is normal, 31 is strong compression
set "FPS=30" :: Output framerate
set "OUT=meteors.mp4" :: Output filename

ffmpeg -start_number %SN% -i IMG_%%[Link] ^

-filter_complex "eq=contrast=%CONTRAST%:brightness=%BRIGHT%:gamma=%GAMMA%,deflicker=size=%DEF%,split[a][b];
[b]lagfun=decay=%DECAY%[c];[a][c]hstack" -r 30 -codec:v mpeg4 -q:v %QU% -y %OUT%

pause

The lagfun filter has a "planes" option, but this option doesn't work with pixel format RGB24. You must use GBRP pixel format.
See also the workaround in the next chapter.

144
In this example the lagfun filter is only applied to the green channel:

set "IN=[Link]" :: Input video

set "DECAY=0.95" :: Decay factor
set "OUT=out.mp4" :: Output video

ffmpeg -i %IN% -vf "format=gbrp,lagfun=decay=%DECAY%:planes=1" -y %OUT%

pause

planes= color
1 green
2 blue
3 cyan
4 red
5 yellow
6 magenta
7 white

145
2.84 Deflicker a video

This is an example for the deflicker filter:

rem Make a flickering video
ffmpeg -f lavfi -i color=gray -vf geq=lum='128+mod(13*N,20):cb=128:cr=128' -t 10 -y flicker.mp4

rem Deflicker the video

ffmpeg -i flicker.mp4 -vf deflicker -y out.mp4

pause

Note: A much better deflickered result can be obtained with DaVinci Resolve (ResolveFX Revival --> Deflicker). Surprisingly, the default "Timelapse"
mode is better than the "Fluoro Light" mode, although in my example it clearly was flicker from fluorescent lights.

146
If the video was made with fluorescent light and rolling shutter, the deflickered video won't be acceptable because flicker isn't uniform over the whole
frame. In this case it helps to split the video into many horizontal stripes, deflicker all of them separately, and then stitch the deflickered stripes together.
This is an example for input size 1280x720:
set "S=10" :: number of frames for deflicker filter
set "T=6" :: duration

rem This is the simple version without stripes:

ffmpeg -i in.mp4 -lavfi deflicker=%S% -t %T% -y deflicker1.mp4

rem 2 stripes:
ffmpeg -i in.mp4 -lavfi split=2[a][b];[a]crop=[Link],deflicker=%S%[aa];[b]crop=[Link],deflicker=%S%[bb];
[aa][bb]vstack=2 -t %T% -y deflicker2.mp4

rem 4 stripes:
ffmpeg -i in.mp4 -lavfi split=4[a][b][c][d];[a]crop=[Link],deflicker=%S%[aa];[b]crop=[Link],deflicker=%S%
[bb];[c]crop=[Link],deflicker=%S%[cc];[d]crop=[Link],deflicker=%S%[dd];[aa][bb][cc][dd]vstack=4 -t %T%
-y deflicker4.mp4

rem 8 stripes:
ffmpeg -i in.mp4 -lavfi split=8[a][b][c][d][e][f][g][h];[a]crop=[Link],deflicker=%S%[aa];
[b]crop=[Link],deflicker=%S%[bb];[c]crop=[Link],deflicker=%S%[cc];[d]crop=[Link],deflicker=%S%[dd];
[e]crop=[Link],deflicker=%S%[ee];[f]crop=[Link],deflicker=%S%[ff];[g]crop=[Link],deflicker=%S%[gg];
[h]crop=[Link],deflicker=%S%[hh];[aa][bb][cc][dd][ee][ff][gg][hh]vstack=8 -t %T% -y deflicker8.mp4

rem 15 stripes:
ffmpeg -i in.mp4 -lavfi split=15[a][b][c][d][e][f][g][h][i][j][k][l][m][n][o];[a]crop=[Link],deflicker=%S%[aa];
[b]crop=[Link],deflicker=%S%[bb];[c]crop=[Link],deflicker=%S%[cc];[d]crop=[Link],deflicker=%S%[dd];
[e]crop=[Link],deflicker=%S%[ee];[f]crop=[Link],deflicker=%S%[ff];[g]crop=[Link],deflicker=%S%[gg];
[h]crop=[Link],deflicker=%S%[hh];[i]crop=[Link],deflicker=%S%[ii];[j]crop=[Link],deflicker=%S%[jj];
[k]crop=[Link],deflicker=%S%[kk];[l]crop=[Link],deflicker=%S%[ll];[m]crop=[Link],deflicker=%S%[mm];
[n]crop=[Link],deflicker=%S%[nn];[o]crop=[Link],deflicker=%S%[oo];[aa][bb][cc][dd][ee][ff][gg][hh][ii][jj]
[kk][ll][mm][nn][oo]vstack=15 -t %T% -y deflicker15.mp4

rem Compare the simple deflickered video with the 15-stripes version:
ffmpeg -i deflicker1.mp4 -i deflicker15.mp4 -lavfi hstack -y out.mp4

pause

Note: Further improvement might be possible if the RGB channels are deflickered separately, because fluorescent light does also have color flicker.
Note: Why didn't I use 16 stripes instead of 15? Because 720 / 16 = 45, that's an invalid odd number for the height of a video stream.

147
This is an example for splitting the video into 8 horizontal stripes and RBG colors and deflickering all 24 stripes separately. Unfortunately the result isn't
better than the previous example.
set "S=10" :: number of frames for deflicker filter
set "T=6" :: duration

rem This is the simple version without stripes:

ffmpeg -i in.mp4 -lavfi deflicker=%S% -t %T% -y deflicker1.mp4

rem 8 stripes rgb:

ffmpeg -i in.mp4 -lavfi format=rgb24,extractplanes=r+g+b[r][g][b];
[r]split=8[r1][r2][r3][r4][r5][r6][r7][r8];
[g]split=8[g1][g2][g3][g4][g5][g6][g7][g8];
[b]split=8[b1][b2][b3][b4][b5][b6][b7][b8];
[r1]crop=[Link],deflicker=%S%[r1d];[r2]crop=[Link],deflicker=%S%[r2d];
[r3]crop=[Link],deflicker=%S%[r3d];[r4]crop=[Link],deflicker=%S%[r4d];
[r5]crop=[Link],deflicker=%S%[r5d];[r6]crop=[Link],deflicker=%S%[r6d];
[r7]crop=[Link],deflicker=%S%[r7d];[r8]crop=[Link],deflicker=%S%[r8d];
[r1d][r2d][r3d][r4d][r5d][r6d][r7d][r8d]vstack=8[rr];
[g1]crop=[Link],deflicker=%S%[g1d];[g2]crop=[Link],deflicker=%S%[g2d];
[g3]crop=[Link],deflicker=%S%[g3d];[g4]crop=[Link],deflicker=%S%[g4d];
[g5]crop=[Link],deflicker=%S%[g5d];[g6]crop=[Link],deflicker=%S%[g6d];
[g7]crop=[Link],deflicker=%S%[g7d];[g8]crop=[Link],deflicker=%S%[g8d];
[g1d][g2d][g3d][g4d][g5d][g6d][g7d][g8d]vstack=8[gg];
[b1]crop=[Link],deflicker=%S%[b1d];[b2]crop=[Link],deflicker=%S%[b2d];
[b3]crop=[Link],deflicker=%S%[b3d];[b4]crop=[Link],deflicker=%S%[b4d];
[b5]crop=[Link],deflicker=%S%[b5d];[b6]crop=[Link],deflicker=%S%[b6d];
[b7]crop=[Link],deflicker=%S%[b7d];[b8]crop=[Link],deflicker=%S%[b8d];
[b1d][b2d][b3d][b4d][b5d][b6d][b7d][b8d]vstack=8[bb];
[gg][bb][rr]mergeplanes=0x001020:gbrp -t %T% -y deflicker8rgb.mp4

rem Compare the simple deflickered video with the 8-stripes-rgb version:
ffmpeg -i deflicker1.mp4 -i deflicker8rgb.mp4 -lavfi hstack -y out.mp4

pause

Note: Line feeds were inserted only for clarity. Of course it must all be written in one command line.

148
2.85 Star trails

The lagfun filter can also be used for making startrail videos:
rem Make a small white star
ffmpeg -f lavfi -i color=white:s=2x2 -y -frames 1 [Link]

rem Make a 10 seconds video of a moving white star over a black background
ffmpeg -f lavfi -i color=black:s=1920x1080 -loop 1 -i [Link] -lavfi overlay=x=10+190*t:y=10+100*t -t 10 -y star.mp4

rem Make a startrail video

ffmpeg -i star.mp4 -vf lagfun=decay=1 -y startrail.mp4

pause

Note: The first two command lines in this batch file are only for generating a simulated star in front of a black background. If you have a real input video,
you can directly feed it to the third command line.
If you set the decay option to 1, the trains will remain for intinite time.
If you set the value slightly smaller than 1.0, for example 0.95, then the trails will decay.

It's also possible to make star trails of finite length with the tmedian filter:
set "R=10" :: Set radius for tmedian filter

rem Make star trails of finite length

ffmpeg -i star.mp4 -vf tmedian=radius=%R%:percentile=1 -y startrail.mp4

pause

Note: The number of frames seems to be twice the number that is specified as "radius".

149
2.86 Bird trails

Paul Bourke did make a nice image of bird trails here: [Link]
It's also possible to do this with FFmpeg's lagfun filter. Because the filter works only with bright objects in front of a dark background, I'm using here a
trick: Negate the input video, apply lagfun with decay=1, then negate again.
set "S=5" :: Specify that only each n-th frame is used

rem Make a small black bird

ffmpeg -f lavfi -i color=black:s=6x6 -y -frames 1 [Link]

rem Make a 10 seconds video of a moving black bird over a white background
ffmpeg -f lavfi -i color=white:s=1920x1080 -loop 1 -i [Link] -lavfi overlay=x=10+190*t:y=10+100*t -t 10 -y bird.mp4

rem Make a bird trail video

ffmpeg -i bird.mp4 -vf select='not(mod(n,%S%))',negate,lagfun=decay=1,negate -y birdtrail.mp4

pause

Note: The first two command lines in this batch file are only for generating a simulated black bird in front of a white background. If you have a real input
video, you can directly feed it to the third command line.

A similar effect can be achieved with the "tmedian" filter, which picks the smallest pixel value out of the last n frames. In this case the trails have a finite
length. Please note that the number seems to be twice the number that you specified.
set "S=5" :: Specify that only each n-th frame is used
set "R=10" :: Set radius for tmedian filter

rem Make a bird trail video, with trails of finite length

ffmpeg -i bird.mp4 -vf select='not(mod(n,%S%))',tmedian=radius=%R%:percentile=0 -y birdtrail.mp4

pause

150
This is an example for a bird trails video from a Kodak PIXPRO SP360 4K camera:
set "IN=116_0002.mp4" :: Input video
set "N=6" :: Specify that only each n-th frame is used
set "FOV=235" :: Input field of view in degrees
set "YAW=-25" :: Yaw angle in degrees
set "PITCH=-50" :: Pitch angle in degrees
set "HFOV=60" :: Output horizontal field of viev in degrees
set "VFOV=60" :: Output vertical field of viev in degrees
set "W=800" :: Output width
set "H=800" :: Output height
set "CONTR=1.5" :: Contrast
set "BRIGHT=-0.2" :: Brightness
set "S=0" :: Start point
set "T=32" :: Duration
set "OUT=birdtrail.mp4" :: Output video

rem Make a bird trail video

ffmpeg -ss %S% -i %IN% -vf select='not(mod(n,%N%))',v360=input=fisheye:output=rectilinear:ih_fov=%FOV%:iv_fov=%FOV%:yaw=

%YAW%:pitch=%PITCH%:h_fov=%HFOV%:v_fov=%VFOV%:w=%W%:h=%H%,negate,lagfun=decay=1:planes=1,negate,eq=contrast=%CONTR
%:brightness=%BRIGHT% -t %T% -y %OUT%

pause

Note: The planes=1 option for the lagfun filter means that the filter is only applied to the luminance plane. The colors stay as they are.

151
This is an image from the output video:

152
2.87 Rainbow-trail effect

I found the original version of this effect here: [Link]

This is my version as a Windows batch file:
rem Make a 10 seconds test video of a white dot moving over a bluescreen

ffmpeg -f lavfi -i color=blue:s=1920x1080 -f lavfi -i color=white:s=60x60 -lavfi

overlay=x=960+800*sin(t):y=540+300*sin(2*t) -t 10 -y dot.mp4

rem Rainbow-trail effect

set "IN=dot.mp4" :: Input video

set "KEY=0x0000FF" :: Color key, use 0x00FF00 for greenscreen or 0x0000FF for bluescreen
set "D=0.1" :: Delay time per color
set "OUT=out.mp4" :: Output video
set "VIOLET=colorchannelmixer=[Link]"
set "INDIGO=colorchannelmixer=.[Link]"
set "BLUE=colorchannelmixer=[Link]"
set "GREEN=colorchannelmixer=[Link]"
set "YELLOW=colorchannelmixer=[Link]"
set "ORANGE=colorchannelmixer=[Link].[Link]"
set "RED=colorchannelmixer=[Link]"

ffmpeg -i %IN% -lavfi "split[a][b];[b]colorkey=%KEY%:0.3:0.1,extractplanes=a,split=7[b1][b2][b3][b4][b5][b6][b7];

[b1]%RED%,setpts=PTS+%D%*7/TB[b1];[b2]%ORANGE%,setpts=PTS+%D%*6/TB,chromakey=black:0.01:0.1[b2];[b1][b2]overlay[b1];
[b3]%YELLOW%,setpts=PTS+%D%*5/TB,chromakey=black:0.01:0.1[b3];[b1][b3]overlay[b1];[b4]%GREEN%,setpts=PTS+%D
%*4/TB,chromakey=black:0.01:0.1[b4];[b1][b4]overlay[b1];[b5]%BLUE%,setpts=PTS+%D%*3/TB,chromakey=black:0.01:0.1[b5];[b1]
[b5]overlay[b1];[b6]%INDIGO%,setpts=PTS+%D%*2/TB,chromakey=black:0.01:0.1[b6];[b1][b6]overlay[b1];[b7]%VIOLET
%,setpts=PTS+%D%/TB,chromakey=black:0.01:0.1[b7];[b1][b7]overlay[b1];[a]colorkey=%KEY%:0.4:0.1[a];[b1][a]overlay" -y
%OUT%

pause

153
2.88 Temporal slice-stacking effect

In this example the video is split into 6 horizontal slices, which are delayed by 0-5 frames. Nice effect for dancing videos.
rem Make a 10 seconds video of a white vertical bar

ffmpeg -f lavfi -i color=black:s=1920x1080 -f lavfi -i color=white:s=20x1080 -lavfi overlay=x=960+800*sin(t):y=0 -t 10

-y bar.mp4

ffmpeg -i bar.mp4 -vf "split=6[a0][a1][a2][a3][a4][a5];[a0]crop=h=ih/6:y=0[b0];[a1]setpts=PTS+1/

(FR*TB),crop=h=ih/6:y=ih/6[b1];[a2]setpts=PTS+2/(FR*TB),crop=h=ih/6:y=2*ih/6[b2];[a3]setpts=PTS+3/
(FR*TB),crop=h=ih/6:y=3*ih/6[b3];[a4]setpts=PTS+4/(FR*TB),crop=h=ih/6:y=4*ih/6[b4];[a5]setpts=PTS+5/
(FR*TB),crop=h=ih/6:y=5*ih/6[b5];[b0][b1][b2][b3][b4][b5]vstack=6" -y out.mp4

pause

For a different approach see also [Link]

The same can also be done with the tpad filter instead of setpts:
ffmpeg -i bar.mp4 -vf "split=6[a0][a1][a2][a3][a4][a5];[a0]crop=h=ih/6:y=0[b0];[a1]tpad=1,crop=h=ih/6:y=ih/6[b1];
[a2]tpad=2,crop=h=ih/6:y=2*ih/6[b2];[a3]tpad=3,crop=h=ih/6:y=3*ih/6[b3];[a4]tpad=4,crop=h=ih/6:y=4*ih/6[b4];
[a5]tpad=5,crop=h=ih/6:y=5*ih/6[b5];[b0][b1][b2][b3][b4][b5]vstack=6" -y out.mp4

pause

The main idea in the above script is to combine the video with one or more delayed versions of itself:
ffmpeg -i test.mp4 -vf "split[a][b];[b]setpts=PTS+5/(FR*TB)[c];[a][c]vstack" -y out.mp4

pause

154
Or with tpad filter:
ffmpeg -i test.mp4 -vf "split[a][b];[b]tpad=start=5:start_mode=clone[c];[a][c]vstack" -y out.mp4

pause

The two above examples consume less memory if the split filter is omitted, and instead the same input video is loaded twice:
ffmpeg -i test.mp4 -i test.mp4 -vf "[0]setpts=PTS+5/(FR*TB)[a];[a][1]vstack" -y out.mp4

pause

2.89 Extract and merge planes, split planes

Extract RGB channels, apply a filter to the G channel, then merge all channels to the output video:
set "IN=[Link]" :: Input video
set "DECAY=0.95" :: Decay factor
set "OUT=out.mp4" :: Output video

ffmpeg -i %IN% -lavfi "format=rgb24,extractplanes=r+g+b[r][g][b];[g]lagfun=decay=%DECAY%[gg];[gg][b]

[r]mergeplanes=0x001020:gbrp" -y %OUT%

pause

Use different delay for RGB planes:

set "IN=test.mp4" :: Input video
set "DELAY_R=2" :: Number of delayed frames for red
set "DELAY_G=1" :: Number of delayed frames for green
set "OUT=out.mp4" :: Output video

ffmpeg -i %IN% -lavfi "format=rgb24,extractplanes=r+g+b[r][g][b];[r]tpad=%DELAY_R%[rr];[g]tpad=%DELAY_G%[gg];[gg][b]

[rr]mergeplanes=0x001020:gbrp" -y %OUT%

pause

155
2.90 Extract the alpha channel

rem Make a short test video with alpha channel

ffmpeg -f lavfi -i testsrc2=s=1920x1080:d=3 -lavfi format=rgba,geq=a='255*gt(X,Y)':r='r(X,Y)':g='g(X,Y)':b='b(X,Y)' -c:v

prores_ks -y [Link]

rem Extract the RGB channels

ffmpeg -i [Link] -pix_fmt rgb48le -y [Link]

rem Extract the alpha channel

ffmpeg -i [Link] -lavfi extractplanes=a -y [Link]

pause

Note: alpha=0 means transparent, alpha=255 means opaque

2.91 Backgroundkey

I haven't yet figured out how the "backgroundkey" filter is expected to work. This short test doesn't give acceptable results:
ffmpeg -f dshow -video_size 1280x720 -framerate 10 -pixel_format yuyv422 -i video="BisonCam,NB Pro" -f lavfi -i
color=red:size=1280x720 -lavfi backgroundkey=threshold=0.08:similarity=0.1[a];[1][a]overlay -f sdl2 -

pause

Note: The default values are threshold=0.08, similarity=0.1, blend=0

156
2.92 Colorkey

This filter works in RGB colorspace. The color is defined as a RGB color and a similarity value.
Example for "colorkey" filter:
ffmpeg -f lavfi -i nullsrc=s=1536x512 -vf geq=r='st(0,clip(512-X,0,255)+clip(X-
1024,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)':g='st(0,lt(X,512)*clip(X,0,255)+gte(X,512)*clip(1024-
X,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)':b='st(0,lt(X,1024)*clip(X-
512,0,255)+gte(X,1024)*clip(1536-X,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)',scale=iw/4:ih/4 -frames
1 -y [Link]

ffmpeg -i [Link] -f lavfi -i color=lightblue:size=384x128 -lavfi [0]colorkey=color=orange:similarity=0.1[a];[1]

[a]overlay -frames 1 -y [Link]

pause

These are the input and output images:

Input Output

157
2.93 Chromakey

This filter works in YUV colorspace. The color is defined alternativley as a RGB or YUV color and a similarity value.
Example for "chromakey" filter:
ffmpeg -f lavfi -i nullsrc=s=1536x512 -vf geq=r='st(0,clip(512-X,0,255)+clip(X-
1024,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)':g='st(0,lt(X,512)*clip(X,0,255)+gte(X,512)*clip(1024-
X,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)':b='st(0,lt(X,1024)*clip(X-
512,0,255)+gte(X,1024)*clip(1536-X,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)',scale=iw/4:ih/4 -frames
1 -y [Link]

ffmpeg -i [Link] -f lavfi -i color=lightblue:size=384x128 -lavfi [0]chromakey=color=orange:similarity=0.1[a];[1]

[a]overlay -frames 1 -y [Link]

pause

These are the input and output images:

Input Output

158
2.94 HSVkey

In this filter the color is defined as a HSV color (Hue-Saturation-Value) and a similarity value.
Example for "hsvkey" filter:
ffmpeg -f lavfi -i nullsrc=s=1536x512 -vf geq=r='st(0,clip(512-X,0,255)+clip(X-
1024,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)':g='st(0,lt(X,512)*clip(X,0,255)+gte(X,512)*clip(1024-
X,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)':b='st(0,lt(X,1024)*clip(X-
512,0,255)+gte(X,1024)*clip(1536-X,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)',scale=iw/4:ih/4 -frames
1 -y [Link]

ffmpeg -i [Link] -f lavfi -i color=lightblue:size=384x128 -lavfi

[0]hsvkey=hue=45:sat=0.7:val=0.5:similarity=0.2[a];[1][a]overlay -frames 1 -y [Link]

pause

These are the input and output images:

Input Output

See also [Link]

159
2.95 Lumakey

In this filter the color is defined by a luminance value (threshold) and a tolerance value. The edges can be softened by a "softness" parameter.
Example for "lumakey" filter:
ffmpeg -f lavfi -i nullsrc=s=1536x512 -vf geq=r='st(0,clip(512-X,0,255)+clip(X-
1024,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)':g='st(0,lt(X,512)*clip(X,0,255)+gte(X,512)*clip(1024-
X,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)':b='st(0,lt(X,1024)*clip(X-
512,0,255)+gte(X,1024)*clip(1536-X,0,255));if(lt(Y,256),255+Y*(ld(0)/255-1),(511-Y)*ld(0)/255)',scale=iw/4:ih/4 -frames
1 -y [Link]

ffmpeg -i [Link] -f lavfi -i color=yellow:size=384x128 -lavfi

[0]lumakey=threshold=0.5:tolerance=0.02:softness=0.05[a];[1][a]overlay -frames 1 -y [Link]

pause

These are the input and output images:

Input Output

The difference between luma and luminance is described here: [Link]

160
2.96 Bluescreen / greenscreen

set "BG=[Link]" :: Background video

set "S1=10" :: Start time for background video
set "BGSAT=1.4" :: Saturation for background
set "BLUE=[Link]" :: Foreground video with blue screen
set "S2=13" :: Start time for foreground video
set "CW=800" :: Crop width
set "CH=1080" :: Corp height
set "CX=500" :: Crop X
set "CY=0" :: Crop Y
set "COLOR=0x223395" :: Measured average blue screen color in format 0xRRGGBB
set "SIM=0.11" :: Similarity for blue color
set "SC=0.35" :: Scale factor for foreground video
set "X=800" :: X Position of foreground
set "Y=310" :: Y Position of foreground
set "T=28" :: Duration
set "OUT=out.mp4" :: Output video

rem Extract an image from the bluescreen video, for measuring the average bluescreen color

ffmpeg -ss 5 -i %BLUE% -vf crop=%CW%:%CH%:%CX%:%CY% -frames 1 -y [Link]

ffmpeg -ss %S1% -i %BG% -ss %S2% -i %BLUE% -filter_complex "[0]eq=saturation=%BGSAT%[BG];[1]crop=%CW%:%CH%:%CX%:%CY

%,chromakey=%COLOR%:%SIM%,scale=iw*%SC%:ih*%SC%[FG];[BG][FG]overlay=%X%:%Y%" -t %T% -y %OUT%

pause

Note: For measuring the average color of the bluescreen, you can extract an image and save is as PNG. Open this image with Fitswork, draw a rectangle
and then make a right click in the rectangle and choose "Statistik für den Bereich anzeigen".

Note: The normalization of chromakey's "similarity" value was changed in May 2020. Old values must now be divided by sqrt(2) to get the same result as
before.
Note: It's much easier to make the greenscreen and despill process in DaVinci Recolve.
Note: There is also a "colorkey" filter which is similar to the "chromakey" filter, but works in RBG (Red-Green-Blue) range.
Note: There is also a "hsvkey" filter which is similar to the "chromakey" filter, but works in HSV (Hue-Saturation-Value) range.

161
I did try to insert an "eq=gamma=1.4" filter after the scale filter, but that didn't work. It seems that the eq filter destroys the alpha channel. This
workaround works with "alphaextract" and "alphamerge" filters. This workaround is no longer required because the "eq" filter was changed in October
2021 and does now support streams with alpha channel (the alpha channel remains unchanged):
ffmpeg -ss %S1% -i %BG% -ss %S2% -i %BLUE% -filter_complex "[0]eq=saturation=%BGSAT%[BG];[1]crop=%CW%:%CH%:%CX%:%CY
%,chromakey=%COLOR%:%SIM%,scale=iw*%SC%:ih*%SC%,format=rgba,split[FG1][FG2];[FG1]alphaextract
[A];[FG2]eq=gamma=1.4[FG3];[FG3][A]alphamerge[FG4];[BG][FG4]overlay=%X%:%Y%" -t %T% -y %OUT%

pause

Note: When the person in the bluescreen video makes fast movements, it's best to use short exposure times. Otherwise the fast moving object gets
smeared with the blue background, and in extreme cases might become so blue that it's detected as background.

Note about "alphamerge" filter: If the input video has no alpha channel, then an alpha channel is added, using the greyscale data from the second input.
If the input video has an alpha channel, then the alpha channel is replaced by the greyscale data from the second input.

See also: despill filter, colorkey filter, hsvkey filter

The documentation for the "despill" filter is rather incomplete:

This filter accepts the following options:
‘type’ Set what type of despill to use.
‘mix’ Set how spillmap will be generated.
‘expand’ Set how much to get rid of still remaining spill.
‘red’ Controls amount of red in spill area.
‘green’ Controls amount of green in spill area. Should be -1 for greenscreen.
‘blue’ Controls amount of blue in spill area. Should be -1 for bluescreen.
‘brightness’ Controls brightness of spill area, preserving colors.
‘alpha’ Modify alpha from generated spillmap.

Some more informations are available through the command ffmpeg -h filter=despill
type <int> ..FV...... set the screen type (from 0 to 1) (default green)
green 0 ..FV...... greenscreen
blue 1 ..FV...... bluescreen
mix <float> ..FV...... set the spillmap mix (from 0 to 1) (default 0.5)
expand <float> ..FV...... set the spillmap expand (from 0 to 1) (default 0)
red <float> ..FV...... set red scale (from -100 to 100) (default 0)
green <float> ..FV...... set green scale (from -100 to 100) (default -1)
blue <float> ..FV...... set blue scale (from -100 to 100) (default 0)

162
brightness <float> ..FV...... set brightness (from -10 to 10) (default 0)
alpha <boolean> ..FV...... change alpha component (default false)
This filter has support for timeline through the 'enable' option.

This is the same bluescreen example as before, with additional despill filter:

set "BG=[Link]" :: Background video

set "S1=10" :: Start time for background video
set "BGSAT=1.4" :: Saturation for background
set "BLUE=[Link]" :: Foreground video with blue screen
set "S2=13" :: Start time for foreground video
set "CW=800" :: Crop width
set "CH=1080" :: Corp height
set "CX=500" :: Crop X
set "CY=0" :: Crop Y
set "COLOR=0x223395" :: Measured average blue screen color
set "SIM=0.12" :: Similarity for blue color
set "SC=0.35" :: Scale factor for foreground video
set "D_TYPE=blue" :: Despill type, blue or green
set "D_MIX=0.7" :: Despill mix parameter
set "D_EXP=1.0" :: Despill expand parameter
set "D_BR=1.0" :: Despill brightness parameter
set "D_GREEN=0" :: Despill green parameter, must be -1 for greenscreen
set "D_BLUE=-1" :: Despill blue parameter, must be -1 for bluescreen
set "X=800" :: X Position
set "Y=310" :: Y Position
set "T=28" :: Duration
set "OUT=out.mp4" :: Output video

rem Extract an image from the bluescreen video, for measuring the average bluescreen color

rem ffmpeg -ss 10 -i %BLUE% -vf crop=%CW%:%CH%:%CX%:%CY% -frames 1 -y [Link]

ffmpeg -ss %S1% -i %BG% -ss %S2% -i %BLUE% -filter_complex "[0]eq=saturation=%BGSAT%[BG];[1]crop=%CW%:%CH%:%CX%:%CY

%,chromakey=%COLOR%:%SIM%,despill=type=%D_TYPE%:mix=%D_MIX%:expand=%D_EXP%:brightness=%D_BR
%:green=%D_GREEN%:blue=%D_BLUE%,scale=iw*%SC%:ih*%SC%[FG];[BG][FG]overlay=%X%:%Y%" -t %T% -y %OUT%

pause

163
This is an example for real-time bluescreen processing. The background video comes from a file and the foreground video comes from the Panasonic
GH5S camera via a HDMI to USB converter. I'm here using FFplay instead of FFmpeg, so that the result is visible in real time:
set "BG=[Link]" :: Background video
set "LOOP_N=50" :: Background video: Number of frames in loop
set "COLOR=0x0000ff" :: Bluescreen color
set "SIM=0.35" :: Similarity for blue color: larger value means more is recognized as background
set "SC=0.6" :: Scale factor for foreground video
set "D_TYPE=blue" :: Despill type, blue or green
set "D_MIX=0.7" :: Despill mix parameter
set "D_EXP=1.0" :: Despill expand parameter
set "D_BR=1.0" :: Despill brightness parameter
set "D_GREEN=0" :: Despill green parameter, must be -1 for greenscreen
set "D_BLUE=-1" :: Despill blue parameter, must be -1 for bluescreen
set "X=0" :: X Position
set "Y=0" :: Y Position

rem ffplay -f dshow -video_size 1920x1080 -framerate 30 -vcodec mjpeg video="USB Video"

rem ffplay -f lavfi movie=filename="video=USB Video":f=dshow

ffplay -f lavfi movie=filename="video=USB Video":f=dshow:discontinuity=0.5,scale=iw*0.5*%SC%:ih*0.5*%SC%,chromakey=

%COLOR%:%SIM%,despill=type=%D_TYPE%:mix=%D_MIX%:expand=%D_EXP%:brightness=%D_BR%:green=%D_GREEN%:blue=%D_BLUE%
[FG];movie=%BG%,loop=-1:%LOOP_N%,scale=960x540[BG];[BG][FG]overlay=%X%:%Y%

pause

Why is the "movie" source used in this example? That's because FFplay doesn't allow "filter_complex", which means you have only one input stream.
But the workaround with the "movie" source inside "-lavfi" allows multiple inputs. The drawback of this method is that you can't specify the properties of
the input device, which means you can't tell the HDMI to USB converter which size, framerate and codec it shall use. It seems it uses some default
values.

It's better to use FFmpeg with the sdl2 output devive:

ffmpeg -f dshow -video_size 1920x1080 -framerate 30 -vcodec mjpeg -i video="USB Video" -i %BG% -lavfi [0]scale=iw*0.5*
%SC%:ih*0.5*%SC%,chromakey=%COLOR%:%SIM%,despill=type=%D_TYPE%:mix=%D_MIX%:expand=%D_EXP%:brightness=%D_BR%:green=
%D_GREEN%:blue=%D_BLUE%[FG];[1]loop=-1:%LOOP_N%,scale=960x540[BG];[BG][FG]overlay=%X%:%Y%,format=rgb24 -window_x 0
-window_y 0 -f sdl2 -

Same as before, but use low-framerate uncompressed output from the HDMI to USB converter:

164
ffmpeg -f dshow -video_size 1920x1080 -framerate 5 -pixel_format yuyv422 -i video="USB Video" -i %BG% -lavfi
[0]scale=iw*0.5*%SC%:ih*0.5*%SC%,chromakey=%COLOR%:%SIM%,despill=type=%D_TYPE%:mix=%D_MIX%:expand=%D_EXP%:brightness=
%D_BR%:green=%D_GREEN%:blue=%D_BLUE%[FG];[1]loop=-1:%LOOP_N%,scale=960x540[BG];[BG][FG]overlay=%X%:%Y%,format=rgb24
-window_x 0 -window_y 0 -f sdl2 -

How does the "despill" algorithm work?

factor = (1 - spillmix) * (1 - spillexpand)

if (type == "bluescreen")
spillmap = blue - (red * spillmix + green * factor)
else
spillmap = green - (red * spillmix + blue * factor)

if (spillmap < 0)
spillmap = 0;

red = red + spillmap * (redscale + brightness)

green = green + spillmap * (greenscale + brightness)
blue = blue + spillmap * (bluescale + brightness)

if (alpha == true)
alpha = 1 - spillmap

It's difficult to understand, and it sems to be totally different from the algorithm described here (in german):
[Link]

This table shows the spillmap value for 7 input colors and different values for "mix" and "expand", for type = bluescreen and brightness = 0.
All non-zero spillmap values are marked in yellow.
"spillmap" is for the original formula: spillmap = blue - (red * spillmix + green * factor)
"spillmap2" is for a modified formula: spillmap2 = blue - (red * spillmix + blue * factor)
Differences between "spillmap" and "spillmap2" are markend with <--

Input: R=0.30 G=0.30 B=0.30 gray mix=0.00 expand=0.00 spillmap=0.00 spillmap2=0.00

Input: R=0.30 G=0.30 B=0.40 blue-gray mix=0.00 expand=0.00 spillmap=0.10 spillmap2=0.00 <--
Input: R=0.30 G=0.40 B=0.30 green-gray mix=0.00 expand=0.00 spillmap=0.00 spillmap2=0.00

165
Input: R=0.30 G=0.40 B=0.40 cyan-gray mix=0.00 expand=0.00 spillmap=0.00 spillmap2=0.00
Input: R=0.40 G=0.30 B=0.30 red-gray mix=0.00 expand=0.00 spillmap=0.00 spillmap2=0.00
Input: R=0.40 G=0.30 B=0.40 magenta-gray mix=0.00 expand=0.00 spillmap=0.10 spillmap2=0.00 <--
Input: R=0.40 G=0.40 B=0.30 yellow-gray mix=0.00 expand=0.00 spillmap=0.00 spillmap2=0.00

Input: R=0.30 G=0.30 B=0.30 gray mix=0.00 expand=0.50 spillmap=0.15 spillmap2=0.15

Input: R=0.30 G=0.30 B=0.40 blue-gray mix=0.00 expand=0.50 spillmap=0.25 spillmap2=0.20 <--
Input: R=0.30 G=0.40 B=0.30 green-gray mix=0.00 expand=0.50 spillmap=0.10 spillmap2=0.15 <--
Input: R=0.30 G=0.40 B=0.40 cyan-gray mix=0.00 expand=0.50 spillmap=0.20 spillmap2=0.20
Input: R=0.40 G=0.30 B=0.30 red-gray mix=0.00 expand=0.50 spillmap=0.15 spillmap2=0.15
Input: R=0.40 G=0.30 B=0.40 magenta-gray mix=0.00 expand=0.50 spillmap=0.25 spillmap2=0.20 <--
Input: R=0.40 G=0.40 B=0.30 yellow-gray mix=0.00 expand=0.50 spillmap=0.10 spillmap2=0.15 <--

Input: R=0.30 G=0.30 B=0.30 gray mix=0.00 expand=1.00 spillmap=0.30 spillmap2=0.30

Input: R=0.30 G=0.30 B=0.40 blue-gray mix=0.00 expand=1.00 spillmap=0.40 spillmap2=0.40
Input: R=0.30 G=0.40 B=0.30 green-gray mix=0.00 expand=1.00 spillmap=0.30 spillmap2=0.30
Input: R=0.30 G=0.40 B=0.40 cyan-gray mix=0.00 expand=1.00 spillmap=0.40 spillmap2=0.40
Input: R=0.40 G=0.30 B=0.30 red-gray mix=0.00 expand=1.00 spillmap=0.30 spillmap2=0.30
Input: R=0.40 G=0.30 B=0.40 magenta-gray mix=0.00 expand=1.00 spillmap=0.40 spillmap2=0.40
Input: R=0.40 G=0.40 B=0.30 yellow-gray mix=0.00 expand=1.00 spillmap=0.30 spillmap2=0.30

Input: R=0.30 G=0.30 B=0.30 gray mix=0.50 expand=0.00 spillmap=0.00 spillmap2=0.00

Input: R=0.30 G=0.30 B=0.40 blue-gray mix=0.50 expand=0.00 spillmap=0.10 spillmap2=0.05 <--
Input: R=0.30 G=0.40 B=0.30 green-gray mix=0.50 expand=0.00 spillmap=0.00 spillmap2=0.00
Input: R=0.30 G=0.40 B=0.40 cyan-gray mix=0.50 expand=0.00 spillmap=0.05 spillmap2=0.05
Input: R=0.40 G=0.30 B=0.30 red-gray mix=0.50 expand=0.00 spillmap=0.00 spillmap2=0.00
Input: R=0.40 G=0.30 B=0.40 magenta-gray mix=0.50 expand=0.00 spillmap=0.05 spillmap2=0.00 <--
Input: R=0.40 G=0.40 B=0.30 yellow-gray mix=0.50 expand=0.00 spillmap=0.00 spillmap2=0.00

Input: R=0.30 G=0.30 B=0.30 gray mix=0.50 expand=0.50 spillmap=0.08 spillmap2=0.08

Input: R=0.30 G=0.30 B=0.40 blue-gray mix=0.50 expand=0.50 spillmap=0.18 spillmap2=0.15 <--
Input: R=0.30 G=0.40 B=0.30 green-gray mix=0.50 expand=0.50 spillmap=0.05 spillmap2=0.08 <--
Input: R=0.30 G=0.40 B=0.40 cyan-gray mix=0.50 expand=0.50 spillmap=0.15 spillmap2=0.15
Input: R=0.40 G=0.30 B=0.30 red-gray mix=0.50 expand=0.50 spillmap=0.03 spillmap2=0.03
Input: R=0.40 G=0.30 B=0.40 magenta-gray mix=0.50 expand=0.50 spillmap=0.13 spillmap2=0.10 <--
Input: R=0.40 G=0.40 B=0.30 yellow-gray mix=0.50 expand=0.50 spillmap=0.00 spillmap2=0.03 <--

Input: R=0.30 G=0.30 B=0.30 gray mix=0.50 expand=1.00 spillmap=0.15 spillmap2=0.15

Input: R=0.30 G=0.30 B=0.40 blue-gray mix=0.50 expand=1.00 spillmap=0.25 spillmap2=0.25
Input: R=0.30 G=0.40 B=0.30 green-gray mix=0.50 expand=1.00 spillmap=0.15 spillmap2=0.15
Input: R=0.30 G=0.40 B=0.40 cyan-gray mix=0.50 expand=1.00 spillmap=0.25 spillmap2=0.25
Input: R=0.40 G=0.30 B=0.30 red-gray mix=0.50 expand=1.00 spillmap=0.10 spillmap2=0.10
Input: R=0.40 G=0.30 B=0.40 magenta-gray mix=0.50 expand=1.00 spillmap=0.20 spillmap2=0.20
Input: R=0.40 G=0.40 B=0.30 yellow-gray mix=0.50 expand=1.00 spillmap=0.10 spillmap2=0.10

166
Input: R=0.30 G=0.30 B=0.30 gray mix=1.00 expand=0.00 spillmap=0.00 spillmap2=0.00
Input: R=0.30 G=0.30 B=0.40 blue-gray mix=1.00 expand=0.00 spillmap=0.10 spillmap2=0.10
Input: R=0.30 G=0.40 B=0.30 green-gray mix=1.00 expand=0.00 spillmap=0.00 spillmap2=0.00
Input: R=0.30 G=0.40 B=0.40 cyan-gray mix=1.00 expand=0.00 spillmap=0.10 spillmap2=0.10
Input: R=0.40 G=0.30 B=0.30 red-gray mix=1.00 expand=0.00 spillmap=0.00 spillmap2=0.00
Input: R=0.40 G=0.30 B=0.40 magenta-gray mix=1.00 expand=0.00 spillmap=0.00 spillmap2=0.00
Input: R=0.40 G=0.40 B=0.30 yellow-gray mix=1.00 expand=0.00 spillmap=0.00 spillmap2=0.00

Input: R=0.30 G=0.30 B=0.30 gray mix=1.00 expand=0.50 spillmap=0.00 spillmap2=0.00

Input: R=0.30 G=0.30 B=0.40 blue-gray mix=1.00 expand=0.50 spillmap=0.10 spillmap2=0.10
Input: R=0.30 G=0.40 B=0.30 green-gray mix=1.00 expand=0.50 spillmap=0.00 spillmap2=0.00
Input: R=0.30 G=0.40 B=0.40 cyan-gray mix=1.00 expand=0.50 spillmap=0.10 spillmap2=0.10
Input: R=0.40 G=0.30 B=0.30 red-gray mix=1.00 expand=0.50 spillmap=0.00 spillmap2=0.00
Input: R=0.40 G=0.30 B=0.40 magenta-gray mix=1.00 expand=0.50 spillmap=0.00 spillmap2=0.00
Input: R=0.40 G=0.40 B=0.30 yellow-gray mix=1.00 expand=0.50 spillmap=0.00 spillmap2=0.00

Input: R=0.30 G=0.30 B=0.30 gray mix=1.00 expand=1.00 spillmap=0.00 spillmap2=0.00

Input: R=0.30 G=0.30 B=0.40 blue-gray mix=1.00 expand=1.00 spillmap=0.10 spillmap2=0.10
Input: R=0.30 G=0.40 B=0.30 green-gray mix=1.00 expand=1.00 spillmap=0.00 spillmap2=0.00
Input: R=0.30 G=0.40 B=0.40 cyan-gray mix=1.00 expand=1.00 spillmap=0.10 spillmap2=0.10
Input: R=0.40 G=0.30 B=0.30 red-gray mix=1.00 expand=1.00 spillmap=0.00 spillmap2=0.00
Input: R=0.40 G=0.30 B=0.40 magenta-gray mix=1.00 expand=1.00 spillmap=0.00 spillmap2=0.00
Input: R=0.40 G=0.40 B=0.30 yellow-gray mix=1.00 expand=1.00 spillmap=0.00 spillmap2=0.00

Even after seeing these results, it's still difficult to describe what the mix and expand parameters do:
• If mix=0, then more or less all colors are despilled (not only blue).
• If mix=1, then the expand value doesn't care.
• Useful mix values seem to be in the range 0.5 to 1.0
• Using mix=0 and expand=0 doesn't deactivate the despill filter with the original formula. But it does so with the modified formula.
• If expand=1, the results are identical for the original and the modified formula.

167
Here is the C# source code for making the above table:
using System;
using [Link];
using [Link];

namespace despill
{
public partial class Form1 : Form
{
public Form1()
{
InitializeComponent();
}

private void Form1_Load(object sender, EventArgs e)

{
block(0.0, 0.0);
block(0.0, 0.5);
block(0.0, 1.0);
block(0.5, 0.0);
block(0.5, 0.5);
block(0.5, 1.0);
block(1.0, 0.0);
block(1.0, 0.5);
block(1.0, 1.0);
}

void block(double mix, double exp)

{
desp(0.3, 0.3, 0.3, "gray ", mix, exp);
desp(0.3, 0.3, 0.4, "blue-gray ", mix, exp);
desp(0.3, 0.4, 0.3, "green-gray ", mix, exp);
desp(0.3, 0.4, 0.4, "cyan-gray ", mix, exp);
desp(0.4, 0.3, 0.3, "red-gray ", mix, exp);
desp(0.4, 0.3, 0.4, "magenta-gray", mix, exp);
desp(0.4, 0.4, 0.3, "yellow-gray ", mix, exp);
[Link]("\n");
}

void desp(double r, double g, double b, string color, double mix, double exp)

168
 {
CultureInfo invC = [Link];
[Link]("Input: ");
[Link]("R=" + [Link]("F2", invC) + " ");
[Link]("G=" + [Link]("F2", invC) + " ");
[Link]("B=" + [Link]("F2", invC) + " ");
[Link](color + " ");
[Link]("mix=" + [Link]("F2", invC) + " ");
[Link]("expand=" + [Link]("F2", invC) + " ");
double factor = (1 - mix) * (1 - exp);
double map = b - (r * mix + g * factor);
if (map < 0) map = 0;
[Link]("spillmap=" + [Link]("F2", invC) + " ");
map = b - (r * mix + b * factor);
if (map < 0) map = 0;
[Link]("spillmap2=" + [Link]("F2", invC) + " ");
[Link]("\n");
}
}
}

169
In this example I did try different values (0, 0.3, 0.5, 0.7, 1.0) for the "mix" and "expand" parameters. The "brightness" parameter was set to 0 and the
"blue" parameter was -1. My arm was moving fast in front of a bluescreen, and so it got smeared with the blue color. The three images marked in red

170
rectangles show a small improvement.
This is a test for different values of the brightness parameter (0, 1 and 2), for mix = 0.7, expand = 1.0, red = 0, green = 0, blue = -1.

171
2.97 Real-time bluescreening

This is an example of a C# program as a real-time GUI for FFmpeg. FFmpeg gets a live foreground video from the GH5S camera via HDMI to USB
converter, and the background video is looped from a file. The GUI has scrollbars for scaling and moving the foreground video, and for choosing the
parameters for colorkey and despill. The parameters can be changed in real time and are sent to the FFmpeg process via ZMQ.
Hint: If you click on "Start FFmpeg" and nothing happens, you may have forgotten to plug in the HDMI to USB converter. You must plug it in after you
have started the computer. It doesn't work if it's already plugged in when you start the computer. However it's not necessary to plug in a HDMI signal
from a camera, because the converter has a built-in test image (8 color bars).

The source code can be downloaded here: [Link]

172
As you can see in the source code, I didn't find a way to move the console window to another position (to make sure that it doesn't overlap the video
output and the GUI of this program).
Moving the video output is no problem, there are even two ways how to do it. You can use the window_x and window_y options in the FFmpeg command
line, or you can use this code:

Process[] allProcesses = [Link]("ffmpeg"); // this is unnecessary, if you already know the

IntPtr ffmpegHandle = [Link](); // process handle because you have started the process

[Link]([Link], 960, 0, 960, 540, true); // move the video output window

public class WinAPI

{
[DllImport("[Link]")]
public static extern bool MoveWindow(IntPtr hWnd, int X, int Y, int nWidth, int nHeight, bool bRepaint);
}

If you know how the programmatically move the console window, please let me know.

Note: If you want to find out to which process a window belongs, you can use the ProcessExplorer:
[Link]

According to ProcessExplorer, both FFmpeg windows (console and video output) belong to the "ffmpeg" process (it's not the "conhost" process!). There
is only one "ffmpeg" process running. I don't know how to get the handle of the console window.

But there is an easy workaround for moving the console window:

Open a console window and move it to the right of the desktop, then right click in the title line, then choose "layout" and then set the "Window position"
and uncheck the box "Let system position window". This adjustment is only required one time and now the console window will always appear at the
same position.

Note: I did try to scale the foreground video dynamically before the colorkey filter, but that didn't work. In general, Most FFmpeg filters don't support
changing the size of a video stream while it is running. In some cases it works (for example if no other filters are between "scale" and "overlay"), but in
many other cases it doesn't work.

My results of using this bluescreening program:

By far the most important thing is to set the colorkey parameters correctly: color and similarity.
The despill filter is not as important and the possible improvements are quite small. The "modify alpha" option is useless and should be deactivated. In
many cases it totally destroys the background image.
Recommended settings for despill filter: type = blue, blue = -1, alpha = false, all other options = 0. The most important parameter is "brightness".

173
2.98 Extract moving objects from a video

ffmpeg -ss 2 -i [Link] -lavfi tmix=10 -frames 1 -y [Link]

ffmpeg -i [Link] -i [Link] -i [Link] -lavfi "[0]smartblur=3,format=rgb24[blurred_ref];[1]format=rgb24,split[a]

[b];[2]format=rgb24[c],[a]smartblur=3,format=rgb24[blurred_in];[blurred_ref]
[blurred_in]blend=all_mode=difference,eq=brightness=0.4:saturation=0,eq=contrast=1000,format=rgb24[mask];[c][b]
[mask]maskedmerge,format=rgb24" -shortest -pix_fmt yuv422p -y [Link]

pause
Note: It's not working very good.
See also: [Link]
See also: [Link]

Another example with webcam input:

rem Create the background image without a person in front of it:

ffmpeg -f dshow -video_size 640x480 -framerate 30 -pixel_format yuyv422 -i video="BisonCam,NB Pro" -vf tmix=25 -frames 1
-y [Link]

rem Replace the original background by a green screen:

ffmpeg -loop 1 -i [Link] -f lavfi -i color=green:size=640x480,format=rgb24 -f dshow -video_size 640x480 -framerate 30

-pixel_format yuyv422 -i video="BisonCam,NB Pro" -lavfi [2]format=rgb24,split[cam1][cam2];
[cam1]smartblur=5,format=rgb24[blurredcam];[0]smartblur=5,format=rgb24[blurredbg];[blurredbg]
[blurredcam]blend=all_mode=difference,curves=m="0/0 .03/0 .04/1 1/1",format=gray,smartblur=5,format=rgb24[mask];[1]
[cam2][mask]maskedmerge,format=rgb24 -f sdl2 -

pause
Note: It's also not working very good.

174
2.99 Datascope

The "datascope" filter can be used to measure the RGB color components of the bluescreen. In this example the input video comes from the HDMI to
USB converter:
ffmpeg -f dshow -video_size 1920x1080 -framerate 5 -pixel_format yuyv422 -i video="USB Video" -lavfi
format=rgb24,scale=64:20,datascope=mode=color2 -f sdl -

Note: The default size seems to be 1280x720 pixels.

Note: Make sure that the pixel format is correct before using datascope. For example, the crop filter might change the pixel format to GBR, and then you
see unexpected results in the datascope output.

This is an analyzer for one pixel in the center of the field of view. The input video comes from the HDMI to USB converter:
ffmpeg -f dshow -video_size 1920x1080 -framerate 5 -pixel_format yuyv422 -i video="USB Video" -lavfi
format=rgb24,crop=1:1,datascope=s=20x36:mode=color2,scale=iw*10:ih*10:flags=gauss -f sdl -
Note: One block consisting of three hex digits has the size 20x36 pixels, which is in this example enlarged by a factor 10 to 200x360 pixels.

Datascope does enlarge each input pixel to this size in the output:

Number of planes 8-bit (2 hex digits) 8-bit (3 decimal digits) 16-bit (4 hex digits) 16-bit (5 decimal digits)
1 (gray) 20 x 12 30 x 12 40 x 12 50 x 12
3 (RGB or YUV) 20 x 36 30 x 36 40 x 36 50 x 36
4 (RGBA or YUVA) 20 x 48 30 x 48 40 x 48 50 x 48

See also: "pixscope" filter

Alternatively you can also use this color picker (although it can only grab 8-bit data from the screen): [Link]

175
2.100 Video line jitter effect

I wanted to create a special effect, adding jitter to the video lines. Like in this video at 3:40 [Link]
This is the first version:
ffmpeg -f lavfi -i testsrc2=size=vga -vf format=gray,geq=lum='lum(X-5+10*random(0),Y)' -t 3 -y out.mp4

pause

The problem is that this is a pixel-wise jitter. Each pixel gets it's own jitter value. That's not what I want. I want that all pixels in the same line get the same
random jitter value. This should be possible by setting ga seed value for the random generator. The seed value must be a function of N (frame number)
and Y (line number). This is me next (unsuccesful) test:
ffmpeg -f lavfi -i testsrc2=size=vga -vf format=gray,geq=lum='st(0,mod(0.001*(N+Y),1));lum(X-5+10*random(0),Y)' -t 3 -y
out.mp4

pause

The random() function uses the variable 0 to save it's seed value. But it seems that it's impossible to write a seed value for the random function. I don't
understand why it doesn't work.

Finally I replaced the random() function by a selfmade workaround. It's not a perfect random function, but good enough for this purpose:
ffmpeg -f lavfi -i testsrc2=size=vga -vf format=gray,geq=lum='st(0,mod(PI*(N+Y*(Y-N+PI)),1));lum(X-5+10*ld(0),Y)' -t 3
-y out.mp4

pause

176
2.101 Vertical jitter effect

This is a simulation of vertical jitter, like in a defect film projector.

set "IN=test.mp4" :: Input video
set "J=0.05" :: Maximum amplitude of vertical jitter as fraction of image height
set "F=15" :: Maximum speed of vertical jitter in pixels from one frame to the next frame
set "OUT=out.mp4" :: Output video

ffmpeg -i %IN% -lavfi crop=w=(1-%J%)*iw:h=(1-%J%)*ih:y='st(0,clip(ld(0)+2*%F%*(random(1)-0.5),0,%J%*ih))' -y %OUT%

pause

This is an interesting example because it shows that a variable can be saved with st(0, expr) and that it keeps its value from one frame to the next. It's
currently not possible to share variables between expressions.

This is another approach for creating a vertical jitter, using the "rgbashift" filter::
ffmpeg -f lavfi -i testsrc2=d=10 -lavfi sendcmd=f=[Link],rgbashift=rv=0:gv=0:bv=0 -y jitter.mp4

pause

This is the content of the file [Link]:

0 [expr] rgbashift rv 'mod(3*N,10)';
0 [expr] rgbashift gv 'mod(3*N,10)';
0 [expr] rgbashift bv 'mod(3*N,10)';

177
2.102 CRT Screen Effect

Found here: [Link]

2.103 Macroblock motion vectors

Found here: [Link]

See also: [Link]
See also: [Link]

2.104 Deblock filter

This filter removes unwanted blocking artefacts from low-quality input images or videos.

178
2.105 Gradfun filter

This filter removes unwanted banding artefacts that appear in backgrounds with a brightness gradient, especially in the sky towards the horizon.
set "IN=MVI_2562.mov" :: Input video
set "OUT=output.mp4" :: Output vodeo

ffmpeg -i %IN% -vf gradfun=3.5:8 -y %OUT%

pause

The first parameter is the strength, this is the maximum amout the filter will change any one pixel. Allowed values are from 0.51 to 64, the default value is
1.2
The second parameter is the radius, which defines the neighborhood to fit the gradient to. Accepted values are from 8 to 32, the default is 16.
Don't use this filter before lossy compression.

2.106 Dilation filter

This filter replaces each pixel by the brightest pixel in the 3x3 neighborhood. It's very useful if you have fisheye images of the night sky (taken with
Canon 6D, height 3648 pixels) and want to scale them down to height 1200 pixels (for projection in the planetarium). Scaling down would remove the
fainter stars, because each pixel in the resulting image would be the average of 3x3 pixels in the original limage. You can avoid this by using the dilation
filter prior to scaling down.
See also the "morpho" filter (which may be faster) and especially the "pixelize" filter, which has a "max" mode.

179
2.107 Morphological transforms

See also: [Link]

This is an example for morphological transforms. The structure is a 9x9 image with black background and a white circular mask:
rem Make an input image for testing

ffmpeg -f lavfi -i color=black:s=400x160,format=rgb24 -lavfi drawtext=text="Test":font="arial

black":fontcolor=white:fontsize=140:x=20:y=25,rgbashift=gh=20:bh=10:bv=10 -frames 1 -y rgb_text.png

rem Make a 9x9 structure with a white circle

ffmpeg -f lavfi -i color=black:s=9x9,format=gray8 -lavfi geq=lum='255*lte(hypot(((2*X+1)/H-1),((2*Y+1)/H-

1)),1)',format=rgb24 -frames 1 -y [Link]

rem Test all modes of the "morpho" filter

ffmpeg -i rgb_text.png -i [Link] -lavfi morpho=erode -y [Link]

ffmpeg -i rgb_text.png -i [Link] -lavfi morpho=dilate -y [Link]
ffmpeg -i rgb_text.png -i [Link] -lavfi morpho=open -y [Link]
ffmpeg -i rgb_text.png -i [Link] -lavfi morpho=close -y [Link]
ffmpeg -i rgb_text.png -i [Link] -lavfi morpho=gradient -y [Link]
ffmpeg -i rgb_text.png -i [Link] -lavfi morpho=tophat -y [Link]
ffmpeg -i rgb_text.png -i [Link] -lavfi morpho=blackhat -y [Link]

pause

180
These are the input and output images:

Structure (enlarged)

Input erode

dilate open close

gradient tophat blackhat

181
If the structure is green instead of white, then the "morpho" filter is only applied to the green channel:
ffmpeg -f lavfi -i color=black:s=400x160,format=rgb24 -lavfi drawtext=text="Test":font="arial
black":fontcolor=white:fontsize=140:x=20:y=25,rgbashift=gh=20:bh=10:bv=10 -frames 1 -y rgb_text.png

ffmpeg -f lavfi -i color=black:s=9x9,format=rgb24 -lavfi geq=g='255*lte(hypot(((2*X+1)/H-1),((2*Y+1)/H-1)),1)' -frames 1

-y [Link]

ffmpeg -i rgb_text.png -i [Link] -lavfi morpho=erode -y [Link]

pause

These are the input and output images:

Structure (enlarged)

Input erode

182
2.108 Correct the radial distortion of (fisheye-) lenses

This can be done with the "lenscorrection" filter, which has the following options:

cx Relative x coordinate of the center of distortion, in range [0...1], default is 0.5

cy Relative y coordinate of the center of distortion, in range [0...1], default is 0.5
k1 Coefficient of the quadratic correction term, in range [-1...1]. 0 means no correction, default is 0.
k2 Coefficient of the ^4 correction term, in range [-1...1]. 0 means no correction, default is 0.

The formula that generates the correction is:

r_src = r_tgt * (1 + k1 * (r_tgt / r_0)^2 + k2 * (r_tgt / r_0)^4)
where r_0 is half of the image diagonal and r_src and r_tgt are the distances from the focal point in the source and target images, respectively.

For fisheye images, it's a little bit more complicated because the coefficients k1, k2 aren't given for half of the image diagonal.
Let w and h be the dimensions of the rectangular input image.
The square of the ratio of the diagonal of the rectangular image to the diagonal of the circular fisheye image is:
ratio = (w^2 + h^2) / h^2

Let c1 and c2 be the given coefficients for the fisheye lens. Then the coefficients k1 and k2 can be calculated as follows:
k1 = c1 * ratio
k2 = c2 * ratio^2

See also [Link]

183
This is an example for the Entaniya M12 280° lens, from Paul Bourke's website:
y = 0.5229 x - 0.043 x^2 + 0.0253 x^3 – 0.0109 x^4

y is the radial coordinate in the image plane in the [0...1] range, and x is the field angle in radians. The maximum x value is FOV * PI / 360°.

Let's assume we have a perfect fisheye image and we want to simulate how this image would look like, if it was taken through the not-so-perfect fisheye
lens.
set "IN=[Link]" :: Input image (fisheye test image)
set "S=1200" :: Size of input image
set "FOV=280" :: Field of view of fisheye lens in degrees
set "A=0.5229" :: First order coefficient, for a perfectly linear fisheye lens this is (360/FOV/PI)
set "B=-0.043" :: Second order coefficient
set "C=0.0253" :: Third order coefficient
set "D=-0.0109" :: Fourth order coefficient

rem Step 1:
rem Apply the fisheye distortion Example for Entaniya M12 280° lens
rem y = A * x + B * x^2 + C * x^3 + D * x^4
rem where x is in the [0...FOV*PI/360] range

rem Create the xmap file

ffmpeg -f lavfi -i nullsrc=size=%S%x%S% -vf format=pix_fmts=gray16le,geq='st(1,hypot(X-%S%/2,Y-%S%/2)/%S%*PI*%FOV

%/180);st(2,atan2(X-%S%/2,Y-%S%/2));st(3,%A%*ld(1)+%B%*pow(ld(1),2)+%C%*pow(ld(1),3)+%D%*pow(ld(1),4));%S%/2+0.5+ld(3)*
%S%/2*sin(ld(2))' -frames 1 -y [Link]

rem Create the ymap file

ffmpeg -i [Link] -vf transpose -y [Link]

rem Apply the remap filter to the video

ffmpeg -i %IN% -i [Link] -i [Link] -lavfi "format=pix_fmts=rgb24,remap" -y [Link]

pause
Note: For all rotational-symmetric problems the ymap file can simply be generated by transposing the xmap file.

184
The inverse problem is much more complicated. Given is an image that was taken through a not-so-perfect fisheye lens, and you want to transform this
image into an undistorted fisheye image, with other words: You want to linearize it.
In this case you need the inverse of the above 4th degree function, which is very complicated to derive.
The trick is to use the root(expr, max) function. But please be warned that this is an extremely slow solution, because it requires to find the root of an
expression for each pixel.
set "IN=[Link]" :: Input image (fisheye test image)
set "S=1200" :: Size of input image
set "FOV=280" :: Field of view of fisheye lens in degrees
set "A=0.5229" :: First order coefficient, for a perfectly linear fisheye lens this is (360/FOV/PI)
set "B=-0.043" :: Second order coefficient
set "C=0.0253" :: Third order coefficient
set "D=-0.0109" :: Fourth order coefficient

rem Step 2:
rem Apply the inverse function to [Link], and then the result should be same as the original image

rem Create the xmap file

ffmpeg -f lavfi -i nullsrc=size=%S%x%S% -vf format=pix_fmts=gray16le,geq='st(1,hypot(X-%S%/2,Y-%S%/2)/%S

%*2);st(2,atan2(X-%S%/2,Y-%S%/2));st(3,root(-ld(1)+%A%*ld(0)+%B%*pow(ld(0),2)+%C%*pow(ld(0),3)+%D%*pow(ld(0),4),1));%S
%/2+0.5+ld(3)*%S%/PI/%FOV%*180*sin(ld(2))' -frames 1 -y [Link]

rem Create the ymap file

ffmpeg -i [Link] -vf transpose -y [Link]

rem Apply the remap filter to the video

ffmpeg -i [Link] -i [Link] -i [Link] -lavfi "format=pix_fmts=rgb24,remap" -y [Link]

pause

Note: It might be possible that the second argument of the root function (1) must be replaced by ( %FOV%/360*PI), I'm not sure.

185
Here are the linearization formulas for a few fisheye lenses:
Sigma 4.5mm f/2.5 180° y = 0.7159 x - 0.0048 x^2 - 0.032 x^3 + 0.0021 x^4
Sigma 8mm f/3.5 180° y = 0.723 x + 0.0252 x^2 - 0.0499 x^3 - 0.0004325 x^4
Canon 8-15mm f/4 180° @8mm y = 0.7189 x + 0.00079342 x^2 - 0.0289 x^3 - 0.001 x^4
Canon 8-15mm f/4 180° @12mm y = 0.6645 x + 0.0071 x^2 - 0.0223 x^3 - 0.0024 x^4
Meike 3.5mm 220° y = 0.6475 x - 0.002 x^2 - 0.0331 x^3 - 0.00010171 x^4
Meike 6.5mm 190° y = 0.5165 x + 0.1106 x^2 + 0.0617 x^3 - 0.0601 x^4
Laowa 4mm 210° y = 0.6241 x - 0.0252 x^2 + 0.024 x^3 - 0.0183 x^4

y is the radial coordinate in the image plane in the [0...1] range, and x is the field angle in radians. The maximum x value is FOV * PI / 360°.

Source of the linearization formulas: Paul Bourke's Website, [Link]

and [Link]

186
2.109 Curve fitting with GeoGebra

Curve fitting can be done with a CAS (Computer Algebra System), for example the "FitPoly" command in Geogebra.
GeoGebra can be used online: [Link] or as a standalone application: [Link]

[Link] Creates list of points from a list of two-element lists.

Example: PointList({{1,2},{3,4}})
[Link] Calculates the regression polynomial of degree n.
Example: FitPoly({(-1, -1), (0, 1), (1, 1), (2, 5)}, 3)
[Link] Calculates the exponential regression curve in the form aℯbx.
[Link] Calculates the logarithmic regression curve a + b ln(x).
[Link] Calculates a function of the form a bx to the points in the list.
[Link] Calculates the regression curve in the form a xb.
[Link] Calculates the regression curve in the form a + b sin (c x + d).
[Link] Calculates the y on x regression line of the points.
[Link] Calculates the x on y regression line of the points.
[Link] Calculates the regression curve in the form a / (1 + b e^(-kx)).
[Link] Calculates a linear combination of the functions that best fit the points in the list.

187
2.110 Lensfun

Lensfun is a library for correcting geometric distortion, chromatic aberration and vignetting of lenses.
The lensfun filter needs one or more database files. These are *.xml files and can be found in the data/db folder at Github:
[Link]
Copy one or more of these files to a local folder and specify the path in the "db_path" option. I did use the path c:/ffmpeg/lensfun
If this option isn't specified, lensfun will search at a default path (which I don't know).

The escaping in the path is tricky and hard to understand. The following versions are all working in a Windows batch file, however there are many more
versions that don't work:
ffmpeg -i [Link] -vf lensfun=db_path=C\\:/ffmpeg/lensfun [Link] Without " " double quotes
ffmpeg -i [Link] -vf lensfun=db_path=\'C:/ffmpeg/lensfun\' [Link] Without " " double quotes, but path encapsulated in
ffmpeg -i [Link] -vf lensfun=db_path=\'C://ffmpeg//lensfun\' [Link] single ' ' quotes
ffmpeg -i [Link] -vf lensfun=db_path=\'C:\\ffmpeg\\lensfun\' [Link]
ffmpeg -i [Link] -vf lensfun=db_path="C\\:/ffmpeg/lensfun" [Link] Path encapsulated in " " double quotes
ffmpeg -i [Link] -vf lensfun=db_path="C\\://ffmpeg//lensfun" [Link]
ffmpeg -i [Link] -vf "lensfun=db_path=C\\:/ffmpeg/lensfun" [Link] Whole filtergraph encapsulated in " " double quotes
ffmpeg -i [Link] -vf "lensfun=db_path=\'C:/ffmpeg/lensfun\'" [Link] Whole filtergraph encapsulated in " " double quotes and
ffmpeg -i [Link] -vf "lensfun=db_path=\'C://ffmpeg//lensfun\'" [Link] path encapsulated in single ' ' quotes
ffmpeg -i [Link] -vf "lensfun=db_path=\'C:\\ffmpeg\\lensfun\'" [Link]

A path must be encapsulated in " " double quotes if it contains space characters. Colons ":" must be escaped as "\:", commas "," must be escaped as "\,"
and backslashes "\" must be escaped as "\\".

If one of both of the "make" or "model" options aren't specified, the lensfun filter will print a list of all available values in the console window:
rem List all available values for "make" and "model":

ffmpeg -i [Link] -vf lensfun=db_path=C\\:/ffmpeg/lensfun -y [Link]

pause

188
If "make" and "model" are specified, but "lens_model" isn't specified, the lensfun filter will print a list of all available lenses in the console window:
rem List all available values for "lens_model":

ffmpeg -i [Link] -vf lensfun=make=Canon:model="Canon EOS 5D Mark IV":db_path=C\\:/ffmpeg/lensfun -y [Link]

pause

Some lenses may be listed multiple times. That's because they are listed in the *.xml file for several different crop factors. Unfortunately FFmpeg doesn't
list the crop factor. It's important that the crop factor of the lens' data fits to the crop factor of the camera. If in doubt, check the *.xml file with an editor.

Now if you have confirmed that data for the camera and lens is available, you can correct the distortion of an image. If it's a zoom lens, the actual focal
length must be specified, because geometric distortion of the lens depends on the actual focal length. "scale=0" means automatic scaling, so that no-
data areas at the borders are automatically cropped. "scale=1" means no scaling. "scale" is a floating point value and can also be set to other values.

rem Correct the geometrical distortion of a lens:

ffmpeg -i [Link] -vf lensfun=make=Canon:model="Canon EOS 5D Mark IV":lens_model="Canon EF 100-400mm f/4.5-5.6L IS II

USM":focal_length=100:scale=1:db_path=C\\:/ffmpeg/lensfun -y [Link]

pause

189
Distortion correction can also be made with Darktable (I didn't test it myself): [Link]

There is a "Lensfun" plugin for Gimp, but it seems to be outdated and doesn't work with the latest version of Gimp (April 2022):
[Link]

Lensfun is also available within RawTherapee: [Link]

The Lensfun database used by RawTherapee is located in this path: C:\Program Files\RawTherapee\5.8\share\lensfun
How to update the Lensfun database (this didn't work correctly when I tested it):
[Link]
More details about Lensfun in RawTherapee: [Link]

If the Adobe DNG Converter is installed, then camera profiles (*.dcp) and lens profiles (*.lcp) are available in this path:
C:\ProgramData\Adobe\CameraRaw
This is a completely different approach for distortion correction and has nothing to do with Lensfun.

Lens distortion can also be corrected in DaVinci Resolve, use the "LensDistort" node in Fusion. The various correction models are explained here:
[Link]

190
2.111 Replace the lensfun filter by the remap filter

If the lensfun filter doesn't work as expected, it's possible to replace it by the remap filter.
The Lensfun library provides many different mathematical models for lens distortion. The most important models are "ptlens" and "poly3".

The "ptlens" model uses a fourth-order polynomial:

Rd = a * Ru^4 + b * Ru^3 + c * Ru^2 + (1 - a - b - c) * Ru

Here Ru is the radius of the undistorted pixel and Rd is the radius of the distorted pixel. The radius is normalized to half of the smaller size of the image,
or with other words to the radius of a circle that fully fits inside the image. That means all pixels on this unit circle remain stationary.
Source: [Link]
Warning: It seems that Lensfun does internally use a simplified model, where the term (1 - a - b - c) is approximated by 1. This results in a slightly
different scale of the output image.

Another useful model is the “poly3” model, which is a simplified 3rd order polynomial:
Rd = k1 * Ru^3 + (1 - k1) * Ru

"poly3" is a simplified variant of the ptlens model, with a=0, b=k1 and c=0.
Source: [Link]

The coefficients for the distortion models can be found in the Lensfun database. These are *.xml files and can be found in the /data/db folder:
[Link]

191
For example, in the *.xml file the section for my Canon zoom lens looks like this (I did remove the additional chromatic aberration and vignetting data).
It's very important that the correct crop factor is listed. It must be the same crop factor as for the camera, in this case 1 for full frame:
<lens>
<maker>Canon</maker>
<model>Canon EF 100-400mm f/4.5-5.6L IS II USM</model>
<mount>Canon EF</mount>
<cropfactor>1</cropfactor>
<calibration>
<distortion model="ptlens" focal="100" a="-0.00104" b="-0.00264" c="-0.00045"/>
<distortion model="ptlens" focal="135" a="-0.0049" b="0.0167" c="-0.02249"/>
<distortion model="ptlens" focal="200" a="0.00118" b="-0.00395" c="0.00649"/>
<distortion model="ptlens" focal="300" a="-0.0007" b="0.00406" c="-0.00001"/>
<distortion model="ptlens" focal="400" a="-0.00355" b="0.01547" c="-0.00989"/>
</calibration>
</lens>

You see that the "ptlens" coefficients a, b and c do strongly depend of the focal length. In the following example I'm correcting an image that was taken at
100mm focal length:
set "IN=[Link]" :: Input image
set "SX=6720" :: Size X
set "SY=4480" :: Size Y
set "A=-0.00104" :: Distortion coefficient a for ptlens model
set "B=-0.00264" :: Distortion coefficient b for ptlens model
set "C=-0.00045" :: Distortion coefficient c for ptlens model
set "R=0.003" :: Rotation angle in radians, positive is counter-clockwise
set "OUT=[Link]" :: Output image

rem Rd = a * Ru^4 + b * Ru^3 + c * Ru^2 + (1 - a - b - c) * Ru

rem Ru is the radius of the undistorted pixel, Rd is the radius of the distorted pixel
rem The radius is normalized to half of the smaller size of the image, with other words
rem to the radius of a circle that fully fits inside the image.

rem Create the xmap file

ffmpeg -f lavfi -i nullsrc=size=%SX%x%SY% -vf format=pix_fmts=gray16le,geq='^

st(0,hypot(Y-(%SY%-1)/2,X-(%SX%-1)/2)/(0.5*%SY%));^
st(1,%R%+atan2(Y-(%SY%-1)/2,X-(%SX%-1)/2));^
st(2,%A%*pow(ld(0),4)+%B%*pow(ld(0),3)+%C%*pow(ld(0),2)+(1-%A%-%B%-%C%)*ld(0));^

192
%SX%/2+ld(2)*0.5*%SY%*cos(ld(1))' -frames 1 -y [Link]

rem Create the ymap file

ffmpeg -f lavfi -i nullsrc=size=%SX%x%SY% -vf format=pix_fmts=gray16le,geq='^

st(0,hypot(Y-(%SY%-1)/2,X-(%SX%-1)/2)/(0.5*%SY%));^
st(1,%R%+atan2(Y-(%SY%-1)/2,X-(%SX%-1)/2));^
st(2,%A%*pow(ld(0),4)+%B%*pow(ld(0),3)+%C%*pow(ld(0),2)+(1-%A%-%B%-%C%)*ld(0));^
%SY%/2+ld(2)*0.5*%SY%*sin(ld(1))' -frames 1 -y [Link]

rem Apply the remap filter to the image

ffmpeg -i %IN% -i [Link] -i [Link] -lavfi "format=pix_fmts=rgb24,remap" -frames 1 -y %OUT%

pause

This is not the fastest solution, but it works fine. The only drawback is that there is no pixel interpolation.

The scale of the resulting image is a little bit different from the result of the "lensfun" filter (with option scale=1).
After I did set scale to approximately 0.996, both images were identical. It seems that in Lensfun the term (1 - a - b - c) is approximated by 1.
Hint: In my example (1 + a + b + c) = 0.99587

If you want that all points on the unit circle remain stationary, you should use the term (1 - a - b - c).
Hovever if you want exactly the same result as with Lensfun (with scale=1), then replace the term (1 - a - b - c) by 1,
or alternatively use the term (1 - a - b - c) and set scale to (1 + a + b + c).

193
2.112 Deep-Zoom-In

still working on this chapter...

Let's assume you habe two images of the same scene, one taken with 100mm focal length and the other with 200mm.
The task is to enlarge a region of the first image so that it becomes identical to the second image. This is only possible if lens distortion is corrected
perfectly in both images. Unfortunately the correction with Lensfun isn't good enough, as the remaining errors are still visible.

set "SX=6720" :: Image size X

set "SY=4480" :: Image size Y
set "X1A=3210" :: Point A in image 1
set "Y1A=1480" ::
set "X1B=1747" :: Point B in image 1
set "Y1B=1338" ::
set "X2A=3016" :: Point A in image 2
set "Y2A=1213" ::
set "X2B=126" :: Point B in image 2
set "Y2B=911" ::

set "ZOOM=(hypot(%Y2A%-%Y2B%,%X2A%-%X2B%)/hypot(%Y1A%-%Y1B%,%X1A%-%X1B%))"
set "ROT=(atan2(%Y2A%-%Y2B%,%X2A%-%X2B%)-atan2(%Y1A%-%Y1B%,%X1A%-%X1B%))"

rem Print the zoom factor and rotation angle:

ffmpeg -f lavfi -i nullsrc=size=1x1 -vf format=gray16le,geq='print(%ZOOM%);print(%ROT%)' -frames 1 -y -f null NUL

rem Apply the zoom factor and rotation angle:

ffmpeg -i [Link] -vf zoompan=zoom='%ZOOM%':x=1682:y=880:s=%SX%x%SY% -frames 1 -y [Link]

pause

194
2.113 V360 filter for rotation of equirectangular 360° videos

This video filter converts equirectangular 360° panoramic videos between various formats, and it can also rotate them.
The default rotation order is yaw --> pitch --> roll, but can be changed by setting the "rorder" parameter. Positive yaw moves the line of sight towards the
right, positive pitch moves the line of sight up, positive roll rotates the image clockwise (or rotates the observer's head counter-clockwise).
set "IN=test1.mp4" :: Input video

ffmpeg -ss 10 -i %IN% -vf v360=yaw=0:output=e -frames 1 -y t_original.jpg

ffmpeg -ss 10 -i %IN% -vf v360=yaw=90:output=e -frames 1 -y t_yaw90.jpg
ffmpeg -ss 10 -i %IN% -vf v360=pitch=90:output=e -frames 1 -y t_pitch90.jpg
ffmpeg -ss 10 -i %IN% -vf v360=roll=90:output=e -frames 1 -y t_roll90.jpg
ffmpeg -ss 10 -i %IN% -vf v360=yaw=90:pitch=90:output=e -frames 1 -y t_yaw90_pitch90.jpg
ffmpeg -ss 10 -i %IN% -vf v360=yaw=90:roll=90:output=e -frames 1 -y t_yaw90_roll90.jpg
ffmpeg -ss 10 -i %IN% -vf v360=pitch=90:roll=90:output=e -frames 1 -y t_pitch90_roll90.jpg

pause

195
Parameters of the v360 filter:
input, output e, equirect Equirectangular projection, see [Link]
Warning: There is a small rounding error in v360 if the input is "e", but in most cases this error can be
neglected. See [Link]
c3x2 3x2 cubemap projection, also used by Youtube, see [Link]
c6x1 6x1 cubemap projection
c1x6 1x6 cubemap projection
eac Equi-angular cubemap
flat, gnomonic, rectilinear Regular video projection, see [Link]
Warning: There is a small rounding error in v360 if the input is "flat", but in most cases this error can be
neglected. See [Link]
dfisheye Double fisheye projection
barrel, fb, barrelsplit Facebook's "transform360" projection, see also [Link]
sg Stereographic fisheye projection, see [Link]
mercator Mercator projection, see [Link]
ball Ball projection, this means the 360° content of the input video is shown as a reflection on a mirror sphere.
Similar to single-fisheye with 360° field of view, but has a different mapping function:
With ball projection all points with 90° distance from the center point are mapped to the circle with 70.7%
radius, however with 360° single fisheye projection they are mapped to the circle with 50% radius.
In both cases the point with 180° distance from the center point is mapped to the edge of the circle.
hammer Hammer-Aitoff map projection, see [Link]
sinusoidal Sinusoidal map projection projection, see [Link]
fisheye Single fisheye projection (equidistant)
pannini Pannini projection (output only), see [Link]
cylindrical Cylindrical projection
Warning: There is a small rounding error in v360 if the input is "cylindrical", but in most cases this error can
be neglected. See [Link]
cylindricalea Cylindrical equal area projection, see [Link]
perspective Perspective projection, this is like watching a sphere from large distance (output only).
Warning: Don't use the v360 filter's "perspective" projection if you need mathematically correct behaviour.
The "perspective" projection depends somehow on the option "v_fov", but the exact meaning of this option
in this context is unknown. Better use the workaround with "remap" filter. I did try to reverse-engineer the

196
 source code to find out the meaning of the "v_fov" option, but this approach wasn't successful. Probably the
implementation of the "v_fov" option is wrong.
tetrahedron Tetrahedron projection
octahedron Octahedron projection
tsp Truncated square pyramid projection
he, hequirect Half equirectangular projection
equisolid Equisolid fisheye projection, see [Link]
og Orthographic fisheye projection, see [Link]
interp near, nearest Nearest neighbour interpolation
line, linear Bilinear interpolation, this is the default
cube, cubic Bicubic interpolation
lanc, lanczos Lanczos interpolation
sp16, spline16 Spline16 interpolation
gauss, gaussian Gaussian interpolation
mitchell Mitchell interpolation
w, h in pixels Width and height of the output image or video, default size depends on output format
yaw, pitch, roll in degrees Rotation angles
Warning: In the command line these are absolute angles, however if they are sent via sendcmd, then they are
interpreted as relative angles. That means you can't test the angles in the command line and then use the
same values for sendcmd. What a stupid idea!
rorder 'ypr', 'yrp', 'pyr', 'pry', 'ryp', 'rpy' Set the rotation order, default is 'ypr'
h_flip, v_flip 0, 1 Flip the output horizontally or vertically
d_flip Flip the output back / forward
ih_flip, iv_flip Flip the input horizontally or vertically
in_trans Transpose the input
out_trans Transpose the output
h_fov, v_fov, d_fov in degrees Set the horizontal, vertical or diagonal field of view for output
ih_fov, iv_fov, id_fov in degrees Set the horizontal, vertical or diagonal field of view for input
alpha_mask 0, 1 Make all unmapped pixels transparent

197
h_offset, v_offset -1 ... 1 Output horizontal and vertical off-axis offset, the default values are 0.
Note: In fact, this isn't output offset correction but input offset correction. Output offset correction isn't yet
implemented in the V360 filter. For more details see chapter "How to replace the v360 filter by the remap
filter".

Undocumented feature of the v360 filter: The top left pixel of the input video is mapped to all those pixels in the output video, which get no input data. If
you want to give the unused area a specific color, you can just fill the top left pixel of the input video with this color:
-vf drawbox=w=1:h=1:color=green,v360=...

Note: If the transformation is from "equirectangular" to "ball", you must use different coordinates for the filled pixel:
-vf drawbox=x=3*iw/4:y=ih/2:w=1:h=1:color=green,v360=input=e:output=ball...

Note: Even if the input pixel format is rgb24, the output format is gbrp which is a planar pixel format.

See also:
List of map projections: [Link]
Fisheye mapping functions: [Link]
Paul Bourke's classification of fisheye mappings: [Link]
Paul Bourke's dome projection with spherical mirror: [Link]

r = 2 * f * sin(theta / 2) equisolid
r = f * theta equidistant
These are all fisheye projections
r = 2 * f * tan(theta / 2) stereographic
r = f * sin(theta) orthographic
r = f * tan(theta) rectilinear Normal "flat" projection

Theta is the angle between the incoming ray of light and the optical axis (in radians), f is the focal length, and r is the distance of the resulting dot on the
sensor from the sensor center.

198
2.114 Using "v360" as a workaround for "rotate"

The "rotate" filter has a few issues if bilinear interpolation is enabled and/or if 14-bit or 16-bit data is used. See tickets 9767 and 9799.
ffmpeg -f lavfi -i allrgb -vf format=rgb48,rotate=0.1,format=rgb48 -frames 1 [Link]

pause

In those cases you can use the "v360" filter as a workaround. It has bilinear interpolation by default:
ffmpeg -f lavfi -i allrgb -vf format=rgb48,v360=flat:flat:roll=0.1*-180/PI:id_fov=45:d_fov=45,format=rgb48 -frames 1 -y
[Link]

pause

Note: In the "rotate" filter the unit of the angle is radians, however in the "v360" filter it's degrees, and the rotation direction is different.

2.115 Equirectangular images of the night sky

Equirectangular images of the night sky can be found here:

[Link]
[Link]
[Link]
[Link]
[Link]
[Link]
[Link]

199
2.116 Equirectangular images of the surface of earth and planets

Solar system, high resolution images: [Link]

Mercury: [Link]
Earth without clouds: [Link]
Earth with clouds, video: [Link]
Earth with clouds, timelapse video over a whole year, without darkness on the night side: [Link]
Moon: [Link]
[Link]
Mars: [Link]

200
2.117 Undo spherical rotations

If you want to undo spherical rotations, you must apply the same rotation angles with different sign, and you must apply the rotations in the reverse
order. For example, this spherical rotation
v360=[Link]yaw=10:pitch=20:roll=30

can be undone by this spherical rotation:

v360=[Link]yaw=-10:pitch=-20:roll=-30:rorder=rpy

Note: It doesn't care in which order you write the angles in the command line. The rotation order is only determined by the "rorder" option. The default
rotation order is yaw-pitch-roll.
Note: The order of the rotations is important. We are talking about a spherical coordinate system, like the geographical coordinates on earth.
Case 1: You walk 1km north, then turn 90° left and walk 1km to the west.
Case 2: You walk 1km to the west, then turn 90° right and walk 1km north.
You might think that in both cases you arrive at the same place. But that's wrong. It becomes clear when you walk longer distances:
Case 3: You start at the equator in Africa and walk north until you reach the north pole, then turn 90° left and walk the same distance. You will arrive
somewhere in central America.
Case 4: You start at the equator in Africa and walk west until you are in central America, then turn 90° right and walk the same distance. You will arrive at
the north pole.
That means the order of rotations is important. When you have made yaw and pitch rotations (in this order) and want to undo these rotations, you must
undo them in reverse order. First pitch and then yaw.

See also: [Link]

201
2.118 Remap a fisheye video to an equirectangular video

In this example the xmap and ymap files for the remap filter are created by FFmpeg (no C# code required).
The size of the equirectangular video is defined by the user and can be different from 2:1.
set "IN=110_0001.mp4" :: Input video
set "SQ=2880" :: Size of square input video
set "SR=1440" :: Radius that is actually used from the source video, must be SQ/2 or smaller
set "PW=1920" :: Width of panorama video
set "PH=550" :: Height of panorama video
set "OUT=out.mp4" :: Output video

rem Create the xmap file

ffmpeg -f lavfi -i nullsrc=size=%PW%x%PH% -vf format=pix_fmts=gray16le,geq='%SQ%/2-Y*%SR%/%PH%*sin(X*2*PI/%PW%)' -frames

1 -y [Link]

rem Create the ymap file

ffmpeg -f lavfi -i nullsrc=size=%PW%x%PH% -vf format=pix_fmts=gray16le,geq='%SQ%/2-Y*%SR%/%PH%*cos(X*2*PI/%PW%)' -frames

1 -y [Link]

rem Apply the remap filter to the video

ffmpeg -i %IN% -i [Link] -i [Link] -lavfi "format=pix_fmts=rgb24,remap" -c:v mpeg4 -q:v 2 -y %OUT%

pause

If the fisheye lens has more than 180° field of view, but you want only 180° visible in the panorama, set the SR variable to a value smaller than SQ/2.

A lot of informations about fisheye projections can be found on Paul Bourke's website: [Link]/dome/

More informations about the remap filter can be found here: [Link]

The color of unmapped pixels can be specified with the "fill" option, the default color is black.

The remap filter uses framesync, which by default repeats the last frame of the secondary input if the primary input is still alive. That's why "-loop 1" is
not required before the mapping file inputs.

202
What's better, -loop 1 before the input files, or "loop" filter in the filtergraph? For single image input, filter looping is better as it avoids file I/O and repeat
decoding of the input. (Source: Gyan Doshi in FFmpeg user list, June 16, 2022)

Fisheye input (from Kodak Pixpro SP360 camera): Panorama output:

The VLC player won't recognize the output video as a spherical equirectangular video, because some special metadata is missing.
This metadata can't be inserted with FFmpeg, but it can be done with the "Spatial Media Metadata Injector":
[Link]

203
In this example the fisheye's field of view can be set to any value up to 360°, and the width/height ratio of the equirectangular output video is always 2:1.
The lower part is filled with black if the fisheye has less than 360° field of view.

set "IN=IMG_077.jpg" :: Fisheye input image or video, must be square

set "SQ=3648" :: Size of square fisheye input image
set "FOV=220" :: Fisheye field of view in degrees
set "Q=2" :: Size divider for output image, use 1 for best quality,
:: or a bigger value for faster computing
set /a "H=%SQ%/%Q%" :: Height of equirectangular image
set /a "W=2*%H%" :: Width of equirectangular image is always twice the height
set /a "A=%H%*%FOV%/360" :: Height of equirectangular image that is actually filled with data,
:: the lower part of the output image remains black
set "OUT=[Link]" :: Equirectangular output image or video

rem Create the xmap file for remapping from fisheye to equirectangular

ffmpeg -f lavfi -i nullsrc=size=%W%x%H% -vf format=pix_fmts=gray16le,^

geq='%SQ%/2*(1-Y/%A%*sin(X*2*PI/%W%))' -frames 1 -y [Link]

rem Create the ymap file for remapping from fisheye to equirectangular

ffmpeg -f lavfi -i nullsrc=size=%W%x%H% -vf format=pix_fmts=gray16le,^

geq='%SQ%/2*(1-Y/%A%*cos(X*2*PI/%W%))' -frames 1 -y [Link]

rem Remap from fisheye to equirectangular

ffmpeg -i %IN% -i [Link] -i [Link] -filter_complex "format=pix_fmts=rgb24,remap" -y %OUT%

pause

204
For a square 180° single-fisheye video the conversion to an equirectangular video can also be done with the V360 filter. The second hemisphere is filled
with a user-defined color. This example is obsolete, please use the next example.
set "IN=in.mp4" :: Fisheye input video (square, camera pointing upwards)
set "OUT=out.mp4" :: Equirectangular output video

ffmpeg -i %IN% -lavfi "pad=w=2*iw:color=darkgreen,v360=input=dfisheye:output=e:pitch=90" -y %OUT%

pause

Square single-fisheye images or videos with any field of view can be converted to equirectangular images or videos:
set "IN=[Link]" :: Input image or video
set "FOV=180" :: Input field of view in degrees
set "C=green" :: Color for filling unused area
set "OUT=[Link]" :: Equirectangular output image or video

ffmpeg -i %IN% -vf drawbox=w=1:h=1:color=%C%,v360=input=fisheye:ih_fov=%FOV%:iv_fov=%FOV%:output=equirect:pitch=-90 -y

%OUT%

pause

Note: For image output, add "-frames 1"

Note: If required, the lower part of the equirectangular output can be cut off with the crop filter.

205
2.119 Remap an equirectangular video to a fisheye video

The field of view can be set between 1 and 360 degrees. The sky is in the center of the fisheye video, and the ground is at the circular edge.
The input video must have 2:1 width/height ratio.
set "IN=test1.mp4" :: Input video
set "H=960" :: Height of equirectangular input video
set "S=1080" :: Size of square fisheye output video
set "FOV=220" :: Set the field of view in degrees
set "OUT=fish.mp4" :: Output video

rem Create the xmap file

ffmpeg -f lavfi -i nullsrc=size=%S%x%S% -vf format=pix_fmts=gray16le,^

geq='%H%*(0.9999+atan2(X-%S%/2,Y-%S%/2)/PI)' -frames 1 -y [Link]

rem Create the ymap file

ffmpeg -f lavfi -i nullsrc=size=%S%x%S% -vf format=pix_fmts=gray16le,^

geq='%H%/360*%FOV%*(hypot((2*X/%S%)-1,(2*Y/%S%)-1))' -frames 1 -y [Link]

rem Apply the remap filter to the video

ffmpeg -i %IN% -i [Link] -i [Link] -lavfi "format=pix_fmts=rgb24,remap" -q:v 2 -y %OUT%

pause

The same thing can also be done with the v360 filter:
set "IN=[Link]" :: Input image
set "FOV=220" :: Output field of view in degrees
set "OUT=[Link]" :: Output image

ffmpeg -i %IN% -vf v360=input=e:fisheye:h_fov=%FOV%:v_fov=%FOV%:pitch=90 -y %OUT%

pause

206
Unfortunately the v360 filter with fisheye output fills not only the image circle with data, but instead the whole quadratic image. The workaround is to
overlay a circular mask:
set "IN=[Link]" :: Input image
set "SIZE=1200x1200" :: Size of the mask image
set "FOV=180" :: Output field of view in degrees
set "OUT=[Link]" :: Output image

ffmpeg -f lavfi -i color=black:s=%SIZE% -lavfi format=argb,geq=a='255*gt(hypot(((2*X+1)/H-1),((2*Y+1)/H-

1)),1)':r=0:g=0:b=0 -frames 1 -y [Link]

ffmpeg -i %IN% -i [Link] -lavfi v360=e:fisheye:h_fov=%FOV%:v_fov=%FOV%:pitch=90,overlay -y %OUT%

pause

207
2.120 Realtime remapping from equirectangular to fisheye

This example is for a computer with two or more monitors. That means the desktop is wider than the monitor. It's assumed that the main monitor
contains an equirectangular content (for example SpaceEngine). This content is grabbed by the "gdigrab" input device, converted to 180° fisheye format
and then shown in the center of the other monitor (or beamer).
ffmpeg -f gdigrab -video_size 1920x1080 -offset_x 0 -offset_y 0 -i desktop -lavfi
v360=e:fisheye:ih_fov=360:iv_fov=180:h_fov=180:v_fov=180:w=1080:h=1080,format=bgra -window_borderless 1 -window_x 2340
-window_y 0 -f sdl2 -

pause

Note: "-offset_x 0 -offset_y 0" can be omitted because the top left corner is the default, but "-video_size 1920x1080" is important because in
a multi-monitor system the desktop is larger than the monitor, however we want to grab only the first screen.
Note: You can see the framerate at the bottom of the console window.
Note: The "format=bgra" conversion is probably required because that's the screen pixel format which is set in the operating system. "rgba" does also
work. "rgb0" or "bgr0" can also be used. These are pixel formats without alpha channel, but they are a little bit slower than "bgra".

The above example is running on my computer with 20fps and can be improved to 27fps by grabbing only the square central part of the equirectangular
input video:
ffmpeg -f gdigrab -video_size 1080x1080 -offset_x 420 -i desktop -lavfi
v360=e:fisheye:ih_fov=180:iv_fov=180:h_fov=180:v_fov=180:w=1080:h=1080,format=bgra -window_borderless 1 -window_x 2340
-window_y 0 -f sdl2 -

pause

Note: The v360 filter becomes faster if interpolation is deactivated by using "interp=near".

208
The remap filter is faster than the v360 filter. This example runs with 30fps (the mapping files are simple identity maps, but that shouldn't affect the
speed):
rem Create the xmap file (this is a simple identity map only for testing)

ffmpeg -f lavfi -i nullsrc=size=1080x1080 -vf format=gray16,geq='X' -frames 1 -y [Link]

rem Create the ymap file (this is a simple identity map only for testing)

ffmpeg -f lavfi -i nullsrc=size=1080x1080 -vf format=gray16,geq='Y' -frames 1 -y [Link]

ffmpeg -f gdigrab -video_size 1080x1080 -offset_x 420 -i desktop -i [Link] -i [Link] -lavfi fps=30,remap
-window_borderless 1 -window_x 2340 -window_y 0 -f sdl2 -

pause

Note: Before I inserted the "fps" filter, there were numerous errors "Application provided invalid, non-monotonically increasing dts to muxer". It wasn't
possible to remove them with the bitstream filter -bsf:v setts=dts=DTS-STARTDTS. But with fps=30 it works fine. But now you can't see any more the
maximum possible framerate in the console window.
Note: Bitstream filters transform encoded media data without decoding it.

209
This is an example with fisheye to equirectangular conversion (wrong direction...)
rem Create the xmap file

ffmpeg -f lavfi -i nullsrc=size=1080x1080 -vf format=pix_fmts=gray16le,geq='st(0,PI/2*(X/539.5-1));st(1,PI/2*(Y/539.5-

1));st(2,cos(ld(1))*cos(ld(0)));st(3,cos(ld(1))*sin(ld(0)));st(4,sin(ld(1)));st(5,ld(2));st(6,ld(3));st(7,ld(4));st(8,hy
pot(ld(6),ld(7)));st(9,atan(ld(8)/ld(5))/(PI/2));539.5+539.5*ld(6)/ld(8)*ld(9)' -frames 1 -y [Link]

rem Create the ymap file

ffmpeg -f lavfi -i nullsrc=size=1080x1080 -vf format=pix_fmts=gray16le,geq='st(0,PI/2*(X/539.5-1));st(1,PI/2*(Y/539.5-

1));st(2,cos(ld(1))*cos(ld(0)));st(3,cos(ld(1))*sin(ld(0)));st(4,sin(ld(1)));st(5,ld(2));st(6,ld(3));st(7,ld(4));st(8,hy
pot(ld(6),ld(7)));st(9,atan(ld(8)/ld(5))/(PI/2));539.5+539.5*ld(7)/ld(8)*ld(9)' -frames 1 -y [Link]

rem Realtime remapping

ffmpeg -f gdigrab -video_size 1080x1080 -offset_x 420 -i desktop -i [Link] -i [Link] -lavfi fps=30,remap
-window_borderless 1 -window_x 2340 -window_y 0 -f sdl2 -

pause

210
This is an example for half-equirectangular (960x1080) to 180° fisheye (1080x1080) conversion. The input is the 1920x1080 screen from SpaceEngine.
because the aspect ratio isn't 2:1, only the central 960x1080 rectangle is grabbed. The output can be sent to a beamer with 180° fisheye lens. The
framerate is about 30fps on my notebook.
rem From equirectangular to fisheye

set "IN_W=960" :: Input width in pixels

set "IN_H=1080" :: Input height in pixels
set "IN_H_FOV=180" :: Input horizontal field of view in degrees
set "IN_V_FOV=180" :: Input vertical field of view in degrees
set "OUT_W=1080" :: Output width in pixels
set "OUT_H=1080" :: Output height in pixels
set "OUT_H_FOV=180" :: Output horizontal field of view in degrees
set "OUT_V_FOV=180" :: Output vertical field of view in degrees

rem Create the xmap file

ffmpeg -f lavfi -i nullsrc=size=%OUT_W%x%OUT_H% -vf format=pix_fmts=gray16le,geq='st(0,%OUT_H_FOV%/180*(2*X/(%OUT_W%-1)-

1));st(1,%OUT_V_FOV%/180*(2*Y/(%OUT_H%-1)-1));st(2,atan2(ld(1),ld(0)));st(3,PI/2*(1-
hypot(ld(0),ld(1))));st(4,cos(ld(3))*cos(ld(2)));(%IN_W%-1)/2*(1+atan2(ld(4),sin(ld(3)))/PI*360/%IN_H_FOV%)' -frames 1
-y [Link]

rem Create the ymap file

ffmpeg -f lavfi -i nullsrc=size=%OUT_W%x%OUT_H% -vf format=pix_fmts=gray16le,geq='st(0,%OUT_H_FOV%/180*(2*X/(%OUT_W%-1)-

1));st(1,%OUT_V_FOV%/180*(2*Y/(%OUT_H%-1)-1));st(2,atan2(ld(1),ld(0)));st(3,PI/2*(1-
hypot(ld(0),ld(1))));st(4,asin(cos(ld(3))*sin(ld(2))));(%IN_H%-1)/2*(1+ld(4)/PI*360/%IN_V_FOV%)' -frames 1 -y [Link]

rem Apply the remap filter in realtime

ffmpeg -f gdigrab -video_size %IN_W%x%IN_H% -offset_x 480 -i desktop -i [Link] -i [Link] -lavfi fps=30,remap
-window_borderless 1 -window_x 2400 -window_y 0 -f sdl2 -

pause

211
2.121 How to replace the v360 filter by the remap filter

In some cases it's better to make spherical transformations with the "remap" filter instead of "v360" filter, for example if input or output formats or
special effects are required that aren't available in "v360", or as a workaround for some errors in "v360". Also the "remap" filter is faster than "v360",
although the speed improvement is quite small if the v360 filter is used with the "interp=near" option. The following formulae have been carefully tested
for correct rounding and absence of +-1 errors.

Comments For which input / Command line in batch file

output format is it?
Specify the parameters for For all formats set "IN_W=1080" :: Input width in pixels
the spherical set "IN_H=1080" :: Input height in pixels
transformation. set "IN_H_FOV=180" :: Input horizontal field of view in degrees
Of course, you only have to set "IN_V_FOV=180" :: Input vertical field of view in degrees
specify those parameters set "OUT_W=1080" :: Output width in pixels
set "OUT_H=1080" :: Output height in pixels
that are actually used.
set "OUT_H_FOV=180" :: Output horizontal field of view in degrees
set "OUT_V_FOV=180" :: Output vertical field of view in degrees
set "YAW=0" :: Yaw angle in degrees
set "PITCH=0" :: Pitch angle in degrees
set "ROLL=0" :: Roll angle in degrees
set "OFF_IN_X=0" :: Input X offset, normalized to the dome's radius
set "OFF_IN_Y=0" :: Input Y offset, normalized to the dome's radius
set "OFF_IN_Z=0" :: Input Z offset, negative is closer to dome's vertex
set "OFF_OUT_X=0" :: Output X offset, normalized to the dome's radius
set "OFF_OUT_Y=0" :: Output Y offset, normalized to the dome's radius
set "OFF_OUT_Z=0" :: Output Z offset, negative is closer to dome's vertex
set "LENS_RAD=60" :: Radius of forward enlarging effect in degrees (1-90°)
set "LENS_PWR=0" :: Power of forward enlarging effect, 0 = neutral
set "DIST=2" :: Distance from camera to center of sphere, normalized
:: to the sphere's radius, must be larger than 1
set "ELLIPSE=0.5" :: Ellipsoid distortion factor
The size of the xmap and For all formats rem Create the xmap and ymap files
ymap files is the size of the
output image. The size of ffmpeg -f lavfi -i nullsrc=size=%OUT_W%x%OUT_H% -vf
the input image may be format=pix_fmts=gray16le,geq='^
different.

212
Convert from pixel For equirectangular st(0,PI/360*%OUT_H_FOV%*((2*X+1)/%OUT_W%-1));^
coordinates to X,Y,Z space, output st(1,PI/360*%OUT_V_FOV%*((2*Y+1)/%OUT_H%-1));^
depending on the desired (see equirect_to_xyz st(4,cos(ld(1))*sin(ld(0)));^
output format. in v360.c) st(5,sin(ld(1)));^
st(6,cos(ld(1))*cos(ld(0)));^
The X,Y,Z output For fisheye st(0,%OUT_H_FOV%/180*((2*X+1)/%OUT_W%-1));^
coordinates are saved in (equidistant) output st(1,%OUT_V_FOV%/180*((2*Y+1)/%OUT_H%-1));^
variables 4,5,6. (see fisheye_to_xyz st(2,atan2(ld(1),ld(0)));^
in v360.c) st(3,PI/2*hypot(ld(0),ld(1)));^
Note: If the pixel is st(4,sin(ld(3))*cos(ld(2)));^
unmapped, variable (9) is st(5,sin(ld(3))*sin(ld(2)));^
set to OUT_H which means st(6,cos(ld(3)));^
st(9,if(lt(ld(6),cos(%OUT_H_FOV%/360*PI)),%OUT_H%,0));^
out of range. The line
st(9, ...) can be omitted if the For nonlinear fisheye st(0,%OUT_H_FOV%/180*((2*X+1)/%OUT_W%-1));^
color of unmapped pixels output according to st(1,%OUT_V_FOV%/180*((2*Y+1)/%OUT_H%-1));^
doesn't care. Paul Bourke's st(2,atan2(ld(1),ld(0)));^
formula, see st(3,PI/2*hypot(ld(0),ld(1)));^
[Link] st(9,if(gt(ld(3),PI/2*%OUT_H_FOV%/180),%OUT_H%,0));^
st(3,%A%*ld(3)+%B%*pow(ld(3),2)+%C%*pow(ld(3),3)+%D%*pow(ld(3),4));^
[Link]/dome/fisheyeco
st(3,ld(3)*%OUT_H_FOV%/360*PI);^
rrect/ st(4,sin(ld(3))*cos(ld(2)));^
Set the parameters A, st(5,sin(ld(3))*sin(ld(2)));^
B, C, D and the st(6,cos(ld(3)));^
correct field of view.
For double fisheye st(0,%OUT_H_FOV%/180*((4*if(lt(X,%OUT_W%/2),X,%OUT_W%-1-X)+1)/%OUT_W%-1));^
(equidistant) output st(1,%OUT_V_FOV%/180*((2*Y+1)/%OUT_H%-1));^
(see dfisheye_to_xyz st(2,atan2(ld(1),ld(0)));^
in v360.c) st(3,PI/2*hypot(ld(0),ld(1)));^
st(9,if(gt(ld(3),PI/2),%OUT_H%,0));^
if(gte(X,%OUT_W%/2),st(3,PI-ld(3)));^
st(4,sin(ld(3))*cos(ld(2)));^
st(5,sin(ld(3))*sin(ld(2)));^
st(6,cos(ld(3)));^
For flat (rectilinear) st(0,tan(%OUT_H_FOV%/360*PI)*((2*X+1)/%OUT_W%-1));^
output st(1,tan(%OUT_V_FOV%/360*PI)*((2*Y+1)/%OUT_H%-1));^
(see flat_to_xyz in st(6,1/sqrt(1+ld(0)*ld(0)+ld(1)*ld(1)));^
v360.c) st(4,ld(0)*ld(6));^
st(5,ld(1)*ld(6));^

213
For cylindrical output st(0,PI/360*%OUT_H_FOV%*((2*X+1)/%OUT_W%-1));^
(see st(1,atan(tan(%OUT_V_FOV%/360*PI)*((2*Y+1)/%OUT_H%-1)));^
cylindrical_to_xyz in st(4,cos(ld(1))*sin(ld(0)));^
v360.c) st(5,sin(ld(1)));^
st(6,cos(ld(1))*cos(ld(0)));^
For planet output, st(0,(2*X+1)/%OUT_W%-1);^
this is a sphere seen st(1,(2*Y+1)/%OUT_H%-1);^
from infinite distance. st(4,ld(0));^
Exactly half of the st(5,ld(1));^
sphere is visible. st(6,sqrt(1-ld(4)*ld(4)-ld(5)*ld(5)));^
st(9,if(gt(ld(4)*ld(4)+ld(5)*ld(5),1),%OUT_H%,0));^
It's the same as
orthographic output
with 180° field of
view.
For planet output, st(0,((2*X+1)/%OUT_W%-1));^
this is a sphere seen st(1,((2*Y+1)/%OUT_H%-1));^
from the finite st(2,hypot(ld(0),ld(1)));^
distance %DIST%. st(9,if(gt(ld(2),1),%OUT_H%,0));^
Less than half of the st(2,ld(2)*tan(asin(1/%DIST%)));^
st(3,1+ld(2)*ld(2)*(1-%DIST%*%DIST%));^
sphere is visible.
st(2,2*atan((1-sqrt(abs(ld(3))))/((1+%DIST%)*ld(2))));^
(The algorithm st(3,atan2(ld(0),ld(1)));^
perspective_to_xyz in st(4,sin(ld(2))*sin(ld(3)));^
v360.c is probably st(5,sin(ld(2))*cos(ld(3)));^
wrong) st(6,cos(ld(2)));^
For mirrorsphere st(0,((2*X+1)/%OUT_W%-1));^
output, this is a st(1,((2*Y+1)/%OUT_H%-1));^
reflecting sphere st(9,if(gt(hypot(ld(0),ld(1)),1),%OUT_H%,0));^
seen from infinite st(2,2*asin(clip(hypot(ld(0),ld(1)),0,1)));^
distance. st(3,atan2(ld(0),ld(1)));^
st(4,sin(ld(2))*sin(ld(3)));^
(see ball_to_xyz in
st(5,sin(ld(2))*cos(ld(3)));^
v360.c) st(6,cos(ld(2)));^

214
For mirrorsphere st(0,((2*X+1)/%OUT_W%-1));^
output, this is a st(1,((2*Y+1)/%OUT_H%-1));^
reflecting sphere st(2,hypot(ld(0),ld(1)));^
seen from the finite st(9,if(gt(ld(2),1),%OUT_H%,0));^
distance %DIST%. st(2,ld(2)*tan(asin(1/%DIST%)));^
st(3,1+ld(2)*ld(2)*(1-%DIST%*%DIST%));^
This isn't
st(2,4*atan((1-sqrt(abs(ld(3))))/((1+%DIST%)*ld(2))));^
implemented in the st(3,atan2(ld(0),ld(1)));^
v360 filter. st(4,sin(ld(2))*sin(ld(3)));^
st(5,sin(ld(2))*cos(ld(3)));^
st(6,cos(ld(2)));^
For equisolid output, st(0,sin(%OUT_H_FOV%/720*PI)*((2*X+1)/%OUT_W%-1));^
(see equisolid_to_xyz st(1,sin(%OUT_V_FOV%/720*PI)*((2*Y+1)/%OUT_H%-1));^
in v360.c) st(2,atan2(ld(1),ld(0)));^
st(3,hypot(ld(0),ld(1)));^
st(3,if(lt(ld(3),1),2*asin(ld(3)),PI));^
st(9,if(gt(ld(3),%OUT_H_FOV%/360*PI),%OUT_H%,0));^
st(4,sin(ld(3))*cos(ld(2)));^
st(5,sin(ld(3))*sin(ld(2)));^
st(6,cos(ld(3)));^
For stereographic st(0,tan(%OUT_H_FOV%/720*PI)*((2*X+1)/%OUT_W%-1));^
output, st(1,tan(%OUT_V_FOV%/720*PI)*((2*Y+1)/%OUT_H%-1));^
(see st(2,atan2(ld(1),ld(0)));^
stereographic_to_xyz st(3,hypot(ld(0),ld(1)));^
in v360.c) st(3,2*atan(ld(3)));^
st(9,if(gt(ld(3),%OUT_H_FOV%/360*PI),%OUT_H%,0));^
st(4,sin(ld(3))*cos(ld(2)));^
st(5,sin(ld(3))*sin(ld(2)));^
st(6,cos(ld(3)));^
For orthographic st(0,sin(%OUT_H_FOV%/360*PI)*((2*X+1)/%OUT_W%-1));^
output, st(1,sin(%OUT_V_FOV%/360*PI)*((2*Y+1)/%OUT_H%-1));^
(see st(2,atan2(ld(1),ld(0)));^
orthographic_to_xyz st(3,hypot(ld(0),ld(1)));^
in v360.c) st(3,if(lt(ld(3),1),asin(ld(3)),PI));^
st(9,if(gt(ld(3),%OUT_H_FOV%/360*PI),%OUT_H%,0));^
st(4,sin(ld(3))*cos(ld(2)));^
st(5,sin(ld(3))*sin(ld(2)));^
st(6,cos(ld(3)));^

215
Optional check for For all formats if(gt(abs(1-sqrt(ld(4)*ld(4)+ld(5)*ld(5)+ld(6)*ld(6))),1e-
debugging: If the X,Y,Z 6),print(66666666);print(X);print(Y);print(ld(4));print(ld(5));print(ld(6));wh
vector in variables 4,5,6 isn't ile(1,0));^
normalized, then print some
variables and enter an
endless loop.
Optional input offset For all formats st(4,ld(4)+%OFF_IN_X%);^
correction, this is usable if st(5,ld(5)+%OFF_IN_Y%);^
the input image was taken This is the same as st(6,ld(6)+%OFF_IN_Z%);^
with a fisheye lens off- the h_offset and st(0,sqrt(ld(4)*ld(4)+ld(5)*ld(5)+ld(6)*ld(6)));^
center in a dome. Input and v_offset options in st(4,ld(4)/ld(0));^
st(5,ld(5)/ld(0));^
output are in variables 4,5,6. v360.
st(6,ld(6)/ld(0));^
For details see below.
Insert optional spherical For all formats
effects here, see next
chapter
Optional roll rotation For all formats st(0,ld(4)*cos(%ROLL%/180*PI)-ld(5)*sin(%ROLL%/180*PI));^
(around Z axis), input and st(5,ld(5)*cos(%ROLL%/180*PI)+ld(4)*sin(%ROLL%/180*PI));^
output are in variables 4,5,6. st(4,ld(0));^

Optional pitch rotation For all formats st(0,ld(5)*cos(%PITCH%/180*PI)-ld(6)*sin(%PITCH%/180*PI));^

(around X axis), input and st(6,ld(6)*cos(%PITCH%/180*PI)+ld(5)*sin(%PITCH%/180*PI));^
output are in variables 4,5,6. st(5,ld(0));^

Optional yaw rotation For all formats st(0,ld(4)*cos(%YAW%/180*PI)+ld(6)*sin(%YAW%/180*PI));^

(around Y axis), input and st(6,ld(6)*cos(%YAW%/180*PI)-ld(4)*sin(%YAW%/180*PI));^
output are in variables 4,5,6. st(4,ld(0));^

216
Optional output offset For all formats st(0,%OFF_OUT_X%*ld(4)+%OFF_OUT_Y%*ld(5)+%OFF_OUT_Z%*ld(6));^
correction, this is usable if st(1,2*(%OFF_OUT_X%*%OFF_OUT_Y%*ld(4)*ld(5)));^
the fisheye projector is This isn't st(1,ld(1)+2*(%OFF_OUT_X%*%OFF_OUT_Z%*ld(4)*ld(6)));^
placed off-center in the implemented in the st(1,ld(1)+2*(%OFF_OUT_Y%*%OFF_OUT_Z%*ld(5)*ld(6)));^
dome. v360 filter. st(1,ld(1)+ld(4)*ld(4)*(1-%OFF_OUT_Y%*%OFF_OUT_Y%-%OFF_OUT_Z%*%OFF_OUT_Z%));^
st(1,ld(1)+ld(5)*ld(5)*(1-%OFF_OUT_X%*%OFF_OUT_X%-%OFF_OUT_Z%*%OFF_OUT_Z%));^
Input and output are in
st(1,ld(1)+ld(6)*ld(6)*(1-%OFF_OUT_X%*%OFF_OUT_X%-%OFF_OUT_Y%*%OFF_OUT_Y%));^
variables 4,5,6. st(0,ld(0)+sqrt(ld(1)));^
st(4,ld(4)*ld(0)-%OFF_OUT_X%);^
For details see below. st(5,ld(5)*ld(0)-%OFF_OUT_Y%);^
st(6,ld(6)*ld(0)-%OFF_OUT_Z%);^
Convert from X,Y,Z space to For equirectangular This is only for the xmap file:
desired input format. input st(7,atan2(ld(4),ld(6)));^
(see xyz_to_equirect 0.5*%IN_W%*(1+ld(7)/%IN_H_FOV%*360/PI)' -frames 1 -y [Link]
Here the formulas are in v360.c)
different for xmap and ymap This is only for the ymap file:
st(8,asin(ld(5)));^
files, the differences are
0.5*%IN_H%*(1+ld(8)/%IN_V_FOV%*360/PI)' -frames 1 -y [Link]
marked in yellow.
This is only for the ymap file, if non-mapped pixels are possible:
Adding ld(9) to the output in st(8,asin(ld(5)));^
the last line is only required ld(9)+0.5*%IN_H%*(1+ld(8)/%IN_V_FOV%*360/PI)' -frames 1 -y [Link]
in one file, either xmap or
ymap, and it's only required For fisheye st(7,atan2(ld(5),ld(4)));^
(equidistant) input st(8,acos(ld(6)));^
if unmapped pixels are
possible for the selected (see xyz_to_fisheye
in v360.c) This is only for the xmap file:
output format. If variable 9 is 0.5*%IN_W%*(1+360/PI/%IN_H_FOV%*ld(8)*cos(ld(7)))' -frames 1 -y [Link]
set to a value larger than the
output size, the color will be This is only for the ymap file:
determined by remap=fill=... ld(9)+0.5*%IN_H%*(1+360/PI/%IN_V_FOV%*ld(8)*sin(ld(7)))' -frames 1 -y [Link]

217
For nonlinear fisheye st(7,atan2(ld(5),ld(4)));^
input according to st(8,acos(ld(6)));^
Paul Bourke's st(8,ld(8)/%IN_H_FOV%*360/PI);^
formula, see st(8,root(-ld(8)+%A%*ld(0)+%B%*pow(ld(0),2)+%C%*pow(ld(0),3)+%D
[Link] %*pow(ld(0),4),1));^
[Link]/dome/fisheyeco
This is only for the xmap file:
rrect/ 0.5*%IN_W%*(1+360/PI/%IN_H_FOV%*ld(8)*cos(ld(7)))' -frames 1 -y [Link]
Set the parameters A,
B, C, D and the This is only for the ymap file:
correct field of view. ld(9)+0.5*%IN_H%*(1+360/PI/%IN_V_FOV%*ld(8)*sin(ld(7)))' -frames 1 -y [Link]
This is slow because
"root" is used.
For double fisheye st(7,hypot(ld(4),ld(5)));^
(equidistant) input
(see xyz_to_dfisheye This is only for the xmap file:
in v360.c) st(7,if(lt(ld(6),0),-1,1));^
st(8,hypot(ld(4),ld(5)));^
st(8,atan2(ld(7)*ld(8),ld(7)*ld(6))/ld(8));^
0.25*%IN_W%*(2-ld(7)+ld(4)*ld(8)/%IN_H_FOV%*360/PI)' -frames 1 -y [Link]

This is only for the ymap file:

st(8,atan2(ld(7),abs(ld(6)))/ld(7));^
0.5*%IN_H%*(1+ld(5)*ld(8)/%IN_V_FOV%*360/PI)' -frames 1 -y [Link]
For flat (rectilinear) st(7,tan(acos(ld(6))));^
input st(8,hypot(ld(4),ld(5))/ld(7));^
(see xyz_to_flat in
v360.c) This is only for the xmap file:
0.5*%IN_W%*(1+ld(4)/ld(8)/tan(%IN_H_FOV%*PI/360))' -frames 1 -y [Link]

This is only for the ymap file:

0.5*%IN_H%*(1+ld(5)/ld(8)/tan(%IN_V_FOV%*PI/360))' -frames 1 -y [Link]
For cylindrical input This is only for the xmap file:
(see st(7,atan2(ld(4),ld(6)));^
xyz_to_cylindrical in 0.5*%IN_W%*(1+ld(7)/%IN_H_FOV%*360/PI)' -frames 1 -y [Link]
v360.c)
This is only for the ymap file:
st(8,asin(ld(5)));^
0.5*%IN_H%*(1+tan(ld(8))/tan(%IN_V_FOV%/360*PI))' -frames 1 -y [Link]

218
For planet input, this This is only for the xmap file:
is a sphere seen from 0.5*%IN_W%*(1+ld(4))' -frames 1 -y [Link]
infinite distance.
Exactly half of the This is only for the ymap file:
sphere is visible. st(9,if(lt(ld(6),0),%OUT_H%,0));^
ld(9)+0.5*%IN_H%*(1+ld(5))' -frames 1 -y [Link]
It's the same as
orthographic input
with 180° field of
view.
For planet input, this st(7,acos(ld(6)));^
is a sphere seen from st(9,if(gt(ld(7),PI/2-asin(1/%DIST%)),%OUT_H%,0));^
the finite distance st(7,sin(ld(7))/(%DIST%-cos(ld(7))));^
%DIST%. st(7,ld(7)/tan(asin(1/%DIST%)));^
Less than half of the st(8,atan2(ld(4),ld(5)));^
sphere is visible.
This is only for the xmap file:
This isn't 0.5*%IN_W%*(1+ld(7)*sin(ld(8)))' -frames 1 -y [Link]
implemented in the
v360 filter. This is only for the ymap file:
ld(9)+0.5*%IN_H%*(1+ld(7)*cos(ld(8)))' -frames 1 -y [Link]
For mirrorsphere st(7,atan2(ld(5),ld(4)));^
input, this is a st(8,0.5*(PI/2+asin(ld(6))));^
reflecting sphere
seen from infinite This is only for the xmap file:
distance. 0.5*%IN_W%*(1+cos(ld(8))*cos(ld(7)))' -frames 1 -y [Link]
(see xyz_to_ball in
This is only for the ymap file:
v360.c) ld(9)+0.5*%IN_H%*(1+cos(ld(8))*sin(ld(7)))' -frames 1 -y [Link]

219
For mirrorsphere st(7,0.5*acos(ld(6)));^
input, this is a st(9,if(gt(ld(7),PI/2-asin(1/%DIST%)),%OUT_H%,0));^
reflecting sphere st(7,sin(ld(7))/(%DIST%-cos(ld(7))));^
seen from the finite st(7,ld(7)/tan(asin(1/%DIST%)));^
distance %DIST%. st(8,atan2(ld(4),ld(5)));^
This isn't
This is only for the xmap file:
implemented in the 0.5*%IN_W%*(1+ld(7)*sin(ld(8)))' -frames 1 -y [Link]
v360 filter.
This is only for the ymap file:
ld(9)+0.5*%IN_H%*(1+ld(7)*cos(ld(8)))' -frames 1 -y [Link]
For equisolid input, st(7,atan2(ld(5),ld(4)));^
(see xyz_to_equisolid st(8,acos(ld(6)));^
in v360.c)
This is only for the xmap file:
st(8,sin(ld(8)/2)/sin(%IN_H_FOV%/720*PI));^
0.5*%IN_W%*(1+ld(8)*cos(ld(7)))' -frames 1 -y [Link]

This is only for the ymap file:

st(8,sin(ld(8)/2)/sin(%IN_V_FOV%/720*PI));^
ld(9)+0.5*%IN_H%*(1+ld(8)*sin(ld(7)))' -frames 1 -y [Link]
For stereographic st(7,atan2(ld(5),ld(4)));^
input, st(8,acos(ld(6)));^
(see
xyz_to_stereographic This is only for the xmap file:
in v360.c) st(8,tan(ld(8)/2)/tan(%IN_H_FOV%/720*PI));^
0.5*%IN_W%*(1+ld(8)*cos(ld(7)))' -frames 1 -y [Link]

This is only for the ymap file:

st(8,tan(ld(8)/2)/tan(%IN_V_FOV%/720*PI));^
ld(9)+0.5*%IN_H%*(1+ld(8)*sin(ld(7)))' -frames 1 -y [Link]

220
 For orthographic st(7,atan2(ld(5),ld(4)));^
input, st(8,acos(ld(6)));^
(see
xyz_to_orthographic This is only for the xmap file:
in v360.c) st(8,sin(ld(8))/sin(%IN_H_FOV%/360*PI));^
0.5*%IN_W%*(1+ld(8)*cos(ld(7)))' -frames 1 -y [Link]

This is only for the ymap file:

st(8,sin(ld(8))/sin(%IN_V_FOV%/360*PI));^
ld(9)+0.5*%IN_H%*(1+ld(8)*sin(ld(7)))' -frames 1 -y [Link]
Apply the remap filter For all formats ffmpeg -i [Link] -i [Link] -i [Link] -lavfi remap=fill=green -frames 1 -y
[Link]

Note: You can choose the order of the yaw, pitch and roll rotations as desired, or you can omit the rotations if you don't need them. But keep in mind that
here in "remap" the order is reversed. You have the output at the beginning and the input at the end. As shown above, the rotation order is yaw, pitch,
roll.
Note: For clarity, each equation is written here in a new line. However in the command line there are no line feeds allowed. Either it must all be written in
one long line, or you must terminate each line with a suitable character, depending on your batch interpreter. For example ^ for Windows batch files, as
shown here.
Note: For double fisheye, the parameters IN_W, IN_H, OUT_W and OUT_H refer to the size of the whole image, however the parameters IN_H_FOV, IN_V
FOV, OUT_H_FOV and OUT_V_FOV refer to the field of view of each of the fisheye images. Not the total horizontal angle of the whole image.

Convert from pixel coordinates [0 ... W-1] to angle = FOV * PI / 360 * ((2 * X + 1) / W - 1)
angle in radians [-FOV/2 ... +FOV/2] with
The angle is calculated for the center of the FOV = field of view in radians
pixel. X = pixel coordinate
W = image width in pixels

Convert from pixel coordinates [0 ... W-1] to XN = (2 * X + 1) / W - 1

normalized coordinates [-1 ... +1]: with
Result for X = 0 is (-1 + 1/W) X = pixel coordinate
Result for X = (W-1) is (1 - 1/W) W = image width in pixels

Convert from normalized coordinates [-1 ... +1] angle = FOV * PI / 360 * XN
to angle in radians [-FOV/2 ... +FOV/2]

221
Convert from angle in radians [-FOV/2 ... X = 0.5 * W * (1 + angle * 360 / PI / FOV)
+FOV/2] to pixel coordinates [0 ... W-1]: with
W = image width in pixels
FOV = field of view in degrees
angle = angle in degrees

When writing the numbers to a *.pgm file, is it safe to assume that the numbers are always rounded down to the next integer? Yes it is, as can be shown
with this simple test:
ffmpeg -f lavfi -i nullsrc=size=20x1 -vf format=pix_fmts=gray16le,geq='X/10' -frames 1 -y [Link]

pause

This is the resulting file in a hex editor. It contains 10 pixels with 16-bit value 0 followed by 10 pixels with 16-bit value 1:
0x00: 50 35 0A 32 30 20 31 0A 36 35 35 33 35 0A 00 00
0x10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x20: 00 00 00 01 00 01 00 01 00 01 00 01 00 01 00 01
0x30: 00 01 00 01 00 01

222
These are the formulas for the planet projection, with the camera at the left side:

See also: [Link]

Distance d (%DIST% in the batch file) gamma epsilon

1.0353 75° 15°
1.1547 60° 30°
1.4142 45° 45°
2 30° 60°
3.8637 15° 75°
221 (Moon seen from earth, average distance) 0.26° 89.74°

223
2.122 Optional Spherical Effects

Effect (input and output are in variables 4,5,6) Output Image

roll1

left: vertical shift

front: roll
right: vertical shift
back: --
up: shift
down: shift

set "EFF=30" :: Effect in degrees

st(7,%EFF%*cos(ld(4)));^
st(0,ld(4)*cos(ld(7)/180*PI)-ld(5)*sin(ld(7)/180*PI));^
st(5,ld(5)*cos(ld(7)/180*PI)+ld(4)*sin(ld(7)/180*PI));^
st(4,ld(0));^

roll2

left: vertical shear

front: roll
right: vertical shear
back: roll
up: --
down: --

set "EFF=30" :: Effect in degrees

st(7,%EFF%*sin(ld(6)));^
st(0,ld(4)*cos(ld(7)/180*PI)-ld(5)*sin(ld(7)/180*PI));^
st(5,ld(5)*cos(ld(7)/180*PI)+ld(4)*sin(ld(7)/180*PI));^
st(4,ld(0));^

224
roll3

left: vertical shift

front: roll
right: vertical shift
back: roll
up: shift
down: shift

set "EFF=30" :: Effect in degrees

st(7,%EFF%*cos(ld(5)));^
st(0,ld(4)*cos(ld(7)/180*PI)-ld(5)*sin(ld(7)/180*PI));^
st(5,ld(5)*cos(ld(7)/180*PI)+ld(4)*sin(ld(7)/180*PI));^
st(4,ld(0));^

neutral1

left: vertical shift

front: --
right: vertical shift
back: --
up: --
down: --

set "EFF=30" :: Effect in degrees

st(7,%EFF%*sin(ld(4)));^
st(0,ld(4)*cos(ld(7)/180*PI)-ld(5)*sin(ld(7)/180*PI));^
st(5,ld(5)*cos(ld(7)/180*PI)+ld(4)*sin(ld(7)/180*PI));^
st(4,ld(0));^

225
neutral2

left: horizontal shift

front: --
right: horizontal shift
back: --
up: --
down: --

set "EFF=30" :: Effect in degrees

st(7,%EFF%*sin(ld(4)));^
st(0,ld(4)*cos(ld(7)/180*PI)-ld(6)*sin(ld(7)/180*PI));^
st(6,ld(6)*cos(ld(7)/180*PI)+ld(4)*sin(ld(7)/180*PI));^
st(4,ld(0));^

neutral3

left: horizontal shear

front: --
right: horizontal shear
back: --
up: --
down: --

set "EFF=30" :: Effect in degrees

st(0,atan2(ld(4),ld(6)));^
st(1,asin(ld(5)));^
st(0,ld(0)+%EFF%/180*PI*sin(ld(0))*sin(ld(1)));^
st(4,cos(ld(1))*sin(ld(0)));^
st(5,sin(ld(1)));^
st(6,cos(ld(1))*cos(ld(0)));^

226
neutral4

left: horizontal shift

front: --
right: horizontal shift
back: --
up: --
down: --

set "EFF=30" :: Effect in degrees

st(0,atan2(ld(4),ld(6)));^
st(1,asin(ld(5)));^
st(0,ld(0)+%EFF%/180*PI*sin(ld(0))*cos(ld(1)));^
st(4,cos(ld(1))*sin(ld(0)));^
st(5,sin(ld(1)));^
st(6,cos(ld(1))*cos(ld(0)));^
neutral5

left: horizontal shift

front: --
right: horizontal shift
back: horizontal shift
up: --
down: --

set "EFF=30" :: Effect in degrees

st(0,atan2(ld(4),ld(6)));^
st(1,asin(ld(5)));^
st(0,ld(0)+%EFF%/180*PI*0.5*(1-cos(ld(0)))*cos(ld(1)));^
st(4,cos(ld(1))*sin(ld(0)));^
st(5,sin(ld(1)));^
st(6,cos(ld(1))*cos(ld(0)));^

227
neutral6

left: horizontal shear

front: --
right: horizontal shear
back: horizontal shear
up: --
down: --

set "EFF=30" :: Effect in degrees

st(0,atan2(ld(4),ld(6)));^
st(1,asin(ld(5)));^
st(0,ld(0)+%EFF%/180*PI*0.5*(1-cos(ld(0)))*sin(ld(1)));^
st(4,cos(ld(1))*sin(ld(0)));^
st(5,sin(ld(1)));^
st(6,cos(ld(1))*cos(ld(0)));^
h_shear1

left: horizontal shear

front: horizontal shear
right: horizontal shear
back: horizontal shear
up: rotate
down: rotate

set "EFF=30" :: Effect in degrees

st(7,%EFF%*sin(ld(5)));^
st(0,ld(4)*cos(ld(7)/180*PI)-ld(6)*sin(ld(7)/180*PI));^
st(6,ld(6)*cos(ld(7)/180*PI)+ld(4)*sin(ld(7)/180*PI));^
st(4,ld(0));^

228
h_shear2

left: horizontal shear

front: horizontal shear
right: horizontal shear
back: horizontal shear
up: rotate
down: rotate

set "EFF=30" :: Effect in degrees

st(0,atan2(ld(4),ld(6)));^
st(1,asin(ld(5)));^
st(0,ld(0)+%EFF%/180*PI*sin(ld(1)));^
st(4,cos(ld(1))*sin(ld(0)));^
st(5,sin(ld(1)));^
st(6,cos(ld(1))*cos(ld(0)));^st(0,atan2(ld(4),ld(6)));^

h_shear3

left: --
front: horizontal shear
right: --
back: horizontal shear
up: --
down: --

set "EFF=30" :: Effect in degrees

st(0,atan2(ld(4),ld(6)));^
st(1,asin(ld(5)));^
st(0,ld(0)+%EFF%/180*PI*cos(ld(0))*sin(ld(1)));^
st(4,cos(ld(1))*sin(ld(0)));^
st(5,sin(ld(1)));^
st(6,cos(ld(1))*cos(ld(0)));^

229
h_shear4

left: horizontal shear

front: horizontal shear
right: horizontal shear
back: --
up: --
down: --

set "EFF=30" :: Effect in degrees

st(0,atan2(ld(4),ld(6)));^
st(1,asin(ld(5)));^
st(0,ld(0)+%EFF%/180*PI*0.5*(1+cos(ld(0)))*sin(ld(1)));^
st(4,cos(ld(1))*sin(ld(0)));^
st(5,sin(ld(1)));^
st(6,cos(ld(1))*cos(ld(0)));^
v_shear1

left: roll
front: vertical shear
right: roll
back: vertical shear
up: --
down: --

set "EFF=30" :: Effect in degrees

st(7,%EFF%*sin(ld(4)));^
st(0,ld(5)*cos(ld(7)/180*PI)-ld(6)*sin(ld(7)/180*PI));^
st(6,ld(6)*cos(ld(7)/180*PI)+ld(5)*sin(ld(7)/180*PI));^
st(5,ld(0));^

230
v_shear2

left: vertical shift

front: vertical shear
right: vertical shift
back: vertical shear
up: --
down: --

set "EFF=30" :: Effect in degrees

st(5,ld(5)+%EFF%/180*PI*sin(ld(4)));^
st(0,sqrt(ld(4)*ld(4)+ld(5)*ld(5)+ld(6)*ld(6)));^
st(4,ld(4)/ld(0));^
st(5,ld(5)/ld(0));^
st(6,ld(6)/ld(0));^

v_shear3

left: vertical shift

front: vertical shear
right: vertical shift
back: vertical shear
up: discontinuity
down: discontinuity

set "EFF=30" :: Effect in degrees

st(0,atan2(ld(4),ld(6)));^
st(1,asin(ld(5)));^
st(1,ld(1)+%EFF%/180*PI*sin(ld(0)));^
st(4,cos(ld(1))*sin(ld(0)));^
st(5,sin(ld(1)));^
st(6,cos(ld(1))*cos(ld(0)));^

231
v_shear4

left: vertical shift

front: vertical shear
right: vertical shift
back: vertical shear
up: --
down: --

set "EFF=30" :: Effect in degrees

st(0,atan2(ld(4),ld(6)));^
st(1,asin(ld(5)));^
st(1,ld(1)+%EFF%/180*PI*sin(ld(0))*cos(ld(1)));^
st(4,cos(ld(1))*sin(ld(0)));^
st(5,sin(ld(1)));^
st(6,cos(ld(1))*cos(ld(0)));^
v_shear5

left: vertical shift

front: vertical shear
right: vertical shift
back: vertical shear
up: --
down: --

set "EFF=30" :: Effect in degrees

st(0,atan2(ld(4),ld(6)));^
st(1,asin(ld(5)));^
st(1,ld(1)+%EFF%/180*PI*sin(ld(0))*0.5*(1+cos(2*ld(1))));^
st(4,cos(ld(1))*sin(ld(0)));^
st(5,sin(ld(1)));^
st(6,cos(ld(1))*cos(ld(0)));^

232
h_shift1

left: --
front: horizontal shift
right: --
back: horizontal shift
up: --
down: --

set "EFF=30" :: Effect in degrees

st(7,%EFF%*sin(ld(6)));^
st(0,ld(4)*cos(ld(7)/180*PI)-ld(6)*sin(ld(7)/180*PI));^
st(6,ld(6)*cos(ld(7)/180*PI)+ld(4)*sin(ld(7)/180*PI));^
st(4,ld(0));^

h_shift2

left: --
front: horizontal shift
right: --
back: horizontal shift
up: --
down: --

set "EFF=30" :: Effect in degrees

st(0,atan2(ld(4),ld(6)));^
st(1,asin(ld(5)));^
st(0,ld(0)+%EFF%/180*PI*cos(ld(0))*cos(ld(1)));^
st(4,cos(ld(1))*sin(ld(0)));^
st(5,sin(ld(1)));^
st(6,cos(ld(1))*cos(ld(0)));^

233
h_shift3

left: horizontal shift

front: horizontal shift
right: horizontal shift
back: --
up: --
down: --

set "EFF=30" :: Effect in degrees

st(0,atan2(ld(4),ld(6)));^
st(1,asin(ld(5)));^
st(0,ld(0)+%EFF%/180*PI*0.5*(1+cos(ld(0)))*cos(ld(1)));^
st(4,cos(ld(1))*sin(ld(0)));^
st(5,sin(ld(1)));^
st(6,cos(ld(1))*cos(ld(0)));^
v_shift1

left: --
front: vertical shift
right: --
back: vertical shift
up: --
down: --

set "EFF=30" :: Effect in degrees

st(7,%EFF%*sin(ld(6)));^
st(0,ld(5)*cos(ld(7)/180*PI)-ld(6)*sin(ld(7)/180*PI));^
st(6,ld(6)*cos(ld(7)/180*PI)+ld(5)*sin(ld(7)/180*PI));^
st(5,ld(0));^

234
v_shift2

left: roll
front: vertical shift
right: roll
back: vertical shift
up: shift
down: shift

set "EFF=30" :: Effect in degrees

st(7,%EFF%*cos(ld(5)));^
st(0,ld(5)*cos(ld(7)/180*PI)-ld(6)*sin(ld(7)/180*PI));^
st(6,ld(6)*cos(ld(7)/180*PI)+ld(5)*sin(ld(7)/180*PI));^
st(5,ld(0));^

v_shift3

left: roll
front: vertical shift
right: roll
back: vertical shift
up: shift
down: shift

set "EFF=30" :: Effect in degrees

st(7,%EFF%*cos(ld(6)));^
st(0,ld(5)*cos(ld(7)/180*PI)-ld(6)*sin(ld(7)/180*PI));^
st(6,ld(6)*cos(ld(7)/180*PI)+ld(5)*sin(ld(7)/180*PI));^
st(5,ld(0));^

235
v_shift4

left: vertical shear

front: vertical shift
right: vertical shear
back: vertical shift
up: --
down: --

set "EFF=30" :: Effect in degrees

st(5,ld(5)+%EFF%/180*PI*sin(ld(6)));^
st(0,sqrt(ld(4)*ld(4)+ld(5)*ld(5)+ld(6)*ld(6)));^
st(4,ld(4)/ld(0));^
st(5,ld(5)/ld(0));^
st(6,ld(6)/ld(0));^

v_shift5

left: vertical shear

front: vertical shift
right: vertical shear
back: vertical shift
up: discontinuity
down: discontinuity

set "EFF=30" :: Effect in degrees

st(0,atan2(ld(4),ld(6)));^
st(1,asin(ld(5)));^
st(1,ld(1)+%EFF%/180*PI*cos(ld(0)));^
st(4,cos(ld(1))*sin(ld(0)));^
st(5,sin(ld(1)));^
st(6,cos(ld(1))*cos(ld(0)));^

236
v_shift6

left: vertical shear

front: vertical shift
right: vertical shear
back: vertical shift
up: --
down: --

set "EFF=30" :: Effect in degrees

st(0,atan2(ld(4),ld(6)));^
st(1,asin(ld(5)));^
st(1,ld(1)+%EFF%/180*PI*cos(ld(0))*cos(ld(1)));^
st(4,cos(ld(1))*sin(ld(0)));^
st(5,sin(ld(1)));^
st(6,cos(ld(1))*cos(ld(0)));^

v_shift7

left: vertical shear

front: vertical shift
right: vertical shear
back: vertical shift
up: --
down: --

st(0,atan2(ld(4),ld(6)));^
st(1,asin(ld(5)));^
st(1,ld(1)+%EFF%/180*PI*cos(ld(0))*0.5*(1+cos(2*ld(1))));^
st(4,cos(ld(1))*sin(ld(0)));^
st(5,sin(ld(1)));^
st(6,cos(ld(1))*cos(ld(0)));^

237
Height over ground correction (for drone videos), only
applied to one hemisphere. Makes a slope discontinuity at
the horizon.

left: --
front: zoomed out
right: --
back: --
up: --
down: --

set "H=2" :: Height factor (1=neutral)

st(7,atan2(ld(5),ld(4)));^
st(8,acos(ld(6)));^
if(lt(ld(8),PI/2),st(8,atan(%H%*tan(ld(8)))));^
st(4,sin(ld(8))*cos(ld(7)));^
st(5,sin(ld(8))*sin(ld(7)));^
st(6,cos(ld(8)));^
Height over ground correction (for drone videos), applied to
both hemispheres.

left: --
front: zoomed out
right: --
back: zoomed out
up: --
down: --

set "H=2" :: Height factor (1=neutral)

st(7,atan2(ld(5),ld(4)));^
st(8,acos(ld(6)));^
st(8,if(lt(ld(8),PI/2),atan(%H%*tan(ld(8))),PI-atan(%H
%*tan(PI-ld(8)))));^
st(4,sin(ld(8))*cos(ld(7)));^
st(5,sin(ld(8))*sin(ld(7)));^
st(6,cos(ld(8)));^

238
Forward looking enlarging effect

left: --
front: zoomed in
right: --
back: --
up: --
down: --

set "LENS_RAD=60" :: Lens radius in degrees

set "LENS_PWR=1" :: Lens power (0=neutral)

st(0,cos(%LENS_RAD%/180*PI));^
if(gt(ld(6),ld(0)),st(6,ld(6)+%LENS_PWR%*pow((ld(6)-ld(0))/
(1-ld(0)),2)));^
st(0,sqrt(ld(4)*ld(4)+ld(5)*ld(5)+ld(6)*ld(6)));^
st(4,ld(4)/ld(0));^
st(5,ld(5)/ld(0));^
st(6,ld(6)/ld(0));^
Ellipsoid distortion effect, front and back zoomed in

left: --
front: zoomed in
right: --
back: zoomed in
up: --
down: --

set "ELLIPSE=2" :: Ellipse factor (1=neutral)

st(6,%ELLIPSE%*ld(6));^
st(0,sqrt(ld(4)*ld(4)+ld(5)*ld(5)+ld(6)*ld(6)));^
st(4,ld(4)/ld(0));^
st(5,ld(5)/ld(0));^
st(6,ld(6)/ld(0));^

239
Ellipsoid distortion effect, front and back zoomed out

left: --
front: zoomed out
right: --
back: zoomed out
up: --
down: --

set "ELLIPSE=0.5" :: Ellipse factor (1=neutral)

st(6,%ELLIPSE%*ld(6));^
st(0,sqrt(ld(4)*ld(4)+ld(5)*ld(5)+ld(6)*ld(6)));^
st(4,ld(4)/ld(0));^
st(5,ld(5)/ld(0));^
st(6,ld(6)/ld(0));^

Ellipsoid distortion effect, up and down zoomed in

left: --
front: --
right: --
back: --
up: zoomed in
down: zoomed in

set "ELLIPSE=2" :: Ellipse factor (1=neutral)

st(5,%ELLIPSE%*ld(5));^
st(0,sqrt(ld(4)*ld(4)+ld(5)*ld(5)+ld(6)*ld(6)));^
st(4,ld(4)/ld(0));^
st(5,ld(5)/ld(0));^
st(6,ld(6)/ld(0));^

240
Ellipsoid distortion effect, up and down zoomed out

left: --
front: --
right: --
back: --
up: zoomed out
down: zoomed out

set "ELLIPSE=0.5" :: Ellipse factor (1=neutral)

st(5,%ELLIPSE%*ld(5));^
st(0,sqrt(ld(4)*ld(4)+ld(5)*ld(5)+ld(6)*ld(6)));^
st(4,ld(4)/ld(0));^
st(5,ld(5)/ld(0));^
st(6,ld(6)/ld(0));^

Ellipsoid distortion effect, left and right zoomed in

left: zoomed in
front: --
right: zoomed in
back: --
up: --
down: --

set "ELLIPSE=2" :: Ellipse factor (1=neutral)

st(4,%ELLIPSE%*ld(4));^
st(0,sqrt(ld(4)*ld(4)+ld(5)*ld(5)+ld(6)*ld(6)));^
st(4,ld(4)/ld(0));^
st(5,ld(5)/ld(0));^
st(6,ld(6)/ld(0));^

241
Ellipsoid distortion effect, left and right zoomed out

left: zoomed out

front: --
right: zoomed out
back: --
up: --
down: --

set "ELLIPSE=0.5" :: Ellipse factor (1=neutral)

st(4,%ELLIPSE%*ld(4));^
st(0,sqrt(ld(4)*ld(4)+ld(5)*ld(5)+ld(6)*ld(6)));^
st(4,ld(4)/ld(0));^
st(5,ld(5)/ld(0));^
st(6,ld(6)/ld(0));^

Black hole distortion effect

Deviation angle = 2 * RS / (R - RS) where R is the angular
distance from the black hole.

set "RS=20" :: Schwarzschild radius in degrees

set "S=0.2" :: Factor for making the distortion smaller,
:: 1=realistic

st(7,atan2(ld(5),ld(4)));^
st(8,acos(ld(6)));^
st(9,if(lte(ld(8),%RS%/180*PI),%OUT_W%,0));^
st(8,if(gt(ld(8),%RS%/180*PI),ld(8)-%S%*(%RS%/90*PI/(ld(8)-
%RS%/180*PI)),0));^
st(4,sin(ld(8))*cos(ld(7)));^
st(5,sin(ld(8))*sin(ld(7)));^
st(6,cos(ld(8)));^

242
Example for using the above effects:
set "IN_W=2400" :: Input width in pixels
set "IN_H=1200" :: Input height in pixels
set "IN_H_FOV=360" :: Input horizontal field of view in degrees
set "IN_V_FOV=180" :: Input vertical field of view in degrees
set "OUT_W=2400" :: Output width in pixels
set "OUT_H=1200" :: Output height in pixels
set "OUT_H_FOV=360" :: Output horizontal field of view in degrees
set "OUT_V_FOV=180" :: Output vertical field of view in degrees
set "EFF=30" :: Effect in degrees

rem Create the xmap and ymap files

ffmpeg -f lavfi -i nullsrc=size=%OUT_W%x%OUT_H% -vf format=pix_fmts=gray16le,geq='^

st(0,PI/360*%OUT_H_FOV%*((2*X+1)/%OUT_W%-1));^
st(1,PI/360*%OUT_V_FOV%*((2*Y+1)/%OUT_H%-1));^
st(4,cos(ld(1))*sin(ld(0)));^
st(5,sin(ld(1)));^
st(6,cos(ld(1))*cos(ld(0)));^
Insert the code for the effect here
st(7,atan2(ld(4),ld(6)));^
0.5*%IN_W%*(1+ld(7)/%IN_H_FOV%*360/PI)' -frames 1 -y [Link]

ffmpeg -f lavfi -i nullsrc=size=%OUT_W%x%OUT_H% -vf format=pix_fmts=gray16le,geq='^

st(0,PI/360*%OUT_H_FOV%*((2*X+1)/%OUT_W%-1));^
st(1,PI/360*%OUT_V_FOV%*((2*Y+1)/%OUT_H%-1));^
st(4,cos(ld(1))*sin(ld(0)));^
st(5,sin(ld(1)));^
st(6,cos(ld(1))*cos(ld(0)));^
Insert the code for the effect here
st(8,asin(ld(5)));^
0.5*%IN_H%*(1+ld(8)/%IN_V_FOV%*360/PI)' -frames 1 -y [Link]

ffmpeg -i [Link] -i [Link] -i [Link] -lavfi remap=fill=green -frames 1 -y [Link]

pause

243
2.123 Off-center fisheye projection

See [Link]
and especially [Link] where unfortunately you can't find the formulas...

Example:
Let's assume input and output formats are fisheye. If the fisheye projector is shifted 0.5 to the side (halfway between the dome's center and edge), and if
the optical axis of the projection lens is tilted 26.565° so that the optical axis is pointing to the dome's vertex, use these parameters to create the image:
set "YAW=26.565" :: Yaw angle in degrees
set "OFF_IN_X=0" :: Input X offset, normalized to the dome's radius
set "OFF_OUT_X=0.5" :: Output X offset, normalized to the dome's radius
However if you have taken an image with a fisheye lens inside the dome, with the camera at the same place as above and also pointing to the dome's
vertex, then this image can be converted back to a normal fisheye image with these parameters:
set "YAW=-26.565" :: Yaw angle in degrees
set "OFF_IN_X=0.5" :: Input X offset, normalized to the dome's radius
set "OFF_OUT_X=0" :: Output X offset, normalized to the dome's radius

A similar effect can be realized with the "h_offset" and "v_offset" options of the v360 filter. But that's the inverse algorithm. It's usable if a fisheye image
was taken off-center in a dome, and shall be corrected to the center of the dome.
set "IN=[Link]" :: Fisheye test pattern from [Link]
set "OUT=[Link]" :: Fisheye output image

ffmpeg -i %IN% -lavfi v360=fisheye:fisheye:h_offset=0.5 -y %OUT%

pause

At first I thought that the offset correction for a off-center fisheye projector could be done very easy as follows:

244
st(4,ld(4)+%OFFSET_X%);^ (add the offset vector to the normalized input vector)
st(5,ld(5)+%OFFSET_Y%);^
st(6,ld(6)+%OFFSET_Z%);^
st(0,sqrt(ld(4)*ld(4)+ld(5)*ld(5)+ld(6)*ld(6)));^ (calculate the length)
st(4,ld(4)/ld(0));^ (re-normalize the vector)
st(5,ld(5)/ld(0));^
st(6,ld(6)/ld(0));^

The above algorithm is the same as the "h_offset" and "v_offset" options of the v360 filter. But this approach doesn't work. It turned out that the inverse
algorithm is required. The output is given and the input has to be found. First the normalization must be undone by a suitable unknown factor, so that
after subtracting the offsets the result is already normalized. Undoing the normalization is possible, but complicated.
For details, see this discussion (in german): [Link]

Here the projection lens is at the dome's center. This is the 1.5m dome in Here the projection lens it at the OFF_OUT_Y=0.5 position. It's clearly
the Sankt Andreasberg Observatory in the Harz Mountains in Germany. visible that the image becomes brighter at the left side and darker at the
right side.

This is an example for transformation from fisheye to off-center fisheye, where the lens is shifted half the dome's radius to the side and tilted 26.565°, so
that the optical axis is pointing to the dome's vertex:
set "IN_W=1200" :: Input width in pixels
set "IN_H=1200" :: Input height in pixels
set "IN_H_FOV=180" :: Input horizontal field of view in degrees

245
set "IN_V_FOV=180" :: Input vertical field of view in degrees
set "OUT_W=1200" :: Output width in pixels
set "OUT_H=1200" :: Output height in pixels
set "OUT_H_FOV=180" :: Output horizontal field of view in degrees
set "OUT_V_FOV=180" :: Output vertical field of view in degrees
set "YAW=26.565" :: Yaw angle in degrees
set "PITCH=0" :: Pitch angle in degrees
set "ROLL=0" :: Roll angle in degrees
set "OFF_IN_X=0" :: Input X offset, normalized to the dome's radius
set "OFF_IN_Y=0" :: Input Y offset, normalized to the dome's radius
set "OFF_IN_Z=0" :: Input Z offset, negative is closer to dome's vertex
set "OFF_OUT_X=0.5" :: Output X offset, normalized to the dome's radius
set "OFF_OUT_Y=0" :: Output Y offset, normalized to the dome's radius
set "OFF_OUT_Z=0" :: Output Z offset, negative is closer to dome's vertex

rem Create the xmap file

ffmpeg -f lavfi -i nullsrc=size=%OUT_W%x%OUT_H% -vf format=pix_fmts=gray16le,geq='^

st(0,%OUT_H_FOV%/180*((2*X+1)/%OUT_W%-1));^
st(1,%OUT_V_FOV%/180*((2*Y+1)/%OUT_H%-1));^
st(2,atan2(ld(1),ld(0)));^
st(3,PI/2*(1-hypot(ld(0),ld(1))));^
st(4,cos(ld(3))*cos(ld(2)));^
st(5,cos(ld(3))*sin(ld(2)));^
st(6,sin(ld(3)));^
st(4,ld(4)+%OFF_IN_X%);^
st(5,ld(5)+%OFF_IN_Y%);^
st(6,ld(6)+%OFF_IN_Z%);^
st(0,sqrt(ld(4)*ld(4)+ld(5)*ld(5)+ld(6)*ld(6)));^
st(4,ld(4)/ld(0));^
st(5,ld(5)/ld(0));^
st(6,ld(6)/ld(0));^
st(0,ld(4)*cos(%ROLL%/180*PI)-ld(5)*sin(%ROLL%/180*PI));^
st(5,ld(5)*cos(%ROLL%/180*PI)+ld(4)*sin(%ROLL%/180*PI));^
st(4,ld(0));^
st(0,ld(5)*cos(%PITCH%/180*PI)-ld(6)*sin(%PITCH%/180*PI));^
st(6,ld(6)*cos(%PITCH%/180*PI)+ld(5)*sin(%PITCH%/180*PI));^
st(5,ld(0));^
st(0,ld(4)*cos(%YAW%/180*PI)+ld(6)*sin(%YAW%/180*PI));^

246
st(6,ld(6)*cos(%YAW%/180*PI)-ld(4)*sin(%YAW%/180*PI));^
st(4,ld(0));^
st(0,%OFF_OUT_X%*ld(4)+%OFF_OUT_Y%*ld(5)+%OFF_OUT_Z%*ld(6));^
st(1,2*(%OFF_OUT_X%*%OFF_OUT_Y%*ld(4)*ld(5)));^
st(1,ld(1)+2*(%OFF_OUT_X%*%OFF_OUT_Z%*ld(4)*ld(6)));^
st(1,ld(1)+2*(%OFF_OUT_Y%*%OFF_OUT_Z%*ld(5)*ld(6)));^
st(1,ld(1)+ld(4)*ld(4)*(1-%OFF_OUT_Y%*%OFF_OUT_Y%-%OFF_OUT_Z%*%OFF_OUT_Z%));^
st(1,ld(1)+ld(5)*ld(5)*(1-%OFF_OUT_X%*%OFF_OUT_X%-%OFF_OUT_Z%*%OFF_OUT_Z%));^
st(1,ld(1)+ld(6)*ld(6)*(1-%OFF_OUT_X%*%OFF_OUT_X%-%OFF_OUT_Y%*%OFF_OUT_Y%));^
st(0,ld(0)+sqrt(ld(1)));^
st(4,ld(4)*ld(0)-%OFF_OUT_X%);^
st(5,ld(5)*ld(0)-%OFF_OUT_Y%);^
st(6,ld(6)*ld(0)-%OFF_OUT_Z%);^
st(7,hypot(ld(5),ld(4)));^
st(8,atan2(ld(7),ld(6))/ld(7));^
0.5*%IN_W%*(1+ld(4)*ld(8)/%IN_H_FOV%*360/PI)' -frames 1 -y [Link]

rem Create the ymap file

ffmpeg -f lavfi -i nullsrc=size=%OUT_W%x%OUT_H% -vf format=pix_fmts=gray16le,geq='^

st(0,%OUT_H_FOV%/180*((2*X+1)/%OUT_W%-1));^
st(1,%OUT_V_FOV%/180*((2*Y+1)/%OUT_H%-1));^
st(2,atan2(ld(1),ld(0)));^
st(3,PI/2*(1-hypot(ld(0),ld(1))));^
st(4,cos(ld(3))*cos(ld(2)));^
st(5,cos(ld(3))*sin(ld(2)));^
st(6,sin(ld(3)));^
st(4,ld(4)+%OFF_IN_X%);^
st(5,ld(5)+%OFF_IN_Y%);^
st(6,ld(6)+%OFF_IN_Z%);^
st(0,sqrt(ld(4)*ld(4)+ld(5)*ld(5)+ld(6)*ld(6)));^
st(4,ld(4)/ld(0));^
st(5,ld(5)/ld(0));^
st(6,ld(6)/ld(0));^
st(0,ld(4)*cos(%ROLL%/180*PI)-ld(5)*sin(%ROLL%/180*PI));^
st(5,ld(5)*cos(%ROLL%/180*PI)+ld(4)*sin(%ROLL%/180*PI));^
st(4,ld(0));^
st(0,ld(5)*cos(%PITCH%/180*PI)-ld(6)*sin(%PITCH%/180*PI));^
st(6,ld(6)*cos(%PITCH%/180*PI)+ld(5)*sin(%PITCH%/180*PI));^

247
st(5,ld(0));^
st(0,ld(4)*cos(%YAW%/180*PI)+ld(6)*sin(%YAW%/180*PI));^
st(6,ld(6)*cos(%YAW%/180*PI)-ld(4)*sin(%YAW%/180*PI));^
st(4,ld(0));^
st(0,%OFF_OUT_X%*ld(4)+%OFF_OUT_Y%*ld(5)+%OFF_OUT_Z%*ld(6));^
st(1,2*(%OFF_OUT_X%*%OFF_OUT_Y%*ld(4)*ld(5)));^
st(1,ld(1)+2*(%OFF_OUT_X%*%OFF_OUT_Z%*ld(4)*ld(6)));^
st(1,ld(1)+2*(%OFF_OUT_Y%*%OFF_OUT_Z%*ld(5)*ld(6)));^
st(1,ld(1)+ld(4)*ld(4)*(1-%OFF_OUT_Y%*%OFF_OUT_Y%-%OFF_OUT_Z%*%OFF_OUT_Z%));^
st(1,ld(1)+ld(5)*ld(5)*(1-%OFF_OUT_X%*%OFF_OUT_X%-%OFF_OUT_Z%*%OFF_OUT_Z%));^
st(1,ld(1)+ld(6)*ld(6)*(1-%OFF_OUT_X%*%OFF_OUT_X%-%OFF_OUT_Y%*%OFF_OUT_Y%));^
st(0,ld(0)+sqrt(ld(1)));^
st(4,ld(4)*ld(0)-%OFF_OUT_X%);^
st(5,ld(5)*ld(0)-%OFF_OUT_Y%);^
st(6,ld(6)*ld(0)-%OFF_OUT_Z%);^
st(7,hypot(ld(5),ld(4)));^
st(8,atan2(ld(7),ld(6))/ld(7));^
0.5*%IN_H%*(1+ld(5)*ld(8)/%IN_V_FOV%*360/PI)' -frames 1 -y [Link]

ffmpeg -i [Link] -i [Link] -i [Link] -lavfi remap -frames 1 -y [Link]

pause

248
2.124 DLP Beamer output

A DLP beamer does always throw an off-axis image. The optical axis of the beamer is below the image (or above the image, if the beamer is mounted
upside down).

(I'm still working on this chapter)

ffmpeg -f lavfi -i color=red:s=320x180 -f lavfi -i color=yellow:s=320x220 -lavfi vstack -frames 1 -y [Link]

ffmpeg -f lavfi -i color=red:s=320x180 -frames 1 -y [Link]

pause

[Link] has size 320x400 and the optical axis of the beamer is in the center. The red part at the top is the visible part.
Field of view from 500 pixels distance: 35.49° x 43.60° (2 * atan(320 / 2 / 500) = 35.49°
[Link] has size 320x180 (16:9) and is only the visible part.
Field of view from 500 pixels distance: 35.40° x 20.41°

249
2.125 Remap from equirectangular to double-fisheye

This is the simple version with the v360 filter:

set "IN=[Link]" :: Input image
set "FOV=180" :: Output field of view in degrees
set "OUT=double_fish.png" :: Output image

ffmpeg -i %IN% -lavfi v360=e:dfisheye:h_fov=%FOV%:v_fov=%FOV%:pitch=0 -y %OUT%

pause
Note: Pitch can be 0 or 90, depending on your needs.

Unfortunately the v360 filter with dfisheye output fills not only the two image circles with data, but instead also the outer areas. The workaround is to
overlay a double-circular mask:
set "IN=[Link]" :: Input image
set "SIZE=1200x1200" :: Size of half mask
set "FOV=180" :: Output field of view in degrees
set "OUT=double_fish.png" :: Output image

ffmpeg -f lavfi -i color=black:s=%SIZE% -lavfi format=argb,geq=a='255*gt(hypot(((2*X+1)/H-1),((2*Y+1)/H-

1)),1)':r=0:g=0:b=0,split,hstack -frames 1 -y [Link]

ffmpeg -i %IN% -i [Link] -lavfi v360=e:dfisheye:h_fov=%FOV%:v_fov=%FOV%:pitch=0,overlay -y %OUT%

pause
Note: Pitch can be 0 or 90, depending on your needs.

250
2.126 Remap an equirectangular video to a "Little planet" video

Fisheye projection is used. The ground is in the center of the video, and the sky is at the circular edge. The input video must have 2:1 width/height ratio.
set "IN=test3.mp4" :: Equirectangular input video
set "H=960" :: Height of input video (width = 2 * height)
set "S=1080" :: Size of square little planet output video
set "OUT=out.mp4" :: Output video

rem Create the xmap file

ffmpeg -f lavfi -i nullsrc=size=%S%x%S% -vf format=pix_fmts=gray16le,^

geq='%H%*(0.9999+atan2(Y-%S%/2,X-%S%/2)/PI)' -frames 1 -y [Link]

rem Create the ymap file

ffmpeg -f lavfi -i nullsrc=size=%S%x%S% -vf format=pix_fmts=gray16le,^

geq='%H%*(1-hypot((2*X/%S%)-1,(2*Y/%S%)-1))' -frames 1 -y [Link]

rem Apply the remap filter to the video

ffmpeg -i %IN% -i [Link] -i [Link] -lavfi "format=pix_fmts=rgb24,remap=fill=green" -q:v 2 -y %OUT%

pause

The values in the xmap and ymap files can't be negative. If a value is greater than the size of the input image, this pixel is painted with the color that's
specified by the "fill" option.

251
If you want the sky in the center and the ground at the circular edge, use these remap functions instead:
ffmpeg -f lavfi -i nullsrc=size=%S%x%S% -vf format=pix_fmts=gray16le,^
geq='%H%*(0.9999-atan2(Y-%S%/2,X-%S%/2)/PI)' -frames 1 -y [Link]

ffmpeg -f lavfi -i nullsrc=size=%S%x%S% -vf format=pix_fmts=gray16le,^

geq='%H%*(hypot((2*X/%S%)-1,(2*Y/%S%)-1))' -frames 1 -y [Link]

The same thing can also be done with the v360 filter. You can use either "fisheye" or "stereographic" output, the results are different:
set "IN=[Link]" :: Equirectangular input image or video
set "FOV_FI=360" :: Output field of view in degrees, for fisheye
set "FOV_SG=250" :: Output field of view in degrees, for stereographic

ffmpeg -i %IN% -vf v360=input=equirect:output=fisheye:d_fov=%FOV_FI%:pitch=-90 -y littleplanet_fisheye.png

ffmpeg -i %IN% -vf v360=input=equirect:output=sg:d_fov=%FOV_SG%:pitch=-90 -y littleplanet_stereographic.png

pause

See also: [Link]

See also: [Link]
See also the website of Anders Jirås: [Link]

252
2.127 Remap an equirectangular video to a "Mirror sphere" video

Similar to "Little planet", but using a different projection. The 360° world is shown as a reflection on a mirror sphere. The ground is in the center of the
video, and the sky is at the circular edge. The input video must have 2:1 width/height ratio.
set "IN=equirectangular_test.png" :: Equirectangular input video
set "H=1200" :: Height of input video (width = 2 * height)
set "S=900" :: Size of square mirror sphere output video
set "OUT=[Link]" :: Output video

rem Create the xmap file

ffmpeg -f lavfi -i nullsrc=size=%S%x%S% -vf format=pix_fmts=gray16le,^

geq='%H%*(0.9999+atan2(Y-%S%/2,X-%S%/2)/PI)' -frames 1 -y [Link]

rem Create the ymap file

ffmpeg -f lavfi -i nullsrc=size=%S%x%S% -vf format=pix_fmts=gray16le,^

geq='%H%*(1-2/PI*asin(hypot((2*X/%S%)-1,(2*Y/%S%)-1)))' -frames 1 -y [Link]

rem Apply the remap filter to the video

ffmpeg -i %IN% -i [Link] -i [Link] -lavfi "format=pix_fmts=rgb24,remap" -q:v 2 -y %OUT%

pause

If you want the sky in the center and the ground at the circular edge, use these remap functions instead:
ffmpeg -f lavfi -i nullsrc=size=%S%x%S% -vf format=pix_fmts=gray16le,^
geq='%H%*(0.9999-atan2(Y-%S%/2,X-%S%/2)/PI)' -frames 1 -y [Link]

ffmpeg -f lavfi -i nullsrc=size=%S%x%S% -vf format=pix_fmts=gray16le,^

geq='%H%*(2/PI*asin(hypot((2*X/%S%)-1,(2*Y/%S%)-1)))' -frames 1 -y [Link]

253
The same thing can also be done with the "ball" output format of the v360 filter:
set "IN=[Link]" :: Equirectangular input image or video
set "OUT=[Link]" :: Output image or video

ffmpeg -i %IN% -lavfi "v360=input=e:output=ball:pitch=90" -q:v 2 -y [Link]

pause

Pitch=90 is for the sky in the center, pitch=-90 is for the ground in the center.

This batch file converts a double-fisheye video from Ricoh Theta V to a "mirror sphere" video:
set "IN=R0010017.mp4" :: Input video
set "H=1920" :: Height of input video
set "FOV=191.5" :: Horizontal and vertical field of view of the fisheye lenses in degrees
set "C=11.5" :: Width of interpolation band in degrees, must be smaller or equal than (FOV-180°)
set "T=20" :: Duration in seconds
set "S=1000" :: Output size
set "FPS=24" :: Output framerate
set "OUT=out.mp4" :: Output video

rem Create the mergemap file

ffmpeg -f lavfi -i nullsrc=size=%H%x%H% -vf "format=gray8,geq='clip(128-128/%C%*(180-%FOV%/(%H%/2)*hypot(X-%H%/2,Y-%H

%/2)),0,255)',v360=fishey[Link]ih_fov=%FOV%:iv_fov=%FOV%" -frames 1 -y [Link]

rem Merge the two fisheye images from the double-fisheye input video and transform it to a mirror-sphere video

ffmpeg -i %IN% -i [Link] -lavfi "[0]format=rgb24,split[a][b];

[a]crop=ih:iw/[Link],v360=input=fisheye:output=e:ih_fov=%FOV%:iv_fov=%FOV%[c];
[b]crop=ih:iw/2:iw/2:0,v360=fishey[Link]yaw=180:ih_fov=%FOV%:iv_fov=%FOV%[d];[1]format=gbrp[e];[c][d]
[e]maskedmerge,drawbox=x=3*iw/4:y=ih/2:w=1:h=1:color=black,v360=e:ball:roll=-90:w=%S%:h=%S%" -r %FPS% -t %T% -y %OUT%

pause

Note: For "equirectangular" to "ball" transformation, the color that's used for unmapped pixels is not located at position 0, 0 (as with most other
transformations), but instead at 3*iw/4, ih/2.

254
This batch file does the same thing as the previous one, but uses another method for filling the unmapped pixels with a color (alpha_mask, scale2ref,
overlay). Surprisingly this method is a little bit faster.
set "IN=R0010017.mp4" :: Input video
set "H=1920" :: Height of input video
set "FOV=191.5" :: Horizontal and vertical field of view of the fisheye lenses in degrees
set "C=11.5" :: Width of interpolation band in degrees, must be smaller or equal than (FOV-180°)
set "T=20" :: Duration in seconds
set "S=1000" :: Output size
set "FPS=24" :: Output framerate
set "OUT=out17.mp4" :: Output video

rem Create the mergemap file

ffmpeg -f lavfi -i nullsrc=size=%H%x%H% -vf "format=gray8,geq='clip(128-128/%C%*(180-%FOV%/(%H%/2)*hypot(X-%H%/2,Y-%H

%/2)),0,255)',v360=fishey[Link]ih_fov=%FOV%:iv_fov=%FOV%" -frames 1 -y [Link]

rem Merge the two fisheye images from the double-fisheye input video and transform it to a mirror-sphere video

ffmpeg -i %IN% -i [Link] -f lavfi -i color=black:s=2x2 -lavfi "[0]format=rgb24,split[a][b];

[a]crop=ih:iw/[Link],v360=input=fisheye:output=e:ih_fov=%FOV%:iv_fov=%FOV%[c];
[b]crop=ih:iw/2:iw/2:0,v360=fishey[Link]yaw=180:ih_fov=%FOV%:iv_fov=%FOV%[d];[1]format=gbrp[e];[c][d]
[e]maskedmerge,v360=e:ball:roll=-90:w=%S%:h=%S%:alpha_mask=1[f],[2][f]scale2ref,overlay=shortest=1" -r %FPS% -t %T% -y
%OUT%

pause

Note:
• First input of scale2ref: The video that shall be scaled
• Second input of scale2ref: The video which has the reference size
• First output of scale2ref: The scaled video
• Second output of scale2ref: A copy of the second input
• First input of overlay: The main (background) video
• Second input of overlay: The overlay (foreground) video

In this example scale2ref has no labels at its two output, which means overlay uses the same two streams in the same order as inputs.

255
This batch file does the same thing as the previous one, but uses the "geq" filter for filling the unmapped pixels with a color. This method is slower.
set "IN=R0010017.mp4" :: Input video
set "H=1920" :: Height of input video
set "FOV=191.5" :: Horizontal and vertical field of view of the fisheye lenses in degrees
set "C=11.5" :: Width of interpolation band in degrees, must be smaller or equal than (FOV-180°)
set "T=20" :: Duration in seconds
set "S=1000" :: Output size
set "FPS=24" :: Output framerate
set "OUT=out17.mp4" :: Output video

rem Create the mergemap file

ffmpeg -f lavfi -i nullsrc=size=%H%x%H% -vf "format=gray8,geq='clip(128-128/%C%*(180-%FOV%/(%H%/2)*hypot(X-%H%/2,Y-%H

%/2)),0,255)',v360=fishey[Link]ih_fov=%FOV%:iv_fov=%FOV%" -frames 1 -y [Link]

rem Merge the two fisheye images from the double-fisheye input video and transform it to a mirror-sphere video

ffmpeg -i %IN% -i [Link] -lavfi "[0]format=rgb24,split[a][b];

[a]crop=ih:iw/[Link],v360=input=fisheye:output=e:ih_fov=%FOV%:iv_fov=%FOV%[c];
[b]crop=ih:iw/2:iw/2:0,v360=fishey[Link]yaw=180:ih_fov=%FOV%:iv_fov=%FOV%[d];[1]format=gbrp[e];[c][d]
[e]maskedmerge,v360=e:ball:roll=-90:w=%S%:h=%S
%:alpha_mask=1,geq=r='if(gt(alpha(X,Y),127),r(X,Y),0)':g='if(gt(alpha(X,Y),127),g(X,Y),0)':b='if(gt(alpha(X,Y),127),b(X,
Y),0)'" -r %FPS% -t %T% -y %OUT%

pause

256
2.128 Shifting the viewing direction in a fisheye image or video

When you want to create a timelapse of many fisheye images, it may happen that one of the images isn't aligned correctly because the viewing direction
of the camera was off. With normal (non-fisheye) images that isn't a big problem, because you can simply re-align the image by shifting it in x and y
directions. However for fisheye images things are much more complicated. The required procedure is as follows:
1. Remap the fisheye image to an equirectangular 360° image. The lower part of the image remains black.
2. Apply two rotations to this equirectangular image.
3. Remap the equirectangular image back to a fisheye image.
set "IN=IMG_077.jpg" :: Input image or video
set "S=3648" :: Size of square fisheye input image
set "FOV=180" :: Fisheye field of view in degrees
set "X=15" :: Rotation angle around X axis
set "Y=0" :: Rotation angle around Y axis
set "Q=5" :: Size divider for the intermediate equirectangular image,
:: use 1 for best quality, or a bigger value for faster computing
set /a "H=%S%/%Q%" :: Height of equirectangular image
set /a "W=2*%H%" :: Width of equirectangular image is always twice the height
set /a "A=%H%*%FOV%/360" :: Height of equirectangular image that is actually filled with data, the rest remains black
set "OUT=[Link]" :: Output image or video

rem Create the xmap file for remapping from fisheye to equirectangular

ffmpeg -f lavfi -i nullsrc=size=%W%x%H% -vf format=pix_fmts=gray16le,^

geq='%S%/2*(1-Y/%A%*sin(X*2*PI/%W%))' -frames 1 -y [Link]

rem Create the ymap file for remapping from fisheye to equirectangular

ffmpeg -f lavfi -i nullsrc=size=%W%x%H% -vf format=pix_fmts=gray16le,^

geq='%S%/2*(1-Y/%A%*cos(X*2*PI/%W%))' -frames 1 -y [Link]

rem Create the xmap file for remapping from equirectangular to fisheye

257
ffmpeg -f lavfi -i nullsrc=size=%S%x%S% -vf format=pix_fmts=gray16le,^
geq='%H%*(0.9999+atan2(X-%S%/2,Y-%S%/2)/PI)' -frames 1 -y [Link]

rem Create the ymap file for remapping from equirectangular to fisheye

ffmpeg -f lavfi -i nullsrc=size=%S%x%S% -vf format=pix_fmts=gray16le,^

geq='%H%/360*%FOV%*(hypot((2*X/%S%)-1,(2*Y/%S%)-1))' -frames 1 -y [Link]

rem Remap from fisheye to equirectangular, apply the rotations, then remap back to fisheye

ffmpeg -i %IN% -i [Link] -i [Link] -i [Link] -i [Link] -filter_complex

"format=pix_fmts=rgb24,remap,v360=pitch=%Y%:roll=%X%:output=e[5];[5][3][4]remap" -y %OUT%

pause

The same thing can also be done with the v360 filter. In this example the top left pixel of the input image or video is set to a specific color with the
"drawbox" filter. This color is used for all those pixels in the output file, that aren't mapped to a pixel in the input file. Please note that this is an
undocumented feature of the v360 filter and it's not guaranteed that it works in all cases.
set "IN=[Link]" :: Input image or video
set "FOV=180" :: Field of view in degrees
set "PITCH=0" :: Rotation angle around X axis
set "YAW=30" :: Rotation angle around Y axis
set "C=green" :: Color for filling unused area
set "OUT=[Link]" :: Output image or video

ffmpeg -i %IN% -vf drawbox=w=1:h=1:color=%C%,v360=input=fisheye:ih_fov=%FOV%:iv_fov=%FOV%:output=fisheye:h_fov=%FOV

%:v_fov=%FOV%:yaw=%YAW%:pitch=%PITCH% -y %OUT%

pause

The v360 filter does have the "alpha_mask" option. If this option is set, all unused pixels in the output file are set to maximum transparency, so that the
overlay filter can be used for filling this area with a color. This example does exactly the same thing as the previous example. Decide yourself which one
is easier or faster:
set "IN=[Link]" :: Input image or video

258
set "FOV=180" :: Field of view in degrees
set "PITCH=0" :: Rotation angle around X axis
set "YAW=30" :: Rotation angle around Y axis
set "C=green" :: Color for filling unused area
set "OUT=[Link]" :: Output image or video

ffmpeg -i %IN% -f lavfi -i color=%C%:s=1200x1200 -filter_complex v360=input=fisheye:ih_fov=%FOV%:iv_fov=%FOV

%:output=fisheye:h_fov=%FOV%:v_fov=%FOV%:yaw=%YAW%:pitch=%PITCH%:alpha_mask=1[a],[1][a]overlay -frames 1 -y %OUT%

pause

Note: If the input is a video, remove the -frames 1 option.

See also [Link]/dome/fishtilt/

2.129 How the "drawbox" filter works

ffmpeg -f lavfi -i color=gray:s=20x20 -vf format=rgb24,drawbox=x=4:y=4:width=6:height=6:thickness=1:color=red -frames 1

-y [Link]

pause

The top left pixel of the box is at coordinates x, y.

The bottom right pixel of the box is at coordinates (x+width-1), (y+height-1).
The width and height of the box are exactly "width" and "height", independent of the thickness.
Setting the thickness greater than 1 doesn't change the outer dimensions of the box.
The number of pixels inside the box is (width-2*thickness), (height-2*thickness).
If you want a number of A pixels inside the box, you must set width or height to (A+2*thickness).

259
2.130 Stitching together double-fisheye videos

The result is an equirectangular panorama video.

set "IN=double_fisheye.jpg" :: Input video or picture
set "X1=198" :: X coordinate of center of left fisheye image
set "Y1=210" :: Y coordinate of center of left fisheye image
set "X2=595" :: X coordinate of center of right fisheye image
set "Y2=210" :: Y coordinate of center of right fisheye image
set "SR=192" :: Radius that is actually used from the source video
set "PW=1920" :: Width of panorama video
set "PH=960" :: Height of panorama video
set "OUT=[Link]" :: Output video or picture

rem Create the xmap file

ffmpeg -f lavfi -i nullsrc=size=%PW%x%PH% -vf format=pix_fmts=gray16le,geq='if(lt(Y,%PH%/2),%X1%-Y*2*%SR%/%PH

%*sin(X*2*PI/%PW%),%X2%+(%PH%-Y)*2*%SR%/%PH%*sin(X*2*PI/%PW%))' -frames 1 -y [Link]

rem Create the ymap file

ffmpeg -f lavfi -i nullsrc=size=%PW%x%PH% -vf format=pix_fmts=gray16le,geq='if(lt(Y,%PH%/2),%Y1%-Y*2*%SR%/%PH

%*cos(X*2*PI/%PW%),%Y2%-(%PH%-Y)*2*%SR%/%PH%*cos(X*2*PI/%PW%))' -frames 1 -y [Link]

rem Apply the remap filter to the video

ffmpeg -i %IN% -i [Link] -i [Link] -lavfi "format=pix_fmts=rgb24,remap" -q:v 2 -y %OUT%

pause

The parameters X1, Y1, X2, Y2 and SR must be carefully adjusted (by try and error) to get a good stitching result. They depend on the size of the source
video or picture. Use these values as a starting point: X1=width/4, Y1=height/2, X2=width*3/4, Y2=height/2, SR=height/2. the following table shows how
the parameters affect the stitching.
Note: The same thing can also be done with the V360 filter, see the next chapter.

260
Parameter Result when decreasing the parameter Result when increasing the parameter
+-------------------------------+ +-------------------------------+
| upper half from left fisheye | | upper half from left fisheye |
| ← ↑ → ↓ ← | | → ↓ ← ↑ → |
X1 +-------------------------------+ +-------------------------------+
| | | |
| lower half from right fisheye | | lower half from right fisheye |
+-------------------------------+ +-------------------------------+
+-------------------------------+ +-------------------------------+
| upper half from left fisheye | | upper half from left fisheye |
| ↑ → ↓ ← ↑ | | ↓ ← ↑ → ↓ |
Y1 +-------------------------------+ +-------------------------------+
| | | |
| lower half from right fisheye | | lower half from right fisheye |
+-------------------------------+ +-------------------------------+
+-------------------------------+ +-------------------------------+
| upper half from left fisheye | | upper half from left fisheye |
| | | |
X2 +-------------------------------+ +-------------------------------+
| → ↑ ← ↓ → | | ← ↓ → ↑ ← |
| lower half from right fisheye | | lower half from right fisheye |
+-------------------------------+ +-------------------------------+
+-------------------------------+ +-------------------------------+
| upper half from left fisheye | | upper half from left fisheye |
| | | |
Y2 +-------------------------------+ +-------------------------------+
| ↓ → ↑ ← ↓ | | ↑ ← ↓ → ↑ |
| lower half from right fisheye | | lower half from right fisheye |
+-------------------------------+ +-------------------------------+
+-------------------------------+ +-------------------------------+
| upper half from left fisheye | | upper half from left fisheye |
| ↓ ↓ ↓ ↓ ↓ | | ↑ ↑ ↑ ↑ ↑ |
SR +-------------------------------+ +-------------------------------+
| ↑ ↑ ↑ ↑ ↑ | | ↓ ↓ ↓ ↓ ↓ |
| lower half from right fisheye | | lower half from right fisheye |
+-------------------------------+ +-------------------------------+

261
2.131 Remove stitching artefacts

When double-fisheye images are stitched together to an equirectangular image, it's possible that stitching artefacts are visible as two vertical lines where
the luminance fron the two images doesn't fit together. These artefacts can be removed by applying a suitable luminance gradient at one or both sides of
the border. This example applies the gradient to the left side of two vertical borders:
set "IN=[Link]" :: Input image
set "B1=250" :: Right side of first vertical border, left side is at B1-1
set "B2=750" :: Right side of second vertical border, left side is at B2-1
set "W=25" :: Width of interpolation area

ffmpeg -i %IN% -vf "geq=cb_expr='cb(X,Y)':cr_expr='cr(X,Y)':lum_expr='clip(lum(X,Y)+between(X,%B1%-1-%W%,%B1%-

1)*lerp(0,lum(%B1%,Y)-lum(%B1%-1,Y),(X-%B1%-1+%W%)/%W%)+between(X,%B2%-1-%W%,%B2%-1)
*lerp(0,lum(%B2%,Y)-lum(%B2%-1,Y),(X-%B2%-1+%W%)/%W%),0,255)',format=rgb24" -y [Link]

pause

How it works:

In the area of width W to the left side of the vertical border, a ramp is added
to the luminance. The amplitude of this ramp equals the difference of the
luminance values left and right of the border.
You have to know in advance where exactly the vertical borders are.

262
Same as previous example, but now applying the gradient to the left side of the first border and to the right side of the second border:
set "IN=[Link]" :: Input image
set "B1=250" :: Right side of first vertical border, left side is at B1-1
set "B2=750" :: Right side of second vertical border, left side is at B2-1
set "W=25" :: Width of interpolation area

ffmpeg -i %IN% -vf "geq=cb_expr='cb(X,Y)':cr_expr='cr(X,Y)':lum_expr='clip(lum(X,Y)+

between(X,%B1%-1-%W%,%B1%-1)*lerp(0,lum(%B1%,Y)-lum(%B1%-1,Y),(X-%B1%+1+%W%)/%W%)+
between(X,%B2%,%B2%+%W%)*lerp(lum(%B2%-1,Y)-lum(%B2%,Y),0,(X-%B2%)/%W%),0,255)',format=rgb24" -y [Link]

pause

Same as previous examples, but now applying half of the gradient to the left side and the other half to the right side of both borders:
set "IN=[Link]" :: Input image
set "B1=250" :: Right side of first vertical border, left side is at B1-1
set "B2=750" :: Right side of second vertical border, left side is at B2-1
set "W=25" :: Half width of interpolation area

ffmpeg -i %IN% -vf "geq=cb_expr='cb(X,Y)':cr_expr='cr(X,Y)':lum_expr='clip(lum(X,Y)+0.5*(

between(X,%B1%-1-%W%,%B1%-1)*lerp(0,lum(%B1%,Y)-lum(%B1%-1,Y),(X-%B1%-1+%W%)/%W%)+
between(X,%B2%-1-%W%,%B2%-1)*lerp(0,lum(%B2%,Y)-lum(%B2%-1,Y),(X-%B2%-1+%W%)/%W%)+
between(X,%B1%,%B1%+%W%)*lerp(lum(%B1%-1,Y)-lum(%B1%,Y),0,(X-%B1%)/%W%)+
between(X,%B2%,%B2%+%W%)*lerp(lum(%B2%-1,Y)-lum(%B2%,Y),0,(X-%B2%)/%W%)),0,255)',format=rgb24" -y [Link]

pause

Remove the line feeds from the command line, which were only inserted for clarity.
Please note that workarounds with geq filter are quite slow.

263
This is an example for merging two overlapping fisheye videos, realized with the "maskedmerge" filter:
set "IN=double_fisheye.mp4" :: Input video
set "H=640" :: Height of input video
set "FOV=191.5" :: Horizontal and vertical field of view of the fisheye lenses in degrees
set "C=11.5" :: Width of interpolation band in degrees, must be smaller or equal than (FOV-180°)
set "T=10" :: Duration in seconds
set "OUT=out.mp4" :: Output video

rem Create the mergemap file

ffmpeg -f lavfi -i nullsrc=size=%H%x%H% -vf "format=gray8,geq='clip(128-128/%C%*(180-%FOV%/(%H%/2)*hypot(X-%H%/2,Y-%H

%/2)),0,255)',v360=input=fisheye:output=e:ih_fov=%FOV%:iv_fov=%FOV%" -frames 1 -y [Link]

rem Merge the two fisheye images from the double-fisheye input video

ffmpeg -i %IN% -i [Link] -lavfi "[0]format=rgb24,split[a][b];

[a]crop=ih:iw/[Link],v360=input=fisheye:output=e:ih_fov=%FOV%:iv_fov=%FOV%[c];
[b]crop=ih:iw/2:iw/2:0,v360=input=fisheye:output=e:yaw=180:ih_fov=%FOV%:iv_fov=%FOV%[d];[1]format=gbrp[e];[c][d]
[e]maskedmerge" -t %T% -y %OUT%

pause
Tested with this input video, downloaded in 1280x640 size: [Link]
Note: The FOV variable must be set to the correct field of view of the fisheye lenses. The procedure for finding the best value for "FOV" is as follows: Set
"C" to a very small value (for example 0.5 degrees), then find the best FOV value by try and error, then set "C" to a larger value, for example 10 degrees.
Note: The "maskedmerge" filter expects the mergemap in the same pixel format as it processes the first two inputs, and these are (in this case)
automatically converted to the planar gbrp pixel format. This is hard to find out, because it's not well documented. That's why the mergemap must be
converted to gbrp pixel format as well.
Note: Pixel formats can be checked in the filter chain by inserting the "showinfo" filter. Another method for checking where Ffmpeg did auto-insert format
conversions is to use "-v verbose" or (for even more informations) "-v debug". But it's quite hard to find the relevant informations in the long listing.

For comparison, this is the same as the previous example, but it's just hard stitching the two fisheye videos together, without any merging. Tested with
the same input video as the previous example.
set "IN=double_fisheye.mp4" :: Input video
set "FOV=191.5" :: Field ov view of the fisheye lenses, over full image height,
:: find the best value by try and error

264
set "T=10" :: Duration in seconds
set "OUT=out.mp4" :: Output video

ffmpeg -i %IN% -vf "v360=input=dfisheye:output=e:ih_fov=%FOV%:iv_fov=%FOV%" -t %T% -y %OUT%

pause

The following example is for converting a dual-fisheye video from a Ricoh Theta camera to an equirectangular video.
The problem with this input video is that the size is 1920x1080, which is not a 2:1 aspect ratio as it should be. The input video has a black border at the
bottom which must be cropped away, so that the height is reduced to 960.
set "IN=theta.mp4" :: Input video
set "H=960" :: Half of the image width = height of input image after cropping
set "FOV=204" :: Horizontal and vertical field of view of the fisheye lenses in degrees
set "C=10" :: Width of interpolation band in degrees, must be smaller or equal than (FOV-180°)
set "OUT=out.mp4" :: Output video

rem Create the mergemap file

ffmpeg -f lavfi -i nullsrc=size=%H%x%H% -vf "format=gray8,geq='clip(128-128/%C%*(180-%FOV%/(%H%/2)*hypot(X-%H%/2,Y-%H

%/2)),0,255)',v360=input=fisheye:output=e:ih_fov=%FOV%:iv_fov=%FOV%" -frames 1 -y [Link]

rem Merge the two fisheye images from the double-fisheye input video

ffmpeg -i %IN% -i [Link] -lavfi "[0]crop=h=%H%:y=0,format=rgb24,split[a][b];

[a]crop=ih:iw/[Link],v360=input=fisheye:output=e:ih_fov=%FOV%:iv_fov=%FOV%[c];
[b]crop=ih:iw/2:iw/2:0,v360=input=fisheye:output=e:yaw=180:ih_fov=%FOV%:iv_fov=%FOV%[d];[1]format=gbrp[e];[c][d]
[e]maskedmerge" -y %OUT%

pause

If you want to rotate another part of the video into the center, insert one more v360 filter after "maskedmerge" and use the rorder/yaw/pitch/roll rotation
options:
... [1]maskedmerge,v360=input=e:output=e:rorder=rpy:roll=-95:pitch=-18" -y %OUT%

265
2.132 Stitch double-fisheye images with alignment errors

If the optical axes of the lenses of a 360° camera aren't aligned to exactly opposite direction, the images don't fit together at the borders. We can assume
that the first camera's axis is perfect and the second camera has three alignment errors: Yaw, pitch and roll. These errors can be corrected before
stitching the images together.
Step 1: Make perfect double-fisheye and equirectangular test images (that means without alignment errors)
set "IN=[Link]" :: Test pattern from [Link]
set "OUT=double_fisheye.png"

ffmpeg -i %IN% -i %IN% -lavfi "[0]transpose=1[left];[1]transpose=2,negate[right];[left][right]hstack" -y %OUT%

set "IN=double_fisheye.png"
set "OUT=[Link]"

ffmpeg -i %IN% -lavfi "v360=input=dfisheye:output=e:ih_fov=180:iv_fov=180" -y %OUT%

pause
These are the perfect double-fisheye and equirectangular test images:

266
Step 2: Make a double-fisheye test image with alignment errors
set "IN=[Link]"
set "OUT=misaligned_double_fisheye.png"
set "FOV=200" :: field of view
set "Y=-8" :: yaw error of right fisheye lens
set "P=5" :: pitch error of right fisheye lens
set "R=7" :: roll error of right fisheye lens

ffmpeg -i %IN% -lavfi split[a][b];[a]v360=e:fisheye:h_fov=%FOV%:v_fov=%FOV%[a];[b]v360=e:fisheye:h_fov=%FOV%:v_fov=%FOV

%:rorder=ypr:yaw='180+%Y%':pitch=%P%:roll=%R%[b];[a][b]hstack -y %OUT%

pause

This is the double-fisheye test image with alignment errors in the right half. You can see that it's shifted to the right and bottom, and rotated counter-
clockwise:

267
Step 3: When you now stich the fisheye images together, you will see the alignment errors:
set "IN=misaligned_double_fisheye.png"
set "OUT=equirect_misaligned.png"
set "FOV=200" :: field of view

ffmpeg -i %IN% -lavfi "split[a][b];[a]crop=ih:ih:0:0,v360=fisheye:fisheye:ih_fov=%FOV%:iv_fov=%FOV

%:h_fov=180:v_fov=180[a];[b]crop=ih:ih:ih:0,v360=fisheye:fisheye:ih_fov=%FOV%:iv_fov=%FOV%:h_fov=180:v_fov=180[b];[b]
[a]hstack,v360=dfishey[Link]ih_fov=180:iv_fov=180" -y %OUT%

pause

This is the stitched equirectangular image, you can clearly see the alignment errors:

268
Step 4: Compensate the alignment errors before stitching the images together:
set "IN=misaligned_double_fisheye.png"
set "OUT=equirect_corrected.png"
set "FOV=200" :: field of view
set "Y=8" :: yaw error of right fisheye lens
set "P=-5" :: pitch error of right fisheye lens
set "R=-7" :: roll error of right fisheye lens

ffmpeg -i %IN% -lavfi "split[a][b];[a]crop=ih:ih:0:0,v360=fisheye:fisheye:ih_fov=%FOV%:iv_fov=%FOV

%:h_fov=180:v_fov=180[a];[b]crop=ih:ih:ih:0,v360=fisheye:fisheye:ih_fov=%FOV%:iv_fov=%FOV
%:h_fov=180:v_fov=180:rorder=rpy:yaw=%Y%:pitch=%P%:roll=%R%[b];[b][a]hstack,v360=dfishey[Link]ih_fov=180:iv_fov=180" -y
%OUT%

pause

This is the corrected equirectangular output image:

The tricky part is to find the best values for FOC, Y, P and R by try and error. In this example I did already know the correct values, they are the same as in
step 2 but with opposite sign. Please note that the rotation order must also be reversed.

269
2.133 Stitching multiple flat images to a 360° image

Found the idea here: [Link]

rem Make a white mask image with a 50% gray border

ffmpeg -f lavfi -i color=color=white:size=1920x1080 -lavfi drawbox=color=gray:t=1 -frames 1 -y [Link]

rem Alternatively make a white mask image with 33% and 67% gray at the border

ffmpeg -f lavfi -i color=color=white:size=1920x1080 -lavfi

drawbox=color=0x555555:t=1,drawbox=color=0xAAAAAA:x=1:y=1:w=iw-2:h=ih-2:t=1 -frames 1 -y [Link]

rem Apply the mask to the input image or video

ffmpeg -f lavfi -i testsrc2=size=1920x1080 -i [Link] -lavfi [0]format=rgb24[a];[1]format=rgb24[b];[a]

[b]blend=all_mode=multiply -frames 1 -y [Link]

rem Alternatively, the same thing cam also be done with the multiply filter:

ffmpeg -f lavfi -i testsrc2=size=1920x1080 -i [Link] -lavfi multiply=offset=0 -frames 1 -y [Link]

rem Stitch together multiple flat images to a 360° image or video

ffmpeg -i [Link] -lavfi ^

[0]v360=input=flat:output=e:id_fov=45:w=5120:h=2560:yaw=0:alpha_mask=1[fg1];^
[0]v360=input=flat:output=e:id_fov=45:w=5120:h=2560:yaw=90:alpha_mask=1[fg2];[fg1][fg2]overlay[a];^
[0]v360=input=flat:output=e:id_fov=45:w=5120:h=2560:yaw=-90:alpha_mask=1[fg3];[fg3][a]overlay[b];^
[0]v360=input=flat:output=e:id_fov=45:w=5120:h=2560:yaw=180:alpha_mask=1[fg4];[fg4][b]overlay[c];^
[0]v360=input=flat:output=e:id_fov=45:w=5120:h=2560:yaw=0:pitch=45:alpha_mask=1[fg5];[fg5][c]overlay[d];^
[0]v360=input=flat:output=e:id_fov=45:w=5120:h=2560:yaw=90:roll=45:alpha_mask=1[fg6];[fg6][d]overlay[e];^
[0]v360=input=flat:output=e:id_fov=45:w=5120:h=2560:yaw=-90:roll=-45:alpha_mask=1[fg7];[fg7][e]overlay[f];^
[0]v360=input=flat:output=e:id_fov=45:w=5120:h=2560:yaw=180:pitch=-45:alpha_mask=1[fg8];[fg8][f]overlay[g];^
[0]v360=input=flat:output=e:id_fov=45:w=5120:h=2560:yaw=0:pitch=-45:alpha_mask=1[fg9];[fg9][g]overlay[h];^

270
[0]v360=input=flat:output=e:id_fov=45:w=5120:h=2560:yaw=90:roll=-45:alpha_mask=1[fg10];[fg10][h]overlay[i];^
[0]v360=input=flat:output=e:id_fov=45:w=5120:h=2560:yaw=-90:roll=45:alpha_mask=1[fg11];[fg11][i]overlay[j];^
[0]v360=input=flat:output=e:id_fov=45:w=5120:h=2560:yaw=180:pitch=45:alpha_mask=1[fg12];[fg12][j]overlay[k];^
[0]v360=input=flat:output=e:id_fov=45:w=5120:h=2560:yaw=0:pitch=90:alpha_mask=1[fg13];[fg13][k]overlay[l];^
[0]drawbox=w=1:h=1:color=black,v360=input=flat:output=e:id_fov=45:w=5120:h=2560:yaw=0:pitch=-90:alpha_mask=1[fg14];
[fg14][l]overlay -frames 1 -y [Link]

pause

271
2.134 Preprocessing a flat video for fulldome projection

If a flat video is to be shown in a fulldome planetarium with a fisheye projector, some preprocessing is required. The video is downscaled to a smaller
size, padded with large black borders to equirectangular 2:1 format, rotated with the v360 filter, and then given out in 180° fisheye output.
set "IN=pk14.mp4" :: Input video
set "UP=35" :: Up-looking angle in degrees (center of the rectangular video)
set "W=480" :: Width of input video after downscaling, this is for 16:9 aspect ratio
set "H=270" :: Height of input video after downscaling, this is for 16:9 aspect ratio
set "S=1200" :: Size of square fisheye output video
set "OUT=out.mp4" :: Output video

ffmpeg -i %IN% -lavfi "scale=%W%:%H%,pad='2*%S%':%S%:-1:-

1,format=pix_fmts=rgb24,v360=input=equirect:output=fisheye:h_fov=180:v_fov=180:pitch='90-%UP%'" -y %OUT%

pause

It's also possible to use the flat video directly as input for the v360 filter. This has the problem that the unused area is filled with a random color (coming
from the top left pixel of the input video). As a workaround, this pixel is filled with black before using the v360 filter:
set "IN=pk14.mp4" :: Input video
set "UP=30" :: Up-looking angle in degrees (center of the rectangular video)
set "H=64" :: Horizontal field of view, this is for 16:9 aspect ratio
set "V=36" :: Vertical field of view, this is for 16:9 aspect ratio
set "OUT=out.mp4" :: Output video

ffmpeg -i %IN% -vf drawbox=w=1:h=1:color=black,v360=input=flat:ih_fov=%H%:iv_fov=%V

%:output=fisheye:h_fov=180:v_fov=180:pitch='90-%UP%' -y %OUT%

pause

With sufficient computing power live processing is possible. Just drag and drop the input video over the icon of this batch file:
set "UP=30" :: Up-looking angle in degrees (center of the rectangular video)
set "H=64" :: Horizontal field of view, this is for 16:9 aspect ratio
set "V=36" :: Vertical field of view, this is for 16:9 aspect ratio

272
ffmpeg -re -i %1 -vf drawbox=w=1:h=1:color=black,v360=input=flat:ih_fov=%H%:iv_fov=%V
%:output=fisheye:h_fov=180:v_fov=180:pitch='90-%UP%' -window_fullscreen 1 -f sdl2 -

Please note that the sdl2 output doesn't play audio. The Windows taskbar remains visible in fullscreen mode. You can hide it as follows: Make a right
click on the taskbar, click on "properties" and then select "automatically hide taskbar".

This is an example for live processing and passing the output to FFplay. Just drag and drop the input video over the icon of this batch file. FFplay has the
advantage that it does also play audio, and the Windows taskbar is automatically hidden:
set "UP=30" :: Up-looking angle in degrees (center of the rectangular video)
set "H=64" :: Horizontal field of view, this is for 16:9 aspect ratio
set "V=36" :: Vertical field of view, this is for 16:9 aspect ratio

ffmpeg -re -i %1 -vf drawbox=w=1:h=1:color=black,v360=input=flat:ih_fov=%H%:iv_fov=%V

%:output=fisheye:h_fov=180:v_fov=180:pitch='90-%UP%' -q:v 2 -c:v mpeg4 -f nut - | c:\ffmpeg
\ffplay -fs -autoexit -

The -fs option means full screen, and -autoexit means that FFplay closes automatically when the end of the video has been reached.

In this example a 180° fisheye image is used as the background and a flat image is overlaid above the horizon. This should also work for videos instead
of images.
set "BG=[Link]" :: Fisheye background image (or video)
set "FG=[Link]" :: Flat foreground image (or video)
set "UP=15" :: This angle is the height of the center of the flat image above the horizon (in degrees)
set "ID=45" :: Diagonal of the foreground image in degrees
set "S=3648" :: Size of the square fisheye input and output images (or videos)
set "OUT=[Link]" :: Output image (or video)

ffmpeg -i %BG% -i %FG% -lavfi "[1]v360=input=flat:output=fisheye:id_fov=%ID%:h_fov=180:v_fov=180:w=%S%:h=%S%:pitch='90-

%UP%':alpha_mask=1[fg];[0][fg]overlay" -y %OUT%

pause

273
This is the same as before, but the flat image is duplicated, so that the same image is shown at the north and south horizon:
set "BG=[Link]" :: Fisheye background image (or video)
set "FG=[Link]" :: Flat foreground image (or video)
set "UP=15" :: This angle is the height of the center of the flat image above the horizon (in degrees)
set "ID=45" :: Diagonal of the foreground image in degrees
set "S=3648" :: Size of the square fisheye input and output images (or videos)
set "OUT=[Link]" :: Output image (or video)

ffmpeg -i %BG% -i %FG% -lavfi "[1]v360=input=flat:output=fisheye:id_fov=%ID%:h_fov=180:v_fov=180:w=%S%:h=%S%:pitch='90-

%UP%':alpha_mask=1,split[fg1][fg2];[fg2]rotate=PI[fg3];[0][fg1]overlay[a];[a][fg3]overlay" -y %OUT%

pause

Some explanations for this example:

[0] and [1] are predefined labels for the input files.
[0] is the first input, in this case "[Link]"
[1] is the second input, in this case "in.mp4"
[fg1] [fg2] [fg3] and [a] are just labels and you can change them if you want.
"alpha_mask" is an option of the v360 filter. In this case it's used to make the outer area of the rectangular video transparent.
[Link]
"split" is a filter that has one input and two (or more) outputs. It is here used to duplicate the foreground video.
[Link]
"rotate" is a filter that rotates the video. In this case it's used to rotate one of the foreground videos to the other side of the dome.
[Link]
"overlay" is a filter that has two inputs and one output. Because in this case we want to overlay two videos, we must use it two times.
[Link]

274
2.135 Rotating the earth, moon or planet

If the surface of the planet is given as an equirectangular image, then things are quite simple:
set "IN=Earth_eq.jpg" :: Equirectangular image of earth or planet surface, for example from:
:: [Link]
set "BG=[Link]" :: Background image
set "P=-50" :: Pitch angle
set "R=30" :: Roll angle
set "S=-0.005" :: Rotation speed, 1.0 means one full revolution per frame
set "D=200" :: Diameter of planet
set "XP=900" :: X position of planet
set "YP=450" :: y position of planet
set "T=10" :: Length of output video

ffmpeg -loop 1 -i %BG% -loop 1 -i %IN% -lavfi "[1]scroll=h=%S%,v360=e:perspective:pitch=%P%:roll=%R%:alpha_mask=1,scale=

%D%:%D%[a],[0][a]overlay=x=%XP%:y=%YP%" -t %T% -y out.mp4

pause

275
This is the same as the previous example, but with day/night and twilight zone added:
set "IN=Earth_eq.jpg" :: Equirectangular image of earth or planet surface, for example from:
:: [Link]
set "BG=[Link]" :: Background image
set "SIZE=1024x512" :: Size for mergemap, must be the same as the equirectangular input video
set "TW=20" :: Width of twilight zone in degrees
set "DEC=23.5" :: Declination of light source (sun), 0° for spring equinox, +23.5° for summer solstice,
:: 0° for fall equinox, -23.5° for winter solstice
set "DS=10" :: Brightness of the dark side, 0 for black, 255 for full brightness
set "S=-0.005" :: Rotation speed, 1.0 means one full revolution per frame
set "Y=90" :: Yaw angle
set "P=50" :: Pitch angle
set "R=0" :: Roll angle
set "D=400" :: Diameter of planet
set "XP=800" :: X position of planet
set "YP=400" :: y position of planet
set "T=10" :: Length of output video

ffmpeg -f lavfi -i nullsrc=size=%SIZE% -vf "format=gray8,geq='clip((1+180*(Y-H/2)*128/(%TW%*(H/2))),%DS

%,255)',v360=[Link]pitch=%DEC%+90,format=rgb24" -frames 1 -y [Link]

ffmpeg -loop 1 -i %BG% -loop 1 -i %IN% -f lavfi -i color=black:size=%SIZE%,format=rgb24 -i [Link] -lavfi

"[1]scroll=h=%S%,format=rgb24[a];[2][a][3]maskedmerge,v360=e:perspective:yaw=%Y%:pitch=%P%:roll=%R%:alpha_mask=1,scale=
%D%:%D%[a],[0][a]overlay=x=%XP%:y=%YP%" -t %T% -y out.mp4

pause

Warning: Don't use the v360 filter's "perspective" projection if you need mathematically correct behaviour. The "perspective" projection depends
somehow on the option "v_fov", but the exact meaning of this option is unknown in this context. Better use a workaround with "remap" filter. I did try to
reverse-engineer the source code to find out the meaning of the "v_fov" option, but this approach wasn't successful.
If a normal perspective image of the planet is given, then things are more complicated. It's clear that the perspectice image contains only half of the
planet's surface. The other half must be replaced by a color.
Another problem is that FFmpeg's "v360" filter allows perspective images only for output, but not for input. In this case a workaround with the "remap"
filter is required. As described above, it's also recommended to use the "remap" workaround for "perspective" output.

276
Now I'd like to describe how to create an image of the moon, as it appears from the side. It's impossible to take such an image from the earth, because
the moon is always facing (roughly) the same side towards the earth. The first step is to take an image of the moon with a telescope or a telephoto lens.
The second step is to measure in this image the moon's center coordinates and its radius. In case of a full moon this is quite simple, but for other moon
phases it's more difficult. One approach is to measure the coordinates of three points on the moon's edge. These points should have wide spacing from
each other. From the coordinates of these three points it's possible to calculate the center coordinates and the radius.
Because it's time consuming to do this with a pocket calculator, I wrote a small C# program for the job. The source code can be downloaded here:
[Link]
Now the center coordinates and the radius are known and can be inserted in this batch file. In the output image the observer is rotated 61° to the left:
set "IN=[Link]" :: Input image
set "XC=927.7" :: X coordinate of center of planet in input image
set "YC=2310.3" :: Y coordinate of center of planet in input image
set "R=2070.0" :: Radius of planet in input image
set "W=4000" :: Width of intermediate equirectangular image
set "H=2000" :: Height of intermediate equirectangular image
set "FILL=blue" :: Fill color for no-data area
set "YAW=61" :: Rotation angle
set "B=250" :: Width of added border
set "OUT=[Link]" :: Output image

rem Create the xmap file, from "perspective" input image to intermediate equirectangular image

ffmpeg -f lavfi -i nullsrc=size=%W%x%H% -vf format=gray16le,geq='st(0,(X-%H%)*PI/%H%);if(between(ld(0),-0.5*PI,0.5*PI),

%XC%+%R%*sin(ld(0))*cos((Y-0.5*%H%)*PI/%H%),-1)' -frames 1 -y [Link]

rem Create the ymap file, from "perspective" input image to intermediate equirectangular image

ffmpeg -f lavfi -i nullsrc=size=%W%x%H% -vf format=gray16le,geq='st(0,(X-%H%)*PI/%H%);if(between(ld(0),-0.5*PI,0.5*PI),

%YC%+%R%*sin((Y-0.5*%H%)*PI/%H%),-1)' -frames 1 -y [Link]

rem Rotate the moon

ffmpeg -i %IN% -i [Link] -i [Link] -lavfi "format=rgb24,remap=fill=%FILL%,v360=e:perspective:yaw=%YAW

%:alpha_mask=1,pad=iw+2*%B%:ih+2*%B%:%B%:%B%" -y %OUT%

pause
Please note that the above batch file uses the incorrect "perspective" output of the v360 filter. The next batch file is better. It uses the "remap"
workaround instead of the "perspective" projection of the "v360" filter:

277
set "IN=[Link]" :: Input image
set "XC=927.7" :: X coordinate of center of planet in input image
set "YC=2310.3" :: Y coordinate of center of planet in input image
set "R=2070.0" :: Radius of planet in input image
set "H=5000" :: Height of intermediate equirectangular image
set "S=5000" :: Width and height of output image
set "R2=2400" :: Radius of moon in output image
set "FILL=blue" :: Fill color for no-data area
set "LON=60" :: Rotation angle in longitude
set "LAT=0" :: Rotation angle in latitude
set "OUT=[Link]" :: Output image
set /a "W=2*%H%"
set /a "S2=%S%/2"

rem Create the xmap1 file, from "perspective" input image to intermediate equirectangular image

ffmpeg -f lavfi -i nullsrc=size=%W%x%H% -vf format=gray16le,geq='st(0,(X-%H%)*PI/%H%);if(between(ld(0),-0.5*PI,0.5*PI),

%XC%+%R%*sin(ld(0))*cos((Y-0.5*%H%)*PI/%H%),-1)' -frames 1 -y [Link]

rem Create the ymap1 file, from "perspective" input image to intermediate equirectangular image

ffmpeg -f lavfi -i nullsrc=size=%W%x%H% -vf format=gray16le,geq='st(0,(X-%H%)*PI/%H%);if(between(ld(0),-0.5*PI,0.5*PI),

%YC%+%R%*sin((Y-0.5*%H%)*PI/%H%),-1)' -frames 1 -y [Link]

rem Create the xmap2 file, from intermediate equirectangular image to "perspective" output image

ffmpeg -f lavfi -i nullsrc=size=%S%x%S% -vf format=gray16le,geq='if(lt(hypot(X-%S2%,Y-%S2%),%R2%),%H%*(1+asin((X-%S2%)/

%R2%)/cos(asin((Y-%S2%)/%R2%))/PI),-1)' -frames 1 -y [Link]

rem Create the ymap2 file, from intermediate equirectangular image to "perspective" output image

ffmpeg -f lavfi -i nullsrc=size=%S%x%S% -vf format=gray16le,geq='if(lt(hypot(X-%S2%,Y-%S2%),%R2%),%H%*(0.5+asin((Y-

%S2%)/%R2%)/PI),-1)' -frames 1 -y [Link]

rem Rotate the moon

ffmpeg -i %IN% -i [Link] -i [Link] -i [Link] -i [Link] -lavfi "format=rgb24[5];[5][1][2]remap=fill=%FILL

%,v360=[Link]yaw=%LON%:pitch=%LAT%[6];[6][3][4]remap" -y %OUT%

pause

278
Here are the input and output images. Input image taken by me 2018-05-20. You can see in the input image that Mare Crisium (at 2 o'clock) appears
elongated in N-S direction, but when the point of view was moved 60° to the right you realize that in fact it's elongated in E-W direction. The no-data area
is shown blue:

279
The script does also work if the input image contains only a part of the moon. The only requirement is that a large enough part of the moon's edge is
visible, for calculating the center coordinates and the radius. It's no problem if the moon's center is outside the image. In this image you see Mare
Marginis and Mare Smythii, which are right of Mare Crisium:

Compare with a real map of the moon (select the projection "Lunar Globe 3D"): [Link]

280
2.136 Live rotation of the moon

A live video of the moon is taken with the GH5S camera, connected to a telescope. The signal is captured with a HDMI to USB converter, and then
processed in realtime so that the libration area becomes visible. The left half of the display shows the input video with a red frame. The telescope must
be positioned so that the moon fits exactly in the frame. The right half of the display shows the rotated moon. Some variables must be set in the batch
file before running: Moon radius, latitude and longitude rotation angles.
set "IW=1920" :: Width of the input video
set "IH=1080" :: Height of the input video
set "R=512" :: Radius of moon in input image, should be less than IH/2
:: R = W * D * pi * F / 21600 / C
:: with W = width in pixels (output of HDMI converter), here 1920
:: D = diameter of moon in arc minutes, here 31.75'
:: F = focal length of telescope in mm, here 1040
:: C = chip width in mm, here 18.8 for GH5S in FHD or 4K mode
set "H=1700" :: Height of intermediate equirectangular image, a good value is input height * pi / 2
set "S=960" :: Width and height of each half of the output image
set "R2=450" :: Radius of moon in output image, should be smaller than S/2
set "FRAME=red" :: Color of frame
set "FILL=blue" :: Fill color for no-data area
set "LON=60" :: Longitude rotation angle
set "LAT=0" :: Latitude rotation angle
set /a "IH2=%IH%/2"
set /a "W=2*%H%"
set /a "S2=%S%/2"

rem Create the xmap1 file, from "perspective" input image to intermediate equirectangular image

ffmpeg -f lavfi -i nullsrc=size=%W%x%H% -vf format=gray16le,geq='st(0,(X-%H%)*PI/%H%);if(between(ld(0),-0.5*PI,0.5*PI),

%IH2%+%R%*sin(ld(0))*cos((Y-0.5*%H%)*PI/%H%),-1)' -frames 1 -y [Link]

rem Create the ymap1 file, from "perspective" input image to intermediate equirectangular image

ffmpeg -f lavfi -i nullsrc=size=%W%x%H% -vf format=gray16le,geq='st(0,(X-%H%)*PI/%H%);if(between(ld(0),-0.5*PI,0.5*PI),

%IH2%+%R%*sin((Y-0.5*%H%)*PI/%H%),-1)' -frames 1 -y [Link]

rem Create the xmap2 file, from intermediate equirectangular image to "perspective" output image

ffmpeg -f lavfi -i nullsrc=size=%S%x%S% -vf format=gray16le,geq='if(lt(hypot(X-%S2%,Y-%S2%),%R2%),%H%*(1+asin((X-%S2%)/

281
%R2%)/cos(asin((Y-%S2%)/%R2%))/PI),-1)' -frames 1 -y [Link]

rem Create the ymap2 file, from intermediate equirectangular image to "perspective" output image

ffmpeg -f lavfi -i nullsrc=size=%S%x%S% -vf format=gray16le,geq='if(lt(hypot(X-%S2%,Y-%S2%),%R2%),%H%*(0.5+asin((Y-

%S2%)/%R2%)/PI),-1)' -frames 1 -y [Link]

rem Live processing

ffmpeg -f dshow -video_size %IW%x%IH% -framerate 5 -pixel_format yuyv422 -i video="USB Video" -i [Link] -i [Link]
-i [Link] -i [Link] -lavfi "crop=ih:ih,format=rgb24,split[a][b];[a]drawbox=x=ih/2-%R%-3:y=ih/2-%R%-3:w=2*%R%
+6:h=2*%R%+6:color=%FRAME%,format=rgb24,scale=%S%:%S%[left];[b][1][2]remap=fill=%FILL%,v360=[Link]yaw=%LON%:pitch=%LAT%
[e];[e][3][4]remap,format=rgb24[right];[left][right]hstack" -window_x 0 -window_y 0 -f sdl2 -

pause

This is a live screenshot:

282
2.137 Insert a crosshair in a live video

A live video is captured with a HDMI to USB converter, and then a crosshair is inserted before the video is shown on the computer screen.
This can be used for checking if a telescope tracks the stars correctly.
set "IW=1920" :: Width
set "IH=1080" :: Height
set "C=white" :: Color of crosshair
set "G=10" :: Radius of clear area in the center

ffmpeg -f dshow -video_size %IW%x%IH% -framerate 5 -pixel_format yuyv422 -i video="USB Video" -lavfi
"format=rgb24,drawbox=x=iw/2:y=0:w=1:h=ih/2-%G%:color=%C%,drawbox=x=iw/2:y=ih/2+%G%:w=1:h=ih/2-%G%:color=%C
%,drawbox=x=0:y=ih/2:w=iw/2-%G%:h=1:color=%C%,drawbox=x=iw/2+%G%:y=ih/2:w=iw/2-%G%:h=1:color=%C%,format=rgb24" -window_x
0 -window_y 0 -f sdl2 -

pause

283
2.138 Enlarge the corners of a live video

This script enlarges the four corners of a live video by an adjustable factor. If you set the factor to 1, then the video is shown unchanged. The live video
is captured by a HDMI to USB converter. Can be used for adjusting telescope optics.
set "IW=1920" :: Width
set "IH=1080" :: Height
set "F=5" :: Enlarging factor

ffmpeg -f dshow -video_size %IW%x%IH% -framerate 5 -pixel_format yuyv422 -i video="USB Video" -lavfi
"format=rgb24,split=4[a][b][c][d];[a]crop=iw/2/%F%:ih/2/%F%:0:0,scale=iw*%F%:ih*%F%[aa],[b]crop=iw/2/%F%:ih/2/%F%:iw-
iw/2/%F%:0,scale=iw*%F%:ih*%F%[bb],[c]crop=iw/2/%F%:ih/2/%F%:0:ih-ih/2/%F%,scale=iw*%F%:ih*%F%[cc],[d]crop=iw/2/%F
%:ih/2/%F%:iw-iw/2/%F%:ih-ih/2/%F%,scale=iw*%F%:ih*%F%[dd],[aa][bb][cc][dd]xstack=inputs=4:layout=0_0|w0_0|0_h0|
w0_h0,format=rgb24" -window_x 0 -window_y 0 -f sdl2 -

pause

This is the same as before, but the input video is grabbed from the first monitor and the result is shown on the second monitor. You can use this script
with any software that uses the full screen of the first monitor.
set "IW=1920" :: Width of first monitor
set "IH=1080" :: Height of first monitor
set "F=5" :: Enlarging factor

ffmpeg -f gdigrab -video_size %IW%x%IH% -i desktop -lavfi "format=rgb24,split=4[a][b][c][d];[a]crop=iw/2/%F%:ih/2/%F

%:0:0,scale=iw*%F%:ih*%F%[aa],[b]crop=iw/2/%F%:ih/2/%F%:iw-iw/2/%F%:0,scale=iw*%F%:ih*%F%[bb],[c]crop=iw/2/%F%:ih/2/%F
%:0:ih-ih/2/%F%,scale=iw*%F%:ih*%F%[cc],[d]crop=iw/2/%F%:ih/2/%F%:iw-iw/2/%F%:ih-ih/2/%F%,scale=iw*%F%:ih*%F%[dd],[aa]
[bb][cc][dd]xstack=inputs=4:layout=0_0|w0_0|0_h0|w0_h0,format=rgb24" -window_x %IW% -window_y 0 -f sdl2 -

pause

Note: "-window_x 1920" near the end of the command line means that the output begins at x=1920, that's the left edge of the second monitor (if the width
of the first monitor is 1920).

284
This is an example for enlarging 9 windows from topleft / topcenter / topright / centerleft / center / centerright / bottomleft / bottomcenter / bottomright:
set "P=200" :: Window size in pixels
set "F=2" :: Enlarging factor

ffmpeg -i input.mp4 -lavfi "format=rgb24,split=3[a][b][c];[a]crop=iw:%P%:0:0,scale=iw:ih*%F%[aa],[b]crop=iw:%P%:0:ih/2-

%P%/2,scale=iw:ih*%F%[bb],[c]crop=iw:%P%:0:ih-%P%,scale=iw:ih*%F%[cc],[aa][bb][cc]vstack=3,split=3[a][b][c];[a]crop=%P
%:ih:0:0,scale=iw*%F%:ih[aa],[b]crop=%P%:ih:iw/2-%P%/2:0,scale=iw*%F%:ih[bb],[c]crop=%P%:ih:iw-%P%:0,scale=iw*%F
%:ih[cc],[aa][bb][cc]hstack=3,format=rgb24" -frames 1 -y [Link]

pause

285
2.139 Tunnel effect

The idea is to horizontally stack many images together, so that we get a very wide image. In this image, connect the upper edge to the lower edge, so that
we get a long cylindrical tube. Now fly through this tube with the camera.
set "IN=image%%[Link]" :: Input filenames
set "N=26" :: Number of images
set "T=3" :: Time in seconds for scrolling from one image to the next image
set /a "D=%T%*(%N%-2)" :: Duration (Warning: /a supports only integer arithmetic!)
set "IH=400" :: Height of input images
set "PW=1200" :: Width of output video
set "PH=800" :: Height of output video
set "A=1200" :: Distance from viewing point to image plane
set "E=10" :: Radius of central black dot
set "FPS=30" :: Output framerate
set "OUT=tunnel.mp4" :: Output filename

rem Create the xmap file

ffmpeg -f lavfi -i nullsrc=size=%PW%x%PH% -vf format=pix_fmts=gray16le,geq='st(0,hypot(%PW%/2-X,%PH%/2-Y));%A%*(1-%E

%/ld(0))' -frames 1 -y [Link]

rem Create the ymap file

ffmpeg -f lavfi -i nullsrc=size=%PW%x%PH% -vf format=pix_fmts=gray16le,geq='%IH%*(0.5-atan2(%PH%/2-Y,%PW%/2-X)/(2*PI))'

-frames 1 -y [Link]

rem Create the tunnel video

ffmpeg -framerate 1/%T% -start_number 2 -i %IN% -framerate 1/%T% -start_number 1 -i %IN% -framerate 1/%T% -start_number
0 -i %IN% -i [Link] -i [Link] -filter_complex [0][1][2]hstack=inputs=3,fps=%FPS%,crop=w=2*iw/3:x='iw/3*(1-mod(t,%T
%)/%T%)',format=pix_fmts=rgb24[5];[5][3][4]remap -t %D% -y %OUT%

pause

An alternative approach is to project the images on a cone instead of a cylinder. Only the xmap file must be changed:

286
set "C=200" :: Distance from image plane to the vanishing point

rem Create the xmap file

ffmpeg -f lavfi -i nullsrc=size=%PW%x%PH% -vf format=pix_fmts=gray16le,geq='st(0,hypot(%PW%/2-X,%PH%/2-Y));%A%*%C

%*(ld(0)-%E%)/(ld(0)*%C%+%A%*%E%)*sqrt(1+%E%*%E%/%C%/%C%)' -frames 1 -y [Link]

pause
Note: For C→∞ the formula is the same as the cylinder case.

A drawback of the above projections is the discontinuity where the upper edge touches the lower edge, which is visible as a straight line. This can be
avoided by duplicating the input image, so that the same image appears twice around the cylinder. There is no visible discontinuity because the bottom
edges of both images touch each other, and the upper edges as well. Only the ymap file must be changed:
rem Create the ymap file

ffmpeg -f lavfi -i nullsrc=size=%PW%x%PH% -vf format=pix_fmts=gray16le,geq='%IH%*(abs(atan2(%PW%/2-X,%PH%/2-Y)/PI))-0.5'

-frames 1 -y [Link]

pause

287
This is the tunnel effect with an additional spiral effect:
set "IN=image%%[Link]" :: Input filenames
set "N=6" :: Number of images
set "T=3" :: Time in seconds for scrolling from one image to the next image
set /a "D=%T%*(%N%-2)" :: Duration (Warning: /a supports only integer arithmetic!)
set "IH=400" :: Height of input images
set "PW=1200" :: Width of output video
set "PH=800" :: Height of output video
set "A=1200" :: Distance from viewing point to image plane
set "E=10" :: Radius of central black dot
set "S=500" :: Spiral effect, number of pixels in radial direction for a 360° rotation
set "FPS=30" :: Output framerate
set "OUT=tunnel.mp4" :: Output filename

rem Create the xmap file

ffmpeg -f lavfi -i nullsrc=size=%PW%x%PH% -vf format=pix_fmts=gray16le,geq='st(0,hypot(%PW%/2-X,%PH%/2-Y));%A%*(1-%E

%/ld(0))' -frames 1 -y [Link]

rem Create the ymap file

ffmpeg -f lavfi -i nullsrc=size=%PW%x%PH% -vf format=pix_fmts=gray16le,geq='%IH%*(mod(hypot(%PW%/2-X,%PH%/2-Y)/%S%+0.5-

atan2(%PH%/2-Y,%PW%/2-X)/(2*PI),1))' -frames 1 -y [Link]

rem Create the tunnel video

ffmpeg -framerate 1/%T% -start_number 2 -i %IN% -framerate 1/%T% -start_number 1 -i %IN% -framerate 1/%T% -start_number
0 -i %IN% -i [Link] -i [Link] -filter_complex [0][1][2]hstack=inputs=3,fps=%FPS%,crop=w=2*iw/3:x='iw/3*(1-mod(t,%T
%)/%T%)',format=pix_fmts=rgb24[5];[5][3][4]remap -t %D% -y %OUT%

pause

288
How the mathematics of this filter works:

A is the distance from the viewing point to the output image plane.
E is the radius of the central area in the outpot image, which will remain black.
R is the radial coordinate in the output image.
X is the horizontal coordinate in the input image.
C is a parameter that defines how far the vanishing point is behind the output image plane.
In the special case of a cylindrical tube (C→∞) the formula simplifies to: X = A (1 - E/R)
You must make sure that for the maximum possible value of R the resulting X value doesn't exceed the width of the input image.

289
2.140 Black hole simulation with remap filter

An introduction to Kerr black holes, including mathematics: [Link]

An exact mathematical approach for the simulation of spinning black holes can be found here:
Oliver James, Eugenie von Tunzelmann, Paul Franklin, Kip S. Thorne: "Gravitational Lensing by Spinning Black Holes in Astrophysics, and in the Movie
Interstellar [Link]

There are many informations about black hole simulations on Simon Tyran's website: [Link]

A black hole simulation by Ziri Younsi:

• Falling into a black hole (Realistic Ultra HD 360 VR movie) [8K] [Link]

A video from Alessandro Roussel about how to make an exact simulation of a black hole (in french language):
• [Link]

Analog simulation of black holes with fluids:

• [Link]

Albert Sneppen: "Divergent reflections around the photon sphere of a black hole" [Link]

290
FFmpeg's remap filter can be used to simulate the light deviation near black holes.
As an approximation, when a beam of light passes near a black hole, it will be deviated by angle alpha (in Radians):
alpha = 2 * rs / (r - rs) or alpha = 2 / ((r / rs) - 1)
where rs is the Schwarzschild radius of the black hole, and r is the closest distance between the beam and the center of the black hole.

r / rs alpha in radians alpha in degrees

1 infinite infinite
1.2 10 573°
1.5 4 229°
2 2 115°
2.5 1.333 76.4°
3 1 57.3°
4 0.666 38.2°
5 0.5 28.6°
6 0.4 22.9°
8 0.286 16.4°
10 0.222 12.7°
15 0.143 8,2°
20 0.105 6.0°

291
Assuming we have a 180° fisheye image, we can express the light deviation in pixels: c = height / pi * 2 * rs / (r - rs)

The values for the PGM files (which are required for the remap filter) can be calculated with these formulas:
r = sqrt( (x - xc)^2 + (y - yc)^2 )
c = shape / (r - rs) where shape is a constant that defines the "strength" of the distortion
if r > rs:
x_remap = x - c * (x - xc)
y_remap = y - c * (y - yc)
if r < rs:
x_remap = 0
y_remap = 0
where xc,yc are the pixel coordinates of the center of the black hole, x,y are the pixel coordinates in the source video and r is the distance between the
source pixel and the center of the black hole.

This is the batch file for applying the black-hole-effect to a video:

set "IN=MVI_2562.mov" :: Input video
set "OUT=output.mp4" :: Output video

ffmpeg -i %IN% -i [Link] -i [Link] -lavfi "format=pix_fmts=rgb24,remap" -c:v mpeg4 -q:v 2 -y %OUT%

pause

It's also possible to simulate moving black holes. To do this you need many xmap and ymap files (one for each frame), and loop through them.
set "IN=MVI_2562.mov" :: Input video
set "OUT=output.mp4" :: Output video

ffmpeg -i %IN% -framerate 30 -i xmap%%[Link] -framerate 30 -i ymap%%[Link] -lavfi "format=pix_fmts=rgb24,remap" -c:v

mpeg4 -q:v 2 -y %OUT%

pause

292
Calculate the xmap and ymap files for all frames. This is done by a C# program, which can be downloaded here:
[Link]
Example of a simulated black hole:

293
Black hole simulation with FFmpeg, no C# code required:
set "W=2448" :: Width of image
set "H=2448" :: Height of image
set "CX=2000" :: X center of distortion
set "CY=1200" :: Y center of distortion
set "RS=50" :: Schwarzschild radius
set "SH=0.50" :: Shape parameter

rem Create the xmap file

ffmpeg -f lavfi -i nullsrc=size=%W%x%H% -vf format=pix_fmts=gray16le,geq='st(0,X-%CX%);st(1,hypot(ld(0),%CY%-Y));st(2,X-

(ld(0)*%SH%*2*%RS%/(ld(1)-%RS%)));if(lt(%RS%,ld(1)),clip(ld(2),0,%W%),0)' -frames 1 -y [Link]

rem Create the ymap file

ffmpeg -f lavfi -i nullsrc=size=%W%x%H% -vf format=pix_fmts=gray16le,geq='st(0,Y-%CY%);st(1,hypot(%CX%-X,ld(0)));st(2,Y-

(ld(0)*%SH%*2*%RS%/(ld(1)-%RS%)));if(lt(%RS%,ld(1)),clip(ld(2),0,%H%),0)' -frames 1 -y [Link]

rem Apply the displace filter to the image

ffmpeg -i test3.mp4 -i [Link] -i [Link] -lavfi format=pix_fmts=rgb24,remap -frames 1 -y [Link]

rem Alternatively it can all be written in one command line:

ffmpeg -i test3.mp4 -f lavfi -i nullsrc=size=%W%x%H% -f lavfi -i nullsrc=size=%W%x%H% -lavfi

[0]format=pix_fmts=rgb24[v];[1]format=pix_fmts=gray16le,geq='st(0,X-%CX%);st(1,hypot(ld(0),%CY%-Y));st(2,X-(ld(0)*%SH
%*2*%RS%/(ld(1)-%RS%)));if(lt(%RS%,ld(1)),clip(ld(2),0,%W%),0)'[x];[2]format=pix_fmts=gray16le,geq='st(0,Y-%CY
%);st(1,hypot(%CX%-X,ld(0)));st(2,Y-(ld(0)*%SH%*2*%RS%/(ld(1)-%RS%)));if(lt(%RS%,ld(1)),clip(ld(2),0,%H%),0)'[y];[v][x]
[y]remap -frames 1 -y [Link]

pause

294
This example is for a moving black hole, no C# code required (but unfortunately this is extremely slow):
set "IN=test3.mp4" :: Input video
set "W=2448" :: Width of video
set "H=2448" :: Height of video
set "CX0=2000" :: X center of distortion, T=0
set "CY0=1200" :: Y center of distortion, T=0
set "CX1=1900" :: X center of distortion, T=1
set "CY1=1500" :: Y center of distortion, T=1
set "CX2=1600" :: X center of distortion, T=2
set "CY2=1800" :: Y center of distortion, T=2
set "CX3=1000" :: X center of distortion, T=3
set "CY3=2000" :: Y center of distortion, T=3
set "RS=50" :: Schwarzschild radius
set "SH=0.50" :: Shape parameter
set "OUT=out.mp4" :: Output video

ffmpeg -i %IN% -f lavfi -i nullsrc=size=%W%x%H% -f lavfi -i nullsrc=size=%W%x%H% -lavfi ^

[0]format=pix_fmts=rgb24[v];^
[1]format=pix_fmts=gray16le,geq='^
st(0,between(T+0.001,0,1)*lerp(%CX0%,%CX1%,T)+between(T+0.001,1,2)*lerp(%CX1%,%CX2%,T-1)+between(T+0.001,2,3)*lerp(%CX2%,%CX3%,T-2));^
st(1,between(T+0.001,0,1)*lerp(%CY0%,%CY1%,T)+between(T+0.001,1,2)*lerp(%CY1%,%CY2%,T-1)+between(T+0.001,2,3)*lerp(%CY2%,%CY3%,T-2));^
st(2,X-ld(0));^
st(3,hypot(ld(2),ld(1)-Y));^
st(4,X-(ld(2)*%SH%*2*%RS%/(ld(3)-%RS%)));^
if(lt(%RS%,ld(3)),clip(ld(4),0,%W%),0)'[x];^
[2]format=pix_fmts=gray16le,geq='^
st(0,between(T+0.001,0,1)*lerp(%CX0%,%CX1%,T)+between(T+0.001,1,2)*lerp(%CX1%,%CX2%,T-1)+between(T+0.001,2,3)*lerp(%CX2%,%CX3%,T-2));^
st(1,between(T+0.001,0,1)*lerp(%CY0%,%CY1%,T)+between(T+0.001,1,2)*lerp(%CY1%,%CY2%,T-1)+between(T+0.001,2,3)*lerp(%CY2%,%CY3%,T-2));^
st(2,Y-ld(1));^
st(3,hypot(ld(0)-X,ld(2)));^
st(4,Y-(ld(2)*%SH%*2*%RS%/(ld(3)-%RS%)));^
if(lt(%RS%,ld(3)),clip(ld(4),0,%H%),0)'[y];^
[v][x][y]remap -t 3 -y %OUT%

pause

"T+0.001" is a workaround to avoid the problem that at the segment borders two "between" expressions become simultaneously true.

This method is extremely slow because this expression must be evaluated four times for each pixel, although it would be sufficient to evaluate it only
one time per frame:
st(1,between(T+0.001,0,1)*lerp(%CY0%,%CY1%,T)+between(T+0.001,1,2)*lerp(%CY1%,%CY2%,T-1)+between(T+0.001,2,3)*lerp(%CY2%,%CY3%,T-2));
Recommended workaround: Calculate many xmap and ymap files in advance by C# code.

295
2.141 Wormhole simulation

A wormhole is a hypothetical window to another place in space or time, or even in another universe.
For more informations see [Link]

Short story from Rudy Rucker:

• "The Last Einstein-Rosen Bridge" [Link]

An easy-level book, unfortunately without any mathematics:

• Kip Thorne: "The Science of Interstellar"

The exact mathematical approach can be found in this paper. Unfortunately the mathematics exceed my capabilities by far:
• Oliver James, Eugenie von Tunzelmann, Paul Franklin, Kip S. Thorne: "Visualizing Interstellar's Wormhole" [Link]
• The same paper is also available here: [Link]

Two very interesting videos by Scott Manley:

• What Would Travelling Through A Wormhole Look Like? (VR/360) [Link]
• Wormholes Get Weirder - Watch Other Objects Near Wormholes & Learn How I Created The Images
[Link]
The math is a little bit explained beginning at 7:40. Two pages of the code are visible at 12:40 (slow 3D version) and 12:44 (much faster 2D
version). Somewhere in the comments he mentioned that he did use the "RK4" Runge-Kutta algorithm.
I got his C code and will try to understand that later.

296
An article from Jason Biggs, Wolfram Alpha: (I'm not familiar with this programming language)
• Visualizing Interstellar's Wormhole: from article to programming [Link]
Please note that some characters got lost in the code.
• The same code (without missing characters) is also available here: [Link]
effects

Here are several wormhole simulations by Pierre-Jean Charpin:

• [Link]

A wormhole simulation by Alessandro Roussel:

• 360° Traversing a flat Worm Hole [Link]

A wormhole simulation by Robert Szili:

• Walking around a wormhole [Link]

An interactive wormhole project on Github: [Link]

297
A simplified wormhole can be simulated in a video as follows:
• In the outer area the light rays are distorted in the same way as when passing near a black hole. This can be simulated with the remap filter.
• In the inner area, another video is inserted as a 360° "little planet" video (or even better a mirror-sphere video).
• The drawback of this simplified simulation is that you can't let the camera or an object fly through the wormhole. You can only look at it from a
distance.

298
This is a batch file for wormhole simulation. The xmap0000 and ymap0000 files for the black hole are created in advance by C# code.
set "IN=[Link]" :: Main input video
set "LP=test1.mp4" :: Equirectangular video for little planet
set "H=960" :: Height of equirectangular input video
set "S=1080" :: Size of square little planet output video
set "P=0" :: Pitch angle
set "Y=90" :: Yaw angle
set "R=-90" :: Roll angle
set "LPD=100" :: Little planet diameter
set "LPX=1500" :: X Position of center of little planet
set "LPY=1000" :: Y Position of center of little planet
set "T=8" :: Length of output video

rem Step 1: Convert the equirectangular video to a little planet video

rem Create the xmap and ymap files

ffmpeg -f lavfi -i nullsrc=size=%S%x%S% -vf format=pix_fmts=gray16le,^

geq='%H%*(0.9999+atan2(Y-%S%/2,X-%S%/2)/PI)' -frames 1 -y [Link]

ffmpeg -f lavfi -i nullsrc=size=%S%x%S% -vf format=pix_fmts=gray16le,^

geq='%H%*(1-hypot((2*X/%S%)-1,(2*Y/%S%)-1))' -frames 1 -y [Link]

rem Apply the remap filter to the video

ffmpeg -i %LP% -i [Link] -i [Link] -lavfi "v360=output=e:pitch=%P%:yaw=%Y%:roll=%R%,format=pix_fmts=rgb24,remap"

-q:v 2 -t %T% -y lp.mp4

rem Step 2: Apply the black hole effect to the main video and then overlay the little planet video over the black hole

ffmpeg -i %IN% -i lp.mp4 -i [Link] -i [Link] -filter_complex "[0][2][3]remap[4];[1]scale=%LPD%:%LPD

%,format=argb,geq=a='255*lt(hypot((2*X/W)-1,(2*Y/H)-1),1)':r='r(X,Y)':g='g(X,Y)':b='b(X,Y)'[5];[4][5]overlay=x=%LPX%-
%LPD%/2:y=%LPY%-%LPD%/2" -q:v 2 -t %T% -y out.mp4

pause

The same thing can de done much easier with the v360 filter and the alpha_mask option:

299
set "IN=[Link]" :: Main input video
set "LP=test1.mp4" :: Equirectangular video for mirror-sphere
set "H=960" :: Height of equirectangular input video
set "S=1080" :: Size of square mirror-sphere output video
set "P=30" :: Pitch angle
set "Y=0" :: Yaw angle
set "R=0" :: Roll angle
set "LPD=102" :: Mirror-sphere diameter
set "LPX=1800" :: X Position of center of mirror-sphere
set "LPY=1000" :: Y Position of center of mirror-sphere
set "T=8" :: Length of output video

rem Make only the mirror-sphere video

rem ffmpeg -i %LP% -vf v360=output=ball:pitch=%P%:yaw=%Y%:roll=%R% -q:v 2 -t %T% -y lp.mp4

ffmpeg -i %IN% -i [Link] -i [Link] -i %LP% -filter_complex "[0][1][2]remap[4];[3]v360=output=ball:pitch=%P

%:yaw=%Y%:roll=%R%:alpha_mask=1,scale=%LPD%:%LPD%[5];[4][5]overlay=x=%LPX%-%LPD%/2:y=%LPY%-%LPD%/2" -q:v 2 -t %T% -y
out.mp4

pause

300
2.142 Realtime Wormhole in Planetarium (Hackathon 2022)

Desktop layout of the planetarium computer:

+------------------+------------------------------------------+
| | ------------------------ |
| FHD monitor for | / \ |
| user interface | / \ |
| 1920x1080 | / \ |
| | / \ |
+------------------+ / Rear half of 4K fisheye image \ |
| || ||
| || 4096x2048 ||
| || ||
| || ||
| +..........4Kx4K extended desktop..........+
| || ||
| unused || ||
| || Front half of 4K fisheye image ||
| || This is the main viewing direction ||
| | \ / |
| | \ 4096x2048 / |
| | \ / |
| | \ / |
| | \ / |
| | ------------------------ |
+------------------+------------------------------------------+

Note: Alternatively it's also possible that the extended desktop is placed on the left side and the user interface on the right side.

301
First test with a 4096x4096 image:
rem Image from [Link]

ffplay -noborder -x 4096 -y 4096 -left 1920 -top 0 [Link]

pause

This is a screenshot of the whole desktop. The white rectangle in the top left corner is the 1920x1080 monitor with the user interface. Obviously the
4096x4096 image is shown twice as large as it should be.

The problem was that in Settings --> Display the scale was set to 200%. With the correct setting (100%) it works as expected.

302
Example for sending the stream:
rem Test pattern from [Link]

set "ROT=240" :: Seconds per full revolution

set "FR=4" :: Framerate
set "BR=20M" :: Bitrate

ffmpeg -f lavfi -i sine=frequency=1000:sample_rate=48000 -framerate %FR% -loop 1 -i [Link] -vf

scale=4096:4096,rotate='2*PI*t/%ROT%' -strict -1 -g 1 -b:v %BR% -f mpegts "udp://[Link]:1234"

pause
Note: "-strict -1" is required because multiples of 4096 seem to be forbidden for image size.
Note: "-g 1" means all frames are encoded as keyframes. This is useful for faster startup at the receiver side, but it reduces the framerate.
Note: In this case "-framerate" works much better than "-r".

Example for receiving the stream with FFmpeg (drawback: no audio!):

ffmpeg -i "udp://[Link]:1234" -vf scale=960:960 -window_borderless 1 -window_x 0 -window_y 0 -f sdl2 -

pause

Example for receiving the stream with FFplay (with audio):

ffplay -noborder -x 4096 -y 4096 -left 1920 -top 0 udp://[Link]:1234

pause
Note: In both cases, use the IP adresses from the other computer. Check with ipconfig.

See also [Link]

See also [Link]

See also: "How to Increase NDI Bandwidth for High-res Images and Framerate"
[Link]
[Link]

303
Final results after testing in the planetarium:

• For fulldome projection it's essential to avoid large bright areas in the image, because light is reflected from one side of the dome to the other
side, where it reduces contrast. I knew this in advance and did attenuate the overhead sky. But It would have been even better to attenuate the
whole northern sky, because the audience was looking to the south anyway.
• The transmission of the stream over network turned out to be problematic. I did test the transmission from computer A to computer B in advance
and it worked well, after I set permissions in the Windows firewall settings in computer B. But that doesn't mean the transmission does also work
from the same computer A to computer C (which is the computer in the planetarium). In this configuration the firewall and the antivirus software
had to be deactivated in computer A. I don't understand why.
• The transmission of the audio track (together with the video stream in the same Mpeg transport stream) turned out to be very problematic. Audio
was always received with many short gaps, even when the video resolution was set to very low 1200x1200 pixels. We finally disabled the audio
stream (with option -an in the last FFmpeg command) and sent the audio track from another computer to the planetarium. Synchronization wasn't
critical because there are only bird voices in the audio track. But that meant we could not use the sound effect when the wormhole appears. As a
workaround, luckily we found a piano of the stage, which we used to generate a live sound effect when the wormhole appeared.
• Without the audio stream, it was possible to send the video stream with 2048x2048 resolution with an acceptable framerate. We did not have the
time to check if 2880x2880 (which is the resolution of the background video from the Kodak PixPro camera) would have worked as well.
• If the 360° camera is also located in the planetarium (where it's quite dark), a lamp is required to illuminate the people who are looking into the
camera. It's essential not to direct the lamp towards the audience. People are seen best if they are very close (about 5-10cm) to the camera.
• There was about 1 second delay in the video stream from the 360° camera to the dome.

304
How does the algorithm work?

The background video was recorded with a Kodak PixPro SP360 4K camera with resolution 2880x2880 @ 29.97fps:

This video is scaled down to 2048x2048. Then mask1 is applied to fill the corners with a dark green color:

305
Then mask2 is applied to attenuate the overhead sky. This improves the contrast in fulldome projection:

Then the video is rotated by angle alpha, which is defined by a sine function with 25° amplitude and period 50 seconds.
The resulting video is transformed with the v360 filter from fisheye to equirectangular. The ground is no-data area and filled with dark green color. Why is
it necessary to give the ground a color, if it's not visible in the final fulldome projection? That's because some light rays which pass near the wormhole
are bent towards the ground. The ground becomes in fact visible in some small areas around the wormhole.
Saturation and gamma is corrected. It's always a good idea to use strong saturation for a planetarium projection:

A sound file is added (bird sounds), and a short sound effect is applied at two times when the wormhole appears and disappears.
This video is saved with resolution 4096x2048 @ 12fps as [Link].

Now comes the realtime part.

A live video is captured from a Ricoh Theta camera with FHD resolution, using the live streaming driver. This stream is converted to a mirrorsphere with
resolution 200x200 @ 12fps.
This stream is stacked vertically under the [Link] video. The resulting stream is 4096 pixels wide and 2048+200=2248 pixels high.
The reason why both streams are stacked together is that the pixels from both streams must be combined for the resulting wormhole video, and the
remap filter accepts only one input.

306
The xmap and ymap files for the remap filter come from two streamselect filters with two inputs each. That's because the wormhole is switched on at
time t1 and switched off at time t2. One set of mapping files is for the wormhole simulation (in this case both input streams are used), and the other set of
mapping files is without wormhole (in this case the second input stream isn't used). For the calculation of the mapping files, see the next chapter.
The output of the remap filter has resolution 2048x2048.
Now the video is rotated by angle -alpha.
Why is the background video rotated by angle alpha and the final stream (with wormhole) by angle -alpha? That's because the wormhole simulation uses
static mapping files (the mapping files do not change ober time). Dynamic mapping files would be too large to load in realtime. With this technique of
rotating and de-rotating the background is stationary and the wormhole does move.
The resulting stream is either saved a *.mov file, or is is piped to FFplay, or is is streamed to a UDP adress:

Without wormhole With wormhole

This is the batch file:

set "IN=133_0006.mp4" :: Background video
set "SS=20" :: In point in seconds
set "LENGTH=180" :: Length in seconds
set "T1=15" :: Time where wormhole appears
set "T2=170" :: Time where wormhole vanishes
set "T3=14" :: Time for first sound effect
set "T4=169" :: Time for second sound effect
set "SIZE=2048" :: Main size
set "SIZE2=200" :: Secondary size (this is content of the wormhole)

rem Parameters for wormhole transformation:

set "IN_W=(2*%SIZE%)" :: Main equirectangular input width in pixels

set "IN_H=%SIZE%" :: Main equirectangular input height in pixels
set "IN_W2=%SIZE2%" :: Secondary input size (this is content of the wormhole)
set "OUT_W=%SIZE%" :: Output width in pixels

307
set "OUT_H=%SIZE%" :: Output height in pixels
set "OUT_H_FOV=180" :: Output horizontal field of view in degrees
set "OUT_V_FOV=180" :: Output vertical field of view in degrees
set "RS=8" :: Schwarzschild radius in degrees
set "S=0.06" :: Make the distortion smaller, 1 = realistic
set "PITCH1=18" :: Pitch rotation in degrees before applying wormhole
set "PITCH2=72" :: Pitch rotation in degrees after applying wormhole

rem Parameters for filling the missing ground area:

set "R3=100" :: Fill ground only outside of this diameter (percent of image width)
set "RAMP=10" :: Width of the transition band (percent of image width)
set "GROUND=0x181800" :: Color for filling the ground

rem Parameters for attenuating the overhead sky:

set "R1=12" :: Inner radius in pixels (percent of image width)

set "R2=30" :: Outer radius in pixels (percent of image width)
set "DARK=60" :: Brightness level inside of R1, 0 is black, 255 is original brightness
set "BRIGHT=255" :: Brightness level outside of R2, 0 is black, 255 is original brightness

rem Parameters for making the background video:

set "FOV=200" :: Field of view of the input video (from Kodak Pixpro SP360 4K)
set "GAMMA=1.5" :: Gamma correction
set "SAT=2" :: Saturation
set "ROT_S=-150" :: Rotation angle at start
set "ROT_A=25" :: Amplitude of sine oscillation for rotation
set "ROT_T=50" :: Period of sine oscillation in seconds
set "VOL=3" :: Volume for sound effect
set "FR=8" :: Framerate
set "BR=40M" :: Bitrate

rem Create the xmap and ymap files with wormhole

ffmpeg -f lavfi -i nullsrc=size=%OUT_W%x%OUT_H% -vf format=pix_fmts=gray16le,geq='^

st(0,%OUT_H_FOV%/180*((2*X+1)/%OUT_W%-1));^
st(1,%OUT_V_FOV%/180*((2*Y+1)/%OUT_H%-1));^
st(2,atan2(ld(1),ld(0)));^

308
st(3,PI/2*hypot(ld(0),ld(1)));^
st(4,sin(ld(3))*cos(ld(2)));^
st(5,sin(ld(3))*sin(ld(2)));^
st(6,cos(ld(3)));^
st(1,if(lt(ld(6),cos(%OUT_H_FOV%/360*PI)),32767,0));^
st(0,ld(5)*cos(%PITCH2%/180*PI)-ld(6)*sin(%PITCH2%/180*PI));^
st(6,ld(6)*cos(%PITCH2%/180*PI)+ld(5)*sin(%PITCH2%/180*PI));^
st(5,ld(0));^
st(7,atan2(ld(5),ld(4)));^
st(8,acos(ld(6)));^
st(9,if(lte(ld(8),%RS%/180*PI),1));^
if(ld(9),st(7,ld(8)*cos(ld(7))));^
ifnot(ld(9),st(8,ld(8)-%S%*(2*%RS%/180*PI/(ld(8)-%RS%/180*PI))));^
ifnot(ld(9),st(4,sin(ld(8))*cos(ld(7))));^
ifnot(ld(9),st(5,sin(ld(8))*sin(ld(7))));^
ifnot(ld(9),st(6,cos(ld(8))));^
ifnot(ld(9),st(0,ld(5)*cos(%PITCH1%/180*PI)-ld(6)*sin(%PITCH1%/180*PI)));^
ifnot(ld(9),st(6,ld(6)*cos(%PITCH1%/180*PI)+ld(5)*sin(%PITCH1%/180*PI)));^
ifnot(ld(9),st(5,ld(0)));^
ifnot(ld(9),st(7,atan2(ld(5),ld(4))));^
ifnot(ld(9),st(8,acos(ld(6))));^
ifnot(ld(9),st(7,atan2(sin(ld(8))*cos(ld(7)),cos(ld(8)))));^
if(ld(9),0.5*%IN_W2%*(1+180/PI/%RS%*ld(7)),ld(1)+0.5*%IN_W%*(1+ld(7)/PI))' -frames 1 -y [Link]

ffmpeg -f lavfi -i nullsrc=size=%OUT_W%x%OUT_H% -vf format=pix_fmts=gray16le,geq='^

st(0,%OUT_H_FOV%/180*((2*X+1)/%OUT_W%-1));^
st(1,%OUT_V_FOV%/180*((2*Y+1)/%OUT_H%-1));^
st(2,atan2(ld(1),ld(0)));^
st(3,PI/2*hypot(ld(0),ld(1)));^
st(4,sin(ld(3))*cos(ld(2)));^
st(5,sin(ld(3))*sin(ld(2)));^
st(6,cos(ld(3)));^
st(1,if(lt(ld(6),cos(%OUT_H_FOV%/360*PI)),32767,0));^
st(0,ld(5)*cos(%PITCH2%/180*PI)-ld(6)*sin(%PITCH2%/180*PI));^
st(6,ld(6)*cos(%PITCH2%/180*PI)+ld(5)*sin(%PITCH2%/180*PI));^
st(5,ld(0));^
st(7,atan2(ld(5),ld(4)));^
st(8,acos(ld(6)));^
st(9,if(lte(ld(8),%RS%/180*PI),1));^
if(ld(9),st(8,ld(8)*sin(ld(7))));^

309
ifnot(ld(9),st(8,ld(8)-%S%*(2*%RS%/180*PI/(ld(8)-%RS%/180*PI))));^
ifnot(ld(9),st(4,sin(ld(8))*cos(ld(7))));^
ifnot(ld(9),st(5,sin(ld(8))*sin(ld(7))));^
ifnot(ld(9),st(6,cos(ld(8))));^
ifnot(ld(9),st(0,ld(5)*cos(%PITCH1%/180*PI)-ld(6)*sin(%PITCH1%/180*PI)));^
ifnot(ld(9),st(6,ld(6)*cos(%PITCH1%/180*PI)+ld(5)*sin(%PITCH1%/180*PI)));^
ifnot(ld(9),st(5,ld(0)));^
ifnot(ld(9),st(7,atan2(ld(5),ld(4))));^
ifnot(ld(9),st(8,acos(ld(6))));^
ifnot(ld(9),st(8,asin(sin(ld(8))*sin(ld(7)))));^
if(ld(9),%IN_H%+0.5*%IN_W2%*(1+180/PI/%RS%*ld(8)),ld(1)+0.5*%IN_H%*(1+ld(8)*2/PI))' -frames 1 -y [Link]

rem Create the xmap and ymap files without wormhole

ffmpeg -f lavfi -i nullsrc=size=%OUT_W%x%OUT_H% -vf format=pix_fmts=gray16le,geq='^

st(0,%OUT_H_FOV%/180*((2*X+1)/%OUT_W%-1));^
st(1,%OUT_V_FOV%/180*((2*Y+1)/%OUT_H%-1));^
st(2,atan2(ld(1),ld(0)));^
st(3,PI/2*hypot(ld(0),ld(1)));^
st(4,sin(ld(3))*cos(ld(2)));^
st(5,sin(ld(3))*sin(ld(2)));^
st(6,cos(ld(3)));^
st(1,if(lt(ld(6),cos(%OUT_H_FOV%/360*PI)),32767,0));^
st(0,-ld(6));^
st(6,ld(5));^
st(5,ld(0));^
st(7,atan2(ld(5),ld(4)));^
st(8,acos(ld(6)));^
st(7,atan2(sin(ld(8))*cos(ld(7)),cos(ld(8))));^
ld(1)+0.5*%IN_W%*(1+ld(7)/PI)' -frames 1 -y [Link]

ffmpeg -f lavfi -i nullsrc=size=%OUT_W%x%OUT_H% -vf format=pix_fmts=gray16le,geq='^

st(0,%OUT_H_FOV%/180*((2*X+1)/%OUT_W%-1));^
st(1,%OUT_V_FOV%/180*((2*Y+1)/%OUT_H%-1));^
st(2,atan2(ld(1),ld(0)));^
st(3,PI/2*hypot(ld(0),ld(1)));^
st(4,sin(ld(3))*cos(ld(2)));^
st(5,sin(ld(3))*sin(ld(2)));^
st(6,cos(ld(3)));^

310
st(1,if(lt(ld(6),cos(%OUT_H_FOV%/360*PI)),32767,0));^
st(0,-ld(6));^
st(6,ld(5));^
st(5,ld(0));^
st(7,atan2(ld(5),ld(4)));^
st(8,acos(ld(6)));^
st(8,asin(sin(ld(8))*sin(ld(7))));^
ld(1)+0.5*%IN_H%*(1+ld(8)*2/PI)' -frames 1 -y [Link]

rem Create the mask for filling the ground ([Link])

ffmpeg -f lavfi -i nullsrc=size=%SIZE%x%SIZE% -lavfi format=gray8,geq='clip(256/(%RAMP%*W/100)*(hypot(X-%SIZE%/2,Y-%SIZE

%/2)-(%R3%*W/200)),0,255)' -frames 1 -y [Link]

rem Create the mask for attenuating the overhead sky ([Link])

ffmpeg -f lavfi -i nullsrc=size=%SIZE%x%SIZE% -lavfi format=gray8,geq='clip(lerp(%DARK%,%BRIGHT%,(hypot(X-%SIZE%/2,Y-

%SIZE%/2)-(%R1%*W/100))/((%R2%-%R1%)*W/100)),%DARK%,%BRIGHT%)',format=rgb24 -frames 1 -y [Link]

rem Make an equirectangular background image (only for testing)

ffmpeg -ss %SS% -i %IN% -f lavfi -i color=%GROUND%:size=%SIZE%*%SIZE% -i [Link] -i [Link] -lavfi [0]scale=%SIZE%x
%SIZE%,format=gbrp[fg];[2]format=gbrp[mask];[fg][1][mask]maskedmerge[a];[a][3]multiply=offset=0,rotate=(%ROT_S%+%ROT_A
%*sin(2*PI/%ROT_T%*t))/180*PI:c=%GROUND%,drawbox=w=1:h=1:color=%GROUND%,v360=fishey[Link]ih_fov=%FOV%:iv_fov=%FOV%:pitch=-
90:w=2*%SIZE%:h=%SIZE%,scroll=hpos=0.1,eq=saturation=%SAT%:gamma=%GAMMA% -frames 1 -y [Link]

rem Make the equirectangular background video

rem Add the sound track and two sound effects

ffmpeg -ss %SS% -i %IN% -f lavfi -i color=%GROUND%:size=%SIZE%*%SIZE% -i [Link] -i [Link] -i birds4min.mp3 -i

soundeffect.mp3 -i soundeffect.mp3 -lavfi [0]scale=%SIZE%x%SIZE%,format=gbrp[fg];[2]format=gbrp[mask];[fg][1]
[mask]maskedmerge[a];[a][3]multiply=offset=0,rotate=(%ROT_S%+%ROT_A%*sin(2*PI/%ROT_T%*t))/180*PI:c=%GROUND
%,drawbox=w=1:h=1:color=%GROUND%,v360=fishey[Link]ih_fov=%FOV%:iv_fov=%FOV%:pitch=-90:w=2*%SIZE%:h=%SIZE%,eq=saturation=
%SAT%:gamma=%GAMMA%;[5]volume=%VOL%,adelay=%T3%s:all=1,apad[a];[6]volume=%VOL%,adelay=%T4%s:all=1,apad[b];[4][a]
[b]amix=3:normalize=0 -r %FR% -b:v %BR% -t %LENGTH% -y [Link]

311
rem ******************************
rem Below is the realtime part
rem ******************************

rem Vertically stack the input videos together, the second one may be smaller
rem Apply the wormhole effect, switch wormhole on/off, de-rotate

rem a) With output to a file:

rem ffmpeg -i [Link] -f dshow -rtbufsize 200M -i video="RICOH THETA V/Z1 FullHD" -i [Link] -i [Link] -i
[Link] -i [Link] -lavfi [0]sendcmd="%T1% streamselect@1 map 1",sendcmd="%T1% streamselect@2 map 1",sendcmd="%T2%
streamselect@1 map 0",sendcmd="%T2% streamselect@2 map 0"[a];[2]loop=-1:1[2a];[3]loop=-1:1[3a];[4]loop=-1:1[4a];
[5]loop=-1:1[5a];[3a][2a]streamselect@1=map=0[x];[5a][4a]streamselect@2=map=0[y];[1]fps=%FR%,v360=e:ball:w=%SIZE2%:h=
%SIZE2%[c];[a][c]xstack=2:"0_0|0_h0"[c];[c][x][y]remap=fill=black,rotate=-(%ROT_A%*sin(2*PI/%ROT_T%*t))/180*PI -r %FR%
-b:v %BR% -t %LENGTH% -y [Link]

rem b) With output to FFplay:

rem ffmpeg -i [Link] -f dshow -rtbufsize 200M -i video="RICOH THETA V/Z1 FullHD" -i [Link] -i [Link] -i
[Link] -i [Link] -lavfi [0]sendcmd="%T1% streamselect@1 map 1",sendcmd="%T1% streamselect@2 map 1",sendcmd="%T2%
streamselect@1 map 0",sendcmd="%T2% streamselect@2 map 0"[a];[2]loop=-1:1[2a];[3]loop=-1:1[3a];[4]loop=-1:1[4a];
[5]loop=-1:1[5a];[3a][2a]streamselect@1=map=0[x];[5a][4a]streamselect@2=map=0[y];[1]fps=%FR%,v360=e:ball:w=%SIZE2%:h=
%SIZE2%[c];[a][c]xstack=2:"0_0|0_h0"[c];[c][x][y]remap=fill=black,rotate=-(%ROT_A%*sin(2*PI/%ROT_T%*t))/180*PI -r %FR%
-b:v %BR% -t %LENGTH% -f nut - | ffplay -fs -autoexit -

rem c) With output to UDP stream:

ffmpeg -i [Link] -f dshow -rtbufsize 200M -i video="RICOH THETA V/Z1 FullHD" -i [Link] -i [Link] -i
[Link] -i [Link] -lavfi [0]sendcmd="%T1% streamselect@1 map 1",sendcmd="%T1% streamselect@2 map 1",sendcmd="%T2%
streamselect@1 map 0",sendcmd="%T2% streamselect@2 map 0"[a];[2]loop=-1:1[2a];[3]loop=-1:1[3a];[4]loop=-1:1[4a];
[5]loop=-1:1[5a];[3a][2a]streamselect@1=map=0[x];[5a][4a]streamselect@2=map=0[y];[1]fps=%FR%,v360=e:ball:w=%SIZE2%:h=
%SIZE2%[c];[a][c]xstack=2:"0_0|0_h0"[c];[c][x][y]remap=fill=black,rotate=-(%ROT_A%*sin(2*PI/%ROT_T%*t))/180*PI -r %FR%
-b:v %BR% -t %LENGTH% -f mpegts "udp://[Link]:1234"

pause

312
If audio and video aren't in sync, the audio timestamps can be changed as follows. This brings audio forward by 300ms relative to video. It is audio that
needed to be shifted, because one cannot combine a filter and the copy codec. (This example is from Jan Ceuleers in the FFmpeg-user list, 2023-03-01).

ffmpeg -i "udp://[Link]:1234" -lavfi "asetpts=PTS-0.3/TB" -vcodec copy -codec:a aac -b:a 256k -f mpegts pipe:1

313
2.143 Spherical Transformations with two Inputs

Two input images (or videos) are vertically stacked together, and a remap filter is applied to the resulting frame. Each pixel in the output frame can come
either from input 1 or from input 2.
This can be used for wormhole simulation, where two 360° worlds are combined into one output image:

Equirectangular input 1 Equirectangular input 2 Equirectangular output

It can also be used for overlaying a (rotating) planet in front of a background video.

314
Comments For which input / Command line in batch file
output format is it?
Specify the parameters For all formats set "IN_W=2400" :: Main input width in pixels
for the spherical set "IN_H=1200" :: Main input height in pixels
transformation. set "IN_W2=2400" :: Secondary input width (inside the wormhole)
Of course, you only set "IN_H2=1200" :: Secondary input height (inside the wormhole)
have to specify those set "OUT_W=1800" :: Output width in pixels
set "OUT_H=900" :: Output height in pixels
parameters that are
set "OUT_H_FOV=360" :: Output horizontal field of view in degrees
actually used. set "OUT_V_FOV=180" :: Output vertical field of view in degrees
set "OUT2_H_FOV=180" :: Output2 horizontal field of view in degrees
set "OUT2_V_FOV=180" :: Output2 vertical field of view in degrees
set "RS=30" :: Schwarzschild radius in degrees
set "DIST=2" :: Distance normalized to planet radius
set "S=0.1" :: Make the distortion smaller, 1 = realistic
set "YAW=0" :: Output yaw rotation in degrees
set "PITCH=-30" :: Output pitch rotation in degrees
set "ROLL=0" :: Output roll rotation in degrees
set "YAW1=0" :: Input 1 yaw rotation in degrees
set "PITCH1=30" :: Input 1 pitch rotation in degrees
set "ROLL1=0" :: Input 1 roll rotation in degrees
set "YAW2=0" :: Input 2 yaw rotation in degrees
set "PITCH2=0" :: Input 2 pitch rotation in degrees
set "ROLL2=20" :: Input 2 roll rotation in degrees
The size of the xmap For all formats rem Create the xmap and ymap files
and ymap files is the
size of the output ffmpeg -f lavfi -i nullsrc=size=%OUT_W%x%OUT_H% -vf format=pix_fmts=gray16le,geq='^
image. The size of the
two input images may
be different.
Convert from output For fisheye output st(0,%OUT_H_FOV%/180*((2*X+1)/%OUT_W%-1));^
pixel coordinates to st(1,%OUT_V_FOV%/180*((2*Y+1)/%OUT_H%-1));^
X,Y,Z space, depending st(2,atan2(ld(1),ld(0)));^
on the desired output st(3,PI/2*hypot(ld(0),ld(1)));^
format. st(4,sin(ld(3))*cos(ld(2)));^
st(5,sin(ld(3))*sin(ld(2)));^
The X,Y,Z output
st(6,cos(ld(3)));^
coordinates are saved st(1,if(lt(ld(6),cos(%OUT_H_FOV%/360*PI)),32767,0));^

315
in variables 4,5,6. For st(0,PI/360*%OUT_H_FOV%*((2*X+1)/%OUT_W%-1));^
equirectangular st(1,PI/360*%OUT_V_FOV%*((2*Y+1)/%OUT_H%-1));^
Note: If the pixel is output st(4,cos(ld(1))*sin(ld(0)));^
unmapped, variable (1) st(5,sin(ld(1)));^
is set to 32767 which st(6,cos(ld(1))*cos(ld(0)));^
means out of range.
Optional output roll For all formats st(0,ld(4)*cos(%ROLL%/180*PI)-ld(5)*sin(%ROLL%/180*PI));^
rotation (around Z st(5,ld(5)*cos(%ROLL%/180*PI)+ld(4)*sin(%ROLL%/180*PI));^
axis), input and output st(4,ld(0));^
are in variables 4,5,6.
Optional output pitch For all formats st(0,ld(5)*cos(%PITCH%/180*PI)-ld(6)*sin(%PITCH%/180*PI));^
rotation (around X st(6,ld(6)*cos(%PITCH%/180*PI)+ld(5)*sin(%PITCH%/180*PI));^
axis), input and output st(5,ld(0));^
are in variables 4,5,6.
Optional output yaw For all formats st(0,ld(4)*cos(%YAW%/180*PI)+ld(6)*sin(%YAW%/180*PI));^
rotation (around Y st(6,ld(6)*cos(%YAW%/180*PI)-ld(4)*sin(%YAW%/180*PI));^
axis), input and output st(4,ld(0));^
are in variables 4,5,6.
Decide from which of Based on st(7,atan2(ld(5),ld(4)));^
the two inputs this Schwarzschild st(8,acos(ld(6)));^
pixel comes, and set radius (in degrees) st(9,lte(ld(8),%RS%/180*PI));^
variable 9 accordingly:
Based on distance, st(7,atan2(ld(5),ld(4)));^
0 = from input 1 st(8,acos(ld(6)));^
normalized to
1 = from input 2 st(9,lte(ld(8),asin(1/%DIST%)));^
planet radius
Optional black hole For all formats This block can be omitted, if no distortion is required.
distortion for input 1,
%RS% is the ifnot(ld(9),st(8,ld(8)-%S%*(2*%RS%/180*PI/(ld(8)-%RS%/180*PI))));^
Schwarzschild radius ifnot(ld(9),st(4,sin(ld(8))*cos(ld(7))));^
in degrees and %S% is ifnot(ld(9),st(5,sin(ld(8))*sin(ld(7))));^
ifnot(ld(9),st(6,cos(ld(8))));^
a factor to make the
distortion smaller. For
realistic simulation use
S = 1.

316
Optional input 1 roll For all formats ifnot(ld(9),st(0,ld(4)*cos(%ROLL1%/180*PI)-ld(5)*sin(%ROLL1%/180*PI)));^
rotation (around Z ifnot(ld(9),st(5,ld(5)*cos(%ROLL1%/180*PI)+ld(4)*sin(%ROLL1%/180*PI)));^
axis), input and output ifnot(ld(9),st(4,ld(0)));^
are in variables 4,5,6.
Optional input 1 pitch For all formats ifnot(ld(9),st(0,ld(5)*cos(%PITCH1%/180*PI)-ld(6)*sin(%PITCH1%/180*PI)));^
rotation (around X ifnot(ld(9),st(6,ld(6)*cos(%PITCH1%/180*PI)+ld(5)*sin(%PITCH1%/180*PI)));^
axis), input and output ifnot(ld(9),st(5,ld(0)));^
are in variables 4,5,6.
Optional input 1 yaw For all formats ifnot(ld(9),st(0,ld(4)*cos(%YAW1%/180*PI)+ld(6)*sin(%YAW1%/180*PI)));^
rotation (around Y ifnot(ld(9),st(6,ld(6)*cos(%YAW1%/180*PI)-ld(4)*sin(%YAW1%/180*PI)));^
axis), input and output ifnot(ld(9),st(4,ld(0)));^
are in variables 4,5,6.
Convert from X,Y,Z For ifnot(ld(9),st(7,atan2(ld(5),ld(4))));^
space to the desired equirectangular ifnot(ld(9),st(8,acos(ld(6))));^
input 1 format. input 1
Only for xmap file:
ifnot(ld(9),st(7,atan2(sin(ld(8))*cos(ld(7)),cos(ld(8)))/PI));^
ifnot(ld(9),st(7,ld(1)+0.5*%IN_W%*(1+ld(7))));^

Only for ymap file:

ifnot(ld(9),st(8,2/PI*asin(sin(ld(8))*sin(ld(7)))));^
ifnot(ld(9),st(8,ld(1)+0.5*%IN_H%*(1+ld(8))));^
Transform output 2 Based on if(ld(9),st(5,180/PI/%RS%*ld(8)*cos(ld(7))));^
coordinates to [-1...+1] Schwarzschild if(ld(9),st(6,180/PI/%RS%*ld(8)*sin(ld(7))));^
range, the result is in radius (in degrees)
variables 5 and 6
Based on distance, if(ld(9),st(5,ld(8)/asin(1/%DIST%)*cos(ld(7))));^
normalized to if(ld(9),st(6,ld(8)/asin(1/%DIST%)*sin(ld(7))));^
planet radius
Convert from [-1...+1] For fisheye output if(ld(9),st(5,ld(5)*%OUT2_H_FOV%/180));^
range to X,Y,Z space, 2 if(ld(9),st(6,ld(6)*%OUT2_V_FOV%/180));^
depending on the if(ld(9),st(7,atan2(ld(6),ld(5))));^
desired output format. if(ld(9),st(8,PI/2*hypot(ld(5),ld(6))));^
if(ld(9),st(4,sin(ld(8))*cos(ld(7))));^
if(ld(9),st(5,sin(ld(8))*sin(ld(7))));^
The X,Y,Z output
if(ld(9),st(6,cos(ld(8))));^

317
coordinates are saved For mirrorsphere if(ld(9),st(7,atan2(ld(5),ld(6))));^
in variables 4,5,6. output 2 if(ld(9),st(8,2*asin(clip(hypot(ld(5),ld(6)),0,1))));^
(as seen from if(ld(9),st(4,sin(ld(8))*sin(ld(7))));^
infinite distance) if(ld(9),st(5,sin(ld(8))*cos(ld(7))));^
if(ld(9),st(6,cos(ld(8))));^
For planet output 2 if(ld(9),st(4,ld(5)));^
(as seen from if(ld(9),st(5,ld(6)));^
infinite distance) if(ld(9),st(6,sqrt(1-ld(4)*ld(4)-ld(5)*ld(5))));^

For planet output 2 if(ld(9),st(2,hypot(ld(5),ld(6))));^

(as seen from the if(ld(9),st(2,ld(2)*tan(asin(1/%DIST%))));^
finite distance if(ld(9),st(3,1+ld(2)*ld(2)*(1-%DIST%*%DIST%)));^
%DIST%) if(ld(9),st(8,2*atan((1-sqrt(abs(ld(3))))/((1+%DIST%)*ld(2)))));^
Less than half of if(ld(9),st(7,atan2(ld(5),ld(6))));^
if(ld(9),st(4,sin(ld(8))*sin(ld(7))));^
the sphere is
if(ld(9),st(5,sin(ld(8))*cos(ld(7))));^
visible. if(ld(9),st(6,cos(ld(8))));^
Optional input 2 roll For all formats if(ld(9),st(0,ld(4)*cos(%ROLL2%/180*PI)-ld(5)*sin(%ROLL2%/180*PI)));^
rotation (around Z if(ld(9),st(5,ld(5)*cos(%ROLL2%/180*PI)+ld(4)*sin(%ROLL2%/180*PI)));^
axis), input and output if(ld(9),st(4,ld(0)));^
are in variables 4,5,6.
Optional input 2 pitch For all formats if(ld(9),st(0,ld(5)*cos(%PITCH2%/180*PI)-ld(6)*sin(%PITCH2%/180*PI)));^
rotation (around X if(ld(9),st(6,ld(6)*cos(%PITCH2%/180*PI)+ld(5)*sin(%PITCH2%/180*PI)));^
axis), input and output if(ld(9),st(5,ld(0)));^
are in variables 4,5,6.
Optional input 2 yaw For all formats if(ld(9),st(0,ld(4)*cos(%YAW2%/180*PI)+ld(6)*sin(%YAW2%/180*PI)));^
rotation (around Y if(ld(9),st(6,ld(6)*cos(%YAW2%/180*PI)-ld(4)*sin(%YAW2%/180*PI)));^
axis), input and output if(ld(9),st(4,ld(0)));^
are in variables 4,5,6.

318
Convert from X,Y,Z For if(ld(9),st(7,atan2(ld(5),ld(4))));^
space to the desired equirectangular if(ld(9),st(8,acos(ld(6))));^
input 2 format. input 2
Only for xmap file:
if(ld(9),st(7,atan2(sin(ld(8))*cos(ld(7)),cos(ld(8)))/PI));^
if(ld(9),st(7,0.5*%IN_W2%*(1+ld(7))));^

Only for ymap file:

if(ld(9),st(8,2/PI*asin(sin(ld(8))*sin(ld(7)))));^
if(ld(9),st(8,%IN_H%+0.5*%IN_H2%*(1+ld(8))));^
Finish writing the xmap For all formats Only for xmap file:
and ymap files ld(7)' -frames 1 -y [Link]

Only for ymap file:

ld(8)' -frames 1 -y [Link]
Stack the two images For all formats ffmpeg -i [Link] -i [Link] -f image2 -i [Link] -f image2 -i [Link] -lavfi [0]
together and apply the [1]xstack=2:"0_0|0_h0"[c];[c][2][3]remap -y [Link]
remap filter.
Note: "-f image2" is
only required in rare
cases, if the *.pgm file
isn't auto-detected
correctly as an image.

319
2.144 Simulation of a moving wormhole

If the wormhole shall move in the field of view, two things must move:
1. The black hole distortion must move. This requires many unique xmap and ymap files for each frame. These files are created by a C# program.
2. The inserted mirror-sphere video must move. This can be realized with sendcmd and overlay filters.

Step 1:
In the main video it's recommended to have a small object at that position in space where the wormhole shall later be inserted. This can for example be a
small white ball (about 8mm diameter) on an almost invisible tripod (which can be built with 0.3mm diameter carbon fibers).
The x,y coordinates of this object must be measured in each frame. There are two methods how to measure the x,y coordinates: Either extract many
images from the video and measure the coordinates manually (this is described in step 2), or preferably extract the coordinates automatically with
FFprobe and find_rect filter. In this case continue with step 3.

Step 2:
Extract a suitable number of frames from the main video:
set "IN=in.mp4" :: Input video
set "STEP=1" :: Step width (number of frames)
set "OUT=image%%[Link]" :: Output images filename

ffmpeg -i %IN% -vf framestep=%STEP% -start_number 0 -y %OUT%

pause
Measure the x,y coordinates of the small object in each frame and enter the positions in the "[Link]" file. Set the "offset" variable in the C#
program to 0. Continue with step 4.

320
Step 3:
Create a small 40x40 pixel grayscale image of the object and then automatically extract the x,y coordinates from the main video:
set "IN=in.mp4" :: Input video
set "OBJ=[Link]" :: Image of the object, must be gray8
set "TH=0.4" :: Threshold, 0.01 = only exact matches, 0.99 = almost everything matches
set "XMIN=900" :: Minimum x position of the object's top left corner
set "XMAX=1900" :: Maximum x position of the object's top left corner
set "YMIN=490" :: Minimum y position of the object's top left corner
set "YMAX=510" :: Maximum y position of the object's top left corner

ffprobe -f lavfi movie=%IN%,find_rect=object=%OBJ%:threshold=%TH%:xmin=%XMIN%:xmax=%XMAX%:ymin=%YMIN%:ymax=%YMAX%

-show_entries frame=pkt_pts_time:frame_tags=[Link].x,[Link].y -of csv=p=0 1> [Link]

pause
Note: To speed up the algorithm, make the object image as small as possible (40x40 pixels) and specify a search window with the xmin, xmax, ymin,
ymax options.
This is the resulting logfile. If no coordinates are written in a line, then no object was found for the specified threshold. In this case you can try a larger
threshold value, or you have to enter the coordinates manually.
0.000000
0.040000
0.080000
0.120000,45,1
0.160000,45,1
0.200000,45,1
0.240000,45,1
...

Note: These coordinates are for the top left corner of the 40x40 pixels search window. Set the "offset" variable in the C# program to 20.

Step 4:
Calculate the xmap and ymap files for all frames. This is done by a C# program, which can be downloaded here:
[Link]

321
Step 5:
The file [Link] was also automatically created by the C# program (many lines omitted here):
0.00 overlay x 1665.00, overlay y 454.00, scale w 100.00, scale h 100.00;
0.04 overlay x 1665.00, overlay y 454.00, scale w 100.00, scale h 100.00;
0.08 overlay x 1665.00, overlay y 454.00, scale w 100.00, scale h 100.00;
...
6.00 overlay x 939.00, overlay y 449.00, scale w 100.00, scale h 100.00;
6.04 overlay x 935.20, overlay y 448.20, scale w 101.60, scale h 101.60;
6.08 overlay x 932.40, overlay y 447.40, scale w 103.20, scale h 103.20;
6.12 overlay x 928.60, overlay y 446.60, scale w 104.80, scale h 104.80;
6.16 overlay x 925.80, overlay y 445.80, scale w 106.40, scale h 106.40;
6.20 overlay x 922.00, overlay y 445.00, scale w 108.00, scale h 108.00;
...

Step 6: Run this batch file to create the final moving wormhole video:
set "IN=in.mp4" :: Main input video
set "LP=WL4149.mp4" :: Equirectangular video for mirror-sphere
set "P=0" :: Pitch angle
set "Y=0" :: Yaw angle
set "R=60" :: Roll angle
set "R1=0.00" :: Rotation speed before v360 filter, 1.0 means one revolution per frame
set "R2=0.00" :: Rotation speed after v360 filter, 1.0 means one revolution per second
set "V=9" :: Time when wormhole vanishes
set "LUT=[Link]" :: Look-Up-Table
set "T=12" :: Length of output video

rem Make only the mirror-sphere video

rem ffmpeg -i %LP% -vf v360=output=ball:pitch=%P%:yaw=%Y%:roll=%R% -t %T% -y lp.mp4

ffmpeg -i %IN% -ss 9 -i %LP% -start_number 0 -i xmap%%[Link] -start_number 0 -i ymap%%[Link] -lavfi "[0]sendcmd='%V%
streamselect map 1',split[4][5];[4][2][3]remap=fill=red,sendcmd=f=[Link][6];[1]
fps=25,v360=[Link]pitch=0,scroll=h=%R1%,v360=output=ball:pitch=%P%:yaw=%Y%:roll=%R
%:alpha_mask=1,rotate='%R2%*2*PI*t':[email protected],scale=w=10:h=10:eval=frame,lut3d=%LUT%[7];[6][7]
overlay=x=0:y=0:format=rgb[8];[8][5]streamselect=map=0,format=yuv420p" -t %T% -y out.mp4

pause

322
Notes:
overlay=format=rgb is strongly required, because the default format yuv420 allows only to set the x,y coordinates in multiples of 2.
"remap=fill=red" is used here only to make alignment errors visible, if the overlay isn't exactly at the same position as the black hole distortion. Normally
there should no red pixels be visible. After this check you can replace it by "remap=fill=black".
"fps=25" is used here because the mirror-sphere video has a different framerate (30fps), which resulted in jerky scaling of this video.
It's also possible to let the inner area of the wormhole rotate as a function of time. Two different rotations are applied in this example. The first rotation is
using the scroll filter (applied to an equirectangular video, before the v360 filter) and the other is using the rotate filter after the v360 filter.
Note for scroll filter: scroll=h=1.0 means one full horizontal revolution per frame. So you have to know the framerate to set the rotation speed.

323
2.145 Dynamically change commands with "sendcmd" or "zmq"

In FFmpeg "commands" are a special case of options, which can be changed dynamically while FFmpeg is running. All commands are options, but not
all options are commands. Commands can be changed either with "sendcmd" or with "zmq".

sendcmd zmq
Which options can be controlled? All those that have support for commands. All those that have support for commands.
When can an option be changed? Options can be changed at exact times, but these Options can be changed in real time while FFmpeg is
times must be known in advance, because they must running, but the timing is not as exact as with
be written in the *.cmd file or in the command line. sendcmd.
Timing accuracy exact about 100-200ms, as a rough estimate
Is it possible to decide in real time to no yes
change or not to change an option?
Possible sources where data comes from • *.cmd file • from a second console window via
• FFmpeg command line, as argument of "zmqsend" tool
sendcmd filter • from a batch file via "zmqsend" tool
• zeromq library which is available for many
programming languages
• for example NetMQ library for C#
Is it possible to change an option for yes, with [expr] option no, not with exact timing for each frame. However it's
each frame, using an expression? possible if exact timing for each frame isn't required

Note: For almost all commands, it doesn't matter if you set an option directly in the command line, or via sendcmd, or via zmq. The effect is the same in
all three cases. But there is one exception from this rule: These are the options yaw, pitch and roll in the v360 filter. Directly in the command line that are
absolute angles, however if sent via sendcmd or zmq that are relative angles. Don't try to understand it. It's stupid.

324
2.146 Sendcmd and commands

• sendcmd has many pitfalls and can drive you crazy!

• The sendcmd filter sends commands to another filter. For example in the previous chapter sendcmd was used to to send the x and y coordinates
to the overlay filter. The commands are defined in a file (*.cmd is recommended), or could also be defined in the command line.
• To find out which filters support commands, use the ffmpeg -filters command, or look in the documentation of the filter.
• Normally the sendcmd filter is inserted in the filter chain somewhere before the target filter. A problem arises when the target filter has more than
one input (for example overlay has two inputs). This doesn't work, because sendcmd accepts only one input. In this case sendcmd must be
inserted somewhere earlier in the filter chain, where only one input exists.
• It's important that sendcmd is inserted at a position in the filter chain that has sufficient duration. For example, if the overlaid video is shorter
than the main video, and if sendcmd is inserted in the input of the shorter video, that would give unexpected behaviour, because when the
shorter video has ended, sendcmd will get the wrong time (which stays then constant), and will send wrong commands to the other filters based
on the wrong time. Always insert sendcmd at the longest input.
• It's also possible to have more than one sendcmd in the filter chain, for example at both inputs.
• It's also possible to insert sendcmd after the target filter, for example at the end of the filter chain. The drawback of this method is that the
changes become effective with one frame delay.
• All arguments of the sendcmd target filter must be initialized with valid values, even if these values are never used because sendcmd does
always overwrite them.
• It's also possible to evaluate an expression in sendcmd and send the result to the target filter. To enable expression evaluation the [expr] flag
must be used instead of the default [enter] flag. Inside the expression the "TI" variable can be used, which runs from 0 to 1 in the interval.
• If a line in the *.cmd file begins with a "#" character then it's a comment. Empty lines are allowed as well.
• The *.cmd file must contain at least one command. It's not allowed if it contains only comments.
• If the filter chain contains multiple filters of the same type, they must be given unique names, for example "v360@1", "v360@2".
• (I have not tested this with FFmpeg. It doesn't work with FFplay) There is another (undocumented) way how to send commands. In the same
console window where FFmpeg is running, type "c" or "C" and immediately (without a space character) let the command follow, for example:
Cdrawtext 15.3 reinit 'x=752:y=480'<enter> In this example "15.3" is the time. You can use "all" instead of the filter/class instance if
you want to send the command to all filters that can receive it. Instead of pressing <enter> you can also send "\n". Found here:
[Link]

325
 • FFmpeg accepts a few keys while it's running. This is undocumented and can only be found in the source code in
ffmpeg.c in the function check_keyboard_interaction()

? show this help

+ increase verbosity
- decrease verbosity
c Send command to first matching filter supporting it
C Send/Queue command to all matching filters
dD cycle through available debug modes
h dump packets/hex press to cycle through the 3 states
q quit
s Show QP histogram
ctrl-c The first ctrl-c interrupts the transcoding process and starts flushing.
The second ctrl-c interrupts the flushing itself.
Three ctrl-c in a row interrupts everything immediately.

This is an example for cropping a square region out of a rectangular video, while changing the x,y position of the cropped region as a function of time
with linear interpolation:
ffmpeg -i in.mp4 -lavfi sendcmd=f=[Link],crop@1=w=800:h=800:x=0:y=0 out.mp4

pause

The top left coordinates of the crop window are specified in the file "[Link]":
0.0-1.0 [expr] crop@1 x 'lerp(0,100,TI)';
0.0-1.0 [expr] crop@1 y 'lerp(0,100,TI)';
1.0-2.0 [expr] crop@1 x 'lerp(100,150,TI)';
1.0-2.0 [expr] crop@1 y 'lerp(100,220,TI)';
2.0-3.0 [expr] crop@1 x 'lerp(150,80,TI)';
2.0-3.0 [expr] crop@1 y 'lerp(220,220,TI)';
and so on...

326
Note: In this example "crop@1" can be replaced by "crop", because there is only one crop filter in the filter chain.
Note: The same effect could also be realized with the 'zoompan' filter, but that's more complicated because 'zoompan' doesn't support commands.

327
This is an example where the x,y position of the cropped region changes at certain times, without any interpolation:
ffmpeg -i in.mp4 -vf sendcmd=f=[Link],crop=[Link] out.mp4

pause

The top left coordinates of the crop window are specified in the file "[Link]":
9.50 crop x 30;
9.50 crop y 60;
15.00 crop x 100;
15.00 crop y 100;

In most cases it's not possible to change the size of a video stream dynamically, but in some cases it does work, for example if "scale" is immediately
followed by "overlay":
ffmpeg -f lavfi -i color=red:s=800x600 -f lavfi -i color=yellow:s=2x2 -lavfi [1]sendcmd="2.0 scale w
400",scale=w=200:h=200[a];[0][a]overlay=x=100:y=100 -t 5 -y out.mp4

pause

This is an example of a rotating earth, where the rotation axis rapidly changes (this is physically impossible) and the observer's viewing point changes:
set "IN=Earth_eq.jpg" :: Equirectangular input image of earth
:: from [Link]
set "BG=[Link]" :: Background image 1920x1080
set "R1=0.005" :: Rotation speed, 1.0 means one revolution per frame, 0.005 means 8s per rev. at 25fps
set "D=400" :: Diameter of earth
set "X=760" :: X position of earth
set "Y=340" :: y position of earth
set "T=30" :: Length of output video

ffmpeg -loop 1 -i %BG% -loop 1 -i %IN% -lavfi

[1]sendcmd=f=[Link],v360@1=[Link]pitch=0:yaw=0:roll=0:reset_rot=1,scroll=h=
%R1%,v360@2=e:perspective:pitch=0:yaw=0:roll=0:alpha_mask=1:reset_rot=1,scale=%D%:%D%[a],[0][a]
overlay=x=%X%:y=%Y% -t %T% -y out.mp4

pause

328
[Link]
# set the initial conditions

0.00 v360@1 pitch 0;

0.00 v360@1 yaw 0;
0.00 v360@1 roll 0;
0.00 v360@2 pitch 50;
0.00 v360@2 yaw 0;
0.00 v360@2 roll 0;

# from t = 8s to 9s change the rotation axis of the earth by 60° from the north pole to Cairo in Egypt
# Latitude 30° north, Longitude 30° east

8.00-9.001 [expr] v360@1 yaw 'lerp(0,30,TI)';

8.00-9.001 [expr] v360@1 pitch 'lerp(0,-60,TI)';

# from t = 14s to 15s change the viewing point, so that the observer is on the rotation axis:

14.00-15.001 [expr] v360@2 pitch 'lerp(50,90,TI)';

# from t = 18s to 21s change the rotation axis of the earth from Cairo to New York
# Latitude 41° north, Longitude 74° west

18.00-21.001 [expr] v360@1 yaw 'lerp(30,-74,TI)';

18.00-21.001 [expr] v360@1 pitch 'lerp(-60,-49,TI)';

Note: If specified in the command line, the v360 rotation angles are absolute angles. However if sent via sendcmd, they become relative angles. That's
why the "reset_rot=1" option is used. It automatically resets the rotations to zero before a new relative rotation is applied.

In my optinion "relative rotations per frame" are the same thing as rotational velocity, and I don't understand why the same options (yaw, pitch and roll)
are used for abslotue angles in the command line and rotational velocities in sendcmd. It would be much clearer if different options would be used for
different things.
The "reset_rot" option can be set to 0, 1 or -1. The meaning of -1 is unclear. It's not documented.

329
A few examples for sendcmd and single / double quotes:
ffmpeg -i in1.mp4 -i in2.mp4 -lavfi sendcmd at the beginning of the filter chain, double
"[0]sendcmd='3.0 streamselect map 1'[a];[a][1]streamselect=inputs=2:map=0" quotes for whole filter chain.
out.mp4 Works fine, but would give enexpected results if the
first input is shorter than the second input!
ffmpeg -i in1.mp4 -i in2.mp4 -lavfi This doesn't work because sendcmd accepts only
"[0][1]sendcmd='3.0 streamselect map 1',streamselect=inputs=2:map=0" out.mp4 one input
ffmpeg -i in1.mp4 -i in2.mp4 -lavfi This is the example from the streamselect
sendcmd='3.0 streamselect map 1',streamselect=inputs=2:map=0 out.mp4 documentation, doesn't work under Windows.
ffmpeg -i in1.mp4 -i in2.mp4 -lavfi Single quotes replaced by double quotes, works
sendcmd="3.0 streamselect map 1",streamselect=inputs=2:map=0 out.mp4 under Windows.
ffmpeg -i in1.mp4 -i in2.mp4 -lavfi Double quotes added for the whole filter chain,
"sendcmd='3.0 streamselect map 1',streamselect=inputs=2:map=0" out.mp4 single quotes for sendcmd argument, works under
Windows.
ffmpeg -i in1.mp4 -i in2.mp4 -lavfi [0][1] added, sendcmd at the end of the filter chain,
"[0][1]streamselect@my=inputs=2:map=0,sendcmd='3.0 streamselect@my map 1'" commands become effective with one frame delay.
out.mp4 Double quotes for filter chain, single quotes for
sendcmd argument
ffmpeg -i in1.mp4 -i in2.mp4 -lavfi [0][1] added, sendcmd at the end of the filter chain,
[0][1]streamselect@my=inputs=2:map=0,sendcmd="3.0 streamselect@my map 1" commands become effective with one frame delay.
out.mp4 No quotes for filter chain, double quotes for
sendcmd argument

Note about double quotes around the filter chain:

In Windows it's not required to put the whole filter chain in double quotes, but it seems these double quotes are required on a Mac. Not tested myself.
If a filter chain contains the "|" character, it must be encapsulated in "" double quotes.
A note from Moritz Barsnick in the FFmpeg user list, April 5, 2020:
"Under Windows, " ' " is not a command line quotation character. If parts of the command line need to be quoted in order to be collated, you need to use
the double quotation mark ' " '. The single quotation mark is passed directly to ffmpeg on Windows, making the filter argument unparsable. [...] I believe a
large part of ffmpeg's examples in doc and wiki are "wrong" in this matter, and could or should be changed to also work on Windows, where possible."

330
2.147 Sending commands with ZMQ

This is an example of a command line with the "zmq" filter, which receives messages that were sent from somewhere else. It's possible to send
messages to all filters and all options that accept commands (as indicated in the documentation of the filter). These are the same options as for the
"sendcmd" filter.
ffplay -dumpgraph 1 -f lavfi "color=s=200x200:c=red[l];color=s=200x200:c=blue[r];nullsrc=s=400x200,zmq[bg];[bg]
[l]overlay[bg+l];[bg+l][r]overlay@my=x=200"

pause

Note: "-dumpgraph" is not required in this example. This option is only available in FFplay and not in FFmpeg. It draws a graphical representation of the
filtergraph in the console output.
Note: This example is copied from the documentation of the zmq filter, but the sizes were changed because it didn't work with the original smaller sizes.
Seems to be a bug in FFplay.
Note: the zmq filter has the option "bind_address" or "b" which is by default set to "tcp://*:5555". You can change this value to your needs, but don't
forget to escape any ’:’ signs.
Note: Don't send messages to those options that don't support commands. This could lead to malfunction.

For details about ZMQ (or ZeroMQ), see [Link]

The commands can be sent from a command line in another console window by using the [Link] tool.
Copy the files [Link] and [Link] in the same directory where you have [Link] (In this example I'm using the folder c:\ffmpeg). Open a
second console window and type this command:
echo overlay@my x 150 | c:\ffmpeg\zmqsend

Question: What can be done if this error message appears?

[Parsed_zmq_3 @ 000001d92beb4180] Could not bind ZMQ socket to address 'tcp://*:5555': Address in use
[lavfi @ 000001d92beabb80] Error initializing filter 'zmq'

331
It's also possible to send ZMQ commands from a C# program with the NetMQ library, as in this short example:
using NetMQ;
using [Link];
using System;
using [Link];

namespace netmq
{
public partial class Form1 : Form
{
public Form1()
{
InitializeComponent();
}

private void button1_Click(object sender, EventArgs e)

{
using (var client = new RequestSocket())
{
[Link]("tcp://localhost:5555"); // this works, but I don't understand it:
// what does "tcp://" mean?
// what does "localhost:" mean?
// what does "5555" mean?
// If you can explain it, please let me know
[Link]("overlay@my x 150");
var message = [Link]();
[Link]("Received {0}", message);
}
}
}
}

Note: Use Nuget to add the NetMQ package, and add the two using directives at the beginning.

Please note that not everything that's possible is also useful. For example, a message can be sent to the "width" option of the "scale" filter for changing
the size of a video stream. But changing the size mid-stream isn't supported by many other filters (for example "eq", "colorkey" and "despill"). In some
cases it works (for example "scale" immediately followed by "overlay"), but in most other cases it fails.

332
2.148 Changing an input image while FFmpeg is running

Let's assume you want to change an input image while FFmpeg is running. As a simple example, we want to show the image on the screen.
The trick is not to use the image as input, but instead a symbolic link (symlink) which is pointing to the image:
ffmpeg -framerate 1 -stream_loop -1 -f image2 -i [Link] -window_x 0 -window_y 0 -f sdl2 -

pause

The complicated part is how to create and change the symlink. You need administrator permission to do this. Type "cmd" in the Windows search window,
then click on "Run as administrator". Go to your working directory by typing (for example)
cd C:\Users\astro\Desktop\test

Before you can start FFmpeg, you must create the symlink which is pointing to the first image:
mklink [Link] [Link]

Then FFmpeg can be started and it should show the first image.
As the next step, we want to change the symlink to the second image, while FFmpeg is running. To do this, type the following three commands:
mklink [Link] [Link]
copy /l /y [Link] [Link]
del [Link]

In the first command line a temporary symlink to the new image is created.
In the second command line the symlink is changed to the new target and FFmpeg should immediately show the new image.
In the third command line the temporary symlink is deleted. That must be done because a new symlink can't be created if one with the same name does
already exist.
See also: [Link]
Note: You can also create links by right-clicking in a folder (New --> Shortcut), but those links don't work for this purpose. They are somehow different
from the symlinks which are used here.
Note: It was also suggested to use -readrate 1 (which is the same as -re). This did't work when I tested it, but it might be useful in other cases.

333
2.149 FFmpeg and C# Programming

Start a FFplay process with console window and video window:

Process p;
ProcessStartInfo startInfo = new ProcessStartInfo();
[Link] = "ffplay";
[Link] = "-f lavfi testsrc2=s=vga";
p = [Link](startInfo);
Note: This assumes that you have set the PATH environment variable accordingly, so that FFplay is found. Otherwise you can use
[Link] = ...

Start a FFplay process without console window:

Process p;
ProcessStartInfo startInfo = new ProcessStartInfo();
[Link] = false;
[Link] = true;
[Link] = "ffplay";
[Link] = "-f lavfi testsrc2=s=vga";
p = [Link](startInfo);

See also: [Link]

Stop the FFplay process:

[Link]();

Get a filename from a command line argument, this works also if you start the C# program by dropping a file on the program's icon:
private void Form1_Shown(object sender, EventArgs e)
{
string[] arguments = [Link](); // get the filename by command line argument,
if ([Link] == 2) // this works also for drag-and-drop
myFilename = arguments[1];
}

334
C# Sample program for real-time zmq adjustment of brightness and contrast with scrollbars:
using NetMQ;
using [Link];
using System;
using [Link];
using [Link];
using [Link];

namespace netmq
{
public partial class Form1 : Form
{
Process p;
RequestSocket client;

public Form1()
{
InitializeComponent();
}

private void button2_Click(object sender, EventArgs e) // Start the FFplay process

{
if (p == null)
{
ProcessStartInfo startInfo = new ProcessStartInfo();
[Link] = false;
[Link] = true;
[Link] = "ffplay";
[Link] = "-top 0 -left 0 -f lavfi \"testsrc2=s=960x540,zmq,eq@my\"";
[Link]([Link]);
p = [Link](startInfo);
}
}

private void button3_Click(object sender, EventArgs e) // Stop the FFplay process

{
if ((p != null) && ![Link])
{
[Link]();
p = null;
}

private void Form1_Shown(object sender, EventArgs e) // Create and connect the client for zmq
{
client = new RequestSocket();
[Link]("tcp://localhost:5555"); // This works, but I don't understand it:

335
 // What does "tcp://" mean?
// What does "localhost:" mean?
// Why 5555?
// If you can explain it, please let me know
}

private void hScrollBar1_Scroll(object sender, ScrollEventArgs e) // Scrollbar for brightness adjustment

{
[Link]("eq@my brightness " + ((double)[Link] * 0.02).ToString([Link]));
var message = [Link]();
[Link]("Received {0}", message);
}

private void hScrollBar2_Scroll(object sender, ScrollEventArgs e) // Scrollbar for contrast adjustment

{
[Link]("eq@my contrast " + ((double)[Link] * 0.1).ToString([Link]));
var message = [Link]();
[Link]("Received {0}", message);
}

private void Form1_FormClosing(object sender, FormClosingEventArgs e) // Cleanup

{
if ((p != null) && ![Link])
[Link]();
}
}
}

Note: In the command line for FFplay it's not necessary to specify any values for contrast and brightness. Just writing "eq" or "eq@my" is sufficient.

It might be a problem that the code is blocking if the zmq message can't be sent. In this case it's better to specify a 500ms timeout for receiving the reply:
private void vScrollBar1_Scroll(object sender, ScrollEventArgs e)
{
if ((p != null) && ![Link])
{
string message;
[Link]("overlay@my y " + y);
[Link]([Link](500), out message);
}
}

See also my example in chapter "Real-time bluescreening".

336
2.150 FFmpeg wrapper libraries for C#

[Link] (I haven't tested it yet)

337
2.151 Video stabilization

Videos can be stabilized in a one-pass process with "deshake" filter or (better) in a two-pass process with "vidstabdetect" and "vidstabtransform" filters.
set "IN=[Link]" :: Input video
set "OUT=C0650_stab.MOV" :: Output video

rem Stabilize the video

ffmpeg -i %IN% -vf vidstabdetect -y [Link]

del [Link]
ffmpeg -i %IN% -vf vidstabtransform -y %OUT%

pause
Note: The vidstabdetect filter does by default write the stabilization data to the file "[Link]". The vidstabtransform filter does by default read from
this file.
In the above example the output of the filter chain is encoded to the file "[Link]" which is deleted in the next line, because it's not required. This
can be simplified by encoding to the null device:
ffmpeg -i %IN% -vf vidstabdetect -an -f null -
Note: However if the option show=1 is used, then the output video contains the detected stabilization vectors and should not be deleted.

This is the same thing, but with 10-bit DNxHD (Digital Nonlinear Extensible High Definition) codec for importing in the free DaVinci Resolve version:
set "IN=[Link]" :: Input video
set "OUT=C0645_stab.MOV" :: Output video

rem Stabilize the video

ffmpeg -i %IN% -vf vidstabdetect -an -f null -

ffmpeg -i %IN% -vf vidstabtransform -map_metadata 0 -pix_fmt yuv422p10le -c:v dnxhd -profile:v 4 -c:a pcm_s24le
-color_range pc -movflags write_colr -y %OUT%

pause

338
Two notes from Andrei B.:
• The "vidstab" filter has the drawback that it gets confused by rolling shutter from CMOS sensors.
• A very good (and free) tool that can do much better is VirtualDub with Deshaker 3.0 filter. This filter has a rolling shutter factor input and can
greatly improve on reducing the wabbliness of a stabilized video. It's documentation includes instructions on how to measure your camera's
rolling shutter factor.

Some notes from Steffen Richter:

• "shakiness" option seems to have no effect.
• "tripod" may lead to strong z rotations, if no relevant movements are detected.
• "relative": Different from the documentation, the default seems to be "1", which also makes sense.
• "zoomspeed": Values between 0.01 and 0.1 are useful.

This is a test for comparing the "deshake" filter with the "vidstabdetect/vidstabtransform" filters:
rem deshake
ffmpeg -i [Link] -lavfi split[a][b];[a]deshake=rx=64:ry=64:edge=0[c];[b][c]hstack -y [Link]

rem vidstabdetect
ffmpeg -i [Link] -lavfi vidstabdetect=shakiness=10:show=1 -y [Link]

rem vidstabtransform with optzoom=0 (which means no zoom, so that borders are visible)
ffmpeg -i [Link] -lavfi split[a][b];[a]vidstabtransform=smoothing=50:crop=black:optzoom=0[c];[b][c]hstack -y
[Link]

rem vidstabtransform with optzoom=1 (which means optimized zoom, so that no borders are visible)
ffmpeg -i [Link] -lavfi split[a][b];[a]vidstabtransform=smoothing=50:crop=black:optzoom=1[c];[b][c]hstack -y
vidstab_zoom.mov

pause

By comparing the results, I found that the two-stage solution with "vidstabdetect/vidstabtransform" gives much better results than the one-stage
"deshake" solution.
In "vidstabtransform" it's possible to set the "smoothing" parameter which defines the number of frames (2 * smoothing + 1) for low-pass filtering of the
camera movements. The default "smoothing" value is 10 (which means 21 frames), but i found it useful to use higher values between 20 and 50.
"vidstabtransform" does correct x and y translations and also rotations.
The option "optzoom=1" does automatically choose a suitable zoom factor, so that there are no no-data areas at the borders visible.

339
Note: I think there is a bug in vidstabtransform, the option crop=black doesn't work as described. The no-data borders are filled with the colors of the
edge of the image. Black borders appear only when the image shift becomes very large. But that doesn't matter, because with "optzoom=1" (which is the
default) the no-data borders are anyway cropped away.
See also: [Link]

2.152 Remove linear drift

This is an example for removing a linear drift from a video. The first and the last image is extracted and the x,y coordinates of an object is measured in
these two images. The diffence between the coordinates is the motion vector. Then the "crop" filter is used for cropping a window out of the input video,
where the top left coordinates of the crop window are a linear function of time. The input size is 2048x2048, and the output size is reduced to 1856x1856
(which is the input size minus the larger of the two motion vectors).
rem Extract the first frame
ffmpeg -i [Link] -frames 1 -y first_frame.jpg

rem Extract the last frame

ffmpeg -sseof -0.2 -i [Link] -update 1 -y last_frame.jpg

rem Coordinates of object in first frame: 1026, 1091

rem Coordinates of object in last frame: 1131, 1282
rem Motion vector: +105, +191
rem Duration: 17.23s

ffmpeg -i [Link] -lavfi crop=x='105/17.23*t':y='191/17.23*t':w=1856:h=1856 -y out1.mp4

pause

340
2.153 Stabilization of 360° Videos

Note: This stabilization method does no longer work because the behaviour of the v360 filter was changed 25-Oct-2020. The yaw, pitch and roll rotation
angles are now interpreted as relative angles, if they are sent via sendcmd. See also ticket 9447. There was a new option "reset_rot" added which resets
the rotation angles, so that absolute rotations can be simulated. I'm not sure if it could be used in this example. Recommended workaround: Use the
spherical stabilizer in DaVinci Resolve.

360° videos can't be stabilized the same way as normal (flat) videos. A normal video can be stabilized by following one point and applying x
and y shifts. In a 360° video, two points must be followed and rotations in three axes (yaw, pitch, roll) must be applied.

Let's begin by making a nice 360° test video. First get a fulldome test pattern and create an equirectangular test image:
set "IN=[Link]" :: Test pattern from [Link]

ffmpeg -i %IN% -i %IN% -lavfi "[0]transpose=1[left];[1]transpose=2,negate[right];[left]

[right]hstack,v360=input=dfisheye:output=e:pitch=90" -y equirectangular_test.png

pause

Now use this image for creating a test video with rotations around different axes:
set "IN=equirectangular_test.png" :: Equirectangular input image

ffmpeg -loop 1 -i %IN% -lavfi sendcmd=f=[Link],v360=[Link]pitch=0:yaw=0:roll=0,drawtext=text='':x=(w-

text_w)/2:y=0.7*h:fontsize=160:fontcolor=red:boxcolor=yellow:box=1:boxborderw=20 -t 9 -y test.mp4

pause

The rotations are defined is the file "[Link]": I don't really understand what reset_rot=-1 is doing.
0.0-9.0 [expr] v360 reset_rot '-1';

0.0-1.0 drawtext reinit 'text=';

0.0-1.0 [expr] v360 yaw '0';
0.0-1.0 [expr] v360 pitch '0';
0.0-1.0 [expr] v360 roll '0';

341
1.0-2.0 drawtext reinit 'text=PITCH';
1.0-2.0 [expr] v360 yaw '0';
1.0-2.0 [expr] v360 pitch 'lerp(0,45,TI)';
1.0-2.0 [expr] v360 roll '0';

2.0-3.0 drawtext reinit 'text=YAW';

2.0-3.0 [expr] v360 yaw 'lerp(0,45,TI)';
2.0-3.0 [expr] v360 pitch '45';
2.0-3.0 [expr] v360 roll '0';

3.0-4.0 drawtext reinit 'text=PITCH + YAW';

3.0-4.0 [expr] v360 yaw 'lerp(45,90,TI)';
3.0-4.0 [expr] v360 pitch 'lerp(45,90,TI)';
3.0-4.0 [expr] v360 roll '0';

4.0-5.0 drawtext reinit 'text=ROLL';

4.0-5.0 [expr] v360 yaw '90';
4.0-5.0 [expr] v360 pitch '90';
4.0-5.0 [expr] v360 roll 'lerp(0,45,TI)';

5.0-6.0 drawtext reinit 'text=PITCH + ROLL';

5.0-6.0 [expr] v360 yaw '90';
5.0-6.0 [expr] v360 pitch 'lerp(90,135,TI)';
5.0-6.0 [expr] v360 roll 'lerp(45,90,TI)';

6.0-7.0 drawtext reinit 'text=YAW + ROLL';

6.0-7.0 [expr] v360 yaw 'lerp(90,135,TI)';
6.0-7.0 [expr] v360 pitch '135';
6.0-7.0 [expr] v360 roll 'lerp(90,135,TI)';

7.0-8.0 drawtext reinit 'text=PITCH + YAW + ROLL';

7.0-8.0 [expr] v360 yaw 'lerp(135,180,TI)';
7.0-8.0 [expr] v360 pitch 'lerp(135,180,TI)';
7.0-8.0 [expr] v360 roll 'lerp(135,180,TI)';

8.0-9.0 drawtext reinit 'text=';

8.0-9.0 [expr] v360 yaw '180';
8.0-9.0 [expr] v360 pitch '180';
8.0-9.0 [expr] v360 roll '180';

342
How large is the image shift in this test video, from one image to the next? The image height is 1200 pixels and that's 180 degrees. So the image scale is
0.15° per pixel. The rotation speed is 45° per second. So the image shift at the default 25fps framerate is 1.8° per frame or 12 pixel per frame.
Let's check this and extract small grayscale images from the center of the test video (images [Link]) , and also from a point 90° right of the center
(images [Link]):
set "IN=test.mp4" :: Equirectangular input video
set "B=100" :: Image size in pixels
set "T=10" :: Duration in seconds

ffmpeg -i %IN% -vf crop=w=%B%:h=%B%,format=gray8 -start_number 0 -t %T% -y a%%[Link]

ffmpeg -i %IN% -vf v360=input=e:output=e:yaw=90,crop=w=%B%:h=%B%,format=gray8 -start_number 0 -t %T% -y B%%[Link]

pause
Note: The duration must only be specified if you want to analyze only the beginning of the video, for fast testing. If you want to analyze the whole video,
just set the "T" variable to a value longer than the video.

Let's have a look which movements we can see in the small images:

Time Frames Rotation Movement in image A Movement in image B

[s] (center of equirectangular video) (90° right of center of equirectangular video)

0-1 0-25 -- -- --
1-2 25-50 pitch down counter-clockwise rotation
2-3 50-75 yaw left down and left
3-4 75-100 pitch + yaw down and left first down and left, then down
4-5 100-125 roll counter-clockwise rotation up
5-6 125-150 pitch + roll first down and right, then right up and right
6-7 150-175 yaw + roll first up, then up and left first up, then up and left
7-8 175-200 pitch + yaw + roll first up, then up and left first up, then up and left
8-9 200-224 -- -- --

343
That means we can measure the image shift in the small images in x and y direction and then calculate the rotation angles as follows:
Yaw_angle_in_radians = -c * horizontal_shift_in_image_A
Pitch_angle_in_radians = c * vertical_shift_in_image_A
Roll_angle_in_radians = -c * vertical_shift_in_image_B
where c is a constant, c = Pi / equirectangular_image_height

Before the A and B images can be analyzed for movement, you should make sure that they contain useful details. For example, they should not contain
the blue sky which has no fine details. Also, when you have for example a video of a mountainbiker, make sure that the images contain only the
background and not the bike or the driver (under the assumption that you want to stabilize on the background).
This batch file draws two red rectangles around the small windows, so that you can check if these windows contain useful details:
set "IN=test.mp4" :: Equirectangular input video
set "B=100" :: Box size in pixels

ffmpeg -i %IN% -vf drawbox=x=iw/2-%B%/2:y=ih/2-%B%/2:w=%B%:h=%B%:color=red:thickness=5,drawbox=x=3/4*iw-%B%/2:y=ih/2-%B

%/2:w=%B%:h=%B%:color=red:thickness=5 -y dummy.mp4

pause

If you find that the small windows contain unsuitable content, then use the V360 filter to rotate the video and then repeat the process.

The image shifts in x and y direction in the A and B images are now analyzed by a C# program, which produces a file "[Link]" which can later be
used as input for FFmpeg's sendcmd filter.
(See the C# source code of the improved version in the next chapter)

344
You can see that the accumulated angle error is only a few degees at the end of the video. In the input video the angles were 180°, 180°, 180° at the end of
the video, which is equivalent to 0°, 0°, 0°.
Finally apply the corrections to the test video:
set "IN=test.mp4" :: Equirectangular input video
set "T=10" :: Duration in seconds

ffmpeg -i %IN% -lavfi sendcmd=f=[Link],v360=[Link]pitch=0:yaw=0:roll=0 -t %T% -y out.mp4

pause

When you play this video, you see that most of the rotations are removed. Now I must produce a real 360° video on my mountainbike :-)

Of course, there is room for improvements:

• Use more than two windows for detecting the image shifts. For example 6 windows front, rear, left, right, up, down.
• This would add redundancy and make the algorithm more robust.
• Automatically detect if a window contains no useful details and don't use this data
• Automatically detect if one window contains details that don't move in the same direction as the other windows. Some kind of median filtering.
• Use all R, G and B channels for detecting the image shifts.

345
2.154 Stabilization of 360° Videos, improved

Note: This stabilization method does no longer work because the behaviour of the v360 filter was changed 25-Oct-2020. The yaw, pitch and roll rotation
angles are now interpreted as relative angles, if they are sent via sendcmd. See also ticket 9447. There was a new option "reset_rot" added which resets
the rotation angles, so that absolute rotations can be simulated. I'm not sure if it could be used in this example. Recommended workaround: Use the
spherical stabilizer in DaVinci Resolve.

This version has the following improvements:

• Extract color images and use R, G and B channels for image shift detection, all colors can be individually enabled or disabled
• Use 6 images from each frame for rotation detection: front (XA), right (XB), back (XC), left (XD), up (XE) and down (XF), all 6 images can be
individually enabled or disabled
• Offset angles can be applied to the output file as well
• Use median filtering for the rotation angles
• Faster algorithm for finding the minimum of the sum of absolute differences
• Set the time values in the *.cmd file in the middle between the time stamps of the frames

These are the movements in the 6 image sequences:

Movement in images
Rotation XA XB XC XD XE XF
pitch down up down down
yaw left left left left
roll up down left right

There is a lot of redundancy: 12 independent ways to calculate pitch (4 for each color channel), and the same for yaw and roll as well.

This is the batch file for extracting the 6 image sequences from the input video:
set "IN=test.mp4" :: Equirectangular input video

346
set "B=100" :: Image size in pixels
set "T=9" :: Duration in seconds

ffmpeg -i %IN% -vf crop=w=%B%:h=%B% -start_number 0 -t %T% -y XA%%[Link]

ffmpeg -i %IN% -vf v360=input=e:output=e:yaw=90,crop=w=%B%:h=%B% -start_number 0 -t %T% -y XB%%[Link]
ffmpeg -i %IN% -vf v360=input=e:output=e:yaw=180,crop=w=%B%:h=%B% -start_number 0 -t %T% -y XC%%[Link]
ffmpeg -i %IN% -vf v360=input=e:output=e:yaw=-90,crop=w=%B%:h=%B% -start_number 0 -t %T% -y XD%%[Link]
ffmpeg -i %IN% -vf v360=input=e:output=e:pitch=90,crop=w=%B%:h=%B% -start_number 0 -t %T% -y XE%%[Link]
ffmpeg -i %IN% -vf v360=input=e:output=e:pitch=-90,crop=w=%B%:h=%B% -start_number 0 -t %T% -y XF%%[Link]

pause

My C# source code can be downloaded here: [Link]

This is the batch file for applying the rotations to the input video:
set "IN=test.mp4" :: Equirectangular input image
set "T=9" :: Duration in seconds

ffmpeg -i %IN% -lavfi sendcmd=f=[Link],v360=[Link]pitch=0:yaw=0:roll=0 -t %T% -y out.mp4

pause

Note: There is a very good spherical stabilizer in DaVinci Resolve, highly recommended.

347
2.155 Remap Video-in-Video with perspective filter

Suppose you have a video in which a TV or computer screen is visible, and in postprocessing you want another video to be shown on that screen.
Or you have a video in which a beamer projects an image on a wall, which is almost impossible to capture flicker-free in a video. It's better to overlay the
projected image in postprocessing.
The perspective filter can be used to remap a rectangular video into the distorted screen (which is an irregular quadrangle).
The coordinates of the corners of the screen are x0,y0 (top left), x1,y1 (top right), x2,y2 (bottom left) and x3,y3 (bottom right) and must me measured in
the original video.
set "X0=500" :: Top left corner
set "Y0=250"
set "X1=1250" :: Top right corner
set "Y1=150"
set "X2=400" :: Bottom left corner
set "Y2=750"
set "X3=1150" :: Bottom right corner
set "Y3=850"

rem Make a color test video

ffmpeg -f lavfi -i testsrc2=s=hd1080 -t 5 -y video1.mp4

rem Make a black and white test video

ffmpeg -f lavfi -i testsrc2=s=hd1080 -vf eq=saturation=0 -t 5 -y video2.mp4

rem Embed the black and white video into the color video

ffmpeg -i video1.mp4 -i video2.mp4 -lavfi "[1]format=argb,pad=w=iw+2:h=ih+2:x=1:y=1:[email protected],perspective=x0=

%X0%:y0=%Y0%:x1=%X1%:y1=%Y1%:x2=%X2%:y2=%Y2%:x3=%X3%:y3=%Y3%:sense=1[2];[0][2]overlay" -q:v 2 -y out.mp4

pause

Before I discovered the perspective filter, I thought that I had to use the remap filter for this purpose, and I figured out the formulas myself. Here they are:

348
The coordinates of the point to be remapped are x,y.
We draw a vertical line through point x,y and calculate the intersection points with the upper and lower edge of the quadrangle:
a = (x - x0) / (x1 - x0) The parameter a describes where the line intersects the upper edge. For a=0 it's at the top left corner, for a=1 it's at the top right
corner. For 0<a<1 the intersection point is somewhere between these two corners. But there are also cases possible a<0 or a>1 where the intersection
point is outside the finite line segment.
The intersection point is x4,y4
x4 = x
y4 = y0 + a * (y1 - y0)
We do the same thing for the lower edge:
b = (x - x2) / (x3 - x2)
x5 = x
y5 = y2 + b * (y3 - y2)
Parameter c describes where the point x,y lies on the line segment between pints 4 and 5:
c = (y - y4) / (y5 - y4)
Now we remap these points into a unit quadrat with the top left corner at 0,0:
Point 4 is at coordinates a,0 and point 5 is at coordinates b,1
Point x,y is remapped to coordinates
xmap = (a + c * (b - a))
ymap = c

349
2.156 Image warping with displace filter

set "W=751" :: Width of image

set "H=853" :: Height of image
set "CX=347" :: X center of distortion
set "CY=451" :: Y center of distortion
set "A=15" :: Maximum amplitude of displacement, positive displaces outwards and negative inwards,
:: allowed range is [0..127], best values are below 20
set "D=30" :: Radius from center of distortion, where the maximum displacement occurs

rem Create the displace_x file

ffmpeg -f lavfi -i nullsrc=size=%W%x%H% -vf format=pix_fmts=gray8,geq='st(0,2*%A%*%D%/(pow((%CX%-X),2)+pow((%CY%-Y),2)+
%D%*%D%));128-ld(0)*(X-%CX%)' -frames 1 -y displace_x.pgm

rem Create the displace_y file

ffmpeg -f lavfi -i nullsrc=size=%W%x%H% -vf format=pix_fmts=gray8,geq='st(0,2*%A%*%D%/(pow((%CX%-X),2)+pow((%CY%-Y),2)+
%D%*%D%));128-ld(0)*(Y-%CY%)' -frames 1 -y displace_y.pgm

rem Apply the displace filter to the image

ffmpeg -i [Link] -i displace_x.pgm -i displace_y.pgm -lavfi format=pix_fmts=rgb24,displace -frames 1 -y bigger_nose.jpg

set "A=-15" :: Now let's try the other sign

rem Create the displace_x file

ffmpeg -f lavfi -i nullsrc=size=%W%x%H% -vf format=pix_fmts=gray8,geq='st(0,2*%A%*%D%/(pow((%CX%-X),2)+pow((%CY%-Y),2)+
%D%*%D%));128-ld(0)*(X-%CX%)' -frames 1 -y displace_x.pgm

rem Create the displace_y file

ffmpeg -f lavfi -i nullsrc=size=%W%x%H% -vf format=pix_fmts=gray8,geq='st(0,2*%A%*%D%/(pow((%CX%-X),2)+pow((%CY%-Y),2)+
%D%*%D%));128-ld(0)*(Y-%CY%)' -frames 1 -y displace_y.pgm

rem Apply the displace filter to the image

ffmpeg -i [Link] -i displace_x.pgm -i displace_y.pgm -lavfi format=pix_fmts=rgb24,displace -frames 1 -y smaller_nose.jpg

pause
Here is the input image and the two output images:

350
[Link] bigger_nose.jpg smaller_nose.jpg

It might be dangerous to use this kind of processing for images of women without prior asking them for permission :-)
The "displace" filter expects mapping files with relative values in the range [0..255], where 128 is the neutral value for no displacement. Larger
displacements than 127 pixels aren't possible.
I recommend to set the format to rgb24 before using the displace filter.
The displace filter isn't fully compatible with 10-bit data. Dithering is introduced.

351
This is an example for enlarging the eyes:
set "W=751" :: Width of image
set "H=853" :: Height of image
set "LX=256" :: left eye x
set "LY=362" :: left eye y
set "RX=445" :: right eye x
set "RY=325" :: right eye y
set "A=10" :: Maximum amplitude of displacement, positive displaces outwards and negative inwards,
:: allowed range is [0..127], best values are below 20
set "D=25" :: Radius from center of distortion, where the maximum displacement occurs

rem Create the displace_x file

ffmpeg -f lavfi -i nullsrc=size=%W%x%H% -vf format=pix_fmts=gray8,geq='st(0,2*%A%*%D%/(pow((%LX%-X),2)+pow((%LY%-Y),2)+
%D%*%D%));st(1,2*%A%*%D%/(pow((%RX%-X),2)+pow((%RY%-Y),2)+%D%*%D%));128-ld
(0)*(X-%LX%)-ld(1)*(X-%RX%)' -frames 1 -y displace_x.pgm

rem Create the displace_y file

ffmpeg -f lavfi -i nullsrc=size=%W%x%H% -vf format=pix_fmts=gray8,geq='st(0,2*%A%*%D%/(pow((%LX%-X),2)+pow((%LY%-Y),2)+
%D%*%D%));st(1,2*%A%*%D%/(pow((%RX%-X),2)+pow((%RY%-Y),2)+%D%*%D%));128-ld
(0)*(Y-%LY%)-ld(1)*(Y-%RY%)' -frames 1 -y displace_y.pgm

rem Apply the displace filter to the image or video

ffmpeg -i [Link] -i displace_x.pgm -i displace_y.pgm -lavfi format=pix_fmts=rgb24,displace -frames 1 -y big_eyes.jpg

pause

If the output is a video, remove "-frames 1" in the last command line.

352
Here are the input and output images:

[Link] big_eyes.jpg

353
Mathematics for this distortion:

2A Dr d 2 AD d d
d= r =√ (x−cx)2 +( y−cy)2 = dx=(x−cx) dy=( y−cy)
r 2+D 2 r (x−cx) +( y−cy)2+ D 2
2
r r
with d = displacement distance
A = maximum amplitude of displacement
r = distance from pixel x,y to center of distortion cx,cy
D = distance where the largest displacement occurs
cx,cy = coordinates of center of the distortion
dx,dy = displacement values

354
2.157 Noise reduction

FFmpeg has several filters for video noise reduction (denoising):

Filter Description Notes and Examples
atadenoise Apply an Adaptive Temporal Averaging Denoiser to the video very fast, temporal only with no motion compensation; LGPL
input Example: atadenoise=0a=0.2:1a=0.2:2a=0.2:0b=0.3:1b=0.3:2b=0.3
bm3d Denoise frames using Block-Matching 3D algorithm very very slow, currently implemented as spatial only, algorithm considered as one of the
state of art denoisers; LGPL
chromanr Reduce chrominance noise This filter calculates the absolute difference of the Y components (contrary to the official
documentation! ) of the current pixel and a neighbour pixel from a rectangular
neighbourhood. Absolute differences are also calculated for the U and V components. A
neighbour pixel is used for averaging, if the sum of all three absolute differences is lower
than the threshold. Only the U and V components are averaged. The Y component
remains unchanged. With the "stepw" and "steph" options it's possible to use only a
subset of the neighbour pixels for averaging.
dctdnoiz Denoise frames using 2D DCT (frequency domain filtering) very very slow: spatial only, blurs too much; LGPL
fftdenoiz Denoise frames using 3D FFT (frequency domain filtering) slow, spatial and limited temporal, using Fast Fourier Transform, may have introduce
ringing with bad settings; LGPL
hqdn3d This is a high precision/quality 3d denoise filter. It aims to fast, both spatial and temporal, does basically lowpass by destroying high frequencies,
reduce image noise, producing smooth images and making blurs with extreme settings; GPL
still images really still. It should enhance compressibility. Example: hqdn3d=[Link]
nlmeans Denoise frames using Non-Local means algorithm very slow, currently implemented as spatial only, algorithm considered as one of the
state of art denoisers; LGPL
owdenoise Apply Overcomplete Wavelet denoiser very very very slow, spatial only, wavelet; GPL
Example: owdenoise=ls=25
removegrain Spatial denoiser for progressive video fast, spatial only, limited usecase
vaguedenoiser Apply a wavelet based denoiser slow, spatial only, pretty good, wavelet; LGPL
tmix Noise reduction by averaging up to 1024 successive frames Not suitable for moving objects. Example: tmix=frames=20
tmedian Noise reduction by calculating the median out of up to 127 Example: tmedian=radius=20
successive frames
Special thanks to Paul B Mahol who posted most of these notes in the FFmpeg-user list on October 27, 2019

355
2.158 -filter_complex_script

The complex filtergraph can be loaded from an external script file.

Line feeds and empty lines are allowed in the script file. This makes the script much more readable.
The drawbacks are that you can't use variables and comments as in a batch file.

2.159 Time delay within a filter chain

This is an example for a time delay within a filter chain:

ffmpeg -f lavfi -i testsrc=duration=10:size=vga -filter_complex split[a][b];[a]setpts=PTS-5/TB[c];[b]
[c]hstack=shortest=1 -y out.mp4

pause

Hint: Subtracting a constant from PTS works fine. However if you try to add a constant to PTS (e.g. setpts=PTS+5/TB), this may lead to the problem that
the true length of the output video isn't equal to the length in the metadata.

In this example the same thing is done with tpad and trim filters:
ffmpeg -f lavfi -i testsrc=duration=10:size=vga -filter_complex split[a][b];[a]tpad=start_duration=5[c];[b]
[c]hstack=shortest=1,trim=start=5,setpts=PTS-5/TB -y out.mp4

pause

356
2.160 Recording one-time events with pre-trigger

Let's assume you want to record a video of a one-time event and you don't know when this event will happen. You get a trigger signal when the event
starts, but in the video you also want to see 10 seconds before the trigger signal. This can be realized by capturing the camera signal continuously,
delaying it by 10 seconds and then streaming it to a UDP adress. This is the first process which is running continuously:
set "SIZE=640x480" :: Size of camera frames
set "FR=30" :: Framerate of camera
set "DELAY=10" :: Delay time in seconds

ffmpeg -f dshow -video_size %SIZE% -vcodec mjpeg -framerate %FR% -i video="BisonCam,NB Pro":audio="Mikrofon (Realtek(R)
Audio)" -lavfi "[0:0]tpad=%FR%*%DELAY%,format=yuv420p;[0:1]adelay=%DELAY%s" -q:v 2 -f mpegts udp://[Link]:1234
Note: This doesn't work if you omit the audio channel.
Note: The argument of the "adelay" filter is normally in milliseconds, but you can also specify seconds if you add "s", as in this example.
For informations about UDP (User Datagram Protocol) see [Link]

The first process should be running for at least 10 seconds. Then the second process is started when the event occurs. It saves a 20 second video,
which consists of 10 seconds before and 10 seconds after the trigger signal:
set "T=20" :: Duration in seconds

ffmpeg -f mpegts -i udp://[Link]:1234 -t %T% -y out.mp4

For testing the stream you can use this batch file:
ffmpeg -f mpegts -i udp://[Link]:1234 -f sdl2 -

or alternatively this batch file:

ffplay -f mpegts -i udp://[Link]:1234

See also [Link] (that's a complicated solution with external software)

357
2.161 Chroma subsampling, pixel formats of images or videos

When you make a video from many JPG images, all images must have the same pixel format. But sometimes they are different. For example I has many
images that came from the camera with [Link] pixel format. But I had to edit one of the images with IrfanView, it then it was saved with pixel format [Link].
This example changes the pixel format of an image from [Link] to [Link]
ffmpeg -i IMG_044x.jpg -pix_fmt yuvj422p -q 0 IMG_044.jpg

pause

Set the pixel format of a video to [Link] and scale the video to 1920x1080
ffmpeg -i [Link] -pix_fmt yuv444p -s 1920x1080 [Link]

pause

In a filter chain the format can be set as follows:

ffmpeg -i [Link] -lavfi "format=pix_fmts=rgb24" -y out.mp4

pause

Which is the same as:

ffmpeg -i [Link] -lavfi "format=rgb24" -y out.mp4

pause

See also: [Link]

358
Chroma 8-bit 10-bit format Notes
subsampling format
[Link] yuv444p yuv444p10le Each of the three Y'CbCr components have the same sample rate, thus there is no chroma subsampling. This
scheme is sometimes used in high-end film scanners and cinematic post production.
Note that "[Link]" may instead be referring to R'G'B' color space, which implicitly also does not have any
chroma subsampling.
[Link] yuv422p yuv422p10le The horizontal chroma resolution is halved. This reduces the bandwidth of an uncompressed video signal by
one-third with little to no visual difference.
[Link] yuv420p yuv420p10le Chroma is subsampled at a factor of 2 both horizontally and vertically.
Source: Wikipedia

RGB and gray pixel formats (this is only a subset of the available formats):

NB_COMPONENTS BITS_PER_PIXEL Notes

rgb8 3 8 2 bits red, 3 bits green, 3 bits blue
rgb24, bgr24 3 24 packed
gbrp 3 24 planar
gray 1 8
argb, rgba, abgr, bgra 4 32 packed
gray10be, gray10le, gray10 1 10 Any pixel format > 8bit have alias (here: gray10,
gray12be, gray12le, gray12 gray12, gray14 and gray16), on LE system it is le
1 12
variant, on BE system it is be variant of pixel, in
gray14be, gray14le, gray14 1 14 other words native to system endianess.
gray16be, gray16le, gray16 1 16
rgb48be, rgb48le, rgb48, bgr48be, bgr48le, bgr48 3 48 packed
gbrp10be, gbrp10le, gbrp10 3 48 planar
gbrp12be, gbrp12le, gbrp12 3 48 planar
gbrp14be, gbrp14le, gbrp14 3 48 planar
gbrp16be, gbrp16le, gbrp16 3 48 planar

359
grayf32be, grayf32le 1 32 float grayscale

Example for creating an image with float grayscale:

ffmpeg -f lavfi -i nullsrc=size=100x100 -vf format=pix_fmts=grayf32le,geq='(X-50)/25' -frames 1 -y [Link]

All pixel formats are described in the source code in the folder libavutil in file pixfmt.h .

You can also print out the list of all pixel formats with this command:
ffmpeg -pix_fmts

pause

List of all pixel formats:

Pixel formats:
I.... = Supported Input format for conversion
.O... = Supported Output format for conversion
..H.. = Hardware accelerated format
...P. = Paletted format
....B = Bitstream format
FLAGS NAME NB_COMPONENTS BITS_PER_PIXEL BIT_DEPTHS
-----
IO... yuv420p 3 12 8-8-8
IO... yuyv422 3 16 8-8-8
IO... rgb24 3 24 8-8-8
IO... bgr24 3 24 8-8-8
IO... yuv422p 3 16 8-8-8
IO... yuv444p 3 24 8-8-8
IO... yuv410p 3 9 8-8-8
IO... yuv411p 3 12 8-8-8
IO... gray 1 8 8
IO..B monow 1 1 1
IO..B monob 1 1 1
I..P. pal8 1 8 8
IO... yuvj420p 3 12 8-8-8
IO... yuvj422p 3 16 8-8-8
IO... yuvj444p 3 24 8-8-8
IO... uyvy422 3 16 8-8-8

360
..... uyyvyy411 3 12 8-8-8
IO... bgr8 3 8 3-3-2
.O..B bgr4 3 4 1-2-1
IO... bgr4_byte 3 4 1-2-1
IO... rgb8 3 8 2-3-3
.O..B rgb4 3 4 1-2-1
IO... rgb4_byte 3 4 1-2-1
IO... nv12 3 12 8-8-8
IO... nv21 3 12 8-8-8
IO... argb 4 32 8-8-8-8
IO... rgba 4 32 8-8-8-8
IO... abgr 4 32 8-8-8-8
IO... bgra 4 32 8-8-8-8
IO... gray16be 1 16 16
IO... gray16le 1 16 16
IO... yuv440p 3 16 8-8-8
IO... yuvj440p 3 16 8-8-8
IO... yuva420p 4 20 8-8-8-8
IO... rgb48be 3 48 16-16-16
IO... rgb48le 3 48 16-16-16
IO... rgb565be 3 16 5-6-5
IO... rgb565le 3 16 5-6-5
IO... rgb555be 3 15 5-5-5
IO... rgb555le 3 15 5-5-5
IO... bgr565be 3 16 5-6-5
IO... bgr565le 3 16 5-6-5
IO... bgr555be 3 15 5-5-5
IO... bgr555le 3 15 5-5-5
..H.. vaapi 0 0 0
IO... yuv420p16le 3 24 16-16-16
IO... yuv420p16be 3 24 16-16-16
IO... yuv422p16le 3 32 16-16-16
IO... yuv422p16be 3 32 16-16-16
IO... yuv444p16le 3 48 16-16-16
IO... yuv444p16be 3 48 16-16-16
..H.. dxva2_vld 0 0 0
IO... rgb444le 3 12 4-4-4
IO... rgb444be 3 12 4-4-4
IO... bgr444le 3 12 4-4-4
IO... bgr444be 3 12 4-4-4

361
IO... ya8 2 16 8-8
IO... bgr48be 3 48 16-16-16
IO... bgr48le 3 48 16-16-16
IO... yuv420p9be 3 13 9-9-9
IO... yuv420p9le 3 13 9-9-9
IO... yuv420p10be 3 15 10-10-10
IO... yuv420p10le 3 15 10-10-10
IO... yuv422p10be 3 20 10-10-10
IO... yuv422p10le 3 20 10-10-10
IO... yuv444p9be 3 27 9-9-9
IO... yuv444p9le 3 27 9-9-9
IO... yuv444p10be 3 30 10-10-10
IO... yuv444p10le 3 30 10-10-10
IO... yuv422p9be 3 18 9-9-9
IO... yuv422p9le 3 18 9-9-9
IO... gbrp 3 24 8-8-8
IO... gbrp9be 3 27 9-9-9
IO... gbrp9le 3 27 9-9-9
IO... gbrp10be 3 30 10-10-10
IO... gbrp10le 3 30 10-10-10
IO... gbrp16be 3 48 16-16-16
IO... gbrp16le 3 48 16-16-16
IO... yuva422p 4 24 8-8-8-8
IO... yuva444p 4 32 8-8-8-8
IO... yuva420p9be 4 22 9-9-9-9
IO... yuva420p9le 4 22 9-9-9-9
IO... yuva422p9be 4 27 9-9-9-9
IO... yuva422p9le 4 27 9-9-9-9
IO... yuva444p9be 4 36 9-9-9-9
IO... yuva444p9le 4 36 9-9-9-9
IO... yuva420p10be 4 25 10-10-10-10
IO... yuva420p10le 4 25 10-10-10-10
IO... yuva422p10be 4 30 10-10-10-10
IO... yuva422p10le 4 30 10-10-10-10
IO... yuva444p10be 4 40 10-10-10-10
IO... yuva444p10le 4 40 10-10-10-10
IO... yuva420p16be 4 40 16-16-16-16
IO... yuva420p16le 4 40 16-16-16-16
IO... yuva422p16be 4 48 16-16-16-16
IO... yuva422p16le 4 48 16-16-16-16

362
IO... yuva444p16be 4 64 16-16-16-16
IO... yuva444p16le 4 64 16-16-16-16
..H.. vdpau 0 0 0
IO... xyz12le 3 36 12-12-12
IO... xyz12be 3 36 12-12-12
IO... nv16 3 16 8-8-8
..... nv20le 3 20 10-10-10
..... nv20be 3 20 10-10-10
IO... rgba64be 4 64 16-16-16-16
IO... rgba64le 4 64 16-16-16-16
IO... bgra64be 4 64 16-16-16-16
IO... bgra64le 4 64 16-16-16-16
IO... yvyu422 3 16 8-8-8
IO... ya16be 2 32 16-16
IO... ya16le 2 32 16-16
IO... gbrap 4 32 8-8-8-8
IO... gbrap16be 4 64 16-16-16-16
IO... gbrap16le 4 64 16-16-16-16
..H.. qsv 0 0 0
..H.. mmal 0 0 0
..H.. d3d11va_vld 0 0 0
..H.. cuda 0 0 0
IO... 0rgb 3 24 8-8-8
IO... rgb0 3 24 8-8-8
IO... 0bgr 3 24 8-8-8
IO... bgr0 3 24 8-8-8
IO... yuv420p12be 3 18 12-12-12
IO... yuv420p12le 3 18 12-12-12
IO... yuv420p14be 3 21 14-14-14
IO... yuv420p14le 3 21 14-14-14
IO... yuv422p12be 3 24 12-12-12
IO... yuv422p12le 3 24 12-12-12
IO... yuv422p14be 3 28 14-14-14
IO... yuv422p14le 3 28 14-14-14
IO... yuv444p12be 3 36 12-12-12
IO... yuv444p12le 3 36 12-12-12
IO... yuv444p14be 3 42 14-14-14
IO... yuv444p14le 3 42 14-14-14
IO... gbrp12be 3 36 12-12-12
IO... gbrp12le 3 36 12-12-12

363
IO... gbrp14be 3 42 14-14-14
IO... gbrp14le 3 42 14-14-14
IO... yuvj411p 3 12 8-8-8
I.... bayer_bggr8 3 8 2-4-2
I.... bayer_rggb8 3 8 2-4-2
I.... bayer_gbrg8 3 8 2-4-2
I.... bayer_grbg8 3 8 2-4-2
I.... bayer_bggr16le 3 16 4-8-4
I.... bayer_bggr16be 3 16 4-8-4
I.... bayer_rggb16le 3 16 4-8-4
I.... bayer_rggb16be 3 16 4-8-4
I.... bayer_gbrg16le 3 16 4-8-4
I.... bayer_gbrg16be 3 16 4-8-4
I.... bayer_grbg16le 3 16 4-8-4
I.... bayer_grbg16be 3 16 4-8-4
..H.. xvmc 0 0 0
IO... yuv440p10le 3 20 10-10-10
IO... yuv440p10be 3 20 10-10-10
IO... yuv440p12le 3 24 12-12-12
IO... yuv440p12be 3 24 12-12-12
IO... ayuv64le 4 64 16-16-16-16
..... ayuv64be 4 64 16-16-16-16
..H.. videotoolbox_vld 0 0 0
IO... p010le 3 15 10-10-10
IO... p010be 3 15 10-10-10
IO... gbrap12be 4 48 12-12-12-12
IO... gbrap12le 4 48 12-12-12-12
IO... gbrap10be 4 40 10-10-10-10
IO... gbrap10le 4 40 10-10-10-10
..H.. mediacodec 0 0 0
IO... gray12be 1 12 12
IO... gray12le 1 12 12
IO... gray10be 1 10 10
IO... gray10le 1 10 10
IO... p016le 3 24 16-16-16
IO... p016be 3 24 16-16-16
..H.. d3d11 0 0 0
IO... gray9be 1 9 9
IO... gray9le 1 9 9
IO... gbrpf32be 3 96 32-32-32

364
IO... gbrpf32le 3 96 32-32-32
IO... gbrapf32be 4 128 32-32-32-32
IO... gbrapf32le 4 128 32-32-32-32
..H.. drm_prime 0 0 0
..H.. opencl 0 0 0
IO... gray14be 1 14 14
IO... gray14le 1 14 14
IO... grayf32be 1 32 32
IO... grayf32le 1 32 32
IO... yuva422p12be 4 36 12-12-12-12
IO... yuva422p12le 4 36 12-12-12-12
IO... yuva444p12be 4 48 12-12-12-12
IO... yuva444p12le 4 48 12-12-12-12
IO... nv24 3 24 8-8-8
IO... nv42 3 24 8-8-8
..H.. vulkan 0 0 0
..... y210be 3 20 10-10-10
IO... y210le 3 20 10-10-10
IO... x2rgb10le 3 30 10-10-10
..... x2rgb10be 3 30 10-10-10
IO... x2bgr10le 3 30 10-10-10
..... x2bgr10be 3 30 10-10-10
IO... p210be 3 20 10-10-10
IO... p210le 3 20 10-10-10
IO... p410be 3 30 10-10-10
IO... p410le 3 30 10-10-10
IO... p216be 3 32 16-16-16
IO... p216le 3 32 16-16-16
IO... p416be 3 48 16-16-16
IO... p416le 3 48 16-16-16
IO... vuya 4 32 8-8-8-8
I.... rgbaf16be 4 64 16-16-16-16
I.... rgbaf16le 4 64 16-16-16-16
IO... vuyx 3 24 8-8-8
IO... p012le 3 18 12-12-12
IO... p012be 3 18 12-12-12
..... y212be 3 24 12-12-12
IO... y212le 3 24 12-12-12
....B xv30be 3 30 10-10-10
IO... xv30le 3 30 10-10-10

365
..... xv36be 3 36 12-12-12
IO... xv36le 3 36 12-12-12
..... rgbf32be 3 96 32-32-32
..... rgbf32le 3 96 32-32-32
..... rgbaf32be 4 128 32-32-32-32
..... rgbaf32le 4 128 32-32-32-32

Any pixel format > 8bit have alias (for example gray10, gray12, gray16). On LE system it is le variant, on BE system it is be variant of pixel, in other words
native to system endianess. These alias pixel formats are not listed with "ffmpeg -pix_fmts".

366
2.162 Automatic format conversions

Automatic format conversions can be disabled by the option "-noauto_conversion_filters".

Use "-v verbose" for checking where FFmpeg did auto-insert format conversions. Search for the green lines in the console listing.
You can also add "graphmonitor=f=format". This output is shown as text overlaid to the video.

If you are using the "-noauto_conversion_filters" option, you must manually insert the required conversions in the filter chain.
Example without "-noauto_conversion_filters":
ffmpeg -v verbose -f lavfi -i testsrc2=s=svga:d=5,format=yuv422p10le -vf
lut3d="VLog_to_V709.cube" -pix_fmt yuv422p10le -c:v h264 -y [Link]

pause

In this example you can see in the console listing that Ffmpeg did auto-insert two format conversions: Before the lut3d filter from yuv422p10le to rgb48le,
and after the lut3d filter from rgb48le to yuv422p10le.
The same example with "-noauto_conversion_filters":
ffmpeg -v verbose -f lavfi -i testsrc2=s=svga:d=5,format=yuv422p10le -vf
scale,format=rgb48le,lut3d="VLog_to_V709.cube",scale -noauto_conversion_filters -pix_fmt yuv422p10le -c:v h264 -y
[Link]

pause

As you can see, there are also two "scale" filters required. It's hard to understand why. In this case the second format conversion can be omitted because
it's redundant with the following "-pix_fmt" option.
The following explanation was written by Gyan Doshi in the ffmpeg user list on September 14, 2020:
"Each filter presents a list of input formats they can work with and a list of output formats they can directly generate. The framework inspects adjacent
filters and sets a compatible common format for the outputs and inputs when possible. If not, it sets one of the available output formats for the preceding
filter and one from input formats for the following filter and inserts a scale filter to convert between those. This process is format negotiation. The format
filter doesn't carry out the conversion itself - it inserts scale which in turn invokes libswscale. scale without any args defaults to the source W and H. But
for pixel formats, its output format is constrained by the following format filter. That triggers a format conversion by libswscale."

367
For more details about the scale filter see also: [Link]

Some important notes from the above website:

• When going from BGR (not RGB) to yuv420p the conversion is broken (off-by-one). Use -vf scale=flags=accurate_rnd to fix that.
• yuvjxxxp pixel formats are deprecated. Yet for x265 the workaround was implemented, but not for jpeg2000 and AV1. For those -vf
scale=out_range=pc should be used.
• Converion from YCbCr limited to RGB 16 bit is broken, use zscale instead of swscale
• Limited range RGB is not supported at all.
• Dither can be turned off using -vf scale=sws_dither=none
• One should always remember that YCbCr [Link] 8 bit is not enough to preserve RGB 8 bit, YCbCr [Link] 10 bit is required.
• The default for matrix in untagged input and output is always limited BT.601

368
2.163 RGB Color Cube

This is the RGB color cube:

C--------W
/| /|
G--------Y |
| | | |
| | | |
| B------|-M
|/ |/
0--------R

0 = black, R = red, G = green, B = blue, Y = yellow, C = cyan, M = magenta, W = white

The cube has 6 faces:

G----Y Y----W W----C C----G C----W 0----R

| | | | | | | | | | | |
| | | | | | | | | | | |
0----R R----M M----B B----0 G----Y B----M

There are several ways how to connect the faces to a net. Here are two examples:

C----W G----0----R
| | | | |
| | | | |
G----Y----W----C----G G----C----B----M
| | | | | | | |
| | | | | | | |
0----R----M----B----0 G----Y----W----M
| | | | |
| | | | |
B----M 0----R----M

369
The cube does also contain 6 internal diagonal planes:

G------W Y------C B------W C------M Y------M R------W

| | | | | | | | | | | |
| | | | | | | | | | | |
0------M R------B 0------Y G------R G------B 0------C

It's not possible to connect these 6 diagonal planes to a net, because they don't have any common edges.

But it's possible to find nets which contain all 6 faces and all 6 diagonal planes:

C----W------B----C------M W----M------C----W------B 0----R G----C

| | | | | | | | | | | | | |
| | | | | | | | | | | | | |
G----Y------0----G------R Y----R------G----Y------0 G----Y Y----W
| | | | | | | |
| | | | | | | |
| | | | | | | |
B----M C----B----M R------B----M------0----B
| | | | | | | | | |
| | | | | | | | | |
0----R G----0----R Y------C----W------G----C
| | | | | | | |
| | | | | | | |
| | | | | | | |
Y----W------G----Y------C----W G----Y------C----W------G 0----R R----M
| | | | | | | | | | | | | | |
| | | | | | | | | | | | | | |
R----M------0----R------B----M 0----R------B----M------0 B----M Y----W

If you find a more compact net than the middle or right one, please let me know.

370
If you want to show all 6 sides and all 6 diagonal planes in a single image, it's more efficient to split the net in 3 parts:

R----M------Y----W------R
| | | | |
| | | | |
0----B------G----C------0

G----Y------C----W------G
| | | | |
| | | | |
0----R------B----M------0

B----C------M----W------B
| | | | |
| | | | |
0----G------R----Y------0

371
This is an example for creating an image which contains all 6 faces and all 6 diagonal planes of the RGB cube:
ffmpeg -f lavfi -i color=black:size=256x256 -lavfi split=4[a][b][c][d];^
[a]geq=r='lerp(255,0,Y/W)':g='0':b='lerp(0,255,X/W)'[aa];^
[b]geq=r='lerp(255,0,Y/W)':g='lerp(0,255,X/W)':b='lerp(255,0,X/W)'[bb];^
[c]geq=r='lerp(255,0,Y/W)':g='255':b='lerp(0,255,X/W)'[cc];^
[d]geq=r='lerp(255,0,Y/W)':g='lerp(255,0,X/W)':b='lerp(255,0,X/W)'[dd];^
[aa][bb][cc][dd]hstack=4,split=3[e1][e2][e3];^
[e2]colorchannelmixer=[Link][f2];^
[e3]colorchannelmixer=[Link][f3];^
[e1][f2][f3]vstack=3 -frames 1 -y [Link]

pause

This is the output image:

372
2.164 Convert RGB to HSV or HSL

MAX := max(R,G,B), MIN := min(R,G,B) Color table:

H := 0 if MAX = MIN Color HUE R G B

H := 60° * (0 + (G - B) / (MAX - MIN)) if MAX = R red 0° 255 0 0
H := 60° * (2 + (B - R) / (MAX - MIN)) if MAX = G orange 30° 255 128 0
H := 60° * (4 + (R - G) / (MAX - MIN)) if MAX = B yellow 60° 255 255 0
green-yellow 90° 128 255 0
if H < 0 then H := H + 360°
green 120° 0 255 0
SHSV := 0 if MAX = 0 blue-green 150° 0 255 128
SHSV := (MAX - MIN) / MAX if MAX != 0 cyan 180° 0 255 255

V := MAX green-blue 210° 0 128 255

blue 240° 0 0 255
SHSL := 0 if MAX = 0
violet 270° 128 0 255
SHSL := 0 if MIN = 0
magenta 300° 255 0 255
SHSL := (MAX - MIN) / (1 - |MAX + MIN - 1| ) if MAX != 0 and MIN != 0
Please note that this formula blue-red 330° 255 0 128
works only if the RGB values white 255 255 255
are in the [0..1] range. For
the [0..255] range, replace black 0 0 0
"1" by "255".

L := (MAX + MIN) / 2

This is a different method for calculating hue and chroma:

HUE := atan2(sqrt(3) * (G - B), 2 * R - G - B)