The following represents a high-level overview of our 2023 plan. You should be aware that this roadmap may change at any time and the order below does not reflect any type of priority.

We strongly encourage you to comment on our roadmap and provide us feedback on this issue here.

Some of the items mentioned below are the continuation of the 2022 effort (#3774)

Improving Usability:

• eager mode - extending support for using DALI operators as standalone entities and improving their interoperability with other libraries like VPF, CV-CUDA or MONAI
• conditional execution - providing a convenient API to conditionally apply operation based on a predicate, providing AutoAugment style capabilities -
  • conditional execution itself (Add tutorial for using conditionals in DALI  #4569, Add experimental support for if statements in DALI #4561, Make conditionals work in debug mode #4738, Fix classification of argument input-only operators in AutoGraph #4618, Track DataNodes produced by .gpu() in conditionals #4602, Add DALI Conditionals documentation #4589, Support inferring batch size from tensor argument inputs #4617, Add lazy and and or, and not lazy not support  #4629, Fix the logical expression tests to avoid short-cutting them #4676)
  • automatic augmentation module with AutoAugment, RandAugment, and TrivialAugment ([AA] Add auto augmentation wrapper #4694, Add augmentations used by AA #4699, [AA] Add select operator/wrapper #4696, Add AutoAugment and ImageNet policy #4702, Add RandAugment and TrivialAugment to auto_aug module #4704, Do not use numpy.typing when not available #4706, Rename as_param to mag_to_param #4710, Add more AutoAugment policies #4753, Add EfficientNet example using automatic augmentations with DALI #4678)
• support for NVIDIA Grace Hopper Superchip, this includes flexible execution model utilizing fast CPU<->GPU memory transfers, where data can go from CPU to GPU and back to the GPU in single pipeline

Extending input format support:

• Extending support of formats and containers with variable frame rate videos
  • decoding raw H264 and H265 streams from memory (Extend decoding support #4480)
• Support for higher dynamic ranges data (int32, float) through the whole data processing pipelines
• Adding GPU acceleration for more image formats, like TIFF or new profiles of the existing one
  • lossless JPEG decoding on CPU and GPU with fn.experimental.decoders.image (Optimize CPU time of JPEG lossless decoder #4625, Skip JPEG lossless tests for compute capability < SM60 #4600, Improve error message when trying to decode JPEG lossless images on the CPU backend  #4587, Support for JPEG lossless images in GPU fn.experimental.decoders.image #4572, Add nvjpeg calls used for lossless jpeg decoding to the stub generator #4592, Add axes_utils.h #4548)

Performance:

• optimizing memory consumption
  • cudaMallocAsync support (Add a memory resource based on cudaMallocAsync  #4900, Add alignment to cuda_malloc_async_memory_resource. #4923, and Fix number of devices for JAX multigpu test #4921)
  • API for pre-allocation and releasing of memory pools (Add functions to preallocate pools and release unused pool memory #4563, Add release_unused function to memory pools. #4556)
• operators performance optimizations
  • O_DIRECT support mode support to fn.readers.tfrecord (Add O_DIRECT support to the TFRecord reader #4820).
  • O_DIRECT mode support to fn.readers.numpy (Add O_DIRECT support in numpy_reader #4796, Fix race condition in the CPU numpy reader #4848)

New transformations:

We are constantly extending the set of operations supported by DALI. Currently, this section lists the most notable additions to our areas of interest that we plan to do this year. This list is not exhaustive and we plan on expanding the set of operators as the needs or requests arise.

• new transformations for general data processing
  • fn.experimental.tensor_resize operator (Add experimental.tensor_resize operator #4492)
• new transformations for image processing
  • median blur - Add median blur operator #4950, Exclude median_blur test from xavier tests #4975
  • histogram equalization operator (fn.experimental.equalize) (Add equalize CPU variant #4742, Equalization operator #4575, Equalize kernel #4565).
  • 2-D convolution(fn.experimental.filter) (Add CPU filter operator #4764, Add GPU filter kernel #4298, Add GPU filter operator (2D, 3D) #4525).
• new transformations for video processing
  • the above image transformations are applicable to video as well