Jiacheng Li, Feiran Li, and Daisuke Iso
In recent years, neural networks have achieved significant progress in offline image processing. However, in online scenarios, particularly in on-chip implementations, memory usage emerges as a critical bottleneck due to the limited memory resources of integrated image processors. In this study, we focus on reducing the memory footprint of neural networks for on-chip image processing by optimizing network design for efficient memory utilization. Specifically, we consider a typical scenario in which images outputted from an image sensor are processed sequentially using line buffers in a line-by-line manner. This setting necessitates the modeling of both intra-line and inter-line correlations—capturing dependencies among pixels within a single line group and across different line groups, respectively. To model intra-line correlations, we propose a progressive feature enhancement strategy, where line pixels are processed with expanding strip convolutions in multiple stages. For inter-line correlation modeling, we introduce a hierarchical line buffer formulation, where features extracted from previous lines are incrementally reused and compressed across multiple hierarchical levels. Comprehensive experiments on various image processing tasks, including RAW denoising, Gaussian denoising, and super-resolution, demonstrate that the proposed method achieves a superior trade-off between performance and memory efficiency than previous solutions, e.g., up to 1dB PSNR gain in RAW denoising at one-fifth of peak memory usage.
train_denoise_raw.py: The main script for training the model for RAW image denoising.test_denoise_raw.py: The main script for testing the trained model.requirements.txt: A list of Python dependencies required to run the project.models/: Contains the PyTorch model definitions.linebuffernet.py: The main model architecture.linebase.py: Base model components.
datasets/: Contains the dataset loading and processing logic.real_dataset.py: The dataset class for loading real-world noisy images.
utils/: Contains utility functions used throughout the project.utils.py: General utility functions.imgproc.py: Image processing functions.image_quality_assessment.py: Functions for calculating image quality metrics like PSNR and SSIM.
images/: Directory to save output images from testing.checkpoints/: Directory to save model checkpoints during training.logs/: Directory to save training logs.
Clone the repository and install necessary dependencies with pip install -r requirements.txt
Please refer to SID and ELD, respectively.
Please refer to PMN's get_dataset_infos.py script for generating info files as training and testing image lists (please modify --root_dir to anchor the dataset location). The default ones assume the root directory of /data/data/sid_sony/Sony/[short/long]/* for SID and /data/data/ELD/SonyA7S2/scene-* for ELD.
python3 get_dataset_infos.py --dstname ELD --root_dir /data/data/ELD --mode SonyA7S2
python3 get_dataset_infos.py --dstname SID --root_dir /data/data/sid_sony/Sony --mode evaltest
Please refer to MuLUT and KAIR for dataset preparation and example dataloaders.
Under the project directory, create folder logs for training logs and images for inference results.
Train models with the following command:
python3 test_denoise_raw.py --task linebuffer --model LineBufferNet
Test and evaluate models with the following command:
python3 test_denoise_raw.py --task linebuffer --model LineBufferNet --cp_dir ./checkpoints/linebuffer/best_psnr.bin --plot_res True -s [sid|eld]
Reference results are as follows:
# SID
linebuffer PSNR SSIM LPIPS
100 41.69 0.9452 0.0750
250 39.51 0.9290 0.1115
300 36.70 0.9119 0.1396
# ELD
linebuffer PSNR SSIM LPIPS
100 44.38 0.9623 0.0381
200 41.69 0.9174 0.0647
If you find our work helpful, please cite the following paper.
@InProceedings{Li_2025_LineBuffer,
author = {Li, Jiacheng and Li, Feiran and Iso, Daisuke},
title = {Learning Hierarchical Line Buffer for Image Processing},
booktitle = {ICCV},
year = {2025},
}