{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:09:27Z","timestamp":1760144967296,"version":"build-2065373602"},"reference-count":50,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2024,6,1]],"date-time":"2024-06-01T00:00:00Z","timestamp":1717200000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"China Scholarship Council (CSC)","award":["2023HZ025003","2022G02008","GY-Z220233","GY-Z23027"],"award-info":[{"award-number":["2023HZ025003","2022G02008","GY-Z220233","GY-Z23027"]}]},{"name":"Fujian Provincial Department of Science and Technology Major Special Projects","award":["2023HZ025003","2022G02008","GY-Z220233","GY-Z23027"],"award-info":[{"award-number":["2023HZ025003","2022G02008","GY-Z220233","GY-Z23027"]}]},{"name":"Key Scientific and Technological Innovation Projects of Fujian Province","award":["2023HZ025003","2022G02008","GY-Z220233","GY-Z23027"],"award-info":[{"award-number":["2023HZ025003","2022G02008","GY-Z220233","GY-Z23027"]}]},{"name":"Education and Scientific Research Project of Fujian Provincial Department of Finance","award":["2023HZ025003","2022G02008","GY-Z220233","GY-Z23027"],"award-info":[{"award-number":["2023HZ025003","2022G02008","GY-Z220233","GY-Z23027"]}]},{"DOI":"10.13039\/501100008462","name":"Fujian University of Technology","doi-asserted-by":"publisher","award":["2023HZ025003","2022G02008","GY-Z220233","GY-Z23027"],"award-info":[{"award-number":["2023HZ025003","2022G02008","GY-Z220233","GY-Z23027"]}],"id":[{"id":"10.13039\/501100008462","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>The Rich spatial and angular information in light field images enables accurate depth estimation, which is a crucial aspect of environmental perception. However, the abundance of light field information also leads to high computational costs and memory pressure. Typically, selectively pruning some light field information can significantly improve computational efficiency but at the expense of reduced depth estimation accuracy in the pruned model, especially in low-texture regions and occluded areas where angular diversity is reduced. In this study, we propose a lightweight disparity estimation model that balances speed and accuracy and enhances depth estimation accuracy in textureless regions. We combined cost matching methods based on absolute difference and correlation to construct cost volumes, improving both accuracy and robustness. Additionally, we developed a multi-scale disparity cost fusion architecture, employing 3D convolutions and a UNet-like structure to handle matching costs at different depth scales. This method effectively integrates information across scales, utilizing the UNet structure for efficient fusion and completion of cost volumes, thus yielding more precise depth maps. Extensive testing shows that our method achieves computational efficiency on par with the most efficient existing methods, yet with double the accuracy. Moreover, our approach achieves comparable accuracy to the current highest-accuracy methods but with an order of magnitude improvement in computational performance.<\/jats:p>","DOI":"10.3390\/s24113583","type":"journal-article","created":{"date-parts":[[2024,6,3]],"date-time":"2024-06-03T05:58:00Z","timestamp":1717394280000},"page":"3583","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Efficiency\u2013Accuracy Trade-Off in Light Field Estimation with Cost Volume Construction and Aggregation"],"prefix":"10.3390","volume":"24","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1684-7356","authenticated-orcid":false,"given":"Bo","family":"Xiao","sequence":"first","affiliation":[{"name":"State Key Laboratory of Advanced Design and Manufacturing for the Vehicle Body, Hunan University, Changsha 410012, China"},{"name":"School of Electrical and Data Engineering, University of Technology Sydney, Ultimo, NSW 2007, Australia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2794-3178","authenticated-orcid":false,"given":"Stuart","family":"Perry","sequence":"additional","affiliation":[{"name":"School of Electrical and Data Engineering, University of Technology Sydney, Ultimo, NSW 2007, Australia"}]},{"given":"Xiujing","family":"Gao","sequence":"additional","affiliation":[{"name":"School of Smart Marine Science and Engineering, Fujian University of Technology, Fuzhou 350118, China"},{"name":"Fujian Provincial Key Laboratory of Marine Smart Equipment, Fuzhou 350118, China"}]},{"given":"Hongwu","family":"Huang","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Advanced Design and Manufacturing for the Vehicle Body, Hunan University, Changsha 410012, China"},{"name":"School of Smart Marine Science and Engineering, Fujian University of Technology, Fuzhou 350118, China"},{"name":"Fujian Provincial Key Laboratory of Marine Smart Equipment, Fuzhou 350118, China"}]}],"member":"1968","published-online":{"date-parts":[[2024,6,1]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"197:1","DOI":"10.1145\/3272127.3275031","article-title":"A system for acquiring, processing, and rendering panoramic light field stills for virtual reality","volume":"37","author":"Overbeck","year":"2018","journal-title":"ACM Trans. Graph."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"104","DOI":"10.1109\/MMUL.2017.24","article-title":"A Light-Field Journey to Virtual Reality","volume":"24","author":"Yu","year":"2017","journal-title":"IEEE MultiMedia"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Guo, M., Jin, J., Liu, H., and Hou, J. (2021, January 11\u201317). Learning Dynamic Interpolation for Extremely Sparse Light Fields with Wide Baselines. Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV), Montreal, BC, Canada.","DOI":"10.1109\/ICCV48922.2021.00245"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"116052","DOI":"10.1109\/ACCESS.2020.3004477","article-title":"LightGAN: A Deep Generative Model for Light Field Reconstruction","volume":"8","author":"Meng","year":"2020","journal-title":"IEEE Access"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Levinson, J., Askeland, J., Becker, J., Dolson, J., Held, D., Kammel, S., Kolter, J.Z., Langer, D., Pink, O., and Pratt, V. (2011, January 5\u20139). Towards fully autonomous driving: Systems and algorithms. Proceedings of the 2011 IEEE Intelligent Vehicles Symposium (IV), Baden-Baden, Germany.","DOI":"10.1109\/IVS.2011.5940562"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Avidan, S., Brostow, G., Ciss\u00e9, M., Farinella, G.M., and Hassner, T. Neural Light Field Estimation for Street Scenes with Differentiable Virtual Object Insertion. Proceedings of the Computer Vision\u2014ECCV 2022, Lecture Notes in Computer Science.","DOI":"10.1007\/978-3-031-19818-2"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Wang, Y., Wang, L., Liang, Z., Yang, J., An, W., and Guo, Y. (2022, January 18\u201324). Occlusion-Aware Cost Constructor for Light Field Depth Estimation. Proceedings of the 2022 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.01919"},{"key":"ref_8","first-page":"1009","article-title":"Attention-based Multi-Level Fusion Network for Light Field Depth Estimation","volume":"35","author":"Chen","year":"2021","journal-title":"Proc. AAAI Conf. Artif. Intell."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"425","DOI":"10.1109\/TPAMI.2022.3152488","article-title":"Disentangling Light Fields for Super-Resolution and Disparity Estimation","volume":"45","author":"Wang","year":"2023","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Huang, Z., Hu, X., Xue, Z., Xu, W., and Yue, T. (2021, January 10\u201317). Fast Light-field Disparity Estimation with Multi-disparity-scale Cost Aggregation. Proceedings of the 2021 IEEE\/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada.","DOI":"10.1109\/ICCV48922.2021.00626"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"2261","DOI":"10.1109\/TCSVT.2023.3305978","article-title":"Unsupervised Light Field Depth Estimation via Multi-view Feature Matching with Occlusion Prediction","volume":"34","author":"Zhang","year":"2023","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"191139","DOI":"10.1117\/12.7972479","article-title":"Photometric Method For Determining Surface Orientation From Multiple Images","volume":"19","author":"Woodham","year":"1980","journal-title":"Opt. Eng."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Shi, B., Wu, Z., Mo, Z., Duan, D., Yeung, S.K., and Tan, P. (2016, January 27\u201330). A Benchmark Dataset and Evaluation for Non-Lambertian and Uncalibrated Photometric Stereo. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.403"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Ju, Y., Lam, K.M., Xie, W., Zhou, H., Dong, J., and Shi, B. (2024). Deep Learning Methods for Calibrated Photometric Stereo and Beyond. IEEE Trans. Pattern Anal. Mach. Intell., 1\u201319.","DOI":"10.1109\/TPAMI.2024.3388150"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Diebold, M., J\u00e4hne, B., and Gatto, A. (2016, January 27\u201330). Heterogeneous Light Fields. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.193"},{"key":"ref_16","first-page":"1","article-title":"Scene Reconstruction from High Spatio-Angular Resolution Light Fields","volume":"32","author":"Kim","year":"2013","journal-title":"ACM Trans. Graph."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Schilling, H., Diebold, M., Rother, C., and J\u00e4hne, B. (2018, January 18\u201323). Trust Your Model: Light Field Depth Estimation with Inline Occlusion Handling. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00476"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"3257","DOI":"10.1109\/TIP.2015.2440760","article-title":"Continuous Depth Map Reconstruction From Light Fields","volume":"24","author":"Li","year":"2015","journal-title":"IEEE Trans. Image Process."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"102732","DOI":"10.1016\/j.jvcir.2019.102732","article-title":"A novel dark channel prior guided variational framework for underwater image restoration","volume":"66","author":"Hou","year":"2020","journal-title":"J. Vis. Commun. Image Represent."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Chen, C., Lin, H., Yu, Z., Bing Kang, S., and Yu, J. (2014, January 23\u201328). Light Field Stereo Matching Using Bilateral Statistics of Surface Cameras. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.197"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Fleet, D., Pajdla, T., Schiele, B., and Tuytelaars, T. Shape from Light Field Meets Robust PCA. Proceedings of the Computer Vision\u2014ECCV 2014, Lecture Notes in Computer Science.","DOI":"10.1007\/978-3-319-10599-4"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Jeon, H.G., Park, J., Choe, G., Park, J., Bok, Y., Tai, Y.W., and So Kweon, I. (2015, January 7\u201312). Accurate Depth Map Estimation from a Lenslet Light Field Camera. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298762"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Yu, Z., Guo, X., Ling, H., Lumsdaine, A., and Yu, J. (2013, January 1\u20138). Line Assisted Light Field Triangulation and Stereo Matching. Proceedings of the 2013 IEEE International Conference on Computer Vision, Sydney, Australia.","DOI":"10.1109\/ICCV.2013.347"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Tao, M.W., Hadap, S., Malik, J., and Ramamoorthi, R. (2013, January 1\u20138). Depth from Combining Defocus and Correspondence Using Light-Field Cameras. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Sydney, Australia.","DOI":"10.1109\/ICCV.2013.89"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"2484","DOI":"10.1109\/TPAMI.2017.2746858","article-title":"Robust Light Field Depth Estimation Using Occlusion-Noise Aware Data Costs","volume":"40","author":"Williem","year":"2018","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Tao, M.W., Srinivasan, P.P., Malik, J., Rusinkiewicz, S., and Ramamoorthi, R. (2015, January 7\u201312). Depth from shading, defocus, and correspondence using light-field angular coherence. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298804"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Williem, W., and Park, I.K. (2016, January 27\u201330). Robust Light Field Depth Estimation for Noisy Scene with Occlusion. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.476"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"606","DOI":"10.1109\/TPAMI.2013.147","article-title":"Variational Light Field Analysis for Disparity Estimation and Super-Resolution","volume":"36","author":"Wanner","year":"2014","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"148","DOI":"10.1016\/j.cviu.2015.12.007","article-title":"Robust depth estimation for light field via spinning parallelogram operator","volume":"145","author":"Zhang","year":"2016","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"587","DOI":"10.1016\/j.patcog.2017.09.010","article-title":"Occlusion-aware depth estimation for light field using multi-orientation EPIs","volume":"74","author":"Sheng","year":"2018","journal-title":"Pattern Recognit."},{"key":"ref_31","first-page":"12095","article-title":"Attention-Based View Selection Networks for Light-Field Disparity Estimation","volume":"34","author":"Tsai","year":"2020","journal-title":"Proc. AAAI Conf. Artif. Intell."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Yang, X., Deng, J., Chen, R., Cong, R., Ke, W., and Sheng, H. (2023, January 17\u201324). Disentangling Local and Global Information for Light Field Depth Estimation. Proceedings of the 2023 IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Vancouver, BC, Canada.","DOI":"10.1109\/CVPRW59228.2023.00344"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Heber, S., and Pock, T. (2016, January 27\u201330). Convolutional Networks for Shape from Light Field. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.407"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Shin, C., Jeon, H.G., Yoon, Y., Kweon, I.S., and Kim, S.J. (2018, January 18\u201323). EPINET: A Fully-Convolutional Neural Network Using Epipolar Geometry for Depth From Light Field Images. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00499"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Hu, J., Shen, L., and Sun, G. (2018, January 18\u201323). Squeeze-and-Excitation Networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00745"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Mayer, N., Ilg, E., Hausser, P., Fischer, P., Cremers, D., Dosovitskiy, A., and Brox, T. (2016, January 27\u201330). A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.438"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Guo, X., Yang, K., Yang, W., Wang, X., and Li, H. (2019, January 18\u201324). Group-Wise Correlation Stereo Network. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.","DOI":"10.1109\/CVPR.2019.00339"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Chang, J.R., and Chen, Y.S. (2018, January 18\u201323). Pyramid Stereo Matching Network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00567"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Kendall, A., Martirosyan, H., Dasgupta, S., Henry, P., Kennedy, R., Bachrach, A., and Bry, A. (2017, January 22\u201329). End-to-End Learning of Geometry and Context for Deep Stereo Regression. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.17"},{"key":"ref_40","unstructured":"Lai, S.H., Lepetit, V., Nishino, K., and Sato, Y. A Dataset and Evaluation Methodology for Depth Estimation on 4D Light Fields. Proceedings of the Computer Vision\u2014ACCV 2016, Lecture Notes in Computer Science."},{"key":"ref_41","unstructured":"Zhao, H., Gallo, O., Frosio, I., and Kautz, J. (2018). Loss Functions for Neural Networks for Image Processing. arXiv."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"600","DOI":"10.1109\/TIP.2003.819861","article-title":"Image quality assessment: From error visibility to structural similarity","volume":"13","author":"Wang","year":"2004","journal-title":"IEEE Trans. Image Process."},{"key":"ref_43","unstructured":"Kingma, D.P., and Ba, J. (2017). Adam: A Method for Stochastic Optimization. arXiv."},{"key":"ref_44","unstructured":"Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., and Antiga, L. (2019). Advances in Neural Information Processing Systems, Curran Associates, Inc."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Wang, T.C., Efros, A.A., and Ramamoorthi, R. (2015, January 7\u201313). Occlusion-Aware Depth Estimation Using Light-Field Cameras. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile.","DOI":"10.1109\/ICCV.2015.398"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"116764","DOI":"10.1016\/j.image.2022.116764","article-title":"Unsupervised disparity estimation from light field using plug-and-play weighted warping loss","volume":"107","author":"Iwatsuki","year":"2022","journal-title":"Signal Process. Image Commun."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"1606","DOI":"10.1109\/TIP.2019.2944343","article-title":"Unsupervised Monocular Depth Estimation From Light Field Image","volume":"29","author":"Zhou","year":"2020","journal-title":"IEEE Trans. Image Process."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Zhou, W., Liang, L., Zhang, H., Lumsdaine, A., and Lin, L. (2018, January 20\u201324). Scale and Orientation Aware EPI-Patch Learning for Light Field Depth Estimation. Proceedings of the 2018 24th International Conference on Pattern Recognition (ICPR), Beijing, China.","DOI":"10.1109\/ICPR.2018.8545490"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"113","DOI":"10.1016\/j.neucom.2022.06.011","article-title":"Unsupervised learning of light field depth estimation with spatial and angular consistencies","volume":"501","author":"Lin","year":"2022","journal-title":"Neurocomputing"},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep Residual Learning for Image Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/11\/3583\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T14:52:26Z","timestamp":1760107946000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/11\/3583"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,6,1]]},"references-count":50,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2024,6]]}},"alternative-id":["s24113583"],"URL":"https:\/\/doi.org\/10.3390\/s24113583","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2024,6,1]]}}}