{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,28]],"date-time":"2026-03-28T17:14:29Z","timestamp":1774718069292,"version":"3.50.1"},"reference-count":36,"publisher":"MDPI AG","issue":"20","license":[{"start":{"date-parts":[[2022,10,20]],"date-time":"2022-10-20T00:00:00Z","timestamp":1666224000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Institute of Information and Communications Technology Planning and Evaluation (IITP) Grant by the Korean Government through the Ministry of Science and ICT (MSIT)","award":["2022-0-00331"],"award-info":[{"award-number":["2022-0-00331"]}]},{"name":"Institute of Information and Communications Technology Planning and Evaluation (IITP) Grant by the Korean Government through the Ministry of Science and ICT (MSIT)","award":["2022R1F1A1063662"],"award-info":[{"award-number":["2022R1F1A1063662"]}]},{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea (NRF) Grant by the Korean Government through MSIT","doi-asserted-by":"publisher","award":["2022-0-00331"],"award-info":[{"award-number":["2022-0-00331"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea (NRF) Grant by the Korean Government through MSIT","doi-asserted-by":"publisher","award":["2022R1F1A1063662"],"award-info":[{"award-number":["2022R1F1A1063662"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Strengthening Research and Development Capability Program of Sejong University","award":["2022-0-00331"],"award-info":[{"award-number":["2022-0-00331"]}]},{"name":"Strengthening Research and Development Capability Program of Sejong University","award":["2022R1F1A1063662"],"award-info":[{"award-number":["2022R1F1A1063662"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>With the advancement in next-generation communication technologies, the so-called Tactile Internet is getting more attention due to its smart applications, such as haptic-enabled teleoperation systems. The stringent requirements such as delay, jitter, and packet loss of these delay-sensitive and loss-intolerant applications make it more challenging to ensure the Quality of Service (QoS) and Quality of Experience (QoE). In this regard, different haptic codec and control schemes were proposed for QoS and QoE provisioning in the Tactile Internet. However, they maximize the QoE while degrading the system\u2019s stability under varying delays and high packet rates. In this paper, we present a reinforcement learning-based Intelligent Tactile Edge (ITE) framework to ensure both transparency and stability of teleoperation systems with high packet rates and variable time delay communication networks. The proposed ITE first estimates the network challenges, including communication delay, jitter, and packet loss, and then utilizes a Q-learning algorithm to select the optimal haptic codec scheme to reduce network load. The proposed framework aims to explore the optimal relationship between QoS and QoE parameters and make the tradeoff between stability and transparency during teleoperations. The simulation result indicates that the proposed strategy chooses the optimal scheme under different network impairments corresponding to the congestion level in the communication network while improving the QoS and maximizing the QoE. The end-to-end performance of throughput (1.5 Mbps) and average RTT (70 ms) during haptic communication is achieved with a learning rate and discounted factor value of 0.5 and 0.8, respectively. The results indicate that the communication system can successfully achieve the QoS and QoE requirements by employing the proposed ITE framework.<\/jats:p>","DOI":"10.3390\/s22208001","type":"journal-article","created":{"date-parts":[[2022,10,21]],"date-time":"2022-10-21T00:34:30Z","timestamp":1666312470000},"page":"8001","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["Reinforcement Learning-Aided Edge Intelligence Framework for Delay-Sensitive Industrial Applications"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5488-5121","authenticated-orcid":false,"given":"Muhammad","family":"Zubair Islam","sequence":"first","affiliation":[{"name":"School of Intelligent Mechatronics Engineering, Sejong University, Seoul 05006, Korea"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3743-0309","authenticated-orcid":false,"family":"Shahzad","sequence":"additional","affiliation":[{"name":"School of Intelligent Mechatronics Engineering, Sejong University, Seoul 05006, Korea"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9756-1909","authenticated-orcid":false,"given":"Rashid","family":"Ali","sequence":"additional","affiliation":[{"name":"Department of Information and Communication Technologies, Universitat Pompeu Fabra, 08018 Barcelona, Spain"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1534-061X","authenticated-orcid":false,"given":"Amir","family":"Haider","sequence":"additional","affiliation":[{"name":"School of Intelligent Mechatronics Engineering, Sejong University, Seoul 05006, Korea"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2240-1337","authenticated-orcid":false,"given":"Hyung Seok","family":"Kim","sequence":"additional","affiliation":[{"name":"School of Intelligent Mechatronics Engineering, Sejong University, Seoul 05006, Korea"}]}],"member":"1968","published-online":{"date-parts":[[2022,10,20]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"42828","DOI":"10.1109\/ACCESS.2020.2977406","article-title":"5G is real: Evaluating the compliance of the 3GPP 5G new radio system with the ITU IMT-2020 requirements","volume":"8","author":"Henry","year":"2020","journal-title":"IEEE Access"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"23022","DOI":"10.1109\/ACCESS.2020.2970118","article-title":"Internet of things (IoT) for next-generation smart systems: A review of current challenges, future trends and prospects for emerging 5G-IoT scenarios","volume":"8","author":"Shafique","year":"2020","journal-title":"IEEE Access"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"668","DOI":"10.1109\/COMST.2021.3059896","article-title":"A survey on resource allocation for 5G heterogeneous networks: Current research, future trends, and challenges","volume":"23","author":"Xu","year":"2021","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"64","DOI":"10.1109\/MVT.2013.2295069","article-title":"The tactile internet: Applications and challenges","volume":"9","author":"Fettweis","year":"2014","journal-title":"IEEE Veh. Technol. Mag."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"433","DOI":"10.1109\/ACCESS.2020.3046773","article-title":"An overview of physical layer design for Ultra-Reliable Low-Latency Communications in 3GPP Releases 15, 16, and 17","volume":"9","author":"Le","year":"2020","journal-title":"IEEE Access"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"957","DOI":"10.1109\/OJCOMS.2020.3010270","article-title":"6G wireless communication systems: Applications, requirements, technologies, challenges, and research directions","volume":"1","author":"Chowdhury","year":"2020","journal-title":"IEEE Open J. Commun. Soc."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"28","DOI":"10.1109\/MVT.2019.2921208","article-title":"6G wireless networks: Vision, requirements, architecture, and key technologies","volume":"14","author":"Zhang","year":"2019","journal-title":"IEEE Veh. Technol. Mag."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"55","DOI":"10.1109\/MCOM.001.1900411","article-title":"Toward 6G networks: Use cases and technologies","volume":"58","author":"Giordani","year":"2020","journal-title":"IEEE Commun. Mag."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"338","DOI":"10.1109\/TOH.2018.2818134","article-title":"A systematic review of multilateral teleoperation systems IEEE Trans","volume":"11","author":"Shahbazi","year":"2018","journal-title":"Haptics"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"152","DOI":"10.1109\/MNET.011.2000611","article-title":"A Federated Reinforcement Learning Framework for Incumbent Technologies in Beyond 5G Networks","volume":"35","author":"Ali","year":"2021","journal-title":"IEEE Netw."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"103362","DOI":"10.1016\/j.jnca.2022.103362","article-title":"A survey on Zero touch network and Service (ZSM) Management for 5G and beyond networks","volume":"10","author":"Liyanage","year":"2022","journal-title":"J. Netw. Comput. Appl."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Siriwardhana, Y., Porambage, P., Liyanage, M., and Ylianttila, M. (, January 8\u201311). AI and 6G security: Opportunities and challenges. Proceedings of the 2021 Joint European Conference on Networks and Communications & 6G Summit (EuCNC\/6G Summit): 6G Visions (6GV), Porto, Portugal,.","DOI":"10.1109\/EuCNC\/6GSummit51104.2021.9482503"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Ateya, A.A., Muthanna, A., Gudkova, I., Abuarqoub, A., Vybornova, A., and Koucheryavy, A. (2018). Development of intelligent core network for tactile internet and future smart systems. J. Sens. Actuator Netw., 7.","DOI":"10.3390\/jsan7010001"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Bojkovic, Z.S., Bakmaz, B.M., and Bakmaz, M.R. (2017, January 18\u201320). Vision and enabling technologies of tactile internet realization. Proceedings of the 13th International Conference on Advanced Technologies, Systems and Services in Telecommunications (TELSIKS), Nis, Serbia.","DOI":"10.1109\/TELSKS.2017.8246242"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"256","DOI":"10.1109\/JPROC.2018.2885541","article-title":"The IEEE 1918.1 \u201ctactile internet\u201d standards working group and its standards","volume":"107","author":"Holland","year":"2019","journal-title":"Proc. IEEE"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"23502","DOI":"10.1109\/ACCESS.2017.2764181","article-title":"Challenges in haptic communications over the tactile internet","volume":"5","author":"Glans","year":"2017","journal-title":"IEEE Access"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"472","DOI":"10.1109\/COMST.2020.3025995","article-title":"A comprehensive survey of the tactile internet: State-of-the-art and research directions","volume":"23","author":"Promwongsa","year":"2020","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"3034","DOI":"10.1109\/COMST.2018.2851452","article-title":"Toward haptic communications over the 5G tactile Internet","volume":"20","author":"Antonakoglou","year":"2018","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"447","DOI":"10.1109\/JPROC.2018.2867835","article-title":"Haptic codecs for the tactile internet","volume":"107","author":"Steinbach","year":"2018","journal-title":"Proc. IEEE"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Dohler, M. (2021). 5G Networks, Haptic Codecs, and the Operating Theatre. Digital Surgery, Springer.","DOI":"10.1007\/978-3-030-49100-0_6"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Gokhale, V., Nair, J., and Chaudhuri, S. (2016, January 8\u201311). Opportunistic adaptive haptic sampling on forward channel in telehaptic communication. Proceedings of the 2016 IEEE Haptics Symposium (HAPTICS), Philadelphia, PA, USA.","DOI":"10.1109\/HAPTICS.2016.7463180"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Liu, X., and Dohler, M. (2019, January 20\u201323). A data-driven approach to vibrotactile data compression. Proceedings of the 2019 IEEE International Workshop on Signal Processing Systems (SiPS), Nanjing, China.","DOI":"10.1109\/SiPS47522.2019.9020534"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Zeng, C., Zhao, T., Liu, Q., Xu, Y., and Wang, K. (2020, January 12\u201316). Perception-lossless codec of haptic data with low delay. Proceedings of the 28th ACM International Conference on Multimedia, Seattle, WA, USA.","DOI":"10.1145\/3394171.3413728"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"4455","DOI":"10.1109\/TMM.2020.3042674","article-title":"Pvc-slp: Perceptual vibrotactile-signal compression based-on sparse linear prediction","volume":"23","author":"Hassen","year":"2020","journal-title":"IEEE Trans. Multimed."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Noll, A., G\u00fclecy\u00fcz, B., Hofmann, A., and Steinbach, E. (2020, January 28\u201331). A rate-scalable perceptual wavelet-based vibrotactile codec. Proceedings of the 2020 IEEE Haptics Symposium (HAPTICS), Washington, DC, USA.","DOI":"10.1109\/HAPTICS45997.2020.ras.HAP20.6.422bbc6e"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Gui, M., Xu, X., and Steinbach, E. (2020, January 28\u201331). Adaptive Packet Rate Control for the Mitigation of Bursty Haptic Traffic in Teleoperation Systems. Proceedings of the 2020 IEEE Haptics Symposium (HAPTICS), Washington, DC, USA.","DOI":"10.1109\/HAPTICS45997.2020.ras.HAP20.18.6a51e1e1"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Kakade, S., and Chaudhuri, S. (2020, January 6). Perceptually Compressive Communication of Interactive Telehaptic Signal. Proceedings of the International Conference on Human Haptic Sensing and Touch Enabled Computer Applications, Leiden, The Netherlands.","DOI":"10.1007\/978-3-030-58147-3_53"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Li, Z., Hassen, R., and Wang, Z. (2021, January 6\u201311). Autoencoder for Vibrotactile Signal Compression. Proceedings of the 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Toronto, ON, Canada.","DOI":"10.1109\/ICASSP39728.2021.9413370"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"42","DOI":"10.1145\/2209249.2209264","article-title":"Controlling queue delay","volume":"55","author":"Nichols","year":"2012","journal-title":"Commun. ACM"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Hoeiland-Joergensen, T., McKenney, P., Taht, D., Gettys, J., and Dumazet, E. (2018). The Flow Queue CoDel Packet Scheduler and Active Queue Management Algorithm, Internet Engineering Task Force. Technical Report, No. RFC-8290.","DOI":"10.17487\/RFC8290"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Gomez, C.A., Wang, X., and Shami, A. (2019, January 9\u201313). Intelligent active queue management using explicit congestion notification. Proceedings of the 2019 IEEE Global Communications Conference (GLOBECOM), Waikoloa, HI, USA.","DOI":"10.1109\/GLOBECOM38437.2019.9013475"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"2005","DOI":"10.1109\/LCOMM.2020.3001227","article-title":"The LSTM-based advantage actor-critic learning for resource management in network slicing with user mobility","volume":"24","author":"Li","year":"2020","journal-title":"CIEEE Commun. Lett."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Bhardwaj, A., Cizmeci, B., Steinbach, E., Liu, Q., Eid, M., AraUjo, J., El Saddik, A., Kundu, R., Liu, X., and Holland, O. (2017, January 22\u201323). A candidate hardware and software reference setup for kinesthetic codec standardization. Proceedings of the 2017 IEEE International Symposium on Haptic, Audio and Visual Environments and Games (HAVE), Abu Dhabi, United Arab Emirates.","DOI":"10.1109\/HAVE.2017.8240353"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"308","DOI":"10.1016\/j.ijinfomgt.2018.08.004","article-title":"Mobile edge computing based QoS optimization in medical healthcare applications","volume":"45","author":"Sodhro","year":"2019","journal-title":"Int. J. Inf. Manag."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Islam, M.Z., Ali, R., Haider, A., and Kim, H.S. (2021). IoTactileSim: A Virtual Testbed for Tactile Industrial Internet of Things Services. Sensors, 21.","DOI":"10.3390\/s21248363"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"85720","DOI":"10.1109\/ACCESS.2022.3197900","article-title":"QoS Provisioning: Key Drivers and Enablers Towards the Tactile Internet in Beyond 5G Era","volume":"10","author":"Islam","year":"2022","journal-title":"IEEE Access"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/20\/8001\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T00:57:57Z","timestamp":1760144277000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/20\/8001"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,10,20]]},"references-count":36,"journal-issue":{"issue":"20","published-online":{"date-parts":[[2022,10]]}},"alternative-id":["s22208001"],"URL":"https:\/\/doi.org\/10.3390\/s22208001","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,10,20]]}}}