{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,17]],"date-time":"2026-04-17T00:33:54Z","timestamp":1776386034022,"version":"3.51.2"},"reference-count":61,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2022,9,28]],"date-time":"2022-09-28T00:00:00Z","timestamp":1664323200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100000038","name":"Natural Sciences and Engineering Research Council","doi-asserted-by":"publisher","award":["RGPIN-2020-05869"],"award-info":[{"award-number":["RGPIN-2020-05869"]}],"id":[{"id":"10.13039\/501100000038","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>Building trust and transparency in healthcare can be achieved using eXplainable Artificial Intelligence (XAI), as it facilitates the decision-making process for healthcare professionals. Knowledge graphs can be used in XAI for explainability by structuring information, extracting features and relations, and performing reasoning. This paper highlights the role of knowledge graphs in XAI models in healthcare, considering a state-of-the-art review. Based on our review, knowledge graphs have been used for explainability to detect healthcare misinformation, adverse drug reactions, drug-drug interactions and to reduce the knowledge gap between healthcare experts and AI-based models. We also discuss how to leverage knowledge graphs in pre-model, in-model, and post-model XAI models in healthcare to make them more explainable.<\/jats:p>","DOI":"10.3390\/info13100459","type":"journal-article","created":{"date-parts":[[2022,9,28]],"date-time":"2022-09-28T22:53:19Z","timestamp":1664405599000},"page":"459","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":44,"title":["Knowledge Graphs and Explainable AI in Healthcare"],"prefix":"10.3390","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9557-0043","authenticated-orcid":false,"given":"Enayat","family":"Rajabi","sequence":"first","affiliation":[{"name":"Department of Financial and Information Management, Cape Breton University, Sydney, NS B1P 6L2, Canada"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5685-6487","authenticated-orcid":false,"given":"Somayeh","family":"Kafaie","sequence":"additional","affiliation":[{"name":"Mathematics and Computing Science Department, Saint Mary\u2019s University, 912 Robie Street, Halifax, NS B3H 3C3, Canada"}]}],"member":"1968","published-online":{"date-parts":[[2022,9,28]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1016\/j.wneu.2021.10.068","article-title":"Artificial intelligence applications in pediatric brain tumor imaging: A systematic review","volume":"157","author":"Huang","year":"2022","journal-title":"World Neurosurg."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Wohlin, C. (2014, January 13\u201314). Guidelines for snowballing in systematic literature studies and a replication in software engineering. Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering, London, UK.","DOI":"10.1145\/2601248.2601268"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"82","DOI":"10.1016\/j.inffus.2019.12.012","article-title":"Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI","volume":"58","author":"Arrieta","year":"2020","journal-title":"Inf. Fusion"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Amann, J., Blasimme, A., Vayena, E., Frey, D., and Madai, V.I. (2020). Explainability for artificial intelligence in healthcare: A multidisciplinary perspective. BMC Med. Inform. Decis. Mak., 20.","DOI":"10.1186\/s12911-020-01332-6"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"52138","DOI":"10.1109\/ACCESS.2018.2870052","article-title":"Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)","volume":"6","author":"Adadi","year":"2018","journal-title":"IEEE Access"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Antoniadi, A.M., Du, Y., Guendouz, Y., Wei, L., Mazo, C., Becker, B.A., and Mooney, C. (2021). Current challenges and future opportunities for xai in machine learning-based clinical decision support systems: A systematic review. Appl. Sci., 11.","DOI":"10.3390\/app11115088"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"41","DOI":"10.3233\/SW-190374","article-title":"On the Role of Knowledge Graphs in Explainable AI","volume":"11","author":"Lecue","year":"2020","journal-title":"Semant. Web"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"103627","DOI":"10.1016\/j.artint.2021.103627","article-title":"Knowledge Graphs as Tools for Explainable Machine Learning: A Survey","volume":"302","author":"Tiddi","year":"2022","journal-title":"Artif. Intell."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Kejriwal, M. (2019). Domain-Specific Knowledge Graph Construction, Springer.","DOI":"10.1007\/978-3-030-12375-8"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Telnov, V., and Korovin, Y. (2020). Semantic Web and Interactive Knowledge Graphs as an Educational Technology. Cloud Computing Security-Concepts and Practice, IntechOpen.","DOI":"10.5772\/intechopen.92433"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"103692","DOI":"10.1016\/j.compbiomed.2020.103692","article-title":"Network-based prioritization of cancer genes by integrative ranks from multi-omics data","volume":"119","author":"Shang","year":"2020","journal-title":"Comput. Biol. Med."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Crichton, G., Guo, Y., Pyysalo, S., and Korhonen, A. (2018). Neural Networks for Link Prediction in Realistic Biomedical Graphs: A Multi-dimensional Evaluation of Graph Embedding-based Approaches. BMC Bioinform., 19.","DOI":"10.1186\/s12859-018-2163-9"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Wang, Q., Li, M., Wang, X., Parulian, N., Han, G., Ma, J., Tu, J., Lin, Y., Zhang, R.H., and Liu, W. (2021, January 6\u201311). COVID-19 Literature Knowledge Graph Construction and Drug Repurposing Report Generation. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Online.","DOI":"10.18653\/v1\/2021.naacl-demos.8"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Dranc\u00e9, M., Boudin, M., Mougin, F., and Diallo, G. (2021, January 25\u201327). Neuro-symbolic XAI for Computational Drug Repurposing. Proceedings of the 13th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management (IC3K 2021)-Volume 2: KEOD, Online.","DOI":"10.5220\/0010714100003064"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1737","DOI":"10.3389\/fgene.2020.625659","article-title":"Adverse Drug Reaction Discovery Using a Tumor-Biomarker Knowledge Graph","volume":"11","author":"Wang","year":"2021","journal-title":"Front. Genet."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"171","DOI":"10.1186\/s12911-021-01518-6","article-title":"Investigating ADR mechanisms with Explainable AI: A feasibility study with knowledge graph mining","volume":"21","author":"Bresso","year":"2021","journal-title":"BMC Med. Inform. Decis. Mak."},{"key":"ref_17","first-page":"2739","article-title":"KGNN: Knowledge Graph Neural Network for Drug-Drug Interaction Prediction","volume":"380","author":"Lin","year":"2020","journal-title":"IJCAI"},{"key":"ref_18","unstructured":"Shang, J., Xiao, C., Ma, T., Li, H., and Sun, J. (February, January 27). GAMENet: Graph Augmented Memory Networks for Recommending Medication Combination. Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence and Thirty-First Innovative Applications of Artificial Intelligence Conference and Ninth AAAI Symposium on Educational Advances in Artificial Intelligence, Honolulu, HI, USA. AAAI\u201919\/IAAI\u201919\/EAAI\u201919."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"100174","DOI":"10.1016\/j.bdr.2020.100174","article-title":"SMR: Medical Knowledge Graph Embedding for Safe Medicine Recommendation","volume":"23","author":"Gong","year":"2021","journal-title":"Big Data Res."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Cui, L., Seo, H., Tabar, M., Ma, F., Wang, S., and Lee, D. (2020, January 4\u20138). DETERRENT: Knowledge Guided Graph Attention Network for Detecting Healthcare Misinformation. Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Anchorage, AK, USA.","DOI":"10.1145\/3394486.3403092"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"36","DOI":"10.1145\/3492855","article-title":"Hc-Covid","volume":"6","author":"Kou","year":"2022","journal-title":"Proc. ACM Hum.-Comput. Interact."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"114","DOI":"10.1016\/j.sbi.2021.09.003","article-title":"Toward Better Drug Discovery with Knowledge Graph","volume":"72","author":"Zeng","year":"2022","journal-title":"Curr. Opin. Struct. Biol."},{"key":"ref_23","unstructured":"(2022, September 22). Bayes\u2019 Theorem (Stanford Encyclopedia of Philosophy). Available online: https:\/\/plato.stanford.edu\/entries\/bayes-theorem\/."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"867","DOI":"10.3389\/fbioe.2020.00867","article-title":"Explainable Prediction of Medical Codes With Knowledge Graphs","volume":"8","author":"Teng","year":"2020","journal-title":"Front. Bioeng. Biotechnol."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"4624","DOI":"10.1021\/acs.jproteome.0c00316","article-title":"Repurpose Open Data to Discover Therapeutics for COVID-19 Using Deep Learning","volume":"19","author":"Zeng","year":"2020","journal-title":"J. Proteome Res."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"58","DOI":"10.1016\/j.inffus.2021.09.022","article-title":"EXplainable Neural-Symbolic Learning (X-NeSyL) methodology to fuse deep learning representations with expert knowledge graphs: The MonuMAI cultural heritage use case","volume":"79","author":"Lamas","year":"2022","journal-title":"Inf. Fusion"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"ii120","DOI":"10.1093\/bioinformatics\/btac478","article-title":"GNN-SubNet: Disease subnetwork detection with explainable Graph Neural Networks","volume":"38","author":"Pfeifer","year":"2022","journal-title":"Bioinformatics"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"101817","DOI":"10.1016\/j.artmed.2020.101817","article-title":"Real-world data medical knowledge graph: Construction and applications","volume":"103","author":"Li","year":"2020","journal-title":"Artif. Intell. Med."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Ciampaglia, G.L., Shiralkar, P., Rocha, L.M., Bollen, J., Menczer, F., and Flammini, A. (2015). Computational Fact Checking from Knowledge Networks. PLoS ONE, 10.","DOI":"10.1371\/journal.pone.0141938"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Huynh, V.P., and Papotti, P. (2019, January 3\u20137). A Benchmark for Fact Checking Algorithms Built on Knowledge Bases. Proceedings of the 28th ACM International Conference on Information and Knowledge Management, Beijing, China. CIKM \u201919.","DOI":"10.1145\/3357384.3358036"},{"key":"ref_31","unstructured":"Admin (2022, August 07). Knowledge Graphs: Backbone of Data-Driven Culture in Life Sciences. Available online: https:\/\/www.virtusa.com\/perspectives\/article\/knowledge-graphs-backbone-of-data-driven-culture-in-life-sciences."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Ametaj, B.N. (2017). What Are Omics Sciences?. Periparturient Diseases of Dairy Cows: A Systems Biology Approach, Springer International Publishing.","DOI":"10.1007\/978-3-319-43033-1"},{"key":"ref_33","unstructured":"Maloy, C. (2022, July 16). Library Guides: Data Resources in the Health Sciences: Clinical Data. Available online: https:\/\/guides.lib.uw.edu\/hsl\/data\/findclin."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Aggarwal, C.C. (2013). Mining of Sensor Data in Healthcare: A Survey. Managing and Mining Sensor Data, Springer.","DOI":"10.1007\/978-1-4614-6309-2"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1414","DOI":"10.1016\/j.csbj.2020.05.017","article-title":"Constructing Knowledge Graphs and Their Biomedical Applications","volume":"18","author":"Nicholson","year":"2020","journal-title":"Comput. Struct. Biotechnol. J."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Chor, B. (2012). Predicting Protein-Protein Interactions from Multimodal Biological Data Sources via Nonnegative Matrix Tri-Factorization. Proceedings of Research in Computational Molecular Biology, Springer.","DOI":"10.1007\/978-3-642-29627-7"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Wu, Q., Wang, Z., Li, C., Ye, Y., Li, Y., and Sun, N. (2015). Protein Functional Properties Prediction in Sparsely-label PPI Networks through Regularized Non-negative Matrix Factorization. BMC Syst. Biol., 9.","DOI":"10.1186\/1752-0509-9-S1-S9"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"2723","DOI":"10.1093\/bioinformatics\/btx275","article-title":"Neuro-symbolic Representation Learning on Biological Knowledge Graphs","volume":"33","author":"Alshahrani","year":"2017","journal-title":"Bioinformatics"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Kalajdziski, S., and Ackovska, N. (2018, January 17\u201319). Deep Learning the Protein Function in Protein Interaction Networks. Proceedings of the ICT Innovations 2018. Engineering and Life Sciences, Ohrid, Macedonia.","DOI":"10.1007\/978-3-030-00825-3"},{"key":"ref_40","first-page":"2498957","article-title":"miRNA-Disease Association Prediction with Collaborative Matrix Factorization","volume":"2017","author":"Wu","year":"2017","journal-title":"Complexity"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"1452","DOI":"10.1093\/jamia\/ocy117","article-title":"Heterogeneous Network Embedding for Identifying Symptom Candidate Genes","volume":"25","author":"Yang","year":"2018","journal-title":"J. Am. Med Inform. Assoc."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Xu, B., Liu, Y., Yu, S., Wang, L., Dong, J., Lin, H., Yang, Z., Wang, J., and Xia, F. (2019). A Network Embedding Model for Pathogenic Genes Prediction by Multi-path Random Walking on Heterogeneous Network. BMC Med. Genom., 12.","DOI":"10.1186\/s12920-019-0627-z"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Wang, X., Gong, Y., Yi, J., and Zhang, W. (2019, January 18\u201321). Predicting Gene-disease Associations from the Heterogeneous Network using Graph Embedding. Proceedings of the IEEE International Conference on Bioinformatics and Biomedicine (BIBM), San Diego, CA, USA.","DOI":"10.1109\/BIBM47256.2019.8983134"},{"key":"ref_44","unstructured":"Wang, M., Liu, M., Liu, J., Wang, S., Long, G., and Qian, B. (2017). Safe Medicine Recommendation via Medical Knowledge Graph Embedding. arXiv."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1016\/j.artmed.2018.03.005","article-title":"EMR-based Medical Knowledge Representation and Inference via Markov Random Fields and Distributed Representation Learning","volume":"87","author":"Zhao","year":"2018","journal-title":"Artif. Intell. Med."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Choi, E., Bahadori, M.T., Song, L., Stewart, W.F., and Sun, J. (2017, January 13\u201317). GRAM: Graph-based Attention Model for Healthcare Representation Learning. Proceedings of the International Conference on Knowledge Discovery & Data Mining, Halifax, NS, Canada.","DOI":"10.1145\/3097983.3098126"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"3191","DOI":"10.1109\/TKDE.2016.2605687","article-title":"Diagnosis Code Assignment Using Sparsity-Based Disease Correlation Embedding","volume":"28","author":"Wang","year":"2016","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_48","unstructured":"Sarker, M.K. (2020). Towards Explainable Artificial Intelligence (XAI) Based on Contextualizing Data with Knowledge Graphs. [Ph.D. Thesis, Kansas State University]."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"2614","DOI":"10.1093\/bioinformatics\/bty114","article-title":"A Global Network of Biomedical Relationships Derived from Text","volume":"34","author":"Percha","year":"2018","journal-title":"Bioinformatics"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"310","DOI":"10.1016\/j.jbi.2010.11.001","article-title":"Semi-automatic Semantic Annotation of PubMed Queries: A Study on Quality, Efficiency, Satisfaction","volume":"44","author":"Lu","year":"2011","journal-title":"J. Biomed. Inform."},{"key":"ref_51","unstructured":"Haq, H.U., Kocaman, V., and Talby, D. (2021). Deeper Clinical Document Understanding Using Relation Extraction. arXiv."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"2463","DOI":"10.1109\/JBHI.2021.3085003","article-title":"EHR-Oriented Knowledge Graph System: Toward Efficient Utilization of Non-Used Information Buried in Routine Clinical Practice","volume":"25","author":"Shang","year":"2021","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"bbab282","DOI":"10.1093\/bib\/bbab282","article-title":"Deep Learning Methods for Biomedical Named Entity Recognition: A Survey and Qualitative Comparison","volume":"22","author":"Song","year":"2021","journal-title":"Briefings Bioinform."},{"key":"ref_54","unstructured":"Zhao, S., Qin, B., Liu, T., and Wang, F. (2020). Biomedical Knowledge Graph Refinement with Embedding and Logic Rules. arXiv."},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"298","DOI":"10.1016\/j.ins.2016.08.038","article-title":"Ontology-based Deep Learning for Human Behavior Prediction with Explanations in Health Social Networks","volume":"384","author":"Phan","year":"2017","journal-title":"Inf. Sci."},{"key":"ref_56","unstructured":"Cassiman, J. (2022, August 26). How Are Knowledge Graphs and Machine Learning Related?. 2022., Available online: https:\/\/blog.ml6.eu\/how-are-knowledge-graphs-and-machine-learning-related-ff6f5c1760b5."},{"key":"ref_57","unstructured":"Hamilton, W.L., Ying, R., and Leskovec, J. (2017). Representation Learning on Graphs: Methods and Applications. arXiv."},{"key":"ref_58","doi-asserted-by":"crossref","unstructured":"Grover, A., and Leskovec, J. (2016, January 13\u201317). node2vec: Scalable Feature Learning for Networks. Proceedings of the International Conference on Knowledge Discovery & Data Mining, San Francisco, CA, USA.","DOI":"10.1145\/2939672.2939754"},{"key":"ref_59","unstructured":"Burges, C., Bottou, L., Welling, M., Ghahramani, Z., and Weinberger, K. (2013, January 5\u201310). Translating Embeddings for Modeling Multi-relational Data. Proceedings of Advances in Neural Information Processing Systems, Lake Tahoe, NV, USA."},{"key":"ref_60","doi-asserted-by":"crossref","unstructured":"Han, H., and Liu, X. (2022). The Challenges of Explainable AI in Biomedical Data Science. BMC Bioinform., 22.","DOI":"10.1186\/s12859-021-04368-1"},{"key":"ref_61","doi-asserted-by":"crossref","first-page":"101666","DOI":"10.1016\/j.giq.2021.101666","article-title":"The Perils and Pitfalls of Explainable AI: Strategies for Explaining Algorithmic Decision-Making","volume":"39","author":"Warnier","year":"2022","journal-title":"Gov. Inf. Q."}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/13\/10\/459\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T00:41:22Z","timestamp":1760143282000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/13\/10\/459"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,9,28]]},"references-count":61,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2022,10]]}},"alternative-id":["info13100459"],"URL":"https:\/\/doi.org\/10.3390\/info13100459","relation":{},"ISSN":["2078-2489"],"issn-type":[{"value":"2078-2489","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,9,28]]}}}