Authors: Assistant Professor Ramya S, Assistant Professor I. R. Suganya
Abstract: Opacity in deep learning models poses a significant challenge in adopting AI models in healthcare as decisions need to be transparent. In this study, we propose XAI-MedFusion, a deep learning explainable multi-modal framework to enable intelligence in the diagnosis of diseases. Our pro-posed XAI-MedFusion is a hierarchal framework that combines the inputs from medical imaging, electronic health records (EHR), and genomics with explainability. We use modality-specific encoders such as CNNs for images, Transformers for EHRs, and GNNs for genomics. We use cross-modal attention in our system to learn how to aggregate information across the modalities. Finally, we utilize explainability methods such as SHAP, LIME, and Grad-CAM with an aggre-gation approach. We validate our framework using Alzheimer’s and Parkinson’s disease data, achieving a classification accuracy of 94.2% compared to the unimodal approaches (12.8% high-er). Clinically relevant interpretations were obtained that match the known biological markers. Moreover, uncertainty quantification was effective in our model along with increased clinician trust (8.4/10).
