Medical decision support systems (MDSS) are computer-based programs that analyse data within a patient's healthcare records to provide questions, prompts, or reminders to assist clinicians at the point of care. Inputting a patient's data, symptoms, or current treatment regimens into an MDSS, clinicians are assisted with the identification or elimination of the most likely potential medical causes, which can enable faster discovery of a set of appropriate diagnoses or treatment plans. Explainable AI (XAI) is a "white box" model of artificial intelligence in which the results of the solution can be understood by the users, who can see an estimate of the weighted importance of each feature on the model's predictions, and understand how the different features interact to arrive at a specific decision.