Abstract Artificial Intelligence (AI) and Machine Learning (ML) promise to significantly enhance the capabilities of decision support systems in medicine. Yet, if these systems fail to provide an understandable rationale of the decision making pro- cess the adoption of this powerful technology will be diffi- cult. Hence, there is growing interest in Explainable Artificial Intelligence (XAI). Explanations that speak the language of the user are a cornerstone for Fair AI. In this position paper, we reflect on research about XAI and designing decision support in medicine. From there, we lay out an approach to use co-design methodology to explore how users perceive and interact with explanations of decision support systems. PDF