AUTHOR=Yogarajan Vithya , Dobbie Gillian , Leitch Sharon , Keegan Te Taka , Bensemann Joshua , Witbrock Michael , Asrani Varsha , Reith David 

TITLE=Data and model bias in artificial intelligence for healthcare applications in New Zealand

JOURNAL=Frontiers in Computer Science

VOLUME=Volume 4 - 2022

YEAR=2022

URL=https://www.frontiersin.org/journals/computer-science/articles/10.3389/fcomp.2022.1070493

DOI=10.3389/fcomp.2022.1070493

ISSN=2624-9898

ABSTRACT=Developments in Artificial Intelligence (AI) are adopted widely in healthcare. However, the introduction and use of AI may come with biases and disparities, resulting in concerns about healthcare access and outcomes for underrepresented indigenous populations. In New Zealand, Māori experience significant inequities in health compared to the non-Indigenous population. This research explores equity concepts and fairness measures concerning AI for healthcare in New Zealand. It takes early steps toward developing a model of socially responsible and fair AI for New Zealand’s population. To ensure research equality and fair inclusion of M\={a}ori, we combine expertise in Artificial Intelligence (AI), New Zealand clinical context, and te ao  M\={a}ori. The mitigation of inequity needs to be addressed in data collection, model development, and model deployment. In this paper, we analyse data and algorithmic bias concerning data collection and model development, training and testing using health data collected by experts.  We use fairness measures such as disparate impact scores, equal opportunities and equalised odds to analyse tabular data. Furthermore, token frequencies, statistical significance testing and fairness measures for word embeddings, such as WEAT and WEFE frameworks, are used to analyse bias in free-form medical text. The AI model predictions are also explained using SHAP and LIME. We show evidence of bias due to the changes made in algorithmic design. Furthermore, we observe unintentional bias due to the underlying pre-trained models used to represent text data. This research addresses some vital issues while opening up the need for and opportunity for future research.