Impact of COVID-19-related data drift on machine-learning prognostic models predicting 30-day opioid-related emergency department visits, hospitalisation or mortality: a population-level administrative data study in Alberta, Canada

Objective To develop machine-learning (ML) models during the COVID-19 pandemic and adjacent time periods to evaluate the impact of data drift on model performance. Design This prognostic study used population-level administrative health data to develop ML prediction models.

Setting Alberta, Canada during 2019 - 2023. Participants All patients over 18 who received at least one opioid dispensation from a community pharmacy within the province of Alberta between 2019 - 2023.

Exposure Each opioid dispensation served as the unit-of-analysis. Main outcomes/measures Opioid-related outcomes were identified from linked health administrative datasets.

Light Gradient Boosting-machine models were developed on pre-pandemic, pandemic and endemic data and temporally validated on 2023 data (pre-pandemic model was also validated on 2020 - 2021 data) to predict the risk of emergency department visit, hospitalisation or mortality within 30-days of an opioid dispensation. We described key feature distributions across the study time period and changes in model prediction performance on the validation sets using relevant metrics.