Do Local Authorities Benchmark Fairly? Using Machine Learning to Develop a Model of Nearest Neighbours to Improve Benchmarking

Background
To improve services for their population, Local Authorities (LAs) require a method of comparing themselves to other, similar local authorities. A commonly used model, one used by Public Health England, is the CIPFA Nearest Neighbours (NNs) model. This research explores whether defining NNs using machine learning (ML) yields similar results to the aforementioned model; and examine selected health outcomes in Coventry, benchmarking against NNs found using the ML method, should they be different.

Methods
A dataset was curated to include eighteen variables, matched to those used in the CIPFA model, alongside proxy variables where necessary. The optimal number of clusters were determined, after which unsupervised ML by k-medoid clustering was employed to identify similar local authorities. Selected health outcomes for Coventry were then compared between clusters using Kruskal-Wallis and Bonferroni-corrected Mann-Witney tests.

Results
Of the ten indicators compared, seven had the same results between the two models, whilst three had different results between the models. The results of this analysis demonstrate that using ML to identify NNs for LAs, compared to the CIPFA model does yield differences that are statistically significant for some of the health outcomes used in this comparison.

Key Messages

Given that the two models largely agree with one another provides some reassurance that there is potential for the ML method to be “trusted” and become increasingly acceptable. The significance of identifying these differences is that they encourage further research in the area of using ML methods for benchmarking purposes, in the context of public health.