24 Human-AI collaboration in brain tumour assessments improves both human and AI agent performance

Vollständiger Abstract

Worum geht es in dieser Arbeit?

Abstract Introduction The benefits of artificial intelligence (AI) human partnerships—evaluating how AI agents enhance expert human performance—are increasingly studied. Though rarely evaluated in healthcare, an inverse approach is possible: AI benefiting from the support of an expert human agent. Brain tumour MRI characterisation, a technically challenging and specialist process which often necessitates gadolinium contrast administration, offers an ideal setting to evaluate such partnerships. Methods Here, we investigate both human-AI clinical partnership paradigms in the MRI-guided characterisation of patients with brain tumours. We utilised data from 11,089 patients with brain tumours across 10 datasets and 4 countries, evaluating 1,109 held-out test cases spanning 5 neuro-oncology categories: glioma, meningioma, metastases, paediatric lesions, and post-surgical appearances. Eleven consultant neuroradiologists across six London hospitals participated in a multi-reader, randomised crossover study. The task was to predict post-contrast tumour from pre-contrast MRI sequences alone, with and without the corresponding agent. Results We reveal that human-AI partnerships improve accuracy and metacognitive ability not only for radiologists supported by AI, but also for AI agents supported by radiologists. AI supported by radiologists achieved the highest balanced accuracy (0.840), followed by AI alone (0.824), radiologists with AI support (0.743), and radiologists alone (0.698). Inter-rater agreement significantly improved with agent support (Cohen’s kappa 0.338 to 0.484; p < 0.0001). The greatest patient benefit was evident with an AI agent supported by a human one. Synergistic improvements in agent accuracy, metacognitive performance, and inter-rater agreement suggest that AI can create more capable, confident, and consistent clinical agents, whether human or model-based. Conclusions Our work suggests that the maximal value of AI in neuro-oncology and beyond could emerge not from replacing human intelligence, but from AI agents that leverage and amplify it.