What to measure for monitoring and decision-making in a case management intervention for emergency department high-users: different stakeholder priorities

Vollständiger Abstract

Worum geht es in dieser Arbeit?

Abstract Background Multidisciplinary case management (CM) for high users (HU) of the emergency department (ED) is a complex intervention involving multiple stakeholders. CM evaluations typically adopt a provider perspective and rely on hospital utilisation-based indicators. However, different stakeholders bear distinct costs, consider different benefits relevant, and face specific constraints. This study aims to identify, compare, and prioritise the evaluation indicators that different stakeholder groups consider most relevant for monitoring and decision-making in a Portuguese HU CM intervention (GRHU), and to assess their measurability using routinely collected hospital data. Methods A participatory action research approach was used, structured around two action-reflection cycles across four sequential steps: (1) identifying candidate indicators via interviews with ED and primary care professionals, and hospital intermediate management; (2) structuring indicators using the Donabedian framework; (3) prioritising indicators with senior organisational stakeholders (administrative board members, ED leadership, and GRHU management); (4) operationalising indicators and assessing their measurability and costing using hospital data. This final step included an economic evaluation calculating a ratio of the observed pre-post cost difference to delivery cost for the highest-priority, measurable, and monetisable outcome indicator, using a 12-month before-and-after window anchored to the patient’s first GRHU appointment. Changes in the outcome indicators were monetised using standard national unit prices and compared with intervention delivery costs, estimated using Time-Driven Activity-Based Costing. Results Twenty-two indicators were defined (9 process, 13 outcome). Priorities differed: board members ( n = 2) prioritised caseload volume; ED leadership ( n = 3) prioritised intervention duration and change in ED visits; and GRHU management ( n = 3) prioritised care plan reassessment frequency and change in ED visits. Only nine indicators were fully measurable using routine data; among these, change in ED visits was the highest-ranked outcome and was used for economic evaluation. In 73 patients, ED visit costs fell by 54.5%, a pre-post difference of €71 622.48; compared with €58 468.51 in delivery costs, this produced a ratio of the observed pre-post ED cost difference to delivery cost of 1.22 over one year. Conclusions Typically, CM evaluations focused solely on the provider perspective provide an incomplete picture of what different stakeholders consider relevant for monitoring and decision-making. Stakeholders prioritise diverse indicators, many not captured by existing health information systems. This gap is consistent with implementation science evidence suggesting that favourable financial results alone may be insufficient to ensure continued organisational engagement. Therefore, effective governance may benefit from multi-stakeholder evaluation frameworks aligned with decision-making priorities and strengthened health data infrastructures.