Clinical temporal relation extraction with long-context transformers: a robustness study on MIMIC-III and MIMIC-IV

Vollständiger Abstract

Worum geht es in dieser Arbeit?

Introduction Temporal relation extraction from electronic health records is important for patient timeline construction and longitudinal clinical analysis, yet remains challenging because temporally relevant evidence may be distributed across extended clinical narratives. Methods This study investigated whether long-context clinical transformers offer practical advantages for rule-based event-pair temporal relation classification across matched and transfer-oriented evaluation settings. Event-pair datasets were constructed from MIMIC-III and MIMIC-IV discharge summaries, with temporal labels assigned using a deterministic rule-based pipeline. The task was formulated as a four-class classification problem using BEFORE, AFTER, OVERLAP, and NONE labels. Clinical-Longformer and Clinical-BigBird were evaluated as primary long-context models, while BERT-base, BioBERT, Bio_ClinicalBERT and BiomedBERT served as standard-context baselines. To address GPU memory constraints during 4,096-token fine-tuning, a progressive two-stage strategy was used, in which long-context models were first fine-tuned at 1,024 tokens and then continued at 4,096 tokens. Results Experiments covered in-domain, zero-shot inter-dataset, and adaptation-based transfer settings. Clinical-BigBird achieved the best MIMIC-III in-domain macro-F1 of 93.85% and the strongest performance in both adaptation settings. Clinical-Longformer achieved the best MIMIC-IV in-domain macro-F1 of 97.32% and the strongest zero-shot MIMIC-IV-to-MIMIC-III macro-F1 of 92.36%. BERT-base and BioBERT reproduced the same directional transfer pattern, with stronger MIMIC-III-to-MIMIC-IV zero-shot results and improvements after adaptation. The 4,096-token continuation stage consistently outperformed 1,024-token training alone. Physician review confirmed 160 of 200 rule-derived labels as correct, corresponding to 80.0% agreement. Statistical testing supported selected improvements, although not all pairwise differences were significant. Conclusion Long-context modeling was particularly useful for improving robustness to dataset shift and target-domain adaptation in clinical temporal relation extraction. However, the findings should be interpreted in the context of rule-derived silver labels.