Extending Epuskesmas With A Domain-Specific Large Language Model For Orthopaedic Documentation of Osteoarthritis And Osteoporosis: A Proof-Of-Concept Study Toward Indonesian Primary Health Care Strengthening

Vollständiger Abstract

Worum geht es in dieser Arbeit?

Background. Clinical documentation is a major driver of workload in primary care, and Indonesia's mandated transition to electronic medical records has increased the recording burden on community health centres (Puskesmas). A recent proof-of-concept study showed that a browser-based pipeline combining automatic speech recognition (ASR) with large language model (LLM) summarisation can convert Bahasa Indonesia doctor-patient conversations into ePuskesmas fields for general primary care. Musculoskeletal complaints, particularly knee osteoarthritis and osteoporosis, are common, disabling and largely manageable at primary level, yet they require documentation elements that a general-purpose template does not explicitly capture. Objective. To develop a domain-specific orthopaedic extension of the ePuskesmas LLM documentation framework and to evaluate the clinical adequacy of its generated documentation through independent clinician review. Methods. Six scripted roleplay consultations covering knee osteoarthritis and osteoporosis-related conditions were recorded in Bahasa Indonesia, transcribed with a Whisper model, and summarised with an LLM using an orthopaedic-specific prompt mapped to ePuskesmas fields. Transcripts and structured outputs were assessed independently by three practising clinicians (two general practitioners working in Puskesmas and one orthopaedic specialist) against a seven-domain rubric scored 1-5, yielding 126 ratings. Inter-rater reliability was quantified using the intraclass correlation coefficient (ICC). Results. Across 126 ratings the overall mean was 3.29 of 5 (range 2-5), corresponding to the rubric anchor acceptable (usable after moderate editing). Agreement between reviewers was high: ICC(2,k) = 0.96, with all three reviewers assigning an identical score on 73.8% of items and agreeing to within one scale point on every item. Domain means were highest for appropriateness of referral recommendation (4.00) and appropriateness of Puskesmas-level management (3.83), and intermediate for red-flag identification (3.44). Scenario means ranged from 4.19 (suspected fragility fracture) to 2.57 (moderate knee osteoarthritis with obesity and gastritis). Qualitative review identified one materially unsafe analgesic recommendation, systematic loss of pertinent negative findings, incomplete transfer of examination detail, over-triage of one chronic case and conflation of fracture risk with established diagnosis. Conclusion. A domain-specific ePuskesmas LLM is technically feasible and produces documentation drafts of acceptable but not clinically final quality. Performance was adequate for management and referral recommendations, whereas fidelity to the source consultation was the limiting factor.