Authors (including presenting author) :
Lau S(1), Yu MS(1), Kwok PH(1), Man YL(1)
Affiliation :
(1) Department of Occupational Therapy , Pok Oi Hospital
Keyword 1: :
Occupational Therapy
Keyword 2: :
Outreach service
Keyword 3: :
Documentation audit
Introduction :
Documentation auditing in Occupational Therapy (OT) is fundamental to maintaining patient record integrity and reliability. Regular audits enhance accuracy, identify discrepancies, and drive continuous quality improvement while ensuring regulatory compliance and strengthening interdisciplinary communication—ultimately improving patient outcomes and safety. In response to healthcare evolution, the New Territories West Cluster (NTWC) has implemented electronic documentation systems as part of the Hospital Authority's paperless initiative. This transition has significantly improved patient data accessibility, enabling efficient retrieval and sharing of information for clinical and administrative purposes. Concurrently, artificial intelligence technologies through HAgenAI tools present transformative opportunities for documentation audits. AI can streamline data processing while ensuring accuracy by rapidly identifying patterns, trends, and anomalies. This pilot project explored an innovative approach incorporating HAgenAI tools complementary to human input for auditing clinical notes within a community OT team.
Objectives :
The pilot trial aimed to assess the quality and completeness of community OT progress notes for May–December 2025 using HAgenAI tools integrated with human analysis. Additionally, the trial sought to explore AI-assisted auditing feasibility and identify advantages over traditional human-only methods.
Methodology :
Researchers randomly selected 30 cases stratified across three key service areas: IDSP-Expansion, ICM-CM, and COT (10 cases each), ensuring representative sampling of community practice. From these cases, 107 progress notes were extracted from the Clinical Management System, with all personally identifiable information removed to ensure patient privacy and data security. Two independent team members conducted the audit following formal education on 12 standardized criteria: date accuracy, referral source, reason for referral, clinical information, assessment findings, goals and objectives, OT intervention details, patient progress, management plans, legibility, appropriate abbreviation use, and electronic signature compliance. Each criterion received one of three scores: Pass (fully met), Partial Pass (partially met or unclear), or Fail (not met). The AI tool (DeepSeek R1) performed pre-screening tasks such as extracting dates and abbreviations and flagging items for manual review. Human auditors retained final scoring authority, with AI outputs compared against human judgments to evaluate concordance and utility.
Result & Outcome :
The audit revealed positive outcomes, with most notes achieving Pass ratings across all 12 criteria. Partial compliance instances were minimal: Criterion 10 (Legibility) showed one partial pass due to typographical errors, though the entry remained clinically comprehensible; Criterion 11 (Abbreviations) showed two partial passes containing unlisted but harmless abbreviations. Notably, 100% fulfillment was achieved for Criteria 1–9 and 12, reflecting exceptional consistency in clinical reasoning and administrative compliance. This pilot successfully demonstrated an effective audit approach combining AI capabilities with human judgment. AI excelled at rapidly processing extensive electronic records, freeing human auditors to concentrate on complex judgment calls and strategic oversight. The collaboration enhanced workforce development, with AI identifying documentation patterns and common errors to provide training insights for new therapists. Given HAgenAI accessibility, administrative personnel can manage routine auditing tasks, allowing frontline therapists to prioritize patient care. Crucially, AI functioned as a safety net, catching inconsistencies that might escape manual assessment. The result is a hybrid model where technology handles initial screening while humans retain final decision-making authority—a framework with significant implications for future documentation audit practices.
