The efficacy of twelve early warning systems for potential use in regional medical facilities in Queensland, Australia

Aim: Early warning system (EWS) validation studies are conducted predominantly in tertiary metropolitan facilities and are not necessarily applicable to regional hospitals. This study evaluates 12 EWSs for use in regional subcritical hospitals. Method: This is a retrospective case–control study of patients who experienced severe adverse events (SAEs) in two regional private hospitals. Vital signs collected over 72 h preceding the SAE were applied to 12 EWSs representing three classes of EWSs. The EWS area under the receiver operator characteristic curve (AUROC), sensitivity, specificity, and number of alerts were calculated. Results: Data from 159 index and 172 control patients showed no significant differences in demographics, length of stay, and level of comorbidities. Only half of index patients achieved a medical emergency alert threshold score. On average, index patients triggered alerts 20.06 (22.67) hours preceding the SAE and alerted 2.25 (3.87) times over 72 h. The AUROC ranged from 0.628 to 0.747, with a single-parameter EWS having the lowest AUROC and an aggregated weighted EWS, the highest. The sensitivity of the EWS ranges from 0.359 to 0.692. The specificity was greater than 0.9 for all the EWSs tested. Conclusions: Based on the EWS sensitivity and AUROC, there is a lack of conclusive evidence of the efficacy of the 12 EWSs tested. However, because the adoption of the EWS in Australian hospitals is mandatory, the implementation of an aggregated weighted EWS, such as Compass, should be considered in subcritical regional private hospitals. Given that only half of SAE achieved an EWS medical alert threshold score, it is important that good clinical judgement be used with EWS. © 2019 Australian College of Critical Care Nurses Ltd