The effect of computerized physician order entry on medication errors and adverse drug events in pediatric inpatients
W. James King
Medication errors are an internationally recognized source of significant morbidity and mortality in hospitalized patients. In the United States alone, the Institute of Medicine report To Err Is Human: Building a Safer Health System estimated that between 44 000 and 98 000 people die each year from medical errors, whereas in the United Kingdom and Australia adverse events causing patient harm occurred in 10% to 16% of admissions. (1-3) Medication use accounts for at least 20% of adverse events in hospitalized patients. (4) Although the majority of medication errors do not result in patient harm, errors that result in a serious adverse drug event (ADE) are associated with increased length of hospital stay, extra costs, and mortality. (5)
In adults, ADEs occur at a rate of 5 per 100 medication orders. (6) A similar error rate has been observed in children who are at risk for unique medication errors such as large, 10-fold errors in dosing. (7,8) Children are at risk for medication errors attributable to weight-based dosing, off-label drug usage and preparation, limited reserve to withstand a dosing error, and limited ability to communicate with health care personnel to prevent an error or signal that an error has occurred. (9,10) A review of prospective studies of ADEs in hospitalized children noted an overall incidence of 9.5%, with severe reactions accounting for 12% of the total. (11) In 2 academic pediatric hospitals, the medication error rate (MER) was 6 per 100 medication orders, the majority of which occurred during physician medication ordering. (7) Many ADEs are preventable (51% to 93%), as medication errors often occur during drug ordering and may be corrected, especially if the error is detected early in the order process. (5,6,11,12)
The significant MER has focused attention on understanding the causes of ADEs and developing methods to prevent them. Advances in information technology, such as computerized physician order entry (CPOE), hold promise of reducing medication-related errors. CPOE has been given a “medium strength of evidence” rating by the Agency for Healthcare Research and Quality as a safety practice to implement, and the Leapfrog Group is promoting CPOE in hospitals as 1 of 3 changes to improve patient safety in America. (13,14) Despite these recommendations, there has been limited evaluation and publication of the impact of CPOE on ADEs in part because of the methodologic difficulties in trial design. (15) Studies demonstrating CPOE effectiveness have shown a decrease in the rate of serious medication errors but no effect on ADEs. (16-19)
A reduction in the rate of serious medication errors conferred by CPOE would clearly have significant implications for hospitalized children. We assessed the impact of a commercially available CPOE system on medication errors and ADEs in pediatric inpatients at a tertiary care pediatric hospital.
Setting and Study Design
This retrospective cohort study was conducted at the Children’s Hospital of Eastern Ontario (CHEO), a tertiary care pediatric teaching hospital affiliated with the University of Ottawa, Ottawa, Canada. The inpatient units at CHEO average 6000 admissions and 38 000 inpatient days per year.
In March 1996, the CHEO implemented, on 2 medical inpatient units, a commercially available CPOE system developed by Eclipsys. The CPOE system was originally introduced as Carevision and underwent periodic product upgrade and is now commercially available as Sunrise Clinical Manager. Clinical decision sup port was not provided and the CPOE system interfaced with the laboratory system but not with the pharmacy computer. The intervention group consisted of the 2 medical wards on which CPOE was implemented. The control group consisted of 1 medical and 2 surgical wards that continued to use handwritten orders. Each ward unit is clinically independent with respect to nursing staff. Medical patients on the intervention wards are cared for by a team approach consisting of pediatric generalists and subspecialists with assistance from pediatric residents, interns, and medical students. Approximately 30% of the patients cared for by the medical team are admitted off-service to the nonintervention medical and surgical wards.
Medication Error Definition and Surveillance
A medication error was defined as any event involving medication prescription, dispensing, administration, or monitoring of medications irrespective of outcome. Medication errors at CHEO are reported in a standardized fashion by the adverse event reporting system on all inpatient floors. This is a passive reporting system that was constant throughout the duration of the study. After a medication error is detected, an incident report is completed by the nurse and physician involved. The events of the medication error are documented and the severity of patient harm is rated as none, mild, moderate, or severe. The incident report is sent to the Pharmacy Department. Each report is reviewed by the institution Quality Improvement Committee. The medication errors are then entered into a spreadsheet database (Microsoft Excel 97; Microsoft, Redmond, WA). MERs in both the intervention and control groups were analyzed for the 3 years before (April 1, 1993 to March 31, 1996) and 3 years following (January 1, 1997 to December 31, 1999) the implementation of CPOE (Fig 1). From April 1, 1996 to December 31, 1996, following the introduction of CPOE, medication errors were not assessed to allow for the piloting and training of CPOE and a major software upgrade in November 1996.
[FIGURE 1 OMITTED]
Two physicians (N.P., W.J.K.) accessed the medication error database and reviewed all original incident reports. Severity was reclassified based on patient impact as an ADE, potential ADE, or other. An ADE was defined as a medication error resulting in an injury to the patient (example: a 10-fold insulin overdose resulting in hypoglycemia). A potential ADE was defined as a medication error with the potential for patient injury where no actual harm occurred. Twenty random incident reports were independently rated by each of the 2 reviewers with good agreement ([kappa] = 0.64; 95% confidence interval [CI]: 0.45, 0.82). All disagreements were resolved through discussion and consensus.
The total number of patient days and discharges during the study period was obtained from the CHEO discharge database. The CHEO uses Med2020 WinRecs abstracting software to collect inpatient information on discharge from the hospital. This database has a minimum dataset with definitions for each data element to ensure data quality and consistency.
We computed the ratios of rates before and after CPOE implementation, per 1000 patient days for medication error, potential ADE, and ADE. CIs for these rate ratios were computed under the assumption that the number of events per 1000 patient days followed a Poisson distribution. (20) To evaluate the impact of introducing CPOE, we computed the ratio of the rate ratio for the intervention wards and the rate ratio for the control inpatient wards. To provide a clinical interpretation of the effect of CPOE on medication errors, we calculated an analog of the number needed to treat (NNT), representing the expected number of patient days to prevent 1 additional medication error using CPOE. (21) The calculation is as follows: NNT = 1/(B x (1 x R)) where R is the ratio of rates of medication errors in the intervention versus the control group, and B is a baseline rate, which we took to be the rate of medication errors in the control group following the introduction of CPOE. (22)
During the 6-year study period, there were 36 103 discharges and 179 183 patient days. A total of 804 medication errors were identified with 18 adverse events resulting in patient harm, yielding an overall MER of 4.49/1000 patient days (Table 1).
The MERs before the introduction of CPOE for the intervention and control wards were indistinguishable (P value = .50; ratio = 0.93; 95% CI = 0.76-1.13; Table 2). The wards that implemented CPOE experienced a 40% reduction in medication errors when compared with the control wards (P value < .001; ratio = 0.60; 95% CI = 0.48-0.74). This change in rate ratios following the introduction of CPOE is statistically significant (P < .01; ratio of rate ratios = 1.54; 95% CI = 1.27-1.88). Our NNT analog revealed that CPOE would prevent 1 medication error every 490 (95% CI = 382-781) patient days.
Table 2 presents the rate ratios for ADEs and potential ADEs. We did not demonstrate an effect of CPOE implementation on ADE (P value = .6; ratio of rate ratios = 1.30; 95% CI: 0.47-3.52), although the power to detect such an effect was low because of the small number of ADEs (18) reported. We did note a larger decrease in potential ADEs on the control as compared with the intervention wards, which was statistically significant (P value < .001; ratio of rate ratios = 0.24; 95% CI: 0.09-0.68).
To our knowledge, this study is the first evaluation of CPOE on medication errors and ADEs in a pediatric inpatient population. We observed a 40% decrease in the MER on the wards that implemented CPOE; however, we were unable to demonstrate a similar effect on actual or potential patient injury. The lack of effect on patient harm may be explained by insufficient power to demonstrate an effect due to the small number of ADEs observed. Another explanation is that our institution saw a decline in potential ADEs and ADEs in both the intervention (from 11-7) and control wards (from 19-5), perhaps the result of other system changes to reduce medical error, such as ward-based pharmacists.
Despite recommendations to incorporate CPOE into everyday practice and the potential benefit of CPOE in pediatrics, there has been limited evaluation of its effectiveness. Existing studies in adult populations have been performed with “homegrown” systems and have shown a similar beneficial effect on medication errors without a demonstrable decrease in actual patient harm. (16-19) Children are a vulnerable population at risk for medication error from both a physiologic and developmental viewpoint. (23) The rate of medication errors and ADEs for pediatric inpatients are comparable to rates in adults with potential ADEs 3 times more common in children. (7) Thus, as a group, children are at risk for ADE with a need to reduce this risk at least as great as in adults. To place in context the observed decrease in medication errors, at our pediatric tertiary care hospital with 40 000 patient days per year, we saw 1 less medication error every 490 patient days (95% CI: 382-781) and would therefore prevent 82 (95% CI: 51-105) medication errors per year. Although we did not demonstrate an actual decrease in patient harm, all medication errors have the potential to harm a patient. Even at a rate of 1 ADE for every 100 medication errors, with longer exposure to CPOE, we would expect to see a reduction in actual patient injury. (24)
The finding of this study must be interpreted in light of its limitations. We used retrospective cohort study design because a clinical trial in which patients are randomized to receive CPOE would be costly and methodologically difficult to implement. (15) Medication error data were collected prospectively and we included reporting of medication errors before CPOE implementation. The similarity of MERs and ADEs before CPOE implementation (MERs per 1000-patient days, 4.48 vs 4.80) are helpful to assure that the populations were similar with respect to medication error. Another limitation is the passive reporting system for medication error at our institution; however, the inpatient wards were unaware of the study and the policy and procedure for medication error reporting were unchanged and occurred in a standardized fashion throughout the study. Finally, although our MER and number of ADEs are low, they are similar to the rate of medication errors reported in the pediatric literature (0.51-17.7 per 1000 patient days), especially when we did not include medication errors occurring in the pharmacy, emergency department, or intensive care units. (25-27)
The introduction of CPOE is associated with a decrease in the rate of medication error. A similar effect on the more serious but rarer ADE was not discovered. CPOE holds promise as an intervention that may improve patient safety but requires further evaluation of the benefit and costs before widespread adoption.
TABLE 1. Medication Errors and ADEs Before and After
the Introduction of CPOE on Pediatric Inpatient Units
Total inpatients 6674 11 944
Total inpatient days 38 578 50 659
Average patient age (y) 5.5 6.7
Gender: % female 45 41
Medication error type (%)
Prescribing 7 (4) 6 (3)
Transcription 26 (15) 44 (18)
Dispensing 3 (2) 16 (7)
Administration 137 (79) 177 (73)
Total medication errors 173 243
ADEs 6 9
Potential ADEs 5 10
MER/1000 patient days 4.48 4.80
Total inpatients 5786 11 699
Total inpatient days 38 286 51 660
Average patient age (y) 6.0 7.0
Gender: % female 47 41
Medication error type (%)
Prescribing 4 (3) 7 (3)
Transcription 35 (29) 46 (17)
Dispensing 1 (1) 5 (2)
Administration 80 (67) 210 (78)
Total medication errors 120 268
ADEs 1 2
Potential ADEs 6 3
MER/1000 patient days 3.13 5.19
TABLE 2. Intervention Versus Control Rate Ratio and Ratio
of Rate Ratios for ADEs Rate, Potential ADEs, and Medication
Error Before and After CPOE Implementation
Rate Ratio Intervention Versus Control
Before After Rate Ratios
(95% CI) (95% CI) (95% CI); P Value
ADEs 0.89 (-0.32, 0.67 (0.09, 1.30 (0.47,
2.27) 5.11) 3.52); P = .6
Potential 0.66 (0.23, 2.7 (0.74, 0.24 (0.09,
ADEs 1.84) 9.90) 0.68); P < .001
Medication 0.93 (0.76, 0.60 (0.48, 1.54 (1.27,
errors 1.13) 0.74) 1.88); P < .001
ABBREVIATIONS. ADE, adverse drug event; MER, medication error rate; CPOE, computerized physician order entry; CHEO, Children’s Hospital of Eastern Ontario; NNT, number needed to treat; CI, confidence interval.
We thank Nicholas J. Barrowman, PhD, for his statistical assistance and Alex E. MacKenzie, MD, PhD, for his review of the manuscript.
(1.) To Err Is Human: Building a Safer Health System. Washington, DC: National Academy Press; 1999
(2.) An organisation with a memory. Report of an expert group on learning from adverse events in the NHS Department of Health. 2000. Available at: http://www.doh.gov.uk/cmo/orgmem.pdf. Accessed September 26, 2002
(3.) Wilson RM, Runciman WB, Gibberd RW, et al. The quality in Australian health care study. Med J Aust. 1995;163:458-471
(4.) Leape LL, Brennan TA, Laird N, et al. The nature of adverse events in hospitalized patients: results from the Harvard Medical Practice Study II. N Engl J Med. 1991;342:377-384
(5.) Classen DC, Pestotnik SL, Evans RS, et al. Adverse drug events in hospitalized patients. Excess length of stay, extra costs, and attributable mortality. JAMA. 1997;277:301-306
(6.) Bates DW, Boyle DL, Vander Vliet MB, et al. Relationship between medication errors and adverse drug events. J Gen Intern Med. 1995;10: 199-205
(7.) Kaushal R, Bates D, Landrigan C, et al. Medication errors and adverse drug events in pediatric inpatients. JAMA. 2001;285:2114-2120
(8.) Kozer E, Scolnik D, Keays T, et al. Large errors in the dosing of medications for children. N Engl J Med. 2002;346:1175-1176
(9.) Conroy S, Choonara I, Impicciatore P, et al. Unlicensed and off-label drug use in pediatric wards in different European countries. BMJ. 2000;320:79-82
(10.) Levine SR, Cohen MR, Blanchard NR, et al. Guidelines for preventing medication errors in pediatrics. J Pediatr Pharmacol Ther. 2001;6:426-442
(11.) Impicciatore P, Choonara I, Clarkson A, et al. Incidence of adverse drug reactions in pediatric in/out-patients: a systematic review and meta-analysis of prospective studies. Br J Clin Pharmacol 2001;52:77-83
(12.) Bates DW, Cullen DJ, Laird N, Petersen LA, et al. Incidence of adverse drug events and potential adverse drug events: implications for prevention. JAMA. 1995;274:29-34
(13.) Shojania K, Duncan B, MacDonald K, Wachter RM, eds. Making Health Care Safer: A Critical Analysis of Patient Safely Practices. Rockville, MD: Agency for Healthcare Research and Quality; 2001. Evidence Report/ Technology Assessment No. 43; AHRQ Publ. No. 01-E058
(14.) Milstein A, Galvin KS, Delbanco SF, et al. Improving the safety of health care: the leapfrog initiative. Eff Clin Pract. 2000;3:313-316
(15.) Leape LL, Berwick DM, Bates DW. What practices will most improve safety? Evidence-based medicine meets patient safety. JAMA. 2002;288: 501-507
(16.) Overhage JM, Tierney WM, Zhou XH, McDonald CJ. A randomized trial of “corollary orders” to prevent errors of omission. J Am Med Inform Assoc. 1997;4:364-375
(17.) Bates DW, Leape LL, Cullen DJ, et al. Effect of computerized physician order entry and a team intervention on prevention of serious medication errors. JAMA. 1998;280:1311-1316
(18.) Bates DW, Teich JM, Lee J, et al. The impact of computerized physician order entry on medication error prevention. J Am Med Inform Assoc. 1999;6:313-321
(19.) Teich JM, Merchia PR, Schmiz JL, et al. Effects of computerized physician order entry on prescribing practices. Arch Intern Med. 2000;160: 2741-2747
(20.) Hasselblad V, McCrory DC. Meta-analytic tools for medical decision making: a practical guide. Med Decis Making. 1995;15:81-96
(21.) Laupacis A, Sackett DL, Roberts RS. An assessment of clinically useful measures of the consequences of treatment. N Engl J Med. 1988;318: 1728-1733
(22.) Deeks JJ, Altman DG, Bradburn MJ. Statistical methods for examining heterogeneity and combining results from several studies in meta-analysis. In: Egger M, Davey Smith G, Altman DG, eds. Systematic Reviews in Health Care. Meta-analysis in Contert. 2nd ed. London, United Kingdom: BMJ Books; 2001:285-312
(23.) Choonara I, Gill A, Nunn A. Drug toxicity and surveillance in children. Br J Clio Pharmacol. 1996;42:407-410
(24.) Barker KN, Flynn EA, Pepper GA, et al. Medication errors observed in 36 health care facilities. Arch Intern Med. 2002;162:1897-1903
(25.) Folli HL, Poole RL, Benitz WE, Russo JC. Medication error prevention by clinical pharmacists in two children’s hospitals. Pediatrics. 1987;79: 718-722
(26.) Ross LM, Wallace J, Paton JY. Medication errors in a pediatric teaching hospital in the UK: five years operational experience. Arch Dis Child. 2000;83:492-497
(27.) Vincer MJ, Murray JM, Yuill A, Allen AC, Evans JR, Stinson DA. Drugs error and incidents in a neonatal intensive care unit. A quality assurance activity. Am J Dis Child. 1989;143:737-740
W. James King, MSc, MD * ([double dagger]); Naomi Paice, MD *; Jagadish Rangrej, MMath ([double dagger]) ([section]); Gregory J. Forestell, MHA ([parallel]); and Ron Swartz, BScPharm ([paragraph])
From the * Department of Pediatrics, Children’s Hospital of Eastern Ontario, Ottawa, Ontario, Canada; ([double dagger]) Children’s Hospital of Eastern Ontario Research Institute, University of Ottawa, Ottawa, Ontario, Canada; ([section]) Chalmers Research Group, University of Ottawa, Ottawa, Ontario, Canada; ([parallel]) Management Information Services Children’s Hospital of Eastern Ontario, Ottawa, Ontario, Canada; and ([paragraph]) Pharmacy Department, Children’s Hospital of Eastern Ontario, Ottawa, Ontario, Canada.
This work was presented in part at a meeting of the Pediatric Academic Society; May 2002; Baltimore, MD
Received for publication Dec 10, 2002; accepted Feb 21, 2003.
Reprint requests to (W.J.K.) Division of Pediatric Medicine, Children’s Hospital of Eastern Ontario, 401 Smyth Rd, Ottawa, Ontario, Canada K1H 8L1. E-mail: firstname.lastname@example.org ca
COPYRIGHT 2003 American Academy of Pediatrics
COPYRIGHT 2003 Gale Group