Lessons To Be Learned From England About The Potential of GP Computer Systems to Improve Patient Safety
In December 2003, the National Patient Safety Agency in England published a report entitled "Realizing the Potential of GP Computer Systems to Improve Patient Safety." The project team that generated the report was led by Professor Anthony Avery, Head of the Division of Primary Care, School of Community Health Sciences, at the University of Nottingham. The project team also included researchers from the University of Manchester and the University of Edinburgh.
The four-part study had the following objectives:
- To identify the most important safety issues regarding GP computer systems
- To assess GP computer systems in terms of these safety features
- To determine GPs' knowledge, usage and training needs in relation to computerized safety features
- To work with stakeholders to produce specifications for GP computer suppliers and for training practice staff
Computers have been used in general practice in England since the 1970s. The real growth in general practice computing occurred in the early 1990s, primarily due to:
- In 1991, the introduction of the new GP contract and the publication of the "Health of the Nation" report with its emphasis on information collection and analysis, particularly in relation to health-promotion targets
- In 1990, the agreement by the Department of Health to reimburse part of the cost of purchasing and maintaining computers in general practice
As a result, today more than 95% of GP practices in England are automated. The most commonly used clinical application is entering medication prescriptions; these are printed and carried to the pharmacy by the patient. Many English GPs use their systems to maintain an EMR; some of the practices are paperless.
PART 1 - STAKEHOLDER INTERVIEWS
Twenty-six stakeholder interviews were carried out with GPs, GP computer-system suppliers, drug database suppliers, experts in academic health informatics, the Royal College of General Practitioners, the Department of Health, the NHS Information Authority, the NHS Design Authority, the medical defence organizations, patients' representatives and experts in health informatics.
The interviews were deliberately unstructured in order to capture stakeholders' views on key issues concerning the use of GP computer systems to help ensure patient safety. The interviews were audiotaped and then transcribed for qualitative analysis. The interviews were transcribed and coded using qualitative techniques and grounded theory in order to find important concepts and themes.
The stakeholder interviews identified a number of important themes relevant to improving the safety of GP computer systems.
- Many stakeholders stated that it was important to have a single drug dictionary for UK general practice, pointing out that the structure of the clinical record was so different between some of the systems that it would probably not be possible to preserve all information or the links between pieces of information transferred to different systems.
- The development of drug ontologies for use in the NHS was thought to be important in terms of enhancing patient safety.
- There is a need to ensure that users record data so that functionality is available when required. The stakeholders pointed out that the way in which doctors record morbidity data on their computers is inconsistent, and therefore one could not rely on these data generating alerts. Many stakeholders argued for the need to train GPs to record data in a more comprehensive and consistent manner.
- Accurate and safe information is required for decision-making. Decision support should be capable of providing users with relevant information at the right time in order to prompt them to take the right actions. Many suppliers do not appear to make full use of the functionality available on their drug databases.
- Patient referrals and medication monitoring emerged as the fifth theme. Practitioners need to make the best use of computerized systems for ensuring that intended actions, such as patient referrals and medication monitoring, are completed. It was suggested that all systems had facilities for call and recall and reporting, but that it required specialist knowledge within the practice in order to set up these systems and maintain them.
- The theme relating to audit trails focused on GPs recording why they had decided to override an important alert. System suppliers suggested that they would be happy to consider introducing audit trails if users supported this idea.
- Human ergonomics also emerged as an important theme. For instance, there are empirically established effects of the colour, shape, font, position, effect of signal words (such as danger, caution, warning, and so on) in the design of alerts and many other aspects of computer systems. Therefore, knowledge about human factors needs to be incorporated when designing the computer systems.
- The eighth theme related to training. Even for well-designed systems, the clear majority of stakeholders felt that GPs and practice staff needed training in the effective use of safety features. For example, in terms of record-keeping, potential problems were noted with GPs re-cording inadequate information if they were not properly trained in the correct use of the clinical coding system.
- The issue of regulations and guidelines came up as a theme in which some stakeholders pointed out that it is important to specify in clear and unambiguous terms what computer systems suppliers need to do. Others argued that suppliers would be more likely to make changes if mandated to do this rather than if changes were only recommended.
- Finally, some stakeholders identified safety culture as an important theme. Particular emphasis was placed on the managing changes in safety. It was also felt that by developing safety culture, GPs might be more aware of the risks of not making best use of their computer systems.
PART 2 - DELPHI EXERCISE
In order to reach consensus among experts on the most important issues regarding the safety of GP computer systems, a two-stage modified Delphi technique was used with an expert panel of 22 members from different primary-care, academic, government and business establishments throughout the UK. The professional background of the panel members was in general practice (15), pharmacy (6) and primary-care management (1).
After reviewing the research literature, a number of issues were identified as being important for safety when using computer systems. A questionnaire with 55 statements, using a five-point Likert scale for scoring, was circulated by e-mail to panel members in two rounds.
There was 90% or greater agreement for 32 (58%) of the statements, with a median score for these statements of 5 ("very important") and an average agreement of 97.5%. The agreement was below 90% for 23 statements; the average level of agreement for these statements was 71%. The agreement was below 50% for only three statements, which were related to the quantity of medication supplied that might be dangerous in overdose, unlicensed drug indications and requests that the prescriber state the expected length of time between prescription requests.
The following statements are examples of those which had 100% agreement:
"When selecting a drug, the computer should alert the prescriber if …"
- The drug is contraindicated because of a previous allergy, for example, penicillin.
- The drug is contraindicated because of a recorded diagnosis of renal impairment, for example ,oxytetracycline.
- The drug is contraindicated (or should be used with caution) because of a recorded history of peptic ulcer, for example, non-selective NSAIDs.
- The drug is contraindicated because of a recorded diagnosis of asthma, for example, non-cardioselective beta-blockers such as propranolol.
- There is a clinically significant potential interaction with another drug that the patient has recently been prescribed, for example, sildenafil and nitrates.
- The frequency of dose may be inappropriate and dangerous, for example, methotrexate prescribed daily.
Other examples of statements with 100% agreement are:
- When a patient requests a repeat prescription, the computer should make it clear whether the item requested has been authorized as a repeat.
- When practice staff try to print out a repeat prescription, it should be clear whether the item requested has gone beyond its review date.
- Hazard alerts need to be designed so that they do not provide clinicians with irrelevant or spurious information.
- Hazard alert messages should be displayed clearly on the computer screen.
- It should be difficult to override alerts if there is a risk of serious patient harm as a consequence - for example, prescription of sildenafil with a nitrate.
- If a clinician decides to override a clinically relevant hazard alert, he or she should be prompted to record a reason for this.
- When selecting a drug from a drop-down menu, there should be mechanisms in place to make it difficult to erroneously issue a drug with a similar name if this could cause serious patient harm - for example, penicillin and penicillamine.
The key findings emerging from the Delphi exercise were:
- The importance of computerized alerts
- The need to avoid spurious alerts
- The issue of making it difficult to override critical alerts, and creating audit trails of such overrides
- Support for safe repeat prescribing
- Effective computer-user interface
- The importance of call and recall management
- The need to be able to run safety reports
PART 3 - ASSESSING GP COMPUTER SYSTEMS: SIMULATED TEST CASES
The aim of this part of the study was to document how current GP computer systems perform in terms of their functionality in relation to issues identified by the Delphi exercise. Evaluation of clinical systems in general practice is a contentious issue because of commercial sensitivities. The Computer Misuse Act 1990 in the UK, which prevents any form of structural testing, also limits ap-proaches to evaluation. Therefore, the design of this evaluation was confined to what a user is legitimately allowed to do, or, as it is also known, a "black-box" approach.
The 32 statements found most important in the Delphi exercise were used to develop 18 case scenarios, which were tested using dummy patient records on the four most commonly used GP computing systems. The four systems (labelled A, B, C and D to preserve suppliers' anonymity) were independently evaluated at Primary Care Information Services (PRIMIS) laboratories in May 2003. In order to minimize risk of bias, systems were tested with each of the case scenarios in random order, and data were recorded onto pre-piloted data-extraction sheets.
The standards against which the four computing systems were to be evaluated were defined a priori. These included, for example, appropriate alerts when con-traindicated drugs and/or hazardous drug-drug combinations were prescribed. For each scenario, the safety profile of the computing system was categorized as being either "appropriate" or "inappropriate." Evaluators then compared findings and had available a mechanism for resolving disagreement should this arise. Finally, the problems identified were fed back to manufacturers, who were invited to comment in order to ensure that there were no technical problems during setup of the dummy cases that could have accounted for any observed failures. There were no discrepancies in assessing the safety of systems between the evaluators. Also, each of the four system suppliers agreed with the outcome of the assessments.
None of the systems produced alerts for all of the 18 scenarios. System B produced appropriate alerts in 39% of test cases (7/18), while systems A and C responded appropriately in 22% (4/18) of cases and System D in 17% (3/18) of cases. Table 1 reveals the results of the 18 scenarios.
The key findings from the assessment of GP computer systems using simulated test cases were:
- A lack of alerts in relation to contraindications - for example, there was no warning of the risk of Reye's syndrome when prescribing aspirin to an eight-year-old child
- Spurious alerts - for example, a serious alert warning was given for a commonly used and relatively safe drug-drug combination
- Failures of drug allergy warnings - depending on how the allergy history had been recorded (i.e, which Read codes had been used), warnings might or might not be displayed
- Risks of prescribing drugs with similar names - particularly with penicillin (frequently used) and penicillamine (rarely used and likely to do harm to some patients)
- Lack of warning for inappropriate dosages - for example, trying to prescribe methotrexate daily instead of weekly
- It is easy to override most alerts
- Lack of audit trails of when such alerts have been overridden.
- "Hidden" alerts for an important hazard
PART 4 - DETERMINING GPS' KNOWLEDGE, USAGE AND TRAINING NEEDS
A questionnaire survey was conducted to further explore safety issues related to GP computer systems from a user perspective. To inform the survey development, semi-structured interviews with five GPs from the Nottingham area were conducted. This helped ensure that the questionnaire covered issues relevant to practising GPs. The GPs were selected purposefully for their contrasting knowledge, experience and attitudes toward the use of GP computer systems. The interview schedule was designed to explore the most important issues regarding knowledge. Usage and training needs related to the safety of GP computer systems and perceived barriers to the use of computers to help improve patient safety.
The survey was undertaken in contrasting areas of England (Nottingham, central Manchester and Stockport). The questionnaires were sent to a total of 609 GPs, together with a personally signed covering letter and a reply-paid envelope. Two postal reminders (including another copy of the questionnaire) were sent to non-responding GPs. Out of 609 questionnaires sent, 387 valid questionnaires were returned (64%).
There was a strong sense that GPs have come to rely on their computers to provide alerts. More than 90% of GPs regarded computerized alerts (including contraindication alerts) and systems for recall for patient monitoring to be important. The vast majority also agreed with the idea of making it more difficult to override critical alerts. The survey revealed that some GPs are not fully aware of the safety features on their computer systems and only a minority has had training on the use of these safety features.
The preferred method for learning more about the use of safety features was hands-on learning with tuition (either one-on-one or in a group setting). It was also found that more than 90% of GPs would strongly favour "the computer system that was best able to support safe clinical practice" if purchasing a new system, and more than two-thirds of GPs would be willing to pay more for "the system that was best able to support safe clinical practice."
|1||Aspirin prescribed for a child of eight years||N||N||N||N|
|2||Methotrexate prescribed in pregnancy||N||Y||N||N|
|3||Penicillin prescribed in a patient with penicillin allergy||N||Y||Y||Y|
|4||Oxytetracycline prescribed in a patient with renal impairment||N||Y||N||N|
|5||Enalapril prescribed in patient with renal impairment||N||N||N||N|
|6||Microgynon 30 (a combined oral contraceptive pill)
prescribed in a patient with history of DVT
|7||Oxytetracycline prescribed in a patient with a serum creatinine of 160 mmol/l||N||N||N||N|
|8||Propranolol prescribed in a patient with a history of heart failure||N||N||N||N|
|9||Sumatriptan prescribed in a patient with a history of coronary heart disease||N||N||N||N|
|10||Naproxen prescribed in a patient with a history of peptic ulcer disease||N||N||N||N|
|11||Propranolol prescribed in a patient with a history of asthma||N||N||N||N|
|12||Sildenafil prescribed to a patients already taking isosorbide mononitrate||Y||Y||Y||Y|
|13||Methotrexate prescribed on a daily basis||N||Y||N||N|
|14||Salbutamol inhaler marked as a repeat item||Y||Y||N||N|
|15||Repeat prescription of salbutamol inhaler issued before it is scheduled||Y||N||N||N|
|16||Atenolol prescribed to a patient taking amlodipine*||Y||Y||N*||N*|
|17||Amoxicillin prescribed to a patient taking hormone replacement therapy*||N*||N*||N*||Y
|18||The 10 most frequently used drug pairs with similar names**||N||N||N||N|
|Total of appropriate alerts||4||7||4||3|
|*In these situations "No" was the appropriate outcome since these interactions are
clinically irrelevant (but appear as spurious alerts on some systems)
|**"No" was recorded if systems failed to warn prescribers about all of these drugs with similar names|
The English study uncovered clinically important deficiencies in the safety features of the GP computing systems currently in use in approximately three-quarters of GP practices in the UK. In the opinion of the researchers, all of them may fail to warn in situations when a warning might be expected, thus potentially creating a health hazard to patients.
The study concluded that while it is clear that using computers in general practice has many advantages, it is also important to recognize that there may be potential hazardous situations associated with their use. In particular, over-reliance on the system may lead to situations where GPs may not be warned about particular hazards in situations when they may expect to be warned. Examples of such situations include prescribing medication when it is contraindicated due to age (e.g. aspirin in children) or a preexisting health problem (e.g. propranolol in asthma). The researchers found that GPs are being warned on some occasions about the prescription of drugs with similar names (e.g. warning with penicillin/penicillamine or Clobetasol/clobetasone) but this is not always the case (e.g. no warning with Noriday/Norimin or carbamazepine/carbimazole). Problems were also identified in relation to repeat prescribing, including over-prescribing of medications and the ability to over-ride warnings related to laboratory results.
A particular problem was found with one system where in an important alert message (associated with prescribing a betablocker to a patient with asthma) was hidden behind a small red block at the bottom of the screen. It was felt that this alert box was too small and in the wrong position on the screen for a prescription that is contraindicated by the Committee on Safety of Medicines. Also, the alert might have been undetectable to anyone with red-green colour blindness.
There is much to be learned as our Canadian GPs gradually begin to introduce computer technology into their office practices. One wonders if the impending CIHI patient safety report will accelerate the rate at which EMRs - with built-in decision-support capabilities - will be implemented in Canadian GP practices.
About the Author(s)
Denis Protti is Professor of Health Informatics at the University of Victoria, British Columbia.
AcknowledgmentThe author wishes to thank Dr. Maureen Baker, Director of Primary Care for the National Patient Safety Agency (www.npsa.nhs.uk) based in London, and Professor Anthony Avery, Head of the Division of Primary Care, School of Community Health Sciences at the University of Nottingham, for permission to use the report as the basis of this paper.
Avery, A. J. 1998. "The Use of Computers to Aid GP Prescribing." Prescriber 9 (21): 13.
Brahams, D., and J. Wyatt. 1989. Decision Aids and the Law. Clinical Knowledge and Practice in the Information Age: A Handbook for Health Professionals. London: Royal Society of Medicine Press Limited.
Cantrill, J.A., B. Sibbald and S. Buetow. 1998. "Indicators of the Appropriateness of Long-Term Prescribing in General Practice in the United Kingdom: Consensus Development, Face and Content Validity, Feasibility and Reliability." Qual Health Care 7(3): 130-5.
Franke, L., A. J. Avery, L. Groom and P. Horsfield. 2000. "Is There a Role for Computerized Decision Support for Drug Dosing in General Practice? A Questionnaire Survey." Journal of Clinical Pharmacy & Therapeutics 25: 373-377.
Gray, J., D. Orr and A. Majeed. 2003. "Use of Read Codes in Diabetes Management in a South London Primary Care Group: Implications for Establishing Disease Registers." British Medical Journal 326: 1130-2.
Magnus, D., S. Rodgers and A. J. Avery. 2002. "GPs' Views on Computerized Drug Interaction Alerts: Questionnaire Survey." J Clin Pharm Ther 27(5): 377-82.
RCGP. 2003. General Practice Computerisation. Royal College of General Practitioners Information Sheet No. 7, March. Available at http://www.rcgp.org.uk/rcgp/information/publications/information/ rcf0007/Rcf0007_2003.asp. Accessed October 2003.
Sheikh, A., B. Hurwitz, R. Ashcroft, L. Smeaton and M. Holmes. 2002. "Prescribing Errors in General Practice: Qualitative Questionnaire Study." Published conference abstract from WONCA.
Wyatt, J. 2001. Clinical Knowledge and Practice in the Information Age: A Handbook for Health Professionals. London: Royal Society of Medicine Press Limited.
Be the first to comment on this!
Personal Subscriber? Sign In
Note: Please enter a display name. Your email address will not be publically displayed