There is empirical evidence to support the use of standardized order sets. One before-and-after study that implemented an emergency department standard order set for septic shock was associated with statistically more rigorous fluid resuscitation of patients, greater administration of appropriate initial antibiotic treatment and a lower 28-day mortality (Micek et al. 2006). The implementation of best practice is increased with the use of standardized order sets (O'Connor et al. 2005; Ozdas et al. 2006; Santolin and Boyer 2004).

Order sets should be current, accurate, comprehensive and clinically intelligent. They coordinate the actions of the entire healthcare team and must be compatible with hospital workflow. Each order set interacts with many hospital processes and support structures such as the drug formulary, medication policy manual, approved abbreviations and method of undertaking diagnostic procedures.

Order sets must be easy to access and understood by all members of the healthcare team. Many organizations find developing order sets much harder to do than they initially anticipated. Seeking consensus and approval of order set content can be difficult to achieve. Once an order set is developed, its implementation and acceptance can become a challenge for the development team.

Order sets are based on a series of assumptions about their nature and use in practice. One of the assumptions is that the benefits of order sets are so obvious that the sets will be rapidly and easily adopted. As our experience shows, not everyone shares this assumption.

This article recounts the experience of the Grey Bruce Health Network (GBHN), whose diverse members worked with each other to adapt and implement an existing set of physician orders while developing significant new order set content. GBHN is a network of three hospital corporations, composed of 10 rural primary hospital sites and one secondary referral site in Southwestern Ontario.

Background

Prior to the beginning of the order set project in 2007, there were approximately 200 order sets spread across the 11 hospital sites of GBHN. Each hospital department made its own order sets, and each site sometimes had different order sets for the same clinical problem. Some sites even had single physician-specific order sets. Order sets for the same clinical problem often had very different content. Order sets were developed on an ad hoc basis. There was no standardized format to keep order sets consistent across different hospital sites or even within individual hospitals. There was no central process to keep track of or maintain them, and many were used sporadically. Some order sets were approved by a Pharmacy and Therapeutics Committee and some by a Medical Advisory Committee (MAC). Some were not approved by anyone.

GBHN started the Evidence-Based Care Program in 2002 to collaborate on developing and using common clinical pathways. Order sets were included in the pathways as a part of the package. Clinical pathways proved to be a challenge to develop and implement. Pathways were developed internally and the process of their approval proved to be cumbersome. Each corporation's MAC had to approve the content and format of a pathway before it was implemented. The multi-layered development and approval process was slow and unresponsive to clinical needs. A new order set could take a year or longer to be approved. This resulted in considerable frustration and disengagement on the part of clinicians.

GBHN is a rural network of hospitals and has a limited number of specialist physicians. Even including the Evidence-Based Care Program, the network has few resources devoted to keeping up with evidence-based care.

Goals

GBHN wanted to standardize physician order sets to improve the quality of care and patient safety. It desired to create an order set development and approval process that was responsive to clinicians and to engage the clinical community in developing evidence-based best practice order sets. This needed to be done in a cost-effective way and in time to support the implementation of a GBHN-wide electronic medical record. Metrics assessing the adoption and clinical impact of the order set project were to be measured as the project developed.

Project Initiation

GBHN recognized from its experience in operating the clinical pathway project that designing and building order sets with available local resources would be very difficult and unlikely to succeed. Therefore, GBHN used the assistance of an external organization (Open Source Order Sets) to assist with project development, order set structure, order set content and implementation.

GBHN learned about Open Source Order Sets through conversations with the Ontario Guidelines Advisory Committee as well as presentations by the organization made at various conferences in 2006. The plan for an order set project was presented at meetings with the GBHN administrative leaders as well as the chiefs of staff. The project was approved and launched. Critical to the ultimate success of the project was the support of the chief executive officers, the MACs, nursing leadership and a few champions. The clinical champions who emerged were essential to carry the project forward. The Evidence-Based Care Program took on the logistical support of the project. A full-time dedicated project lead, a half-time administrative support person and a two-day-a-week educator/utilization coordinator were utilized to implement the project.

A New Process for Order Set Development

After securing the support of senior management and clinical leaders, the next step was to establish an Order Set Committee (OSC). The OSC is an inter-professional group with physician and nursing representation from each organization. It also contains representatives from pharmacy, laboratory and health information services as core members. Importantly, the membership of the OSC consisted of clinically expert front-line staff with an excellent knowledge and understanding of workflow. Senior administrators and managers were not members of the OSC but were included as ad hoc members. While the mission of the OSC is of strategic importance to the organizations, the actual review and development of order sets is an operational matter and requires a large commitment of time. It was helpful to include administrators as ad hoc members to explain the mission and deal with barriers that were not able to be dealt with by the committee.

The key feature of this committee was that it was given the authority to approve and sign off on all order sets and clinical protocols in use at all GBHN hospitals. This feature represented a huge cultural shift away from the multiple layers of approval previously required at the organizations. The OSC's authority to approve and implement order sets allows for rapid cycle changes to occur. This makes the committee responsive to clinicians' needs and facilitates the incorporation of clinical feedback from all stakeholders in a very short period of time. The OSC reports to the chiefs of staff at each organization.

Initially, the OSC was developed as an oversight committee. The original planning envisioned clinical content expert teams developing or adapting order sets that would then be approved by the committee. As the project unfolded, the OSC became the clinical content committee. The main work of the committee became the adaptation of order set content from the existing repository of generic order sets provided by the partnering organization and merging this with the current corporate order sets. This role developed due to the large workload this merge would have been for staff in the corporations - the hospitals are small and, thus, there were not large numbers of staff available to merge the 200 order sets in a short period of time.

There was another factor that determined the work style of the OSC. This project was undertaken concurrently with the building of a large component of the electronic health records in the three organizations. Due to contract deadlines with this build, the order sets project had to be accelerated and moved very quickly to get as many order sets as possible approved across the network in a four-month time frame. Failure to do so meant that GBHN would lose the opportunity to have order sets built into the electronic medical record software by the software company, and would have to build them in later in-house.

Since reaching the deadline for the electronic health record build, the workload and number of newly introduced order sets has been reduced. Most development work is now done by clinical service groups, and the OSC has moved into its original role of an oversight and approval committee.

An order set's development remains a continuous process until the content seems correct. Formal meetings are used along with electronic communications to achieve consensus. Development teams work closely with the OSC, which acts as a resource to assist with development and to ensure that standardized approaches to care have been used.

GBHN employed a repository of order sets that was available on a searchable web-enabled database. Order sets, once adapted and approved, were re-deposited into this database for use by other members in the partnering organization. GBHN has access to over 250 order sets on the master Open Source Order Set repository and all the order sets from over 50 hospitals in the collaborative. These order sets are an excellent resource for the GBHN project, and the standardized, modular format across the collaborative facilitated the sharing of order set content.

Once an order set was approved by the OSC, the order set was posted on the GBHN order set website, enabling all care providers at GBHN to access the order sets as needed. Although there is movement toward a completely electronic medical record, much of the current record remains paper based. If a clinician wishes to use an order set, it must be printed off the web page. Although this occasionally proves awkward, there are a number of advantages to this system. The web page provides the clinicians at GBHN with an easy way to access and manage order sets across the numerous departments and hospital sites. There is no need to store piles of pre-printed copies or have someone constantly checking to make sure they are the current version. The order set project allows users to print on demand as orders are required, ensuring access to the most up-to-date orders. The web page also contains the library of content and order set best practices, which are accessible to all hospital staff.

Implementation

The order sets were launched with staff and physician education. Various methods were used to achieve this - in-services were held and videoconferenced across the hospital sites, presentations were made to applicable committees in the corporations, newsletters and memos were used to introduce the project and give updates as the project unfolded, physician representatives from the OSC brought information back to their respective corporate physician groups and a part-time educator rotated through the hospital sites providing smaller, informal information sessions about the project and how to use the order sets.

A key feature for implementation was to make use of the order sets voluntary. Physicians were not forced to use them and had the option to stroke out or insert other orders as they felt necessary. This strategy was adopted on the assumption that physicians would find the order sets so useful and to be such time savers that they would readily use them. It was felt that "forcing" the use of the order sets would set up confrontations about content or format. It was felt that, over time, the innovation would spread through word of mouth, and this appears to be what is happening at the GBHN hospitals.

Outcomes

GBHN was able to implement one standardized development and approval process for all 11 hospitals. One standardized modular format for order sets was introduced. Existing order sets were adapted to the new format, and overlapping and personalized order sets were eliminated. Where multiple departmental and corporate order sets for one clinical condition existed, they were merged into one. All order sets were compatible with hospital policies, including the formulary and parenteral drug manual. The central OSC was able to develop and approve over 70 order sets in nine months. The initial adoption of the order sets by clinicians was very high, over 40% across all sites and more than 80% at some of the hospital sites in the first six months of use.

The OSC was composed of front-line staff that were given authority by the MACs and administrations to implement their work. This represents a cultural shift within the hospital corporations and provides a successful example of an alternative method of introducing change into hospital practice. It is anticipated that this will have some spillover into the processes around conversion to a fully electronic health record and, ultimately, computerized physician order entry.

Challenges

General Challenges

This project is essentially one of diffusion of innovation and change of process. There are also elements of knowledge transfer. As with any new project, there were expected challenges to developing and using the new order sets. The team began the project using basic change management principles - expecting opposition to changes in the format of order sets, challenges with communication and education and challenges to the "best practice" and "evidence-based" content of the order sets. Team members made sure to address these issues right from the beginning of the project. Key to ensuring that the implementation went smoothly were (1) attempts to have wide stakeholder involvement to increase buy-in, (2) the use of multiple routes of communication and education and (3) advocacy from administration.

Obtaining Consensus

There was considerable discussion and debate about the content of the order sets. This challenge was not anticipated. The team had expected that there would be ready acceptance of the "evidence" and "best practice" content from the external organization used to develop the core order sets. However, despite reasonable face validity, "evidence" and "best practices" were sometimes contested, especially when they did not fit with the local practice. Although evidence-based decision making is the touted ideal, questions about what constitutes evidence, the lack of empirical-based evidence for much of medical practice and who gets to decide what "best" practice is made the acceptance of some practices contentious. Another factor in the acceptance of off-the-shelf order sets is that practice is context dependent. For example, referrals to some interdisciplinary services are not available in rural sites. Perhaps the terminology evidence based and best practice should be changed to usefulpractice. As Orlikowski noted, "the notion of 'useful practice' suggests the necessarily contextual and provisional nature of such practices and the organizational knowing that they constitute" (Orlikowski 2002; 271). This change in terminology would allow greater ability to adapt useful information and incorporate it into local practice.

For many of the sets, stakeholders participated in a lot of discussion and back-and-forth debate before consensus was achieved. This was particularly the case for order sets that existed in one or more of the corporations before project implementation.

Multiple Sites

The members of the OSC worked in multiple sites of the network. The only practical way of holding committee meetings was by videoconference. Videoconferencing represents its own source of challenges (Kennedy et al. 2001; Van Ast and Larson 2007). However, the members made good use of this technology and were able to work within its constraints. This saved a large amount of travel time and allowed for more frequent meetings of the OSC.

A novel feature of this project was authority granted to the OSC to make final decisions and sign off on each completed order set. In previous network projects, all the MACs across the 11 hospital sites needed to vote and approve the content of an initiative. If one group wanted to make a change, then it was referred back to the steering committee. The result was that projects seemed to get caught in endless loops of feedback and meetings without ever getting implemented.

Adapting a generic order set to become standardized at multiple sites provided another challenge. Variations in formularies, laboratory test availability and support services such as the availability of inter-professional consultations exist among the sites. The team found they had to be creative in building the content into the order set to allow for standardization yet also be able to accommodate reasonable local variations. This challenge emphasizes the importance of local context in the adaptation of generic order sets and useful practices.

Turf Protection

An unexpected challenge did arise regarding turf protection. This came largely from the secondary centre, where the nursing and pharmacy staff defended their turf or level of care given. Some surgical order sets are clearly applicable only to a secondary centre. However, some medical order sets pertaining to elements such as electrolyte replacement were defended as something that should only be done in a secondary centre. Staff at the secondary centre felt that some types of patient care should not be done in rural hospitals and therefore those hospitals should not use the shared order set for those conditions. This feeling was not shared by the staff at the rural sites. It was felt the use of a guideline such as an order set is even more important in a smaller site, where the clinicians may not encounter a diagnosis or a undertake a specific procedure very often. This is exactly when it is helpful to have an order set to guide the provision of care. Ultimately, the team agreed with the rural sites.

Short Time Frame for the Initial Development

This project was completed concurrent with the software build of part of the electronic health record in the three corporations. Contract deadlines imposed by the build resulted in considerable time pressure being applied to order sets project. It was necessary to get as many order sets approved as possible in a four-month window; otherwise, the network would have lost the opportunity to have order sets built into the electronic record by the software company. Failure to meet the deadline would have involved building order sets into the electronic health records in-house. This would have resulted in the electronic health record project falling behind its deadlines and costing a lot more money.

The first way the OSC dealt with this short time frame was to have a member of the electronic health record build team on the OSC. This ensured as order sets were developed and approved, they fit within the constraints of the software program and were "computerized physician order entry" ready at the point of implementation. There was considerable back and forth discussion in this regard, ensuring that the paper-based order sets would fit within the software and also that, as the software was built for the three corporations' electronic health record, it fit the clinical needs of the order sets.

The tight time frame meant it was not possible to involve as many stakeholders in the adaptation of the order sets as the team would have liked. It did not allow stakeholders much time to come to consensus. This was a mixed blessing. Although the initial batch of 60 order sets was fast tracked through the conversion and content upgrade process, it resulted in some issues post-implementation and backlash from stakeholders who were not involved in the initial consultation. On the other hand, the fast track meant the team became responsive to the clinicians' criticisms and implemented a system of rapid cycles of change. The order set committee made sure to make changes to the order sets as feedback came in, helping stakeholders understand both that their feedback was important and that the documents were easily changed (using a change management process). Within a few months of release of the first batches of order sets, these issues were non-existent.

Another challenge created by the short time frame was freeing up time for busy front-line staff to attend the frequent meetings of the OSC. Prior to the software deadline, at the peak of the work, meetings were held every other week. It is a testament to the support the project received from the administration, staff and physicians that the time was made available for members to participate in the meetings.

Conclusions

There were a number of lessons learned in this project. The authority given to the OSC to have final approval over the content and implementation of completed order sets worked well in our multi-corporation context. It greatly decreased the time required for the adaptation and implementation of standardized order sets compared with GBHN's previous experience with clinical pathways.

The multi-professional front-line clinical experts who made up the OSC were aware of content issues and the workflow in their respective sites. Administrators and managers remained ad hoc and were helpful when the OSC ran into system barriers.

It is critical to communicate widely about the project using a variety of mediums. Although this was not emphasised in this article, the communication plan requires much thought. It takes a lot of time to communicate to and educate all stakeholders.

Implementing standardized order sets into multiple hospital corporations is a challenge. Differences in formularies, laboratory test values, work processes and available resources will likely be present. Order sets cannot be completely "standardized" and must be adapted to the local context.

Finally, "best practice" and "evidence based" are concepts that are contested by clinicians - this is especially true if a change in their practice is being proposed. It is important to be prepared to support the recommendations with evidence. However, it may be better to realize in advance that these "best" practices are being introduced into a context where they may not fit. Perhaps it is better to think of and label them "useful" rather than "best" practices.

Comments

Be the first to comment on this!