ElectronicHealthcare 2(4) March 2004 : 72-81

Critical Success Factors Relating to Healthcare's Adoption of New Technology: A Guide to Increasing the Likelihood of Successful Implementation

Kevin J. Leonard


Over the last decade, significant attention has been paid in both academic and professional literature to the healthcare information technology conundrum, which can easily be summarized in the following question: Why have we not seen more successful implementation of information technology in healthcare? While many theories and suggestions have been proposed, there can be no argument that none have been truly effective in explaining or helping to resolve this widespread problem. As a result, the healthcare field is becoming experienced in building not-so-effective systems.

The obvious question facing healthcare is: How do we get out of this cycle of poor systems begetting more poor systems? The recommendation presented herein is that we analyze the process of adopting new technology in other sectors, across different organizations and industries. There are a number of ways of illustrating experiences - through case studies, research papers or conference presentations. Here, we apply storytelling, where the stories are short vignettes that encapsulate a problem, a decision process, the solution selected and the results.

We present a number of stories from within healthcare and elsewhere that illustrate the struggle and lessons learned in many different areas of innovation and new technology. We define the relevant critical success factors and provide a guideline for further adoption of innovation. Whether the information technology creates new functionality or replaces an existing system, the critical fact is that the outcomes resulting from the adoption must be measured - compared to previous statistics or results to illustrate the improvement (or not) provided by the new technology - and ultimately, this change in outcomes must be communicated to stakeholders. While all this may seem obvious and perhaps even trivial, one of the fatal flaws in information systems design is that new technology (regardless of its composition) requires an interface with human beings. If the stakeholders do not have their expectations properly established through effective communication, resistance to change and other factors will often derail an otherwise effective new technology adoption.


The information systems (IS) literature has an abundance of research that describes innovative approaches to designing and developing information and/or decision-support systems. This literature often highlights methods or processes that enhance the likelihood of building and delivering more effective systems (e.g., Gibson 1977; Keen 1981; Ives and Olson 1984; Rockart and Crescenzi 1984; Baroudi et al. 1986; Franz and Robey 1986; Barki and Hartwick 1989; Barki and Hartwick 1994; Chang 1995; Goodhue 1995; Norman 1998; Rogers 2003).

In recent years, the same objective of facilitating information systems development has received much attention within healthcare literature, but with more of a focus on specific problems than on theoretical presentations. As examples, research has focused on multimedia applications, real-time access to electronic patient records (EPRs), patient education and support and knowledge development and knowledge translation relating to best practices (Barry et al. 1997; Berg et al. 1998; Cimino 2000; Ferguson 1997; Ross et al. 2002; Strecher et al. 1999). Further, patient-centred knowledge development and support tools, administered in partnership with traditional physician-centred clinical programs to promote patient education and support, have received attention of late pertaining to the design of information systems and interfaces (Ball and Lillis 2001; Molenaar et al. 2001; Ross and Lin 2003; Young and Chang 1997).

In addition, there have been many research papers concentrating on the development of systems for the different physician groups. These reports emphasize both the slowness of adoption and resistance to change (Leonard 2000; Treister 1998) as well as innovative approaches to system functionality in order to improve adoption (Dixon 1999; Dixon and Dixon 1994). Lorenzi and Riley (1995) found that technology is perceived to interfere with the traditional role of the physician. Treister (1998) gives 11 reasons why physicians fail to accept new systems. These include the failure to begin with an adequate physician base of support, lack of user-friendly interfaces, concern regarding the information collected, failure to collect the most important information, physician "technophobia," exclusion of physician involvement in the financial analysis, failure to include marketing to physicians in the implementation plan, inadequate training of physicians to use the system, lack of strong, centralized IS leadership respected by physicians and lack of control by the organization over physician practices. In summary, it is clear that physician resistance is high when an information system creates no added value to the physician's work (Leonard and Winkelman 2002).


Even though much has been written regarding the adoption of new technology across a wide range of industries and sectors, adopting new technology is still a major problem in healthcare. This topic has continued to be prevalent in the healthcare literature over the last decade (Lorenzi and Riley 1995; Lorenzi et al. 1997; Lorenzi and Riley 2000; Leonard and Winkelman 2002; Stiell et al. 2003). These studies and reports have focused on identifying ways to ensure effective information technology (IT) implementation. Some guidelines include:

  • Identify information problems that are unique to health
  • Identify mainstream, non-health efforts that can be leveraged by healthcare
  • Identify where health informatics can make the most difference
  • Build systems that support the shared objectives of providers and consumers
  • Develop tools that are scalable
  • Design the interface for transferring information from provider to patient
  • Increase research into Internet applications
  • Invest in existing resources

In addition, the literature on new technology adoption in healthcare has consistently referred to related disciplines such as sociology, psychology, social anthropology, organizational behavior and development, management and the cognitive sciences. Within these disciplines, reference is made to the importance of motivation, leadership, culture, management of information, user involvement and acceptance and the usability of the technology. Unfortunately, these many treatises have not resulted in more widespread, efficacious deployment of new technology throughout health and health services delivery.

Some authors have gone so far as to recommend examining other industries in order to provide some insight (Stead and Lorenzi 1999). Perhaps the most significant research around the areas of adoption and diffusion of innovation and new technology belongs to Rogers (1995, 2003). Although there have been others (Moore 1991; Christensen 1997; Norman 1998), much of the terminology development was started by Rogers when he identified five characteristics of adoption. Rogers (1995) concluded that competencies in the following areas will speed up the rate of widespread adoption and diffusion:

  1. Relative advantage
  2. Compatibility
  3. Complexity
  4. Trialability
  5. Observability

Cain and Mittman (2002) have taken these five characteristics and applied them to healthcare. These characteristics have been retitled the "10 critical dynamics of innovation diffusion." They are:

  1. Relative advantage
  2. Trialability
  3. Observability
  4. Communication channels
  5. Homophilous groups (groups with similar characteristics)
  6. Pace of innovation/reinvention
  7. Norms, roles and social networks
  8. Opinion leaders
  9. Compatibility
  10. Infrastructure.

Cain and Mittman (2002) also report an outline of the stages in the decision process to adopt innovation (originating in Rogers 1995). These stages include (1) knowledge, (2) persuasion, (3) decision, (4) implementation and (5) confirmation. Although each of these articles, and the information contained therein, are accurate reflections of the state of the health industry, outcomes to date from both research and practice relating to new technology adoption in healthcare demonstrate that the technology, and the research on how best to use it, result primarily in frustration.

As stated, while these "volumes of research" are detailed and outline creative solutions and management theories, they have done little to facilitate the adoption of technology in healthcare. One reason for this is that there is no single paper that outlines a step-by-step methodology on how to improve IT implementation. The literature presents the many current theories, often at a high level (as above), outlining factors from many (perhaps) disjointed or contradictory perspectives. Second, many papers outline the important factors affecting both the information technology and the systems. This provides a broader perspective; however, it makes the adoption issues more confusing due to the complexities of both of these two diverse entities - the innovation and the people! The result is that there is a large amount of reading material and thought-provoking ideas, but no description on how to operationalize them. In other words, we know the ingredients but not the recipe.

One solution is to outline a concise and precise all-inclusive listing of critical success factors (CSFs) that must be addressed. Although this listing may not constitute a recipe per se, it does provide an exhaustive list of ingredients and the correct order in which they must be addressed. In this way, a set of guidelines is presented to lay the framework for successful design, development and implementation of new technology in healthcare. The challenge, of course, is to present these CSFs within a context that is both informative and illustrative. In this paper, we use stories from other industries as a means of providing both context and content. Further, we limit the discussion to the critical factors that pertain to the "people issues" surrounding new technology adoption. After all, it must be the users' need for better information that ultimately determines the fate of innovation in information systems and technology.


Storytelling is becoming, once again, a popular way to share positive and negative experiences. Once a lost art of passing information down from generations, storytelling is now used as a teaching tool and a method to bridge solutions across industries or settings. The concept of storytelling is similar to the case study teaching method pioneered by Harvard University. Case study methodology is used in almost all undergraduate and graduate business or management programs today (not necessarily exclusively, but as some part of the curriculum in a variety of courses). In a case study, background is given so readers can immerse themselves in the particulars of the case. Then a problem is presented, and students are required to make the best decision(s), given the parameters as they have been laid out. The cases can range in length from one paragraph to 50 pages or more. Whereas case studies require the reader to resolve the "crisis," a story goes one step further by providing the resolution and then offers the connection between the situation and the outcome - thereby employing the story's issues or problems as powerful educational assistants.

Here, we use storytelling to elaborate on the issues relating to the CSFs of technology adoption. In his book, Denning (2001) outlines many principles of storytelling. The main argument is that the story needs to be made relevant in as many ways as possible, so the listener can make the leap from the story to the issue at hand … and can recognize that this leap takes time to complete. The following is the first CSF story, which deals with the issue of resistance to change.

STORY 1: Change Management

In a large academic health science centre around the mid-1990s, the diagnostic imaging department was going through the final stages of the conversion to a completely film-less radiology process - in other words, moving to electronic images of x-rays and the like. During this conversion, the doctors were notified that their patients' x-ray results were ready, and they were then sent their patients' images by e-mails. By late morning of the first day of the transition, many doctors complained that they had yet to receive their patients' x-ray results. Apparently, the doctors did not check their e-mails.

In an effort to ensure the success of the conversion, the imaging department then began to send the doctors the radiology results on a computer diskette or compact discs (CD-ROM, depending on the size of the file). Unfortu-nately, the doctors still continued to complain about the tardiness of the results. Even though they received the diskettes and CDs in a timely manner, they could not match the arrival of these hardware supplies to their own information needs. This is not to say that the physicians were not computer-literate; rather, they were inexperienced at identifying the connection between CDs and patient x-rays. So the imaging department hit upon a great idea: it continued to use the CDs but now sent them to the doctors via internal hospital delivery system in the traditional large x-ray envelopes. Recognizing the en-velopes, the doctors opened them immediately, placed the CDs in their computers and started reviewing the results.

Takeaways from Story 1:

Quite simply, people hate change. This may seem to be an obvious remark or even an overstatement. When it comes to evolving technology, however, the need to focus on people's acceptance to change cannot be overstated. It really comes down to a basic phenomenon - the fight for survival, or at least the perception of it. When change happens, people who do not adjust their work patterns begin to fail. They fail because the old ways of doing things do not work within the new system. Soon their failures mount and more bad things happen. The people who do succeed are the ones who first realize that the way they did things before was not perfect, and that new technology has allowed them to improve. Once they have accepted the change, they begin to operate more efficiently and effectively than ever before. This is an important point: If people operate in the exact same way after the introduction of new technology as they did before, then they have missed a great opportunity to improve both the processes and the outcomes. Change requires work - as well as the important realization that the way things have always been done can, and must, be improved.

In addition, this story is a great illustration of the need to "make the link" between the new technology and the old ways of doing things. It is worth noting that the adoption was not quite complete, however, until the final functionality was arrived at: the quality of the electronic x-ray image had to be as good or better than what was previously available to doctors … or the adoption would never have taken place, no matter how easily the results were accessed.

STORY 2: Amount of Training Before and After the Transition (i.e., Communication of Expectations)

Two teams of telephone pole installation technicians are hired to put telephone poles into a new residential subdivision. The existing crew is the best in the business and needs little guidance. A second crew is brand-new, but also receives little instruction - after all, they are only installing telephone poles (it's hardly brain surgery!).

At the end of the first day, both teams report to their supervisor. The first team reports installing installed 25 poles that day - a new record for that area. When the supervisor asks the second crew team leader, he sheepishly replies that they have installed only two.

"Two? How can that be?" is the boss's immediate retort. "Let's go out right now to see the work of the first team, so that you can see how a real team does professional work!"

They all storm out to the site in the subdivision where the first team had worked. The supervisor looks proudly down the street. "See, that is championship work!"

To which the second team leader responds immediately: "Yeah, sure, but look how much of their poles are sticking out of the ground!"

Takeaways from Story 2:

No matter how simple or straightforward it may appear to the user, the information request to the technology group and system developers is always new and complex. Therefore no instruction, clarification or detail should be considered too small to communicate; overlooking training for even the smallest detail can lead to catastrophic results. Communication is the key to success!

This is another illustration of the need to "make the link" between the new technology and a different way of doing things. Not only is training imperative in order to overcome obstacles, but the details surrounding how the innovation will address the old objectives in a new way must also be made clear. Although this Story 2 is "light" in nature, it does illustrate how a lack of training and communication (around the linkage) can lead to wasteful results.

STORY 3: Amount of Buy-In/Contribution from Stakeholder Groups

A large community in southwestern Ontario has been working on the development of a Community Health Information Network (CHIN). Over the last eight years, much debate has occurred regarding the need for a community network and the potential benefits. However, no matter how hard the executive of the CHIN tried to articulate the potential upside, the only stakeholders interested in the CHIN were those involved in delivering care in the community. It was very difficult to get the local hospitals committed to the CHIN project due to in-hospital initiatives competing for the same internal resources (i.e., funding and personnel). Over time, however, the functionality of the network improved, making data and information exchange feasible. The data evolved from text to data fields to metrics and even to images. Throughout this evolution, the CHIN executive met with the hospitals on various occasions to apprise them of the development and gauge their interest to join the CHIN, but to no avail.

Then, at one of the regular meetings, the executives from the hospitals were asked directly: What information would be useful for the hospitals? The answer was very specific and somewhat surprising: They wanted access to information on discharge summaries regarding their patients awaiting home care and community services. Upon getting this straightforward directive from the hospitals, the CHIN immediately created a "decision-making group" or "steering committee," comprising representatives from the three hospitals, the regional Community Care Access Centre and the public-health sector, in order to develop evaluation measures and establish a budget for ongoing management.

Takeaways from Story 3:

This is an excellent story illustrating the need to involve stakeholders in the design and development of information systems. Once the hospitals were made aware of the functionality achieved in the CHIN through regular meetings and briefings, they were very definite in what types of functionality they desired and had no trouble in identifying next steps, priorities and a plan to get there. As stated in the preceding two CSFs, there has to be a link made between the new system and the organizational or industry objectives that have not been fully (or, perhaps, not optimally) met under the old system. In this example, the old way of keeping up-to-date was to chase down paper copies of patient discharge summaries. The new technology allows better monitoring through information transfer and improved management by facilitating decision-making and implementing action. The significant issue here is recognizing that the healthcare representatives do not always understand their role and the options that they possess. It is only in a safe iterative-style environment that all parties can fulfill their responsibilities to maximum effectiveness.

STORY 4: Effective Reporting on the Status of the Outcome Measures During and After Implementation (i.e., Communication on the Technology Adoption Progress)

In the early 1990s, an innovative hockey coach was looking for ways to get a competitive advantage over other coaches and teams in the National Hockey League. Consultants were brought in and a sophisticated statistical database was built that incorporated full-motion video, as highlight clips, into the database management system. The only problem was that many of his coaching staff would not embrace the new technology. The complaints were that: "The new system is hard to use, too cumbersome, takes too long to enter data and there is so much output that I cannot find what I am looking for!" Therefore, many coaches just stuck with using the old system of watching tapes and entering data manually on legal-size paper forms - and doing all of the statistical calculations by hand.

One day the consultants got an idea and decided to change the layouts of the computer report. They made the computer printouts look exactly like the handwritten reports by incorporating a "handwritten" font and printing the results on legal-size paper (rather than the standard size). Some of the coaches were startled to see the old reports had been completed so fast (i.e., right after the game rather than a day later). Up until this point, some of the coaches had never even looked at the com-puter printouts; with the right layout and font, the reports began receiving recognition for their speed of turnaround, but not yet for their additional content. However, once the coaching staff picked up the reports, they began to see how innovative statistics were being created. They began to seek out the reports and the evaluation right after each period of the game. The most important part was that the new technology not only provided insight into the team performance but also became the only way to ultimately measure the team's success over time.

Takeaways from Story 4:

After an adoption of innovation occurs, the momentum will eventually begin to get stale and wear off. However, the momentum must be kept up throughout the adoption cycle - meaning there must be a way to sustain the energy around the project. The best way to do this is to continually measure the progress of the adoption. If the project is doing well, then the measurement itself will motivate the stakeholders. If not, then an insightful routine report, at the very least, allows for explanations into why there has not been more progress to date.

As before, the issue of "making the link" between the new technology and the old ways of doing things is imperative. Here, this link is fundamental as it was the means by which the new reports were ever looked at - in other words, the consultants needed to make the new reports look exactly the same as the old reports. This helped the coaches "get over the change issues" and begin to look at the information from a new and exciting perspective.

STORY 5: Level of Effectiveness in Dealing with the "Breaks/Good or Bad Fortune"

As patients move to a more self-reliant role in the management of their health, the demand for personalized information has increased. At a large academic health science centre in Toronto, a research team was engaged in one of the first patient-centred research efforts of its kind. The project engaged patients as stakeholders by asking them to design, using innovative technology, an effective electronic patient record interface. The project employs a systematic patient-centred, iterative approach to system design, development and evaluation. The only real hurdle has been that many physicians were skeptical about the value of patients accessing their own records and how the patients would react. The physicians worried about increased workload involved in handling extra questions, and about how patients would react emotionally when they accessed their record. Further, there was serious concern over the integrity and security of the data emanating from the many people with remote access to active health records. In fact, the major obstacle was convincing the doctors that there was a need at all (or even the demand) to access records remotely. This battle was going very slowly.

In early 2003, SARS (Severe Acute Respiratory Syndrome) hit Toronto and had a major impact on the healthcare community. Doctors who entered hospitals to treat patients were exposed to the potential of a forced quarantine, the effects of the virus or both. Suddenly, doctors and other clinicians saw tremendous benefit to being able to access records and all sorts of patient and health information remotely. The research project continues, with a partnership of both clinicians and patients potentially remotely accessing patient records.

Takeaways from Story 5:

Clearly, there are events in life that are unpredictable - some of which will have a positive effect and some which will have a negative effect. Being able to take advantage of both types of outcomes will increase the likelihood of success for any project. In this story, a number of months of further struggle were eliminated due to the intervention of a "break" - one that had an extremely negative impact on the healthcare service delivery sector, but which created an out-come that may benefit the industry for years down the road.

Finally, as in the other stories, there is a need to "make the link" between the new technology and the old ways of doing things. In this case, the link was to illustrate to the doctors that the issues relating to patient access to medical records are exactly the same as they are for physician access. The SARS outbreak was a way for many clinicians to witness first-hand the frustration of not being able to access the information needed to accomplish goals.


Regardless of the industry, when change through the adoption of new technology takes place, the amount of work required to operate the existing system actually increases. This is always true and will last for a period of time. This may seem to be contrary to popular belief, which states that there should be efficiency (and even effectiveness) gains from new technology. Yes, this is true that once the new system has been implemented and is working effectively, efficiencies will increase - thereby reducing overall workload. However, during the transition or adoption of a new information system, the work increases, and this increased work will last for a period of time. Ultimately, change requires work. How much the work increases, and how long this effect lasts, truly depend on five critical success factors (CSF):

  1. Amount of resistance to change or industry experience in using technology
  2. Amount of training before and during the transition
  3. Amount of buy-in (or contribution) from stakeholder groups
  4. Level of effective reporting on outcome measures during and after implementation (i.e., communication on the technology adoption progress)
  5. Level of effectiveness in dealing with the "breaks"

In some detail, if the industry and its personnel have had experience in using IT, then navigating its way through another adoption is much easier. This is true due to the fact that the benefits are known in advance and the industry has already found ways to address concerns or deal with obstacles.

Training is critical, since new employees and even other more experienced personnel need support throughout the development and implementation life cycle. This training provides stakeholders with the skills and confidence required to address concerns, overcome obstacles and ensure success.

The amount of stakeholder buy-in is normally attributed to the interaction between the system developers and the healthcare users. The more users are involved in the design and development of new information technology, the more they will perceive this new technology as their own system rather than one imposed on them.

Nothing succeeds like success, and these words were never more relevant than today relating to IT adoption in healthcare. The more progress is made, the faster will be the next adoption. However, people must be notified of the progress … and this must happen as soon as possible after the intervention (or the implementation in this case) in order to retain momentum and confidence!

Every person or initiative encounters unplanned events that are outside the scope of realistic outcomes. Although they should not happen, they still do; sometimes these things are positive and sometimes negative. The success of any project does not depend on whether the positives outweigh the negatives (contrary to popular belief) but rather on how well the project leadership deals with the "breaks" in the first place.


We have discussed the CSFs in the context of the amount of time and work that is needed to be successful with IT adoption. To this end, a figure or picture can assist greatly in illuminating an idea. Once again, there have been many attempts to represent the process of adoption and diffusion of innovation, with the main emphasis being on the time to adopt (see Rogers 1995, for the Rate of Adoption Curve; Christensen 1997, for the Needs-Satisfaction Curve; or Cain and Mittman 2002 for the Diffusion S-Curve adapted from Rogers 1995). In Figure 1, the novel contribution is a curve plotting the amount of work, or effort, it takes to move through the adoption life cycle. This figure does not define when people come on board and adopt technology (or even attempt to name them) but rather focuses on how much work is required to bring on the entire stakeholder group, from early adopters or innovators through to the late adopters or laggards (terminology from Rogers 2003). Further, it must be emphasized that the workload depicted in the graph (as discussed in the previous section) involves both the work to maintain the overall system and the work to make the change happen, to adopt and implement the new technology and to achieve a new steady state.

[Figure 1]

When technology is first introduced, there is a certain amount of effort expended to operate the current system - represented on the graph at Time 0, and considered a "baseline workload level." At Time 1, a period of time has gone by and the workload has increased to some degree due to running two systems in parallel and overcoming resistance and system failures. It is at this point that resistance to change starts to be articulated. At Time 2, the overall workload is increasing at a faster rate, and even the initial supporters of change are starting to have their doubts. At Time 3, the workload is increasing at an increasing rate; the stress on the information and computer system developers is at its highest. It must be noted here that if a system implementation fails, it usually happens somewhere between Times 2 and 3 (although there are always exceptions).

At Time 4, the innovation starts to win some public favour, as the overall workload is decreasing due to increased efficiencies; there is even the odd comment that maybe things are not unsalvageable! By the time we reach Time 5, the total amount of workload using the new system is approximately about what it was under the old system - we are once again at the "baseline workload level." The jobs and tasks have changed (often dramatically), but the total amount of work at least appears to be the same. Time 6 (and even more so at Time 7, if achieved) sees benefits improving at an increasing rate - often garnering "wins" in areas that were not even predicted when the system was implemented. This is most often due to escalating effects and synergies. Here, people begin to realize the benefits of the new technology and begin using them to create even more opportunities, resulting in new and continued avenues for growth and for working more cooperatively.

Unfortunately, it is difficult, in the abstract, to identify where an organization (or an industry, for that matter) is positioned on this technology adoption curve, or how long it will take until we get to Time 4. Most of the people who resist change do not come up to the systems designers and say: "The innovation is probably good and will help us in the long run, but I am very insecure and I would like all of this go away before I have a nervous breakdown!" If they did, it would be easier to deal with resistance to change. What normally happens is that, before the new system has been implemented, they preach about how bad things will be. Then, once implementation begins and the workload increases (which we know it must), they point to this as evidence that the system cannot work. It should be recognized that there will always be people who resist change simply because it is change! In order to effectively combat this, we must realize that it is the change itself that they are fighting and not the merits of the innovation.
If these "resistors" have any power whatsoever, the system will be shut down before one ever reaches Time 4. The only way to combat this attack from resistors is to find "early adopters." These are people who like technology, embrace change and are not afraid of making mistakes (Rogers 1995). Every successful adoption of new technology has had supporters who ensured that the system had enough time to make it to Time 4, so that the positives could gain momentum and convert the resistors or, at the very least, remove their influence.


Each of the five critical success factors defines the speed through which an organization moves along the technology adoption curve. Unfortunately for the healthcare field, there has been very little successful information technology adoption. Consequently, due to the status of the CSFs in healthcare, there has been little information system support to date to ensure successful implementation going forward.

Any type of technology change must be seen in one of two ways: either it is completely new technology, whereby the benefits are obvious or can be easily and straightforwardly articulated, or it is meant to replace existing technology and must (i) lower costs and/or (ii) improve outcomes enough (either operationally or incrementally through the change) to justify the increased costs. A good example of the first is telemedicine: this allows patients and clinicians to connect over long distances. The operational model is the same, with the only difference being that patients can be evaluated and treated remotely; the technology adoption allows for this remote connectivity that did not exist previously. Most of the time, however, information technology is extremely difficult to justify because it is not of the first type above. Therefore, the new technology must either lower costs in the short term - which we know it does not (see Figure 1) - or it must improve outcomes - which, as stated above, are often not measured … or at least not with any consistency. Hence the dilemma: if we cannot demonstrate that new information technology has value, then, to ensure successful adoption, we must become preachers of a new philosophy borrowed from Kinsella (1982) - if you build it, they will come. Regrettably, people seldom adopt technology on faith alone!


Why have we not been able to see the same successful technology development in healthcare that we have seen in many other industries to date?

The objective of this paper was to highlight some key considerations that must be addressed when organizations adopt new technology - in our discussion, namely information technology. These factors are consistent across industries, and overlooking them has often led to the downfall of IT initiatives in healthcare applications. Although being cognizant of them does not guarantee a successful implementation, it does increase the likelihood of a positive outcome.

These CSFs infer the following conclusions (in no direct order or relationship):

  1. The objective of any new information system must be made clear at the beginning of the project so that everyone involved knows whether this involves the creation of a new system or the revision of current functionality.
  2. Technology performance and system outcome metrics must be established upfront so that the right controls are put in place at the outset - that is, ongoing evaluation is paramount.
  3. Of utmost importance is the fact that all issues, both positives and negatives related to the technology change must be, to the degree that they can be measured or predicted, communicated effectively!
  4. Every project, organization, or person encounters both good luck and bad. At times, taking advantage of the good "bounces" is as critical as good strategic planning. Further, realizing that every project will encounter bad luck is a first step in overcoming obstacles - which must be present, or else the solution would have been created long ago!
  5. The reliance on IT as a cost saviour must be abandoned (at least in the short term): new information systems have yet to be implemented successfully when the benefits are limited solely to lowering costs.

Conclusion 5 should be elaborated upon, as it summarizes this whole theory. While it is true that many technology implementations have first been designed to address cost issues, potential cost savings are often dwarfed by the onset of escalating costs pertaining to the implementation phase - once again, see Figure 1. Therefore, the project life or momentum is only sustained if there are other benefits to be gained - benefits that have to be measured objectively and in conjunction with the goals stated in Conclusion 1. Hence, the need for the development of metrics in Conclusion 2. This, of course, leads to the need for better documentation and communication of improvements (Conclusion 3) and better management of the unknown (Conclusion 4). If not, the technology project will die somewhere between Times 2 and 3 (on the technology adoption curve, Figure 1) as costs escalate … and the resistors to change will have accomplished another victory for the status quo!

The bottom line is that change is necessary if we are to improve the way we do things. This is an important consideration, as the healthcare system needs improvement. Currently, the present system is fraught with inefficiencies. Any new initiative has to focus on developing a health information system that allows information exchange among patients, government agencies, healthcare providers, educational institutions and private sector partners. It is hoped that the guidelines presented here will assist in change management and increase the probability of future successful information system development and adoption.


Let's review the healthcare industry with respect to each of these five CSFs. First, there has been little successful adoption of information technology throughout the health system. As a result, there is a shortage of both experience (making the change management issue even more prevalent) and qualified trainers (leading to infrequent training) around the use of IT. Historically, one of the main contributing factors to the poor success rate has been a lack of stakeholder involvement. Unfortunately, clinicians and patients have seldom been asked what features or functionality would constitute a valuable information system (or, say, an electronic health record) from their point of view. Moreover, there has been little, at least until recently, accomplished in establishing performance metrics to evaluate the healthcare system in general. Therefore, relating an improvement in health outcomes (if there is one) back to IT investment has been even more difficult. Keeping stakeholders apprised of this progress during and after implementation by reporting on these new metrics is still only a theoretical construct in today's healthcare environment.

As Denning (2001) stated, the focal point of storytelling is the need to make a link between the story and the issue at hand; this is the same for each of the CSFs - we must continually make a link between old processes or objectives and the need to improve the system and the innovation. We cannot overemphasize the human factors, components and issues within today's new technology challenge for healthcare.

In conclusion, we propose an evaluation framework for healthcare innovation - for those that have been already developed and implemented, or for those in the preliminary design phase. It is our intent to raise the awareness of areas that may require more specific attention. We outline this process as follows:

  1. Formative evaluation
    This seeks to understand, at the outset, what potential users of the system want and need, how they perceive it should/could work and what their expectations are. Findings of this evaluation component should be presented to each of the stakeholder groups (clinical, administrative and technical staff) for their input and overall evaluation.
  2. Evaluation of prototype system
    The evaluation of a prototype system focuses on the comparison of capabilities of two information systems - the current one and the new technology. In comparison to the future expectations, one should attempt to describe current functionality with quantitative measures. This will require, for the most part, the development of these measures and a methodology for evaluating these indicators as it progresses through time. The evaluation could also include a user satisfaction evaluation.
  3. Evaluation of functionality
    The final component is the evaluation of the functionality of new technology that is in place. This will include open-forum discussions as well as a review of other state-of-the-art systems. Further, functionality should be measured using the indicators already established from phase ii above. The final step will constitute setting strategy at a much higher level than is required at the first two steps, where we are trying to envision an improved system over the one that currently exists. There is no doubt that any information system could benefit from creative solutions and future visioning.

This is an excellent opportunity for the health services industry to use expertise and infrastructure developed over the past several years to evaluate an innovative and effective information-based health system. Moreover, it is possible to look forward (as well as back) to novel ways of documenting the benefits from improved access to information and to developing new infrastructure so as to ultimately increase effectiveness in health outcomes.

About the Author(s)

Kevin J. Leonard, MBA, PhD, CMA, is an Associate Professor, Department of Health Policy, Management and Evaluation, Faculty of Medicine, University of Toronto, and a Research Scientist, Centre for Global eHealth Innovation, Toronto General Hospital, University Health Network, Toronto, Ontario. Address correspondence to: Faculty of Medicine, University of Toronto, McMurrich Building, 12 Queen's Park Crescent West, 2nd Floor, Toronto, Ontario, M5S 1A8, Canada.


Berg, M., C. Langenberg, I. van der Berg and J. Kwakkernaat. 1998. "Considerations for Sociotechnical Design: Experiences with an Electronic Patient Record in a Clinical Context." International Journal of Medical Informatics 52(1-3): 243-251.

Cain, M. and R. Mittman. 2002. "Diffusion of Innovation in Healthcare." Prepared by the Institute for the Future for California HealthCare Foundation ihealth reports, May 2002. http://www.chcf.org

Chang, J.C.J. 1995. "Business Environment, User Involvement, and Information Systems Success: A Case Study." Inaugural America Conference of the Association for Information System. http://www.hsb.baylor.edu/ramsower/acis/papers/changj.htm

Christensen, C.M. 1997. The Innovator's Dilemma: When New Technologies Cause Great Firms to Fail. Boston, Massachusetts: Harvard Business School Press.

Cimino, J.J., J. Li, E. A. Mendonca, S. Sengupta, V. L. Patel and A. W. Kushniruk. 2000. "An Evaluation of Patient Access to Their Electronic Medical Records via the World Wide Web." Proceedings of the American Medical Informatics Association Symposium, pp.151-155.

Denning, S. 2001. The Springboard: How Storytelling Ignites Action in Knowledge-Era Organizations. Butterworth-Heinemann.

Dixon, D.R. 1999. "The Behavioral Side of Information Technology." International Journal of Medical Informatics 56: 117-123.

Dixon, D.R. and B. J. Dixon. 1994. "Adoption of Information Technology-Enabled Innovations by Primary Care Physicians." Journal of the American Medical Informatics Association (Symposium on Computer Applications in Medical Care supplement) 8: 631-634.

Ferguson, T. 1997. "Health Online and the Empowered Medical Consumer." Journal on Quality Improvement 23(5): 55-70.

Franz, C.R. and D. Robey. 1986. "Organizational Context, User Involvement, and the Usefulness of Information Systems." Decision Sciences 17(4): 329-356.

Gibson, H.L. 1977. "Determining User Involvement." Journal of System Management August: 20-22.

Goodhue, D.L. 1995. "Understanding User Evaluations of Information Systems." Management Science 41(12): 1827-1844.

Ives, B. and M. Olson. 1984. "User Involvement and MIS Success: A Review of Research. Management Science 30(5): 586-603.

Keen, P.G.W. 1981. "Information Systems and Organizational Change." Communications of the ACM 14(1): 24-33.

Kinsella, W.P. 1982. Shoeless Joe. Boston, Massachusetts: Mariner Books, Houghton Mifflin Company.

Leonard, K.J. 2000. "Information Systems for Healthcare: Why We Have Not Had More Success - the Top 15 Reasons." Healthcare Management Forum 13(3): 45-51.

Leonard, K.J. and W. Winkelman. 2002. "Developing Electronic Patient Records: Employing Interactive Methods to Ensure Patient Involvement." Proceedings from the 28th meeting of the European Working Group on Operational Research Applied to Health Services (ORAHS). Rio de Janeiro, Brazil, July-August. pp.225-239.

Lorenzi, N.M. and R. T. Riley. 1995. Organizational Aspects of Health Informatics: Managing Technological Change. New York: Springer-Verlag: 224.

Lorenzi, N.M. and R. T. Riley. 2000. "Managing Change: An Overview." Journal of the American Medical Informatics Association 7(2): 116-124.

Lorenzi, N.M., R. T. Riley, A. J. C. Blyth, G. Southon and B. J. Dixon. 1997. "Antecedents of the People and Organizational Aspects of Medical Informatics: Review of the Literature." Journal of the American Medical Informatics Association 4(2): 79-93.

Molenaar, S., M. A. Sprangers, E. J. Rutgers, E. J. Luiten, J. Mulder, P.M. Bossuyt et al. 2001. "Decision Support for Patients with Early-Stage Breast Cancer: Effects of an Interactive Breast Cancer CD-ROM on Treatment Decision, Satisfaction, and Quality of Life." Journal of Clinical Oncology 19(6): 1676.

Moore, G.A. 1991. Crossing the Chasm: Marketing and Selling High-Tech Goods to Mainstream Customers. NewYork: Harper Business.

Norman, D.A. 1998. The Invisible Computer: Why Good Products Can Fail, the Personal Computer Is So Complex and Information Appliances Are the Solution. Cambridge, Massachusetts: MIT Press.

Rockart, J. and A. Crescenzi. 1984. "Engaging Top Manage-ment in Information Technology." Sloan Management Review 25(4): 3-16.

Rogers, E.M. 2003. Diffusion of Innovations. New York: Fifth Edition, The Free Press.

Ross, S.E., M. A. Earnest, C. Lin and L. Wittevrongel. 2002. "Providing Patients Access to Online Medical Records: A Comparison of Physician and Patient Expectations." Society of General Internal Medicine, 25th Annual Meeting. May 2-4.

Ross, S.E. and C. T. Lin. 2003. "The Effects of Promoting Patient Access to Medical Records: A Review." Journal of the American Medical Informatics Association 10(2): 129-138.

Stead, W.W. and N. M. Lorenzi. 1999. "Health Informatics: Linking Investment to Value." Journal of the American Medical Informatics Association 6(5): 341-348.

Stiell, A., A. J. Forster, I. G. Stiell and C. Van Walraven. 2003. "Prevalence of Information Gaps in the Emergency Department and the Effect on Patient Outcomes." Canadian Medical Association Journal 169(10): 1023.

Strecher, V.J., T. Greenwood, C. Wang and D. Dumont. 1999. "Interactive Multimedia and Risk Communication." Journal of the National Cancer Institute Monograph 25: 134-9.

Treister, N.W. 1998. "Physician Acceptance of New Medical Information Systems: The Field of Dreams." Physician Executive 24(3): 20-24.

Weed, L.L. 1991. Knowledge Coupling: New Premises and New Tools for Medical Care and Education. New York: Springer-Verlag, pp. 109-111.

Young, S.T. and J. S. Chang. 1997. "Implementation of a Patient-Centred and Physician-Oriented Healthcare Information System." Medical Informatics 22(3): 207-214.


Be the first to comment on this!

Related Articles

Note: Please enter a display name. Your email address will not be publically displayed