The HIT Report from KLAS: Emergency Department Systems Study; FOCUS: Eight ED Vendor Products Compared. Study Provides Valuable Insight into the Success of ED Systems.
PUTTING CLIENTS COMMENTS INTO PERSPECTIVE
KLAS professionals conducted 122 provider interviews as well as 12 with vendor executives. The titles of the provider respondents ranged from CIO, IT Director/Manager, ED Director/Manager, Medical Director, Nursing Director/Manager, to Business Administrator. The results of each interview were entered into a database where the actual respondent was kept confidential.
Clients surveyed in the study varied widely in size, ranging from under 50 beds to over 1,800 beds and multi-entity IDNs, and varied in ED visits per year from less than 25,000 to more than 100,000, with the majority logging between 25,000 and 75,000 visits per year.
The performance rating measurements address 39 criteria in the following information areas:
14 Primary Indicators of Performance (rating 1-9, where 1=poor and 9=excellent)
- Lived Up to Expectations
- Product Works as Promoted
- Vendor Is Improving
- Quality of Training
- Proactive Service
- Quality of Implementation
- Quality: Money's Worth
- Quality of Telephone Support
- Commitment to Technology
- Quality of Interface Services
- Executives Interested in You
- Third Party Product Works with Vendor Products
- Good Contracting Experience
- Helps Your Job Performance
Detail Performance Indicators (rating 1-9, where 1=poor and 9=excellent)
- Worth the Effort
- Quality of Documentation
- Real Problem Resolution
- Quality of Releases & Updates
- Good Job Selling
- Software Errors Corrected Quickly
- Product Quality Rating
- Interfaces Met Deadlines
- Implementation on Time
- Quality of Custom Work
- Implementation within Budget
- Hardware Vendor Satisfaction
- Quality of Implementation Staff
- Response Times
Business Indicators (rating is a yes or no)
- Implemented in the Last 3 Years
- Contract Is Complete
- Core Part of IS Plan
- Contract Administered Fairly
- Would You Buy It Again?
- Timely Enhancement Releases
- Avoids Nickel and Diming
- Support Costs as Expected
- Keeps All Promises
- Ranked Client's Best or Second Best Vendor
- A Fair Contract
In addition, specific questions were asked surrounding the
subject matter of:
- ED system live components
- Why ED system implemented
- Did you achieve your implementation goals?
- Quantifiable benefits of an ED system
- Exciting new tools
OVERALL PERFORMANCE RATINGThe overall performance rating is calculated from individual scores for each of the 39 performance ratings above. The performance indicators (primary and detail) account for 70% of the rating, while the business indicators account for the remaining 30%. The total score is based on a 100-point scale.
- A careful look at the Live Components of ED Systems (Figure 4)
shows measurable differences between the vendors and "what is
live." It also identifies patient tracking and discharge planning
as the two most popular live components, followed equally by
clinical integration of orders and results and nurse documentation
- The overwhelming reason why ED systems are implemented is for
patient tracking and to streamline operations. The next reason is
to decrease the length of stay (LOS) followed by electronic
- There is evidence that the system chosen is linked to the major
ED system objective.
- ED system users all speak about achieving benefits, but the
degree achieved varies from that which was originally anticipated,
ranging from 46% to 92%.
- Quantifiable benefits were in the double digits for better
tracking, better reporting, decreasing LOS, increasing
reimbursement, better charting/documentation and standardized
discharge instructions, in that order.
- Overwhelmingly, wireless laptops are the exciting new tool in use.
WHY ED AUTOMATION?Aided by recent advancements both in technology and application software, ED automation is now often championed by nurses and physicians alike, with some synergy from the IT department. With prerequisites often in place (data feeds that improve the value of ED automation from areas such as ancillary systems, ADT, order entry, CDRs, etc.) and enough experience with wireless, voice recognition, ergonomic devices and foreign system interfaces, ED implementations are having success. The vision of the value of information at one's fingertips is becoming a reality.
Five out of the eight vendor systems assessed have legacies that involve ED physician founders. This may be one reason why the products appear to meet the needs of the user and have such positive survey results. Also of interest is the fact that each product is known for its own strengths. There is no product that leads the pack in all categories. Clearly, "beauty is in the eyes of the beholder" and the system you are likely to purchase has more to do with your own needs and the primary business issue(s) you are trying to solve. The savvy ED purchaser should be looking for a "best fit" solution.
LESSONS LEARNEDBoth clients and vendors were willing to provide "words of wisdom" and their thoughts on the "key to success":
- Interfacing is one of the toughest pieces. You have to have
highly skilled staff to be successful.
- This system works best when both physicians and nurses are
doing the documentation … The nursing piece makes it easier,
so that the physician can add to or make changes to the nursing
- From our experience we have generally observed that clients
tend to undertrain their staff. Without good training and internal
selling, the success of any system may be subject.
- We look for those prospects with good IT departments. Without
this, a successful implementation is difficult. We also look for
key physician drivers. Without a physician leader, success again
may be limited.
- One huge barrier is that we didn't do our ADT interface
- My recommendation for others considering an EDIS would be to
set appropriate expectations up front and define them in the
contract. Make sure that there is always a written knowledge
- We began using voice recognition, but stopped using it because
it was not working the way we wanted it to.
- We have voice recognition. However, the ER is very loud and the
exterior sounds integrate into the voice file: the outside noise
contaminates the voice recognition.
- We recommend that they do thorough testing on any network that
the system is installed with. We have found that it just does not
work well in multiple domains (for example, a server domain, a
hospital domain and a workstation domain). We had some connectivity
issues when we moved software from one domain to the
- We hired a consultant to assist us with a re-engineering
project in the ED. One part of that process was to move from a
manual process to an automated process.
- You need to watch out for the cost on training. When the only
thing they give you in the contract is an estimate, you end up
paying for what you actually use. That came back to bite us. I
would negotiate this differently.
- It is really tough to make orders through an ED system. If I
were to do this again, I would simply assign orders through our
HIS. The ROI just doesn't seem to be there, when doing this through
- We set too lofty of a goal. Computers cannot solve all of our
- If I had my choice of anybody in the world to pick from for an ED software solution … I would look at functionality, ease of reporting, ease of integration into our systems, clinical data repository and costs.
OFFER TO CANADIAN HEALTHCARE FACILITIESMany Canadian healthcare facilities and organizations use vendors who supply solutions unique to Canada. KLAS - the organization rating IT vendors in this and subsequent issues of ElectronicHealthcare - is willing to gather confidential and candid data from Canadian healthcare executives and provide access to the results to all the data contributors. The vehicle that will be used is the web-based KLAS vendor evaluation form found at www.healthcomputing.com under the "rate your vendor" button. If you are interested in participating, please contact Mr. Ralph Reyes Jr., Senior Vice-President, KLAS Enterprises LLC. Telephone: 801-763-7252. E-mail: firstname.lastname@example.org.
If you have comments, suggestions or questions about this vendor-rating feature, please send an e-mail to email@example.com.
About the Author(s)
WHO WE ARE: KLAS, founded in 1996, is the only research and consulting
firm specializing in monitoring and reporting the performance of healthcare's information technology vendors (HIT). Our staff and advisory board average 25 years of healthcare information technology experience.
HOW WE SERVE THE HEALTHCARE INDUSTRY:
KLAS, in concert with thousands of healthcare executives, CIOs, directors, managers and clinicians, has created a dynamic database of information on the performance of (HIT) vendors. The KLAS database represents the opinions of healthcare executives, managers and clinicians from over 3,500 healthcare facilities on 200+ vendors and 300+ different products. The information is continually refreshed with new performance evaluations and interviews daily. The KLAS database is dynamically and effectively used by:
- Healthcare organizations to align expectations with a vendor's actual performance, to assist in strategic planning and to validate decision processes.
- Vendors to monitor their performance in comparison with competitors.
- Consultants for current performance information on a specific company or product.
- Healthcare investment firms to evaluate publicly traded HIT company trends.
Contact KLAS at www.healthcomputing.com or 866-268-9438.
Be the first to comment on this!
Personal Subscriber? Sign In
Note: Please enter a display name. Your email address will not be publically displayed