ElectronicHealthcare 1(1) May 2001 : 75-82

Research Distilled: Enabling Intelligent Decisions; Focus: Healthcare Information Systems



KLAS enables intelligent decisions in the healthcare industry by compiling reports on areas of interest in Healthcare Information Technology (HIT). One of the most comprehensive HIT reports is focused on Acute-Care Healthcare Information Systems. This article is an extract from that report which focuses on the following vendors: Eclipsys SDK, IDX LastWord, McKessonHBOC Star, Meditech Client/Server, Siemens Invision, and QuadraMed Affinity.


The study utilizes 38 factors to measure the success of vendor HIT solutions. Using these factors the clients of each of the identified vendors ranked/rated their direct post implementation experience utilizing the vendor's product. These ratings represent the opinion of a vendor's most important critic, their own clients.

The integrity of survey data is paramount to KLAS and no single data point is accepted until a personal interview is completed with each person providing the opinion (typically a CIO or I.S. Director). Key areas covered during the interviews include: interfacing, ongoing support, cost, quality of product/implementation and several others. The objective of the personal interview is to obtain candid information regarding: a) What does the vendor do well? b) What areas need improvement? c) What observations/comments can be shared? d) What is the future outlook for this product or company? The specific comments are then divided into eight areas (e.g. implementation, training, support, documentation, functionality, upgrades/releases, technology, and interfacing), which are further divided in positive and negative issues for each area.


The data was gathered from healthcare professionals representing a total of 553 facilities: 45 facilities with QuadraMed Affinity, 34 facilities with Meditech Client/Server (many more using the MAGIC version), 115 facilities with IDX LastWord, 174 facilities with McKessonHBOC Star, 148 facilities with Siemens Invision, and 37 facilities with Eclipsys SDK. KLAS selected three groupings "Partnering," "Quality," and "Business Indicators" from which to report the measurement points relating to these areas. For the purpose of this report only 22 of the 38 measurement points from the KLAS evaluation are used.


I. Partnering: the performance measures in this grouping included:

  • Is the vendor proactive?
  • Are the vendor executives interested in you as a client?
  • Did you have a good contracting experience?
  • Does the product work as promoted?
  • Does the vendor/product improve your job performance?
  • Was it worth the effort?

Final Ratings (scoring is 1 to 9, with 1=very poor and 9=excellent):

  • QuadraMed Affinity 6.95
  • IDX LastWord 6.28
  • Meditech Client/Server 6.24
  • McKessonHBOC Star 6.04
  • Siemens Invision 5.51
  • Eclipsys SDK 4.57

II. Quality: the performance measures in this grouping included:

  • Did you get your money's worth?
  • Quality of the training
  • Quality of implementation
  • Quality of telephone support
  • Quality of interfaces
  • Quality of implementation staff
  • Quality of documentation
  • Quality of releases and updates
  • Quality of custom work

Ratings (scoring is 1 to 9, with 1=very poor and 9=excellent):

  • QuadraMed Affinity 6.85
  • IDX LastWord 6.31
  • McKessonHBOC Star 6.05
  • Meditech Client/Server 5.88
  • Siemens Invision 5.86
  • Eclipsys SDK 4.41

III. Business Indicators: the performance measures in this grouping included:

  • Would you buy again?
  • Does the vendor nickel and dime you?
  • Does the vendor keep promises?
  • Is it a fair contract?
  • Was the contract considered complete?
  • Was the contract administered fairly?
  • Were support costs as expected?

Ratings (scoring is 1 to 9, with 1=very poor and 9=excellent):

  • Meditech Client/Server 83.09%
  • QuadraMed Affinity 80.67%
  • IDX LastWord 75.66%
  • McKessonHBOC Star 67.83%
  • Siemens Invision 62.97%
  • Eclipsys SDK 56.89%


Of the vendors surveyed, the top three ratings went to #1 QuadraMed Affinity, #2 IDX LastWord, and #3 Meditech Client/Server.

  1. QuadraMed Affinity: While many analysts/CIOs have been surprised by the continued success of QuadraMed's Affinity solution, Affinity continues to focus on client needs and for the most part have been responsive. Issues that were felt to be roadblocks for success with Affinity were: mumps language, sagging sales, changes in management, lack of "integration suite of products" have not materialized. It is stated by their clients that while Affinity does not offer a suite of solutions, they do offer companion/partnership solutions that have complimented Affinity's offerings.
  2. IDX LastWord's recent standing in the acute care reports has re-enforced three key factors of their successful ratings: 1) Client references at the CMO, physician and CIO level have, for the most part, solid vendor supporters. LastWord's strong base of academic facilities has provided their referral foundation. 2) Their large multi-entity operations represent leading healthcare systems in the U.S. 3) Their Tandem technology, while not now considered as visionary or bleeding edge, has proven to be a solid and stable product. With generally better system response times and fewer IT FTEs' than their competitor's implementations.
  3. Meditech Client/Server (C/S) continues its forward march. Monitoring Meditech C/S for several years has been fascinating. Their C/S version improved (up 28%) to such an extent in 2000 that it exceeded their tried and true MAGIC version. Their C/S version has attempted to take advantage of the newer architecture and provide the integration sought by their clients. C/S had been quite a challenge to the early adapters, who routinely complained of such simple "technology glitches" as printer problems. A second C/S user concern has been the extent of the server farm necessary to run the system. Meditech C/S' last challenge is to prove its ability to handle the larger integrated delivery networks and still maintain the strategy of keeping user customization controlled.

Upcoming major projects will include detailed studies on PACS, document imaging and EMPI solutions. Other currently available reports include:

  • Five key clinical information systems
  • Clinical Alerts and Rules - How prevalent are they?
  • RIS systems
  • LIS systems
  • Where are we with Data Warehousing?
Many Canadian Healthcare facilities and organizations use vendors who supply solutions unique to Canada. KLAS - the organization rating IT vendors in this and subsequent issues of ElectronicHealthcare - is willing to gather confidential and candid data from Canadian healthcare executives and provide access to the results to the all data contributors. The vehicle that will be used is the web based KLAS vendor evaluation form found at www.healthcomputing.com under "rate your vendor" button. If you are interested in participating please contact Mr. Ralph Reyes Jr.,
Senior Vice President, KLAS Enterprises LLC. Telephone: 281-257-9993. Email: ralph@healthcomputing.com.

If you have comments, suggestions or questions about this vendor-rating feature please send an email to editors@longwoods.com

Additional Notes

KLAS reports that they have over 3000 facilities reporting and that the information comes from CIOs, Department Directors and other IT Executives. Overall some 300 products are represented from over 180 vendors. The material presented in the column "Research Distilled" (pg. 75) is very current while the summary information on this page represents year 2000.

Some of the vendors in the KLAS Performance Database

Vendors # of Products Reports
M-HBOC 1724
Cerner 1025
SMS 707
IDX 589
Sunquest 572
Eclipsys 446
PeopleSoft 304
3M 234
Meditech 219
Per-Se 212
EPIC 174
Lawson 153
ADAC 149
Medscape 144
GEAC 102
SCC 101
Tempus 76
Softmed 72
QuadraMed 71
Mediware 66
Medic 64
Health Man. Sys. 58
Medical Manager 56


Summary Ration for Year 2000- Major Acute Care Vendors

Rank Vendor/
Average Rating

( 1-9)

Rating% Change % Rating as Best Vendor Buy Again Keeps Promises Upport Costs As Expected Fair Contract Nickel andColor Diming
1. Quadra-
Med Affinity
6.92 -1% 50% B B B B C
2. Meditech Client/
6.76 +28% 14% A B A A A
3. IDX LastWord 6.55 Even 40% C B C B A
4. Meditech MAGIC 6.49 -4% 35% B B B B B
6.03 3% - 34% C C C C N
6. SMS Invision 5.99 +3% 24% C C C C N
7. MHBOC Series2000 5.94 +6% 32% C C C B C
8. SMS Med-
5.63 -2% 42% C C N C C
9. Cerner Millen-
5.51 +6% 19% B C C B C
10. MHBOC Pathways CIS 5.50 +10% 12% C N C C C
11. Eclipsys SDK 5.27 -3% 20% N N A C C
12. SMS Unity 4.74 +3% 17% N C N N N
13. MHBOC Health-
4.47 Even 0% C N N N N
14. MHBOC Precision 4.07 -22% 0% N N N N N
15. SMS Allegra 4.03 -12% 6% C N N N N
Column Explanation:  
Average Rating: Mean of all ratings given by participants on all questions. Ratings are from 1-9 with 1 as low and 9 as high.
Rating % Change: % Increase or decrease from Spring 2000 average rating to Fall 2000 average rating.
% Rated Best Vendor: This is the % of the time that CIOs and Directors rate this vendor as their best vendor.
Buy Again: Grade based upon average rating from Key Indicator area "Would you buy from this vendor again?"
Keeps Promises: Grade based upon average rating from Key Indicator area "Has vendor kept ALL promises?"
Support Costs As Expected: Grade based upon average rating from Key Indicator area "Support costs are as expected, no surprises?"
Fair Contract: Grade based upon average rating from Key Indicator area "Has the contract turned out to be fair?"
Nickel and Diming: Grade based upon average rating from Key Indicator area "Vendor avoids charging for every little thing, no nickel and diming?"
Key Indicator Grade: The letter grades come from the combined overall ratings of the Key Indicators Areas: Fair Contract, Would Buy Again, Keeps All Promises, Support Costs As Expected, Contract Administered Fairly, Avoids Nickel and Diming. The top 20% come home with As, the next 30% get Bs, the next 30% Cs, and the last 20% take Ns (for Needs Improvement).

About the Author(s)

KLAS, founded in 1996, is the only research and consulting firm specializing in monitoring and reporting the performance of healthcare's information technology vendors (HIT). Our staff and advisory board average 25 years of healthcare information technology experience.

KLAS, in concert with thousands of Healthcare Executives, CIOs, Directors, Managers and Clinicians has created a dynamic database of information on the performance of (HIT) vendors. The KLAS database represents the opinions of healthcare executives, managers and clinicians from over 3000 healthcare facilities on 180+ vendors and 300+ different products. The information is continually refreshed with new performance evaluations and interviews daily. The KLAS database is dynamically and effectively used by:

  • Healthcare organizations to align expectations with a vendor's actual performance, to assist in strategic planning and to validate decision processes.
  • Vendors to monitor their performance in comparison with competitors.
  • Consultants for current performance information on a specific company or product.
  • Healthcare investment firms to evaluate publicly traded HIT company trends.

Contact KLAS at www.healthcomputing.com or 866-268-9438.


Be the first to comment on this!

Note: Please enter a display name. Your email address will not be publically displayed