8. diy rating

1. Rating the Ratings
Part 8 – How to “DIY”
By Vince Ciotti & Elise Ames
HIS Professionals, LLC

2. Who Should You Believe?
• So who has the best large hospital EMR: Epic, Cerner or Allscripts?

3. © 2016 HIS Professionals, LLC
And the Best Physician Practice System?
• Should your practice select MediTouch, PracticeFusion or Epic?

4. © 2016 HIS Professionals, LLC
“Do It Yourself” (DIY) Rating
• It’s not that hard actually, and employs the very people most
impacted by the new system anyway: your end users.
• Just takes a few steps at minimal cost, possibly less than the
large fees some rating agencies charge for their reports.
• We developed this DIY approach during over 160 system
selections our firm has conducted in 25 years of consulting.
– Each search involved 5-6 vendors, so we vetted thousands
of HIS and MD systems using this “DIY” phone survey.
• Yeah, we’ve made a number of mistakes too, but learned
from them and thank our clients for helping us correct them.
• And there’s no ratings we can share (or sell!): the “right”
vendor varies with every unique hospital & practice…

5. © 2016 HIS Professionals, LLC
Step 1: Starts with an RFI
• First step is to start your selection process by issuing a “Request For
Information” (RFI) to the leading vendors being considered:
- That’s an area where these rating agencies might come in very
handy: identifying the 5-6 leaders for the apps you are buying –
not the “perfect” one to buy, but just the leading contenders.
- In the RFI, make vendors list 3-5 client references each for your:
• State – for EDI with your state’s Medicaid & Medicare Inter-
mediary, exchanging CCD with HIEs & RHIOs, PR taxes…
• Size - # of beds: CAH (<25), small (<100), mid-size (1-300),
Large (300+) or IDN (# of sites); and/or your # of physicians…
• Product – the release they are proposing: not some old version
barely supported, or ones they acquired and are sunsetting…
• Conversions - from your old system – if they ever did any…
• Recent Installs – over the past 1-3 years, not 20 years ago…

6. © 2016 HIS Professionals, LLC
Some Common Vendor Dodges
• We send out an Excel SS with 3-5 rows for each category on the
previous slide, and vertical columns for key info required for each:
- Hospital name, city & state, # of beds, & switchboard phone #.
- In the remaining columns, we list the key apps being considered
so the vendor can place an X under those the site has installed.
• Common tactics some vendor employ to dodge such requests are:
- Provide only their primary contact phone number, usually the CIO,
who they call in advance to make sure she’s happy (no sweat,
you can easily look up the switchboard # on their web site!)
- Stating their contract prevents any such reference calls – which
is bull-roar: nurses can’t talk about their EMR with other nurses?
- Vendors must join any reference calls – you can’t talk to their
clients without them being on the call. What are they afraid of?
- If the vendor persists with such doges – eliminate them. If you
can’t to talk with their clients, they’re hiding something!

7. © 2016 HIS Professionals, LLC
2. Peer to Peer Reference Calls
• Who should then call each client reference given? Not your:
– CIO & IT staff alone – who concentrate mostly on tech issues, or
– C-Suite – when is the last time your COO entered an order?
• But end users – the poor folks stuck with the system after go-live:
– Nurses – Informaticist(s) if you have any, shift supervisors and
any EMR “super users” for various floor and shifts.
– Physicians – CMIO if you have one, but the CMO and any super
users or IT champions on staff who build EMR screens & alerts.
– User Departments – not just the Directors/Managers, but the
super user or IT Coordinator for every department involved:
• Lab, Radiology RX, ED, HIM, Access, RCM, HIM, Finance…
• Why involve so many people in making phone reference calls?
– “Buy In” - if end users pick the system, they will make it work
despite the inevitable problems they’ll know about from the calls.
– If you pick the system, they’ll blame you for every problem…

8. © 2016 HIS Professionals, LLC
Step 3: Tracking the Calls
• It’s hard to get some end users to take time off their busy schedules
to call about a new system, so some techniques to get it done:
– Require users to fill out a checklist for each call and keep a SS
of how many each department turns in (samples follow).
– Share the results with your C-suite so they can harass those
user departments under them who aren’t cooperating.
• For departments who say
they can’t get through to
anyone, suggest they call
vendor users they might
know from state or
regional associations like
HIMSS, HFMA, ACHE,
AONE, etc.
• The more calls the merrier!

9. © 2016 HIS Professionals, LLC
4. Getting Through on the Phone
• Have your users call clients in each category of the RFI list:
– Your state, your size, your product, your apps, etc.
• Call the client’s switchboard and ask for their counterpart:
– Access (Admitting), RCM (Patient Accounting), ERP (finance),
Nursing (RN Informaticist, CNO, Nurse Supervisor…), etc.
• As busy as most healthcare employees are today, odds are they’ll
end up leaving a message, so call & leave many messages!
• Once you get through (or one calls you back), the conversation gets
very granular as users talk shop that only peers appreciate:
– RNs talk about OE & RR, eMAR, BMV, TPR, I&O, notes…
– Lab Directors chat about instrument interfaces, BB, Micro…
– MDs rant about the rotten CPOE, annoying alerts, scribes…
– CIOs & IT staff compare notes on DR, upgrades, interfaces…
• Miles deeper than any rating agency (or consultant) can ever dig!

10. © 2016 HIS Professionals, LLC
5. Sample Phone Reference Checklist
Hospital Caller: ____________________ Vendor: _____________________
Date of Call: __________ Hospital Called: ___________________________
Contact Name: _____________________ Title: ______________________
Answer the questions with 1 2 3 4 5
a according to this grid: Very Poor Ave. Good Very
Poor Good
1. What is your overall satisfaction? How do you rate separately their:
product (programs) vs. service (support)?
2. How would rate user friendliness:
b. Help key?
implement?
budget?
flexibility?
training?
hardware?
how would you rate ongoing support:
Phone support? On-Site Visits ?
User Groups? Periodic releases ?

11. © 2016 HIS Professionals, LLC
Step 6: Score the Results
• Next step is to
score the phone
checklists in a SS
like this one:
• Questions must be
answered in a
numeric scale, e.g.:
1-to-5 or 1-to-10.
• Have a separate
page for each user
dept. so they see
their results.
• Hand out the results
in a meeting where
each user relays
what they heard
about each vendor.

12. © 2016 HIS Professionals, LLC
The Net
• This DIY rating approach may give you far more relevant data on
vendors than just trusting a third-party “objective” rating agency:
- Our scores average about 70 points, with the best in the 80s & the
worst in the 60s, far more realistic numbers than the near-perfect
scores some rating agencies give to paying vendors.
- It heavily involves end users, and teaches them that they are
picking the “least of the evils,” rather than a perfect system.
- You can apply the numeric checklist & scoring to several other
steps in the selection process as well: demos, site visits, etc.
• Hope you enjoyed the series – we bet the raters give it a low score!
- Elise Ames
- 413.329.6925, eames@hispros.com
- Vince Ciotti
- 505.466.4958, vciotti@hispros.com