Princess Health and "Meaningful Use" not so meaningful: Multiple medical specialty societies now go on record about hazards of EHR misdirection, mismanagement and sloppy hospital computing. Princessiccia

Princess Health and "Meaningful Use" not so meaningful: Multiple medical specialty societies now go on record about hazards of EHR misdirection, mismanagement and sloppy hospital computing. Princessiccia

AMIA healthcare IT dissatisfaction healthcare IT safety Kaizen Event Karen De Salvo management mysticism MAUDE meaningful use

Gusty 06:02 Add Comment

The "Meaningful Use" program for EHRs is a mismanaged boondoggle causing critical issues of patient safety, EHR usability, etc. to be sidestepped.

This is on top of the unregulated U.S. boondoggle which should probably be called "the National Programme for IT in the HHS" - in recognition of the now-defunct multi-billion-pound debacle known as the National Programme for IT in the NHS (NPfIT), see my Sept. 2011 post "NPfIT Programme goes PfffT" at http://hcrenewal.blogspot.com/2011/09/npfit-programme-going-pffft.html.

The complaints are not just coming from me now.

As of January 21, 2015 in a letter to HHS at: http://mb.cision.com/Public/373/9710840/9053557230dbb768.pdf, they are now coming from the:

American Medical Association
AMDA � The Society for Post-Acute and Long-Term Care Medicine
American Academy of Allergy, Asthma and Immunology
American Academy of Dermatology Association
American Academy of Facial Plastic
American Academy of Family Physicians
American Academy of Home Care Medicine American Academy of Neurology
American Academy of Ophthalmology
American Academy of Otolaryngology�Head and Neck Surgery
American Academy of Physical Medicine and Rehabilitation
American Association of Clinical Endocrinologists
American Association of Neurological Surgeons
American Association of Orthopaedic Surgeons
American College of Allergy, Asthma and Immunology
American College of Emergency Physicians
American College of Osteopathic Surgeons
American College of Physicians
American College of Surgeons
American Congress of Obstetricians and Gynecologists
American Osteopathic Association
American Society for Radiology and Oncology
American Society of Anesthesiologists
American Society of Cataract and Refractive Surgery and Reconstructive Surgery
American Society of Clinical Oncology
American Society of Nephrology
College of Healthcare Information Management Executives
Congress of Neurological Surgeons
Heart Rhythm Society
Joint Council on Allergy, Asthma and Immunology
Medical Group Management Association
National Association of Spine Specialists
Renal Physicians Association
Society for Cardiovascular Angiography and Interventions
Society for Vascular Surgery

In the letter to Karen B. DeSalvo, National Coordinator for Health Information Technology at HHS, these organizations observe:

Dear Dr. DeSalvo:

The undersigned organizations are writing to elevate our concern about the current trajectory of the certification of electronic health records (EHRs). Among physicians there are documented challenges and growing frustration with the way EHRs are performing. Many physicians find these systems cumbersome, do not meet their workflow needs, decrease efficiency, and have limited, if any, interoperability.

Of course, my attitude is that we need basic operability before the wickedly difficult to accomplish and far less useful (to patients) interoperability. 
 

... Most importantly, certified EHR technology (CEHRT) can present safety concerns for patients. We believe there is an urgent need to change the current certification program to better align end-to-end testing to focus on EHR usability, interoperability, and safety.

Let me state what they're saying more clearly:

"This technology in its present state is putting patients at risk, harming them, and even killing them, is making practice of medicine more difficult, is putting clinicians at liability risk, and the 'certification' program is a joke."

... We understand from discussions with the Office of the National Coordinator for Health Information Technology (ONC) that there is an interest in improving the current certification program. For the reasons outlined in detail below, we strongly recommend the following changes to EHR certification:

1. Decouple EHR certification from the Meaningful Use program;
2. Re-consider alternative software testing methods;
3. Establish greater transparency and uniformity on UCD testing and process results;
4. Incorporate exception handling into EHR certification;
5. Develop C-CDA guidance and tests to support exchange;
6. Seek further stakeholder feedback; and
7. Increase education on EHR implementation.

Patient Safety
Ensuring patient safety is a joint responsibility between the physician and technology vendor and requires appropriate safety measures at each stage of development and implementation.

I would argue that it's the technologists who have butted into clinical affairs with aid from their government friends, thus the brunt of the ill effects of bad health IT should fall on them.  However, when technology-related medical misadventures occur, it's the physicians who get sued.

... While training is a key factor, the safe use of any tool originates from its inherent design and the iterative testing processes used to identify issues and safety concerns. Ultimately, physicians must have confidence in the devices used in their practices to manage patient care. Developers must also have the resources and necessary time to focus on developing safe, functional, and useable systems.

Right now, those design and testing processes compare to those in other mission-critical sectors employing IT quite poorly.

Considering fundamental stunningly-poor software quality that I've observed personally, such as lack of appropriate confirmation dialogs and notification messages supporting teamwork, lack of date constraint checking (see my report to FDA MAUDE at http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfmaude/detail.cfm?mdrfoi__id=1729552 and many others at http://hcrenewal.blogspot.com/2011/01/maude-and-hit-risk-mother-mary-what-in.html), and other fundamentals, I would say grade schoolers could probably have done a better job of safety testing than the vendors and IT amateur-implementers of the major systems I observed did. 

... Unfortunately, we believe the Meaningful Use (MU) certification requirements are contributing to EHR system problems, and we are worried about the downstream effects on patient safety.

In other words, computers and the government thirst for data do not have more rights than patients.  In the current state of affairs, as I have observed prior, computers do seem to have more rights than patients and the clinicians who must increasingly use them.

... Physician informaticists and vendors have reported to us that MU certification has become the priority in health information technology (health IT) design at the expense of meeting physician customers� needs, patient safety, and product innovation. We are also concerned with the lack of oversight ONC places on authorized testing and certification bodies (ATCB) for ensuring testing procedures and standards are adequate to secure and protect electronic patient information contained in EHRs.

Not just security, but patient safety also.  See for example my Feb. 2012 post "Hospitals and Doctors Use Health IT at Their Own Risk - Even if 'Certified'" at http://hcrenewal.blogspot.com/2012/02/hospitals-and-doctors-use-health-it-at.html.

Read the entire letter at http://mb.cision.com/Public/373/9710840/9053557230dbb768.pdf.

Sadly, while on the right track regarding the problems of bad health IT, the societies take a Milquetoast approach to correction:

... In May 2014, stakeholders representing accredited certification bodies and testing laboratories (ACB & ATL), EHR vendors, physicians, and health care organizations provided feedback to ONC on the complexities of the current certification system. Two main takeaways from these comments were for ONC to host a multi-stakeholder Kaizen event and to prioritize security, quality measures, and interoperability in the EHR certification criteria. We strongly support both of these ideas...

A multi-stakeholder "Kaizen event'?  (http://en.wikipedia.org/wiki/Kaizen)

That's one recommendation I find disappointing.  The industry plays hard politics, and organized medicine wants to play touchy-feely "good change" management mysticism with that industry and their government apparatchiks.  That's how organized medicine wants patients and the integrity of the medical profession to be protected from the dysfunctional health IT ecosystem (see http://cci.drexel.edu/faculty/ssilverstein/cases/?loc=cases&sloc=ecosystem)?  

When I originally created my old website called "Medical informatics and leadership of clinical computing" back in 1998, Kaizen events were not exactly what I had in mind.

Finally, the American Medical Informatics Association (http://www.amia.org) was apparently not informed of this letter, nor did it participate in its drafting.  While this is regrettable, as the organization is the best reservoir of true Healthcare Informatics expertise, I opined to that group that this may have been due to the organization's tepid response to bad health IT and to industry control of the narrative, and the problems these issues have caused for physicians and other clinicians. The lack of AMIA leadership regarding bad health IT is an issue I've been pointing out since the late 1990s. AMIA has been largely a non-critical HIT promoter.  That stance has contributed to the need for this multiple-medical specialty society letter in the first place.

Parenthetically, and for a touch of humor about an otherwise drab topic: Here's an example of how management mysticism plays out in pharma.

It's meant to be satirical, but captures reality all too well, in fact scarily so at times:

Management mysticism and muddled thinking.  See https://www.youtube.com/watch?v=kwVjftMMCIE

In pharma, as well as in hospital IT in my days as CMIO, gibberish like this was real.  I imagine it's no different in many hospital management suites these days.

-- SS

1/28/2015  Addendum:

Per a colleague:

FierceHealthIT (1/28) reports, �It�s time for the American Medical Association and more than 30 other organizations urging change in the electronic health record certification process to be part of the solution, former Deputy National Coordinator for Health IT Jacob Reider said in a blog post.� Reider said, �So far, I don�t see much [any?] engagement from the AMA or the others who signed the letter. It�s relatively easy to write a letter saying someone else is responsible for solving problems. Time to step up to the plate and participate in the solutions, folks!"

Regarding the victims of compelled use of bad health IT, this erstwhile health IT leader opines "It's relatively easy to write a letter saying someone else is responsible for solving problems?"

That is simply perverse.

I ask: why are we in the midst of a now-compelled national rollout with Medicare penalties for non-adopters when a former government official once responsible for the technology remarks that it's apparently not the makers' problem and that it's "time to step up to the plate and participate in the solutions, folks [a.k.a. end users]!"

(One wonders if Reider believes those who step up to the plate are entitled to fair compensation for their aid to an industry not exactly known for giving its products away, free.)

It seems to me it's not up to (forced) customers to find solutions to vendor product problems, some deadly.

It's the responsibility of the sellers.

Put more bluntly, Reider's statement is risible and insulting.

I've already opined the following to the AMA contact at the bottom of the letter:

... Relatively milquetoast approaches such as multi-stakeholder Kaizens are not what I had in mind ... A more powerful stance would be to advise society members to begin to avoid conversion, report on bad health IT, and even boycott bad health IT until substantive changes are realized in this industry.

That's "stepping up to the plate" to protect patients, in a very powerful way.

-- SS

Princess Health and A new "Better EHR" book and an observation re: health IT regulation, health IT amateurs, and user centered design (UCD) - "responding to user feature requests or complaints?". Princessiccia

Princess Health and A new "Better EHR" book and an observation re: health IT regulation, health IT amateurs, and user centered design (UCD) - "responding to user feature requests or complaints?". Princessiccia

AMIA healthcare IT amateur healthcare IT regulation user centered design

Gusty 14:39 Add Comment

A new book has appeared on improving usability of electronic health records.  The result of government-sponsored work, the book is available free for download.  It was announced via an AMIA (American Medical Informatics Association, http://www.amia.org/) listserv, among others:

From: Jiajie Zhang [support@lists.amia.org]
Sent: Tuesday, December 02, 2014 6:00 PM
To: implementation@lists.amia.org
Subject: [Implementation] - New Book on EHR Usability - "Better EHR: Usability, Workflow, and Cognitive Support in Electronic Health Records"

Dear Colleagues,

We are pleased to announce the availability of a free new book from the ONC supported SHARPC project: "Better EHR: Usability, Workflow, and Cognitive Support in Electronic Health Records".The electronic versions (both pdf and iBook) are freely available to the public at the following link:https://sbmi.uth.edu/nccd/better-ehr/

First, this book appears to be a very good resource at understanding issues related to EHR usability.  I particularly like the discussion of cognitive issues.

However, this book also holds messages about the state of the industry and the issue of regulation vs. no regulation, and impairment of innovation:

I think it axiomatic that user-centered design (UCD) is a key area for innovation, especially in life-critical software like clinical IT.  (I would opine that UCD is actually critical to safety and efficacy of these sophisticated information systems in a sociotechnically complex setting.)

I think it indisputable that the health IT industry has been largely unregulated for most of its existence, in the manner of other healthcare sectors such as pharma and traditional medical devices.

Yet, even in the absence of regulation, the book authors found this, per Section 5 - EHR Vendor Usability Practices:

a)  A research team of human factors, clinician/human factors, and clinician/informatics experts visited eleven EHR vendors and conducted semi-structured interviews about their UCD processes. "Process" was defined as any series of actions that iteratively incorporated user feedback throughout the design and development of an EHR system. Some vendors developed their own UCD processes while others followed published processes, such as ISO or NIST guidelines.

Vendor recruitment. Eleven vendors based on market position and type of knowledge that might be gained were recruited for a representative sample (Table 1). Vendors received no compensation and were ensured anonymity.

and

b)  RESULTS
Vendors generally fell into one of three UCD implementation categories:

Well-developed UCD: These vendors had a refined UCD process, including infrastructure and the expertise to study user requirements, an iterative design process, formative and summative testing. Importantly, these vendors developed efficient means of integrating design within the rigorous software development schedules common to the industry, such as maintaining a a network of test participants and remote testing capabilities. Vendors typically employed an extensive usability staff.

Basic UCD: These vendors understood the importance of UCD and were working toward developing and refining UCD processes to meet their needs. These vendors typically employed few usability experts and faced resource constraints making it difficult to develop a rigorous UCD process.

Misconceptions of UCD: These vendors did not have a UCD process in place and generally misunderstood the concept, in many cases believing that responding to user feature requests or complaints constituted UCD. These vendors generally did not have human factors/usability experts on staff. Leadership often held little appreciation for usability.

About a third of our vendor sample fell equally into each category.

In other words, a third of health IT sellers lacked the resources to do an adequate job of UCD and testing; and a third did not even understand the concept.

Let me reiterate:

In an unregulated life-critical industry, a third of these sampled sellers thought 'responding to user feature requests or complaints constituted UCD'.  And another third neglected UCD due to a 'lack of resources'.

I find that nothing short of remarkable.

I opine that this is only possible in healthcare in an unregulated healthcare sector.

Regulation, for example, that enforced good design practices and good manufacturing practices (GMP's) could, it follows, actually improve clinical IT innovation considering the observations found by these authors, through ensuring those without the resources either found them or removed themselves from the marketplace, and by making sure those sellers that did not understand such a fundamental concept either became experts it UCD, or also left the marketplace.

I can only wonder in what other fundamental(s) other sellers are lacking, hampering innovation, that could be improved through regulation.

As a final point, arguments that regulation hampers innovation seems to assume a fundamental level of competency and good practices to start with among those to be freed from regulation. In this case, that turns our to be an incorrect assumption. 

As a radio amateur, I often use the term "health IT amateurs" to describe persons and organizations who should not be in leadership roles in health IT, just as I, as a radio amateur, should not be (and would not want to be) in a leadership role in a mission-critical telecommunications project.

I think that, inadvertently, the writers of this book section gave real meaning to my term "health IT amateurs."  User centered design is not a post-accident or post-mortem activity.

-- SS

12/4/2014 Addendum:

I should add that in the terminology of IT, "we don't have enough resources" - a line I've heard numerous times in my CMIO and other IT-related leadership roles - often meant: we don't want to do extra work, to reduce our profits (or miss our budget targets), or hire someone who actually knows what they're doing because we don't really think that the expertise/tasks in question are really that important.

In other cases, the expertise is present. but when those experts opine an EHR product will kill people if released, they find the expert 'redundant', e.g., http://cci.drexel.edu/faculty/ssilverstein/cases/?loc=cases&sloc=lawsuit.

Put in more colloquial terms, this is a slovenly industry that has always made me uncomfortable, perhaps in part due to my experience having been a medical safety manager in public transit (SEPTA in Philadelphia), where lapses in basic safety processes could, and did, result in bloody train wrecks.

Perhaps some whose sole experience with indolence and incompetence-driven catastrophe has been in discussions over coffee in faculty lounges cannot appreciate that viewpoint.

Academic organizations like AMIA could do, and could have done, a whole lot more to help reform this industry, years ago.

-- SS

Princess Health and Australian ED EHR Study: An End to the Line "Your Evidence Is Anecdotal, Thus Worthless?" .Princessiccia

Princess Health and Australian ED EHR Study: An End to the Line "Your Evidence Is Anecdotal, Thus Worthless?" .Princessiccia

AMIA healthcare IT anecdote Jon Patrick OHSU William Hersh

Gusty 06:55 Add Comment

At my Sept. 2010 post "The Dangers of Critical Thinking in A Politicized, Irrational Culture" I lamented that while my early mentor in biomedicine Victor P. Satinsky MD taught the wise credo "critical thinking always, or your patient's dead" in the 1970's, our culture had become so perverse that this credo had been largely supplanted with:

"Critical thinking anytime, and your career's dead."

That post was in reaction to continued heckling on a professional mailing list, the American Medical Informatics Association (AMIA) Clinical Information Systems Working Group (cis-wg). On this list, Oregon Health Sciences University professor William Hersh expounded on how the evidence of health IT dangers was largely "anecdotal" therefore to be discounted, and how I, specifically, "didn't know the literature on health IT."



In that post I put the lie to the latter figment. On the former prevarication Dr. Jon Patrick, author of the recent thorough dissection of problems with the ED EHR system being rolled out in public hospitals in New South Wales, Australia (see my Mar. 5, 2011 post "On an EMR Forensic Evaluation by Professor Jon Patrick from Down Under: More Thoughts"), hit the ball out of the park:

Prof. Patrick to AMIA cis-wg:



I think such defences are particularly unuseful especially with respect to the dismissal of personal stories and experiences as "anecdotes", hence committing them to the realm of folklore. I offer these notions as a counterpoint.



Discounting Anecdotes:



1. Is a perfidious and specious act.



2. It denies early warning signs of problems.



3. It denies a voice and disempowers the working clinical community who have to operationalise decisions made by others.



4. It denies a route to process improvement within an institution - which is most important for EBM and incremental review of local processes.



5. It defends software manufacturers from fault rectification - cuts off even a need to deliberate on it. Critics of the value of anecdotes are squarely on the side of the faulty and deficient manufacturer.



6. A rule of project management is that projects consist of 3 components, cost, quality and time and if their needs to be a compromise it has to be on quality. Anecdotes are early warning signs of such a compromise.

Prof. Patrick had to once again put the lie to this refrain at a comment on the HISTalk blog yesterday, where the anonymous proprietor had written in a review of Patrick's EHR study:

... On the other hand, I wouldn�t say it�s [Prof. Patrick's Cerner FirstNet study] necessarily unbiased, it focuses on implementation of a single department application that didn�t go well for a variety of reasons (despite many successful FirstNet implementations elsewhere), it uses the unchallenged anecdotal comments of unhappy users who make it clear they liked their previous EDIS better, and it nitpicks (I wasn�t moved to find a pitchfork when I learned that the primary keys in the Millennium database aren�t named consistently).



[Considering the complexity and changeability of healthcare and the corresponding software lifecycle, I duly note that that latter attitude about 'failing to find pitchforks' regarding breaches of sound software engineering practices seems to be a symptom of the larger health IT disease that Prof. Patrick writes about - ed.]

Dr. Patrick then knocked the ball out of the Southern Hemisphere with a comment of his own about anecdotalism and a link to an expansion of the aforementioned ideas he'd shared on "discounting anecdotes":

Prof. Patrick to HISTalk owner:



Your Comment �it uses the unchallenged anecdotal comments of unhappy users � is not only unfair but unreasonably inaccurate. The comments made by the users are the Directors of 7 EDs and so they have a right to carry authority by virtue of the experience but also the number of 6 out 7 presenting a view of Firstnet as unfit for purpose has numeric validity, which they justify with about 20+ pages of their comments � see Part 2 Appendix 2.



I would also point readers to my editorial about the role of personal experiences being the most useful information to understand the nature of socio-technical failures. http://aci.schattauer.de/en/contents/archive/issue/1124/manuscript/15463/show.html

The essay at that link, "The Validity of Personal Experiences in Evaluating HIT", is an editorial in Johns Hopkins informaticist Chris Lehmann's brilliant new journal "Applied Clinical Informatics."



The editorial is available free, and is a must-read for anyone in a decision-making or managerial role in mission critical domains, including our elected representatives.



In the editorial Dr. Patrick concludes. similarly to his earlier AMIA mailing list opinion:

... the denial of recounted personal experiences in discussion and analysis of HIT is biased and specious and has the effect of:



1. Denying early warning signs of problems.



2. Denying a voice for the working clinical community who have to operationalise decisions made by others and thus disempowers them.



3. Denying process improvement within an institution � which is most important for Evidence Based Medicine and incremental review of local processes.



4. Discourages staff from engaging in any form of process improvement hence worsening the sense of disenchantment.



Every legitimate personal experience of a HIT deserves to be considered on its merits lest we wish to retreat from process and product improvement. Mechanisms of censorship both implicit due to contrived processes of disinformation and disempowerment or explicit due to contractual specifications will lead to more waste, lost productivity, contempt for the providers, and distress among frontline staff rather than increased productivity and improved patient health and safety as we all desire.

In my view, the drivers or motivators for the "anecdotalist" accusation are these, singly or in combination:

• Too much "education" to see the nose on one's face, as in, to think zebras and unicorns instead of horses when hearing hoofbeats outside one's midwest U.S. abode (eggheads);
  
  
• Too little common sense (fools), as in Scott Adams' example: "IGNORING ALL ANECDOTAL EVIDENCE - Example: I always get hives immediately after eating strawberries. But without a scientifically controlled experiment, it�s not reliable data. So I continue to eat strawberries every day, since I can�t tell if they cause hives";
  
  
• Too much concern for the possible interruption of flow of money or power in one's direction (gonifs).
  
  

In conclusion, the anecdotalist refrain of "your evidence is anecdotal" [therefore of little or no value] when used repetitively against competent observers is the refrain of eggheads, fools and gonifs.



In healthcare, the end result is "your patient's dead."



My "anecdotal relative" injured in a mid-2010 HIT mishap is sadly an example.



[June 2011 addendum: my relative, after much suffering, has now died of complications of the "anecdotal HIT mishap" - ed.]



As for myself, I am a Markopolist (see my Sept. 2010 post "Health IT: On Anecdotalism and Totalitarianism").



-- SS