Checklists are still important, but not a ‘one and done’
A STUDY AT UPMC HEALTH SYSTEM CONFIRMS IT’S TIME TO APPLY USER-CENTRED DESIGN TO CHECKLISTS
Monday, January 29, 2018 – Ellen Gardner
The vibe around checklists – especially after the publication of Atul Gawande’s widely acclaimed The Checklist Manifesto (2009) – is pretty positive. In many industries, particularly aviation where the step-by-step checklist originated, it’s central to the maintenance of safe practice.
There is, however, a fly in the checklist ointment and Gawande found it in his own industry – medicine. In Checklists he says that although all medical procedures involve risk, uncertainty and complexity, many physicians are not using the checklists. He uses his own experience in the operating room to coax physicians to commit to them, saying “I have yet to get through a week in surgery without the checklists leading us to catch something we would have missed.”
For some clinicians Gawande’s checklist evangelism makes him somewhat of an idealist; Dr. Jeremy Kahn admits he has a more cynical approach. He laughingly says the best path is probably somewhere in the middle. “Checklists have caught the zeitgeist in healthcare and they make strong intuitive sense,” says the Director of Health Policy and Management at CRISMA (Clinical Research Investigation and Systems Modeling of Acute Illness) in Pittsburgh.
Dr. Kahn was speaking at the Critical Care Canada Forum, an annual event that he says brings together the most influential clinicians in the world.
Although research is his first love, for six weeks of the year Kahn goes back into the ICU. “I’m glad it’s only for a limited time, but this is also very relevant to what I’m doing in my research life,” he says. “I get to see first-hand how we’re failing to deliver evidence-based care.”
He was completely optimistic when the checklists were rolled out three years ago in the ICUs at the Magee-Women’s Hospital of UPMC (University of Pittsburgh Medical Center). In practice, things did not go as expected. “The reality of critical care is that we know we should be using them, but the default is not to bother with it,” says Kahn. “There’s a lot of guilt, but that isn’t enough to nudge us into doing the right thing.”
So what’s changed since the groundbreaking discovery made by Johns Hopkins critical care specialist Dr. Peter Pronovost in 2001? His insistence that clinicians follow a simple checklist of the five steps necessary to take when inserting a central venous catheter had dramatic results over the year they were being observed: the ten day line infection rate went from 11 per cent to zero. They followed the patients for another 15 months and only two line infections occurred during that period.
THE PRACTICAL AND EXISTENTIAL PROBLEMS
The life-saving checklist that Pronovost was promoting has grown significantly from its simple five steps – and that could be part of the problem. “Many of the topics on the list are not relevant to everyone on the team,” says Kahn. “If it’s outside your field of interest, the checklist will not be perceived as useful.”
As Kahn tells it, the basis for the problems with checklists is grounded in two realms – the practical and the existential. “On the practical side, the checklist is time-consuming and feels redundant because we typically do it after rounds and many of the items on the list have already been discussed,” he says.
The second problem gets to the heart of what distinguishes the world of medicine from aviation, a more existential one according to Kahn. “Medicine is seen as an art and as professionals, doctors need to feel they have the freedom to do the right thing by their patients.” With freedom comes flexibility, the complete opposite of the protocol demanded by checklists. “Airplanes are flown the same way every time, but in the ICU every patient is different and checklists go against that,” he says.
Aviation came up with the checklist at a time when airplanes were becoming too complex for the memory of the average pilot. You could say medicine has arrived at the same point. Technology has enhanced our ability to deliver better care, but has also added complexity that can challenge even the most experienced clinician. Checklists are designed to mitigate the risks associated with human error and even in these enlightened times, that’s something physicians have trouble admitting they’re prone to.
A QUALITATIVE STUDY
In 2016, Kahn was the senior investigator in a CRISMA study of the perceptions of rounding checklists in the ICU at the UPMC Health System. They performed a qualitative study in 32 ICUs in 14 hospitals; 89 people were interviewed.
The results were consistent with what they suspected – checklists were being used inconsistently; there was a lack of applicability to the current patient in the ICU; and many topics were unimportant to some or all healthcare providers. As well, the checklists were not integrated for members of the team who have a sense of urgency. “The inherent purpose of checklists,” says Kahn, “is to raise questions and give clinicians a chance to pause and reflect.”
The study confirmed it was time to stop pretending that the checklists were a useful tool. “We realized if they were going to be used at all, we needed to rebuild them from the ground up,” says Kahn.
They took several key steps:
- The checklists were shortened so they only contain items connected with evidence-based practice. The result is a greatly shortened checklist that is perceived as useful to the whole team.
- The checklists are being completed at the beginning of rounds, not the end as was done previously. “Putting checklists first basically re-envisioned the way rounds are done,” says Kahn.
- Intelligently order items, starting with the most critical.
- Changes were made at the system-level too, so each ICU was encouraged to customize their checklist. This ensures the items are relevant to members of that team.
“I was completely prepared to take a new and creative approach to checklists, such as making them electronic and incorporating them into our EHR,” says Kahn, “but it didn’t go that way. There is a human side to checklists and people want the discussion.”
Making these changes Kahn says, “has charged people’s batteries rather than draining them. People are energized by the checklist, especially the ICU nurses because we’re really listening to them.” The question – still being reviewed – is whether the new checklists are having an impact on clinical outcomes.
The cynic (or realist) in Kahn is quick to push back against the notion that even the re-designed checklists are going to solve all our problems. “Like a hammer sitting on a table – there is no inherent value in checklists. It’s all in how you use them,” he says. “It’s a continuous process, definitely not a one and done.” Which experienced providers like Kahn say is pretty much the way it is with all quality improvement tools in healthcare. “It’s hard and takes time but it can be fun too and that’s what makes it interesting.”
HIROC was a sponsor of the 2017 Critical Care Forum. This session on the CRISMA study of the use of checklists in their ICU was of particular interest because of the value HIROC places on checklists as a critical tool to support quality and safety in healthcare. They are an essential component of our widely adopted RAC (Risk Assessment Checklists) program. A 2014 Ontario study of the impact of the WHO Surgical Safety Checklist in Ontario hospitals found that the checklist was not associated with significant reductions in operative mortality or complications. This aligns with the CRISMA study and other findingsthat for checklists to be effective, they must be implemented thoughtfully, and cannot be regarded as a singular or straightforward solution.
By Ellen Gardner, Senior Specialist, Communications and Marketing, HIROC
