ACEP Covid19 Risk Assessment guideline modelling and analysis

It turns out that our work on Task Planning and Decision Logic Modules is bearing unexpected fruit…

As part of consulting work for the US Veterans Affairs, I am encoding the ACEP/ Covid19 risk assessment guideline into the formal language used by, with a view to a) implementation in openEHR and b) helping a larger group build and validate a definition in BPM+ form. Examples of the first formalism are here.

The guideline is very nicely designed and documented, however there are some small errors which need to be corrected.

In encoding the vital signs ranges in the first step, shown below, there are 3 range errors - i.e. instances where the ranges leave gaps. One might be tempted to say, well it’s obvious to the docs in the room that say ‘<88%’ SpO2 really means ‘<=88%’, … or … did they mean that ‘moderate risk’ is 88-92%? So, already for an alert doc or nurse, a quandary. Probably one would err on the side of caution. But what about a tired doc or nurse?

Now, if these ranges were coded directly into a computational form, as I am doing right now, the values Heart rate = 100 /min; SpO2 = 88%; and respiratory rate = 22 / min would just be missed completely, and the consequence would certainly be missed or wrong alerts. I adjusted the ranges to meet, but of course, I don’t know whether the authors intended to put 88% SpO2 in the ‘severe’ or ‘moderate’ ranges etc.

We can see that the interactive tool also contains these errors - if the heart rate is measured as 100 /min, a nurse is going to have to think: which do I choose - ‘<100’ or ‘101-120’. Erring on the side of caution means having to remember whether rising or falling heart rate is associated with Covid19 risk. Most clinicians will guess correctly, and with the PDF form of the guideline, which maps the numeric ranges under low/mild/moderate etc headings, it is easy. However, on the interactive form, the risk bands are not shown, and a tired clinician or someone not familiar with the clinical course of Covid19 might pick the ‘optimistic’ range. The consequences are unlikely to be catastrophic, but on the other hand, it’s probably better to remove sources of unnecessary doubt especially in the ICU.

However, the consequences of these ranges encoded in a purely rule-based system might be catastrophic, since patients with HR=100, respiratory rate=22 and SpO2=88% might not be classified at all and/or no alerts generated - least for the SpO2, this will be dangerous.

I will report this to ACEP with a view to helping quality improvement.

Very interested in any clinical or other comments on these kinds of errors in guidelines.

Hello Thomas,
I am not sure about which remarks you expect, but anyway there goes my two cents… What I can tell you is that this is not unusual at all in clinical guidelines, even the best guidelines of very prestigious institutions contain many ambiguities, contradictions and logical open ends. That’s one of the reasons why knowledge elicitation in knowledge-based systems is such a lengthy process, knowledge needs to be analyzed and converted into a logically sound representation, that traverses the whole solutions space. If a doctor or clinician needs to make critical decisions based on narrative CGs, it would be very risky indeed. Clinical Decisions Support Systems are a solution but they still have to overcome many challenges. Yo need to go back to the CG authors and clarify the ambiguities and open ends.
Sorry if I misunderstood your question, this is what I can tell you about it.


An update: I have uploaded a partial draft of the ACEP guideline, now with (I think) enough content to see what it may look like as a Task Plan + DLM. See here.