Scores or scales with mixed data types or explicit NULL values

Clinical
1

Some originally paper based scores or scales use mixed data types or explicit choices which evaluate to a NULL value. One example is the NANO scale, which uses this pattern for all its elements:

image
image695×328 53.6 KB

The first four values here would fit right into a DV_ORDINAL, but the final two do not. We’ve been discussing this pattern, and come up with a set of possible solutions, one of which modelling-wise is elegant, and four which … aren’t.

  1. Constrain the null_reason RM element in the archetype, to be a DV_CODED_TEXT containing the two “Not assessed” and “Not evaluable” values. This looks to us to be the most elegant solution, but AFAIK it’s not supported in any modelling tools, implementations or the CKM :sob:
  2. Represent the final two values using a ‘0’ as the ordinal value. This would be simple to model and implement, but is wrong with regards to the data we’d end up persisting. We don’t want to risk mixing up ‘Normal’ and ‘Not evaluable’ :disguised_face:
  3. Separate the two outlier values into a separate data element with a DV_CODED_TEXT data type. This would require the UI to do some smarts to make sure only one of each pair of DV_ORDINAL/DV_CODED_TEXT is persisted. A variation of this is to put them both in an internal cluster with cardinality 1..1, to make sure only one is ever persisted. This solution is clunky and puts a larger workload on implementers :grimacing:
  4. Make the entire data element a DV_CODED_TEXT where the first four values have the codes 0-3, and the final two either NULL if possible, or a letter code if necessary :face_with_spiral_eyes:
  5. Make the element a choice between DV_ORDINAL containing the first four values, and DV_CODED_TEXT containing the final two. We don’t know whether current implementations support constraining a choice data type at run-time, so this could possibly be unimplementable :dizzy_face:

Options 3-5 also adds the complexity of having to make up and insert into the archetype a small “external” terminology, since internal codes in a DV_CODED_TEXT is limited to at-codes as the code values. This also isn’t very well supported in modelling tools :scream:

So we’d really like some input from other modellers, implementers and specifications people. How do we solve this conundrum?

4 Likes
2

This is an interesting topic. Thanks for the good preparation of the topic.

I agree with you on the order of the items.

It seems reasonable to put the “Not assessed” or “Not evaluable” on the null_reason of ELEMENT. It’s an DV_TEXT attribute which perfectly well can be defined as DV_CODED_TEXT. The RM allows this. To use this in applications tooling and software must support this kind of feature. We need to agree if this is the correct way to model such things and then follow up on the software. This might take som time.

If time is as constraint, which it always is, I think candidate two is the best approach. It might be simpler to understand the archetype (for non-openEHR experts) when it is in the same list as others. The atCode will be different for each items and it will be possible to distinguish the data. It will not affect the total score if a NULL item is given the score 0.

Based on this I will go for the second option for this archetype.

For reference: the current draft of the archetype is here Observation Archetype: Neurologic Assessment in Neuro-Oncology (NANO) scale [Nasjonal IKT Clinical Knowledge Manager]

3 Likes
3

:saluting_face:

image

2 Likes
4

As tooling developers, do you have any input regarding this discussion, @borut.fabjan or @sebastian.garde ?

5

That is a tough one, @siljelb !

In your preferred option 1, you MUST set the null_flavour to something anyway for the last two “values” from the score. (Only having a null_reason is not enough as per the spec)

The choice for this is:
253|unknown|, 271|no information|, 272|masked|, and 273|not applicable|

Therefore is simply using the two null_flavours as follows not an option?

  • “Not assessed” ↔ 271| no information and
  • “Not evaluable” ↔ 273| not applicable

I must say that I am always slightly uneasy with the openEHR null_flavour codes because these four codes seem not to be very well defined (or simply not very well understood by me) and my assignments above may not be correct. BTW, it seems I am not the only one with this problem, see e.g. this discussion: Uncertain, unknown and no information - #2 by pablo

The question of course is how to properly present this in GUIs based on templates, etc. and also if you need the exact text of “Not assessed” and “Not evaluable”, especially the more low-code it gets.

Adding the null_reason as constraint to an archetype makes this far more complex (beyond just supporting another field). This is because we then have to constrain the tuple of (null_flavour, null_reason) to two different options above (but e.g. Not assessed ↔ 273 not applicable would not be allowed).

Another reason why constraining the null_flavour and/or null_reason at design time is problematic is that the other null_flavours (and kind of free text reason) may still be required at runtime, e.g. masked, so in essence there seems to be more than one layer of null_flavours possible here. It may well be possible that people want to document e.g. “Not evaluable”, but this information subsequently gets “masked” when presenting documented data. This seems to be problematic when we start constraining null_* at design time already.

For these reasons, I find option 1 - EXPLICITLY constraining null_flavour and/or null_reason at design time, pretty hard to get right.

However, using null_flavours [+ null_reason] at runtime to document these two NULL options, should be possible in theory at least everywhere, including in this case where the score developers have explicitly thought of these possibilities and explicitly allow them.

And if the two codes aboce chosen by me are not accurate, we may need to look into adding others of making the description more explicit.
Looking at NullFlavor - HL7 Terminology (THO) v5.3.0 they support quite a lot more codes, and “Not Asked” may be a more appropriate or at least more specific null_flavour for the score’s “Not Assessed” value.

It may not be enough to put in a comment or hint in the description/comment that these two values are modelled explicitly in the score, but are typically covered by null_flavours in our case? If these two values need to be modelled more explicitly than that I am not sure Option 1 is good.

Option 2 seems very dangerous to me. At least I would consider if -1 is not a better value in these cases, although we’d need to be careful if there is a total score. Maybe 999 is a little better - that would at least come up in any reasonable check if the total score is correct.

Re Option 4: I don’t think having a NULL code here is an option, it would need to be a letter I guess. Generally a bit clunky and not conveying the modelling intention very well.

So, from my point of view, I think making sure that we make null_flavours so useful that we can use them in such situations would be very benefitial. This may require a few extra codes or better documentation of the existing codes, (re-)harmonising with FHIR. More importantly, it requires giving a hint that these two null_flavours (and potentially these two null_reasons) are preferred. And one way of doing this in the future may be by using binding strengths here?

Given all this and also the clunkiness of any alternative approach, it seems to me that using an ORDINAL for this modelling without explicitly constraining null_flavours or null_reasons at design time and providing descriptions/comments/(UI) hints of some sort as to the usage of the two preferred NULL values is the least evil solution. It also seems to me that it is the only one with a pathway to a more proper solution in the future.

2 Likes
6

In this specific archetype there is a score based on the sum of ordinal values. I assume a null value (not present due to some definition) is equivalent with the number 0. Thus it seems safe to represent the ordinal as a zero number.

This is somehow similar to NEWS2 where multiple ordinals have the same number.

1 Like
7

That may well be the case here, just one thing to consider, given that 0 also equals “normal”:

May a score of - say - 20 of 25 possible points have a different clinical interpretation than a score of - say - 20 of 30 possible points?

In that sense, whether you assume min, max, mean or median here as the default value if a value is missing is an interesting choice to make. Also considering that in some scores minimum number of points is “best” like here in NEWS2 and NANO but in others like APGAR from memory it is the other way round.

But as I and probably everybody else has said, there does not seem a perfect choice here, so if the above does not bother clinicians, I am certainly happy.

1 Like
8

Very good thinking @sebastian.garde
These are all things to consider both in design and runtime.

1 Like
9

Just a heads up: The total score element is going to be removed. The NANO score contains no total score, on the contrary the original article actively disencourage it.

2 Likes
10

You just ruined all my arguments @varntzen :rofl:

For this specific archetype. The discussion is relevant for many use-cases. We had an interesting discussion today, with among @siljelb , on the interpretation of NULL flavor and reason. We agreed that we have different views on some parts. Will present the topic after some more thinking.

2 Likes
11

This is definitely the right solution. Pretty much any other solution is just wrong (sorry to say), unless you just don’t want to treat it as a score, which it obviously is.

The problem with choosing solution hacks is that the tooling will catch up soon enough, but modelling hacks are (kind of) forever, at least until they get redone in a new breaking change version.

Null_reason constraining can be done by hand-editing if the current tools don’t support it. I don’t understand why AD wouldn’t support it - as long as the BMM files it imports are up to date, then it should automatically treat null_reason as an available attribute.

3 Likes
13

We have probably had a lucky escape with this particular archetype since it does not compute a total score and therefore a 0 ordinal is probably safe but it would still be good to try to understand the blockers to making null_flavours more useable.

This example is clearly a good use of null_flavours but practically speaking, it is quite difficult to define their use clearly in archetypes and templates. They work well for integration cases where UI is not involved but it is quire difficult to incorporate them into UI and certa9nly form building.

Is this something that might benefit from a re-think or at least some suggestions about how it might all be easier to work with.

I’ll throw up a challenging strawman!!

  • what about if we allowed null_flavours and associated reason codes to ‘appear’ to be added to a Valueset. I say ‘appear’ because I think the underlying separation of ‘nulls’ from real data is correct but perhaps we could find a away to present the use of nulls in a more naturastic way (as seeming added terms as per the scale/score example ) in both archetype tooling and forms building.
1 Like
14

I like this thinking, and it aligns pretty well with what we were envisioning when we thought up option #1 above.

I agree with Sebastian though, that the null_flavours should be defined more thoroughly in the specs. Currently it’s often difficult to tell ‘not applicable’ from ‘no information’, and the possibility of applying a ‘masked’ flavour as a query result makes my head spin. I’m not sure the answer is to add more flavours - I’d prefer if we at least start by more clearly defining the ones we have.

15

Are you sure? I don’t understand how using 0 doesn’t make it difficult to distinguish “Not assessed” from “Normal”. Differentiate by the at-code?

16

ok ‘lucky-ish’ escape - it would have been much more problematic if there had been a computed score - at least the text e.g. ‘not assessesed’ will be visible when assessing the response.

1 Like
17

No more flavours than necessary, I agree…some from FHIR seem to be specialisations of the more generic ones that we already share between FHIR and openEHR.

2 Likes
18

We looked at the flavours in a discussion the other day and it suddenly appeared to me: Do we really need to distinguish the flavours in the RM?
When I read them I surely understand the differences from a logical and theoretical view. But will it be clear for an end-user in a specific entry situation? And what is the benefit of distinguishing them? Perhaps it would be good enough with a NULL_FLAVOR definition and some NULL_REASONS either as text or coded text?

Another perspective that came up in the discussion was about the masked flavour. Some thought this could be used to mask the real value in the data. Like “I have entered the value of some disorder but it is masked by the null flavour”. This would of course be meaningful in a client-server architecture where the client ask the server if it has information about the attribute. The server might say “yes I have some information but it is masked to you… give me some other credentials and I might give them to you…”

This is,AFAIK, not the intention with openEHR “masked”. It should be used to say that somewhere in the real world there exist some information but it’s not present in this EHR-system.

The question is:
Are there any known use of the NULL_FLAVOUR today? If yes - what’s your experiences using the different flavours? Is the current list of flavours good enough? Do you miss any flavours?

1 Like
19

Agree. I don’t know how this particular score is used, whether it’s only once and never again, or it’s repeated to give an idea of progression. If the latter, it’s not unreasonable to graph it. Then values of 0 would really be problematic. “Normal, or I don’t know - might be 3 if we scored.”

2 Likes
20

That’s not exactly what I meant. My thinking was that a null value could be returned on a query where the person or organisation running the query for some reason didn’t have access to the data. The null_flavour would be returned alongside the null value to explain the reason why there’s no data.

But yeah, I’d love for this kind of thing to be further clarified.

1 Like
21

Null flavours is a tough topic that has been discussed for a long time. @thomas.beale presents some excellent arguments in his blog:

We also have some infomation in the wiki:
https://openehr.atlassian.net/wiki/spaces/spec/pages/4915211/Null+Flavours+and+Boolean+data+in+openEHR

And also in Discourse:

I’m not sure which is the best way to discuss this and to try to reach an agreement in the community, but it will require some effort and dedication for sure.

Meanwhile, these are my two cents.

Maybe we should rethink which is the real meaning and value of having null flavours. I have been thinking on how the HTTP response codes work (4XX codes for client errors and 5XX for server errors) and that could be a good inspiration.

  • There are null flavors that will come from the server, during data recovery. For example, the no information and the masked nulls, meaning that the server is not able to serve that information due to a technical or logical reason. These should be generated automatically just to inform of the situation to the receiver of the data (being a human or another system) so it can react accordingly: show an error, retry later, retry with different credentials…
  • There are null flavors that will come from the user, from the person or system that is generating the data. This is the case of not applicable/evaluable, not asked… The difference here is that these null are provided during the data capture process, they are in some way also clinical data, that should be stored, processed and maybe be modified in the future. They could be also useful in the situation when an archetype defines a data point as mandatory but maybe that information cannot be recorded or it doesn’t exist. The user could select a null reason so he/she is able to, at least, store the composition without errors with all the other data. And that means that they could be archetyped.

At the end, we will probably end with more or less the same values of null, but I think that this approach of thinking helps in clarifying that the null flavour should not be understood just as a miscellaneous mess of codes.

4 Likes