# Scores or scales with mixed data types or explicit NULL values **Category:** [Clinical](https://discourse.openehr.org/c/clinical/5) **Created:** 2023-11-07 08:33 UTC **Views:** 1679 **Replies:** 60 **URL:** https://discourse.openehr.org/t/scores-or-scales-with-mixed-data-types-or-explicit-null-values/4620 --- ## Post #1 by @siljelb Some originally paper based scores or scales use mixed data types or explicit choices which evaluate to a NULL value. One example is the [NANO scale](https://pubmed.ncbi.nlm.nih.gov/28453751/), which uses this pattern for all its elements: ![image|690x325](upload://pElPeXet7B9rJfWZQptf2WRJy9Q.png) The first four values here would fit right into a DV_ORDINAL, but the final two do not. We've been discussing this pattern, and come up with a set of possible solutions, one of which modelling-wise is elegant, and four which ... aren't. 1. Constrain the [`null_reason`](https://specifications.openehr.org/releases/RM/latest/data_structures.html#_element_class) RM element in the archetype, to be a DV_CODED_TEXT containing the two "Not assessed" and "Not evaluable" values. This looks to us to be the most elegant solution, but AFAIK it's not supported in any modelling tools, implementations or the CKM :sob: 2. Represent the final two values using a '0' as the ordinal value. This would be simple to model and implement, but is wrong with regards to the data we'd end up persisting. We don't want to risk mixing up 'Normal' and 'Not evaluable' :disguised_face: 3. Separate the two outlier values into a separate data element with a DV_CODED_TEXT data type. This would require the UI to do some smarts to make sure only one of each pair of DV_ORDINAL/DV_CODED_TEXT is persisted. A variation of this is to put them both in an internal cluster with cardinality `1..1`, to make sure only one is ever persisted. This solution is clunky and puts a larger workload on implementers :grimacing: 4. Make the entire data element a DV_CODED_TEXT where the first four values have the codes 0-3, and the final two either NULL if possible, or a letter code if necessary :face_with_spiral_eyes: 5. Make the element a choice between DV_ORDINAL containing the first four values, and DV_CODED_TEXT containing the final two. We don't know whether current implementations support constraining a choice data type at run-time, so this could possibly be unimplementable :dizzy_face: Options 3-5 also adds the complexity of having to make up and insert into the archetype a small "external" terminology, since internal codes in a DV_CODED_TEXT is limited to at-codes as the code values. This also isn't very well supported in modelling tools :scream: So we'd really like some input from other modellers, implementers and specifications people. How do we solve this conundrum? --- ## Post #2 by @bna This is an interesting topic. Thanks for the good preparation of the topic. I agree with you on the order of the items. It seems reasonable to put the "Not assessed" or "Not evaluable" on the `null_reason` of [ELEMENT](https://specifications.openehr.org/releases/RM/latest/data_structures.html#_element_class). It's an `DV_TEXT` attribute which perfectly well can be defined as `DV_CODED_TEXT`. The RM allows this. To use this in applications tooling and software must support this kind of feature. We need to agree if this is the correct way to model such things and then follow up on the software. This might take som time. If time is as constraint, which it always is, I think candidate two is the best approach. It might be simpler to understand the archetype (for non-openEHR experts) when it is in the same list as others. The `atCode` will be different for each items and it will be possible to distinguish the data. It will not affect the total score if a _NULL_ item is given the score `0`. Based on this I will go for the second option for this archetype. For reference: the current draft of the archetype is here https://arketyper.no/ckm/archetypes/1078.36.2952 --- ## Post #3 by @damoca [quote="siljelb, post:1, topic:4620"] Constrain the [`null_reason`](https://specifications.openehr.org/releases/RM/latest/data_structures.html#_element_class) RM element in the archetype, to be a DV_CODED_TEXT containing the two “Not assessed” and “Not evaluable” values. This looks to us to be the most elegant solution, but AFAIK it’s not supported in any modelling tools, implementations or the CKM :sob: [/quote] :saluting_face: ![image|601x311](upload://x1C75Iq9pvj0vWApkBWwR3kPWog.png) --- ## Post #4 by @siljelb As tooling developers, do you have any input regarding this discussion, @borut.fabjan or @sebastian.garde ? --- ## Post #5 by @sebastian.garde That is a tough one, @siljelb ! In your preferred option 1, you MUST set the null_flavour to something anyway for the last two "values" from the score. (Only having a null_reason is not enough as per the spec) The choice for this is: `253|unknown|`, `271|no information|`, `272|masked|`, and `273|not applicable|` Therefore is simply using the two null_flavours as follows not an option? - "Not assessed" <-> 271| no information and - "Not evaluable" <-> 273| not applicable I must say that I am always slightly uneasy with the openEHR null_flavour codes because these four codes seem not to be very well defined (or simply not very well understood by me) and my assignments above may not be correct. BTW, it seems I am not the only one with this problem, see e.g. this discussion: https://discourse.openehr.org/t/uncertain-unknown-and-no-information/1473/2 The question of course is how to properly present this in GUIs based on templates, etc. and also if you need the exact text of "Not assessed" and "Not evaluable", especially the more low-code it gets. Adding the null_reason as constraint to an archetype makes this far more complex (beyond just supporting another field). This is because we then have to constrain the tuple of (null_flavour, null_reason) to two different options above (but e.g. Not assessed <-> 273 not applicable would not be allowed). Another reason why constraining the null_flavour and/or null_reason at design time is problematic is that the other null_flavours (and kind of free text reason) may still be required at runtime, e.g. *masked*, so in essence there seems to be more than one layer of null_flavours possible here. It may well be possible that people want to document e.g. "Not evaluable", but this information *subsequently* gets "masked" when presenting documented data. This seems to be problematic when we start constraining null_* at design time already. For these reasons, I find option 1 - EXPLICITLY constraining null_flavour and/or null_reason at design time, pretty hard to get right. However, using null_flavours [+ null_reason] at runtime to document these two NULL options, should be possible in theory at least everywhere, including in this case where the score developers have explicitly thought of these possibilities and explicitly allow them. And if the two codes aboce chosen by me are not accurate, we may need to look into adding others of making the description more explicit. Looking at https://terminology.hl7.org/5.3.0/ValueSet-v3-NullFlavor.html they support quite a lot more codes, and "Not Asked" may be a more appropriate or at least more specific null_flavour for the score's "Not Assessed" value. It may not be enough to put in a comment or hint in the description/comment that these two values are modelled explicitly in the score, but are typically covered by null_flavours in our case? If these two values need to be modelled more explicitly than that I am not sure Option 1 is good. Option 2 seems very dangerous to me. At least I would consider if -1 is not a better value in these cases, although we'd need to be careful if there is a total score. Maybe 999 is a little better - that would at least come up in any reasonable check if the total score is correct. Re Option 4: I don't think having a NULL code here is an option, it would need to be a letter I guess. Generally a bit clunky and not conveying the modelling intention very well. So, from my point of view, I think making sure that we make null_flavours so useful that we can use them in such situations would be very benefitial. This may require a few extra codes or better documentation of the existing codes, (re-)harmonising with FHIR. More importantly, it requires giving a hint that these two null_flavours (and potentially these two null_reasons) are preferred. And one way of doing this in the future may be by using binding strengths here? Given all this and also the clunkiness of any alternative approach, it seems to me that using an ORDINAL for this modelling **without explicitly constraining** null_flavours or null_reasons at design time and providing descriptions/comments/(UI) hints of some sort as to the usage of the two preferred NULL values is the least evil solution. It also seems to me that it is the only one with a pathway to a more proper solution in the future. --- ## Post #6 by @bna [quote="sebastian.garde, post:5, topic:4620"] Option 2 seems very dangerous to me. At least I would consider if -1 is not a better value in these cases, although we’d need to be careful if there is a total score. Maybe 999 is a little better - that would at least come up in any reasonable check if the total score is correct. [/quote] In this specific archetype there is a score based on the sum of ordinal values. I assume a null value (not present due to some definition) is equivalent with the number 0. Thus it seems safe to represent the ordinal as a zero number. This is somehow similar to NEWS2 where multiple ordinals have the same number. --- ## Post #7 by @sebastian.garde [quote="bna, post:6, topic:4620"] In this specific archetype there is a score based on the sum of ordinal values. I assume a null value (not present due to some definition) is equivalent with the number 0. Thus it seems safe to represent the ordinal as a zero number. [/quote] That may well be the case here, just one thing to consider, given that 0 also equals "normal": *May a score of - say - 20 of 25 possible points have a different clinical interpretation than a score of - say - 20 of 30 possible points?* In that sense, whether you assume *min*, *max*, *mean* or *median* here as the default value if a value is missing is an interesting choice to make. Also considering that in some scores minimum number of points is "best" like here in NEWS2 and NANO but in others like APGAR from memory it is the other way round. But as I and probably everybody else has said, there does not seem a perfect choice here, so if the above does not bother clinicians, I am certainly happy. --- ## Post #8 by @bna Very good thinking @sebastian.garde These are all things to consider both in design and runtime. --- ## Post #9 by @varntzen Just a heads up: The total score element is going to be removed. The NANO score contains no total score, on the contrary the original article actively disencourage it. --- ## Post #10 by @bna You just ruined all my arguments @varntzen 🤣 For this specific archetype. The discussion is relevant for many use-cases. We had an interesting discussion today, with among @siljelb , on the interpretation of NULL flavor and reason. We agreed that we have different views on some parts. Will present the topic after some more thinking. --- ## Post #11 by @thomas.beale [quote="siljelb, post:1, topic:4620"] Constrain the [`null_reason`](https://specifications.openehr.org/releases/RM/latest/data_structures.html#_element_class) RM element in the archetype, to be a DV_CODED_TEXT containing the two “Not assessed” and “Not evaluable” values. This looks to us to be the most elegant solution, but AFAIK it’s not supported in any modelling tools, implementations or the CKM [/quote] This is definitely the right solution. Pretty much any other solution is just wrong (sorry to say), unless you just don't want to treat it as a score, which it obviously is. The problem with choosing solution hacks is that the tooling will catch up soon enough, but modelling hacks are (kind of) forever, at least until they get redone in a new breaking change version. Null_reason constraining can be done by hand-editing if the current tools don't support it. I don't understand why AD wouldn't support it - as long as the BMM files it imports are up to date, then it should automatically treat null_reason as an available attribute. --- ## Post #13 by @ian.mcnicoll We have probably had a lucky escape with this particular archetype since it does not compute a total score and therefore a 0 ordinal is probably safe but it would still be good to try to understand the blockers to making null_flavours more useable. This example is clearly a good use of null_flavours but practically speaking, it is quite difficult to define their use clearly in archetypes and templates. They work well for integration cases where UI is not involved but it is quire difficult to incorporate them into UI and certa9nly form building. Is this something that might benefit from a re-think or at least some suggestions about how it might all be easier to work with. I'll throw up a challenging strawman!! - what about if we allowed null_flavours and associated reason codes to 'appear' to be added to a Valueset. I say 'appear' because I think the underlying separation of 'nulls' from real data is correct but perhaps we could find a away to present the use of nulls in a more naturastic way (as seeming added terms as per the scale/score example ) in both archetype tooling and forms building. --- ## Post #14 by @siljelb [quote="ian.mcnicoll, post:13, topic:4620"] what about if we allowed null_flavours and associated reason codes to ‘appear’ to be added to a Valueset. I say ‘appear’ because I think the underlying separation of ‘nulls’ from real data is correct but perhaps we could find a away to present the use of nulls in a more naturastic way (as seeming added terms as per the scale/score example ) in both archetype tooling and forms building. [/quote] I like this thinking, and it aligns pretty well with what we were envisioning when we thought up option #1 above. I agree with Sebastian though, that the `null_flavour`s should be defined more thoroughly in the specs. Currently it's often difficult to tell 'not applicable' from 'no information', and the possibility of applying a 'masked' flavour as a query result makes my head spin. I'm not sure the answer is to add more flavours - I'd prefer if we at least start by more clearly defining the ones we have. --- ## Post #15 by @varntzen [quote="ian.mcnicoll, post:13, topic:4620"] We have probably had a lucky escape with this particular archetype since it does not compute a total score and therefore a 0 ordinal is probably safe [/quote] Are you sure? I don't understand how using 0 doesn't make it difficult to distinguish "Not assessed" from "Normal". Differentiate by the at-code? --- ## Post #16 by @ian.mcnicoll ok 'lucky-ish' escape - it would have been much more problematic if there had been a computed score - at least the text e.g. 'not assessesed' will be visible when assessing the response. --- ## Post #17 by @sebastian.garde [quote="siljelb, post:14, topic:4620"] I agree with Sebastian though, that the `null_flavour`s should be defined more thoroughly in the specs. Currently it’s often difficult to tell ‘not applicable’ from ‘no information’, and the possibility of applying a ‘masked’ flavour as a query result makes my head spin. I’m not sure the answer is to add more flavours - I’d prefer if we at least start by more clearly defining the ones we have. [/quote] No more flavours than necessary, I agree…some from FHIR seem to be specialisations of the more generic ones that we already share between FHIR and openEHR. --- ## Post #18 by @bna [quote="sebastian.garde, post:17, topic:4620"] No more flavours than necessary, I agree…some from FHIR seem to be specialisations of the more generic ones that we already share between FHIR and openEHR. [/quote] We looked at the flavours in a discussion the other day and it suddenly appeared to me: Do we really need to distinguish the flavours in the RM? When I read them I surely understand the differences from a logical and theoretical view. But will it be clear for an end-user in a specific entry situation? And what is the benefit of distinguishing them? Perhaps it would be good enough with a NULL_FLAVOR definition and some NULL_REASONS either as text or coded text? Another perspective that came up in the discussion was about the *masked* flavour. Some thought this could be used to mask the real value in the data. Like "I have entered the value of some disorder but it is masked by the null flavour". This would of course be meaningful in a client-server architecture where the client ask the server if it has information about the attribute. The server might say "yes I have some information but it is masked to you... give me some other credentials and I might give them to you.." This is,AFAIK, not the intention with openEHR "masked". It should be used to say that somewhere in the real world there exist some information but it's not present in this EHR-system. The question is: Are there any known use of the NULL_FLAVOUR today? If yes - what's your experiences using the different flavours? Is the current list of flavours good enough? Do you miss any flavours? --- ## Post #19 by @varntzen [quote="ian.mcnicoll, post:16, topic:4620"] at least the text e.g. ‘not assessesed’ will be visible when assessing the response. [/quote] Agree. I don't know how this particular score is used, whether it's only once and never again, or it's repeated to give an idea of progression. If the latter, it's not unreasonable to graph it. Then values of 0 would really be problematic. "Normal, or I don't know - might be 3 if we scored." --- ## Post #20 by @siljelb [quote="bna, post:18, topic:4620"] “I have entered the value of some disorder but it is masked by the null flavour”. [/quote] That's not exactly what I meant. My thinking was that a null value could be returned on a query where the person or organisation running the query for some reason didn't have access to the data. The null_flavour would be returned alongside the null value to explain the reason why there's no data. But yeah, I'd love for this kind of thing to be further clarified. --- ## Post #21 by @damoca [quote="bna, post:18, topic:4620"] The question is: Are there any known use of the NULL_FLAVOUR today? If yes - what’s your experiences using the different flavours? Is the current list of flavours good enough? Do you miss any flavours [/quote] Null flavours is a tough topic that has been discussed for a long time. @thomas.beale presents some excellent arguments in his blog: https://wolandscat.net/2011/05/18/the-hl7-null-flavor-debate-part-1/ https://wolandscat.net/2011/05/18/the-hl7-null-flavor-debate-part-2/ We also have some infomation in the wiki: https://openehr.atlassian.net/wiki/spaces/spec/pages/4915211/Null+Flavours+and+Boolean+data+in+openEHR And also in Discourse: https://discourse.openehr.org/t/uncertain-unknown-and-no-information/1473 I'm not sure which is the best way to discuss this and to try to reach an agreement in the community, but it will require some effort and dedication for sure. Meanwhile, these are my two cents. Maybe we should rethink which is the real meaning and value of having null flavours. I have been thinking on how the HTTP response codes work (4XX codes for client errors and 5XX for server errors) and that could be a good inspiration. * **There are null flavors that will come from the server**, during data recovery. For example, the no information and the masked nulls, meaning that the server is not able to serve that information due to a technical or logical reason. These should be generated automatically just to inform of the situation to the receiver of the data (being a human or another system) so it can react accordingly: show an error, retry later, retry with different credentials... * **There are null flavors that will come from the user**, from the person or system that is generating the data. This is the case of not applicable/evaluable, not asked... The difference here is that these null are provided during the data capture process, they are in some way also clinical data, that should be stored, processed and maybe be modified in the future. They could be also useful in the situation when an archetype defines a data point as mandatory but maybe that information cannot be recorded or it doesn't exist. The user could select a null reason so he/she is able to, at least, store the composition without errors with all the other data. And that means that they could be archetyped. At the end, we will probably end with more or less the same values of null, but I think that this approach of thinking helps in clarifying that the null flavour should not be understood just as a miscellaneous mess of codes. --- ## Post #22 by @thomas.beale [quote="ian.mcnicoll, post:13, topic:4620"] null_flavours and associated reason codes to ‘appear’ to be added to a Valueset [/quote] These are formally two different vocabularies - the null_flavours is sourced from openEHR terminology, while reason text/codes are on a per archetype basis. I'd suggest that we don't want people being able to edit the null_flavours vocabulary in an uncontrolled way (this will wreck any computability). I can imagine however that AD and similar tools would make it easy to see the vocabulary and potentially post a change request. [quote="bna, post:18, topic:4620"] We looked at the flavours in a discussion the other day and it suddenly appeared to me: Do we really need to distinguish the flavours in the RM? When I read them I surely understand the differences from a logical and theoretical view. But will it be clear for an end-user in a specific entry situation? And what is the benefit of distinguishing them? Perhaps it would be good enough with a NULL_FLAVOR definition and some NULL_REASONS either as text or coded text? [/quote] Well it depends on how computable we want data marked with a null flavour to be. Not applicable is very different from Not available for example. FYI - 'masked' originally was designed to be a NF that is set by a server when it hides (i.e. removes) some data item because it is 'sensitive', when it provides the data to some client system or application. Whether this still makes sense is an open question, since I don't think we seriously envisage marking data as sensitive or not at the ELEMENT level. --- ## Post #23 by @bna [quote="thomas.beale, post:22, topic:4620"] ‘masked’ originally was designed to be a NF that is set by a server when it hides (i.e. removes) some data item because it is ‘sensitive’, when it provides the data to some client system or application. Whether this still makes sense is an open question, since I don’t think we seriously envisage marking data as sensitive or not at the ELEMENT level. [/quote] Yes - this makes sense and I think it would be good to add this to some documentation/specification. One use-case could be extract of openEHR data where it could make sense to _mask_ free text (DV_TEXT). By using the `masked` code the server/routine could inform that some data exist or not. As you write; data access control on ELEMENT level will be extremely hard to design and implement. If we want to follow this pattern of access controll (to filter data) we could visit the idea of a NULL_FLAVOUR on higher level structures. In some situations it could be relevant to _mask_ an ENTRY or CLUSTER structure. Not that we have seen this in a concrete use-case yet. Still I have from a theoretical view seen this need. --- ## Post #24 by @ian.mcnicoll [quote="thomas.beale, post:22, topic:4620"] I’d suggest that we don’t want people being able to edit the null_flavours vocabulary in an uncontrolled way (this will wreck any computability). I can imagine however that AD and similar tools would make it easy to see the vocabulary and potentially post a change request. [/quote] That's not what I was suggesting .. What if the tooling / visualisation / from build were to allow this **visually** Mild Moderate Severe Not applicable Not asked Essentially the ability to visually add 'null' type terms to a normal list, or as a pesudo alternative datatype. Under the hood these would actually be Null reasons/ null_falvour codes Mild Moderate Severe Not applicable [Reason: Not indicated Null_flavour: not_applicable] Not asked [Reason: Not asked Null_flavour: ] which would be represented at run-time exactly as currently intended. i.e Mild, Moderate, Severe -> value Not applicable -> null.null_flavour = not_applicable, null.reason = "Not applicable" I think we could probably do this with current ADL but I do wonder if this can cope with both 'technical integration' constraints (fall back for null data in integrations) and clinical null constraints, essentially specifying allowed null terms that can be entered. --- ## Post #25 by @thomas.beale [quote="ian.mcnicoll, post:24, topic:4620"] What if the tooling / visualisation / from build were to allow this **visually** Mild Moderate Severe Not applicable Not asked [/quote] That's probably a step too weird for me (since it makes values and non-values look like one value space), but I now get what you are suggesting and can see some attraction in it, since normal people (including docs who design healthcare forms etc) often build this kind of mixed value list. I think to make it work properly, you'd want some way of visually distinguishing the null values, e.g. in a different colour, and to get them added in the first place, you'd have some button for 'add exception value' or similar. --- ## Post #26 by @siljelb [quote="thomas.beale, post:25, topic:4620"] and can see some attraction in it, since normal people (including docs who design healthcare forms etc) often build this kind of mixed value list. [/quote] I do think it's necessary to be able to present this visually as if both the ordinal values and the `null` values belonged to the same value set. But I don't think it needs to be difficult to visually distinguish them from each other. The paper form screenshot I included in the initial post of this topic does this beautifully: the ordinal values are presented with their respective values to the left, and the `null` values are presented with, well, nothing :smile: [quote="siljelb, post:1, topic:4620"] ![image|690x325](upload://pElPeXet7B9rJfWZQptf2WRJy9Q) image695×328 53.6 KB [/quote] --- ## Post #27 by @ian.mcnicoll Agree. I have no problem with making it clear somehow both in tooling and end-user apps that these are nulls but I do think this is probably how they have to be handled to make them useable in archetyping/templating and UI. Under the hood, I 'm happy for these to resolve into the current RM approach. though not sure if anything needs to change to support constraints of null_flavour codes, along with one or more 'Reason' codes - this is a pattern quite similar to ism_transitions, I guess and I wonder if the RM might need to change to make constraining easier. --- ## Post #28 by @thomas.beale [quote="siljelb, post:26, topic:4620"] I do think it’s necessary to be able to present this visually as if both the ordinal values and the `null` values belonged to the same value set [/quote] I wouldn't agree with that: I think it's reasonable that they can be made to *appear* that way (as you have shown), but to make them literally part of the same value set, when the 0-3 are values resulting from assessment, and the other two indicate that no assessment was done (or maybe 'not evaluable' is actually a kind of evaluation?), would greatly confuse any subsequent inferencing. Essentially it is mxing values with non-values, which we humans are good at because we are cognitively very sophisticated, and we are abstracting over the difference. But computers just do what they're told... --- ## Post #29 by @varntzen Conclusion for now, from the modellers perspective: > The published [GOSE archetype](https://ckm.openehr.org/ckm/archetypes/1013.1.6671) uses the ordinal ‘99’ to indicate an unscored value, similar to NANO’s “not assessed” and “not evaluable”. This is similar to option #2, but makes it possible for business logic to tell the actual values apart from the unscored values from the ordinal value itself and not just from the at-codes. We’ll use this pattern for both NANO and [NIHSS](https://ckm.openehr.org/ckm/archetypes/1013.1.2041). Business logic will have to make sure any of the ‘99’s are excluded from any graphing or summing, but this is easier than extracting the numerical value from a string. The plan is to finish review of the archetype and then pause it waiting for the “proper” solution (option 1) to be implemented in tools. --- ## Post #30 by @siljelb Is there any progression in tooling development wrt how to solve this kind of issue? We've now gotten around to doing the NIHSS, which has a similar issue requiring both `null_flavour` and `null_reason`. These elements need to be constrainable on the archetype level, and possible to use/visualise through the CKM, in OPTs, in implementations/lowcode tools and renderers, and in CDRs. @borut.fabjan @sebastian.garde @bna --- ## Post #31 by @ian.mcnicoll I think the answer is no but perhaps we can get together as modellers and express a common clear requirement. So **I** want to be able to specify, in both 1. That null_flavours are explicitly available or explicitly **not** available on an ELEMENT. 2. If they are available, which null_flavours are allowed ( we already have that) Perhaps we can make this work already with the existing constraints. i.e. If you want to permit a null_flavour, you need to make at least one of the null_flavours available, otherwise nulls are not allowed. One issue to think about is whether as well as having a null reason comment, whether there is any value in allowing other types of term to be used instead of, or alongside the official null_flavours Perhaps a bit like current_state and careflow_step in ACTIONs. Is it important that we stick absolutely to the current list for computational reasons? Personally I think not. Allowing local null_flavours would solve the problem as with this example ![image|690x325](upload://jnvuQcpNxVAmFzAlrrXXoKTxFN2.png) which expects 'Not assessed and 'Not evaluable' and not the current terms Whatever, the tooling should (IMO) be able to show the visually combined lists of true valueset + allowed nulls., but agree with Thomas that these are not actually mixed into a single valueset. --- ## Post #32 by @sebastian.garde [quote="ian.mcnicoll, post:31, topic:4620"] That null_flavours are explicitly available or explicitly **not** available on an ELEMENT. [/quote] Not 100% sure why you need this, but would ``` null_flavour existence matches {0} ``` not do this from a specs point of view (tooling is a different issue of course)? To me the spec is not 100% clear whether we can just use other codes as null_flavour: [4.1. Item Structure Package / Overview](https://specifications.openehr.org/releases/RM/latest/data_structures.html#_overview_2) > > Values from the openEHR [`null flavours` vocabulary](https://github.com/openEHR/specifications-TERM/blob/master/computable/XML/en/openehr_terminology.xml), including `253|unknown|`, `271|no information|`, `272|masked|`, and `273|not applicable|` are used to populate it. Only a small number of generic codes are defined, in order to avoid complex processing for most data instances, for which this simple classification of null is sufficient. whereas [5.2.3. ELEMENT Class](https://specifications.openehr.org/releases/RM/latest/data_structures.html#_element_class) > **null_flavour**: `DV_CODED_TEXT` Flavour of null value, **e.g.** `253|unknown|`, `271|no information|`, `272|masked|`, and `273|not applicable|`. The first seems to mandate or at least strongly encourage a fixed set of null_flavour codes, whereas the second uses "e.g." to indicate that these are examples only (in my understanding). Now I guess it depends on the value of having "computable" null_flavours. I suspect that very little can actually be done in terms of computation... If these generic null_flavours are not so useful anyway (due to problems like the ones already described by e.g. @thomas.beale, @bna and myself above already), we might just add custom ones? If not, I think @ian.mcnicoll has suggested that we could think about custom null_flavours mapping to the core ones (roughly similar in spirit to what we have for [ISM_TRANSITION](https://specifications.openehr.org/releases/RM/latest/ehr.html#_ism_transition_class)), if I understood you correctly, @ian.mcnicoll? This of course is a somewhat bigger change. --- ## Post #33 by @emmanuel.eschmann [quote="siljelb, post:1, topic:4620"] [`null_reason` ](https://specifications.openehr.org/releases/RM/latest/data_structures.html#_element_class) RM element [/quote] Hi @siljelb , This thread you started asking how to model the NANO scale containing elements with a mix of ordinal and NULL values has been marked as solved with this post here. The solution described here is based on solution #2 you proposed, with the difference that NULL values are represented by '99' instead of '0'. However, the current version of the NANO scale (https://arketyper.no/ckm/archetypes/1078.36.2952, Rev. 2) lists only the measured / observed values in the ordinal data elements and does not list any substitutes for NULL values (neither '99' nor '0'), contrary to the post marked as 'solution' (I should of course mention that the creation date of this archetype revision is older than this discourse thread). Is that because you eventually decided to follow your proposed solution #1 after all? Have you tried to constrain the null_reason RM element with ADL (as it is not supported by modelling tools)? What are your plans for the modelling of the NANO score? --- ## Post #34 by @siljelb Hi! We haven't decided otherwise, as solution #1 (which we still think is the best) is unimplementable with our current tooling. @varntzen, do you know what happened here? --- ## Post #35 by @emmanuel.eschmann Hi Silje, Thank you for your answer! For the score I'm modelling (Delirium Observation Screening Scale = DOS) I don't need to distinguish between null_reasons. So, I think I can still use the solution #1 you described: * Using DV_ORDINAL for the two values per element/characteristic to be evaluated * Using the RM attribute 'null_flavour' 'unknown' if the user has not been able to score this characteristic. * Restricting the RM attribute 'null_flavour' to 'unknown'. I can't prevent compositions with an ordinal value and the RM attribute 'null_flavour' set to 'unknown' for the same data element from being validated by the openEHR REST API, but at least the DOS scale can be modelled using your solution #1 this way. --- ## Post #36 by @heather.leslie From what I can see there is an unpublished version from Nov 2023 on the branch at https://ckm.openehr.org/ckm/archetypes/1013.1.7080, which contains the two data 'null' elements with values of 99. This is the latest version, which follows the pattern in [GOSE](https://ckm.openehr.org/ckm/archetypes/1013.1.6671) and an adaptation of Option 2. I vaguely remember a conversation about updating this archetype with this pattern as a temporary measure (was it this one :thinking:), and pausing the review/publication process until the tooling catches up to the modeling needs. Or have I misremembered? --- ## Post #37 by @siljelb [quote="heather.leslie, post:36, topic:4620"] I vaguely remember a conversation about updating this archetype with this pattern as a temporary measure (was it this one :thinking:), and pausing the review/publication process until the tooling catches up to the modeling needs. Or have I misremembered? [/quote] I vaguely remember that too, and our meeting notes from November supports it, as referenced by @varntzen above: https://discourse.openehr.org/t/scores-or-scales-with-mixed-data-types-or-explicit-null-values/4620/29?u=siljelb --- ## Post #38 by @emmanuel.eschmann Thank you for today's meeting, Silje, Heather and Vebjørn! Based on the clarifications in this thread here and in our meeting, I will now model the Delirium Observation Screening Scale (DOS) using the same pattern as the current version of the NANO Scale, i.e. with an ordinal value of 99 for ‘null’ values. This means that I will not be modelling the DOS in the way I described a few days ago in this thread (https://discourse.openehr.org/t/scores-or-scales-with-mixed-data-types-or-explicit-null-values/4620/35). The reason for this change is that the modelling I did back then is hidden / not made explicit by the tooling and that the actual tooling doesn’t support the needs. However, using an ordinal value of 99 for ‘null’ values is only a temporary measure and a workaround until there is a better way of supplementing ordinals with null values. --- ## Post #39 by @sebastian.garde One more alternative for discussion: Use a DV_SCALE with a fairly simple spec change making the *value* attribute (but not the *symbol* attribute) optional*. ![image|690x192](upload://morxyKuExujpyv5ntYzt43W324k.png) [DV_SCALE](https://specifications.openehr.org/releases/RM/latest/data_types.html#_dv_scale_class) Rationale: DV_SCALE has been explicitly developed to pragmatically support the oddities of scores/scales and if this what is explicitly modelled in such scales it should be directly supported by the datatype as well. Seems to be more straightforward to me than trying to make null_flavours work for this. --- (\* Minor detail: This may also need a tweak to the wording of *is_strictly_comparable_to* and maybe an invariant to indicate that it is not possible for *symbol/code_string* to be blank and the *value* attribute null) --- ## Post #40 by @heather.leslie Exactly the solution we need. @Lucas @emmanuel.eschmann @Daniel_Ratschiller - an evolution of the specs to match a modelling need, rather than a modelling 'fudge', just as we discussed! --- ## Post #41 by @ian.mcnicoll Just as an addendum, until @sebastian.garde came up with this great suggestion, I was looking at what might need to change in RM/AOM to allow null_flavours to work more easily with DV_ORDINAL, and other datatypes, a was being discussed earlier in the thread. It turns out that this would require some significant changes. Right now all we can really do is constrain the list of null_flavours than can be used as and when an Element.null is required. We can;t say anything about how to force these to sit alongside a list of 'normal' terms as above. While there is very definitely a need for ELEMENT.null to handle integration failures, I am starting to question whether it ever makes sense to try to make this part of 'routine clinical modelling' practice. We already add 'pseudo-nulls' like unknown in questionnaire resources and in fine-grain datapoints e.g. in cancer pathology reports. I'll start a new thread to pick this up. Just for clarity, I think we have nulls and negations broadly correct in most areas but I'm no longer sure that trying to use ELEMENT.null in other places esp within Observations, makes sense. If nothing else, even if ELENT.null does make sense, it is certainly not going to be a simple tweak to the RM or in tooling. --- ## Post #42 by @heather.leslie I'm increasing not sure that null flavours should ever be part of clinical modelling at the archetype level. No information, masked, unknown (in information context, as opposed to a clinical determination of unknown) and not applicable should be available on all data points at all times, including a reason for each null. The current tooling only supports **removal** of the remaining null flavour options in order make one explicit or emphasised, when in fact all null flavours should be available all the time. Making 'unknown' an option in the archetype for a value set, should not remove the ability for 'not applicable' or 'masked' to be available in a template or implementation scenario. Agree about another thread being a better option at this point. --- ## Post #43 by @varntzen So, the solution is ... what? a) Use Ordinal and value '99' for now or; b) Wait for changes in spec's as described by Sebastian, then change to DV_SCALE or; c) Change to DV_SCALE now, and fudge it to something in line of '99' as alternative a) Just a slightly confused... --- ## Post #44 by @ian.mcnicoll The problem will be that if you do *anything* right now, it will require a breaking change to the archetype later, other than using the ELEMNT.null feature. or adding a temporary DV_CODED_TEXT alternative. I suggest @sebastian.garde and I take it to the SEC and see if we can get a quick Spec update agreed. It would also need a change in tooling and ultimately in the CDR validation. Is there a really pressing need? --- ## Post #45 by @heather.leslie If this can go through the SEC quickly it would be appreciated. This archetype is going through review ready for near future implementation. And there is another archetype with this same modelling issue that will likely follow through review in the next few months. --- ## Post #46 by @siljelb Agree, it would be great if this could be fast tracked. However, if I understand correctly it'll have to be changed in tooling and implementations too? --- ## Post #47 by @sebastian.garde [quote="siljelb, post:46, topic:4620"] However, if I understand correctly it’ll have to be changed in tooling and implementations too? [/quote] Yes, this change would definitely not work out of the box for CKM and pretty sure also not for any other tool. --- ## Post #48 by @ian.mcnicoll And it will also need a change in CDR validation --- ## Post #49 by @varntzen So it will take months to be able to use the proposed design. Which means we'll make NANO archetype as alternative a) in my post above, and leave it in v0. Which, by the way, was done some days ago: https://ckm.openehr.org/ckm/archetypes/1013.1.7006 --- ## Post #50 by @varntzen @ian.mcnicoll, @sebastian.garde [quote="sebastian.garde, post:39, topic:4620"] Use a DV_SCALE with a fairly simple spec change making the *value* attribute (but not the *symbol* attribute) optional*. ![image](upload://morxyKuExujpyv5ntYzt43W324k) [/quote] [quote="ian.mcnicoll, post:44, topic:4620"] I suggest @sebastian.garde and I take it to the SEC and see if we can get a quick Spec update agreed. It would also need a change in tooling and ultimately in the CDR validation. [/quote] I wonder if there has been any progression on this? Not to stress people, but if this change is fairly minor and solves the problem, it would be nice to proceed with it. :slight_smile: Kind regards, Vebjørn --- ## Post #51 by @ian.mcnicoll Raised as a Specification PR and I will table specifically at the next SPEC meeting. https://openehr.atlassian.net/jira/software/c/projects/SPECPR/issues/SPECPR-442 --- ## Post #52 by @siljelb This looks like a different issue to me? --- ## Post #53 by @sebastian.garde [SPECPR-442](https://openehr.atlassian.net/jira/software/c/projects/SPECPR/issues/SPECPR-442) is the correct one. --- ## Post #54 by @ian.mcnicoll Apologies - JIRA was messing me about. Fixed now. Please add comments there, so hopefully we can get consensus on this ASAP. --- ## Post #55 by @Seref Thanks to all who contributed for this thread, deserves (and requires) a few readings. My inadequate brain failed to grasp how @sebastian.garde 's suggestion is solving the example @siljelb gave though :slight_smile: As discussed in this thread, there is a requirement to indicate the semantics of lack/unavailability of data and won't we just lose the capability to express that semantics by using a null value ? Clinical modellers seem to be happy with the suggestion, so I must be missing something here. Care to help me catch up with the rest of you? Ps: There is so much to unpack here, worth commenting on, but I'd like to stay on track to be of some use in the SEC :slight_smile: --- ## Post #56 by @ian.mcnicoll A couple of comments that might help. - For these PROMS-type scores the real 'value' is arguably the `symbol` (DV_CODED_TEXT) not the numeric 'value'. And in some Scores that value really needs to be null, for at least some of the responses. - I had suggested making 'value' optional but on further thinking, I can see that this would allow all sorts of crappy data to be collected. - what I think we need is to allow an 'allow null value' option to be associated with a specific Code value constraint which in ADL2 could be done with a tuple, rather than 'allow null value' on the whole Element. ``` value matches { DV_ORDINAL[id9056] matches { [value, symbol,allow_null_value] matches { [{0}, {[at24]}, {{false}}], [{1}, {[at25]},{{false}}], [{2}, {[at26]},{{false}}], [{3}, {[at27]},{{false}}], [{}, {[at28]},{{true}}] // Nullable } } } ``` I'm sure that is not legal ADL but hopefully you get the idea. This is about being able to apply 'optional' to a specific CodedText constraint, not to the whole Element. --- ## Post #57 by @jannisp Thanks to everyone for the insightful input so far — this topic has had my head spinning for a while, too! In the example shared by Silje, the "null" answers are explicitly embedded within the question's ordinal options, which makes the handling of missing data more transparent. However, when it comes to PROMs and the subsequent analysis of PROMs data, properly addressing *patterns of missing data* and the use of **null flavours** becomes especially important. In fact, this might be relevant not just for questions where such options are explicitly listed, but potentially for *all* PROM questions. Consider this example: A male patient completes a PROM questionnaire, and several pregnancy-related questions are automatically hidden by the application logic. These hidden questions don’t display options like "Not applicable" or "Not assessed." A female patient sees the same pregnancy-related questions but chooses not to answer one of them, while submitting the rest of her responses. Later, when analyzing the data, it becomes critical to distinguish whether a question was *not shown* (i.e., not applicable due to system logic), or whether it was shown but *not answered* by the patient. Some clinics in our hospital use REDCap for Surveys, we’ve seen some rather “hacky” solutions to this — such as using codes like `999` or `777` to indicate different types of missing responses. While practical, these workarounds highlight the need for more robust, semantically meaningful approaches. Is there a good way to handle these "Reasons for Missing Data" scenarios? --- ## Post #58 by @ian.mcnicoll This is under discussion by SEC right now and there is an outstanding action to get clinical and tech together so that SEC can better understand the challenge and e,g why null_flavours (IMO) are not a good answer. See the JJIRA PR discussion here https://openehr.atlassian.net/browse/SPECPR-442 It would be great if you could join the discussion --- ## Post #59 by @jannisp Thank you, Ian. I will join. --- ## Post #60 by @ian.mcnicoll Re-reading you post, I actually think that null_flavours probably are a good place to document e.g partial completion for various reasons, especially for those that can be anticipated e.g not answerable by a male. I'm not sure I 'd try to catch and justify every partial-response, as that really complicates the UX --- ## Post #61 by @jannisp Somebody from our Quality-Management Department performing analysis of the datasets of PROMS-data brought this up which was very interesting to view this topic from his perspective: When seeing PROMs-data in a table view it would be very useful to see why some answers were not provided. I was not sure if this is possible with our current modeling: ![image|690x231](upload://2wsopreSqV7w7rAWEdvBx1WG1T3.png) As for the questions where explicit missing data ordinals are present, it would be great if we could render these null flavours as part of the frontend only in these special cases. The user would be unaware that altough he gets 5 possible answer-options 2 of them are semantically different things. Like this the archetype would not have additional fields because we would be using already existing ones. --- ## Post #62 by @thomas.beale One thing everyone needs to remember here is that in some cases, longer questionnaires rather than classic scores like GCS or ABCD etc, can be in a state of partial completion over some time period. Therefore, marking unanswered questions with null flavours probably should only be done at a final completion step. openEHR already has a lifecycle that allows partially completed content to exist and to be committed to the versioning system, i.e. there is no problem committing a half-done questionnaire so that it is visible to carers; it can be marked incomplete. --- **Canonical:** https://discourse.openehr.org/t/scores-or-scales-with-mixed-data-types-or-explicit-null-values/4620 **Original content:** https://discourse.openehr.org/t/scores-or-scales-with-mixed-data-types-or-explicit-null-values/4620