We’re encountering an issue with DV_DURATION types which have been saved and then retrieved in EHRBase compositions. For example, if we save a composition containing a DV_DURATION of 130 minutes, then EHRBase GetComposition retrieves that same data for us at a different precision level (it returns 2 hours), and in so doing it loses information (we’ve lost 10 minutes of precision).
These inaccuracies can get pretty large depending upon the values & units used! And DV_DURATION seems to be a pretty commonly used type in the CKM. So it seems to be a fairly troublesome bug.
This issue has actually been picked up and raised on github a couple of years ago, but there hasn’t been much activity. So I’m wondering if this is actually less severe than we think, or perhaps there’s an easy workaround we’re missing?
Looking forward to some discussion on this one!
Chris
This makes me wonder how DV_DURATION is persisted in EHRbase. Does it in some way normalise the ISO8601 duration string, for example changing “PT130M” to “PT2H10M”? If so that’s a potential problem, as for some use cases it’s not arbitrary which specific time units are used in a duration. For example you don’t want “38 weeks pregnant” changed to “8 months, 3 weeks, 1 day, 12 hours, 7 minutes and 29 seconds pregnant”. This is of course an extreme example, but if durations are automatically normalised, and you’re using average lengths of years or months, this sort of thing can happen.
I’m copying here the message I just posted in GitHub on the indicated issue.
EHRbase relies on openEHR Archie library for parsing durations, which in turn relies on the java standard libraries. Here the value is always converted into years, months, and days;
We consider in the past adjusting this to retain the original string, however we did not proceed with this because of other internal implications.
We will recheck this as in the past couple of months we updated the archie version we use for EHRbase and evaluate if this is still a limitation or not.
That is indeed part of the problem, I think.
The standard java library seems to parse “PT130M” then return the Duration object in string form as “PT2H10M” which iI agree, is definitely not what we would want in a health record. Though that still does not explain any rounding / loss of precision
@siljelb I think the 38 weeks pregnant example is a great one, and helps justify the desire for preservation of the input units!
@vidi42 Thank you for that update, that sounds promising, we will look out for an update when you’ve had a chance to look into it
@ian.mcnicoll We’re saving the composition as canonical JSON. I dug a bit deeper and I have to confess I’ve slightly over-egged the issue, my apologies! We do indeed get 2H10M back from EHRBase as in your test - that’s my mistake. I will edit the title of this issue slightly, to clarify this is more about the time units. The original precision is still there: we’ve just lost the original units.
For context, and because I think it helps to validate our use case: the reason this tripped me up is that our app requests input of a single numeric value (along with its accompanying unit of time). So when we retrieve this duration broken down into multiple units, our UI can only display one of those units (and it is picking the most significant unit). (edit: I can’t seem to edit the topic title, sorry - wanted to update to “Loss of original units…”)
I hope that the core of the original issue (and this discussion) remains: that it would be great to preserve the original input units if we can!
@vidi42 - we can definitely point the problem back to Archie and then to the standard java library behaviour.
I did a little more digging and the JavaDuration.parse does appear to preserve minutes correctly, so it is the .toString() function which is not preserving the original minutes vale
ISO: PT2H10M
Duration.minutes: 130
I’ll see what the AI gods can dig out, although possibly a bit niche to healthcare, it can’t be unique
I would put it stronger than this: It is very important in a healthcare context to preserve original input units. Unless we specifically want to normalise them, in which case we should use business logic to do this in a controlled manner. The CDR should not in any circumstance do this automatically, whether we’re talking about a DV_QUANTITY or a DV_DURATION.
he issue you’re encountering with the Duration.toString() function in Java is related to how it formats the duration. The toString() method of the Duration class returns a string representation of the duration using the ISO-8601 format, which is based on seconds. This format is PTnHnMnS, where n represents the relevant hours, minutes, or seconds part of the duration2.
For example, if you have a duration of 130 minutes, the toString() method will convert it to PT2H10M, which represents 2 hours and 10 minutes.
It then offers some nice code which looks great but actually introduces the same problem!!
Does this mean that EHRbase stores a DV_DURATION only as seconds and not as the original ISO8601 duration string? If so, this is of course useful for implementing the magnitude () function, but I suspect it makes this problem impossible to resolve without changing the storage format, since in many cases information is lost on converting an ISO8601 duration string to seconds.
I donlt know for sure, but from what @vidi42 said, I suspect it store some sort of representation of the Java Duration object. which has Hours, Minutes,Seconds etc, recorded ‘faithfully’ but the problem is that te kava library that converts this back to an ISO string is what seems to try to ‘optimise’ that string into Hrs, Minutes, Seconds, but first converting into ‘seconds’ then working back out.
It is really interesting that I can find nothing on the interwebs about this issue of the ISO translation losing the original representation. Just another indication that this is a very niche requirement in healthcare?
If I’m right about where the probem lies, it should be fixable by replacing the .toString function which converts the native Duration object to an ISO string. Better still if there is java lib that already handles this ‘correctly’ from our POV.
I would recommend running all dates and times through an ISO 8601 parser, for which you can use Java time or other classes, but only to validate the value. If valid, keep the original value as is and persist it.
