AQL to query the same node across multiple versions of any archetype

Dileep_V_S · 8 February 2020 10:15

Hi,
What is the way to write an aql to query the same node in multiple versions of the same archetype? Is there a way to use a wild card or regex to include all versions of an archetype in the contains section of an archetype?

If that is not possible, can we include multiple versions as below and point the data nodes to the same key?

select
b_b/data[at0002]/events[at0003]/data[at0001]/items[at0004]/value/magnitude as weight,
b_c/data[at0002]/events[at0003]/data[at0001]/items[at0004]/value/magnitude as weight,
EHR e CONTAINS COMPOSITION a CONTAINS (OBSERVATION b_b[openEHR-EHR-OBSERVATION.body_weight.v2] OR OBSERVATION b_c[openEHR-EHR-OBSERVATION.body_weight.v1])

In Ethercis the above does not seem to work as the key seems to get overwritten by the 2nd instance always.

regards

birger.haarbrandt · 8 February 2020 10:35

Hi,

the wildcard approach you suggest works in the Better Platform/EHRSCAPE. At EtherCIS/EHRbase, it is not supported, yet. Hope that we can put this on the roadmap but unfortunately it might take some time util it becomes available.

Birger

ian.mcnicoll · 8 February 2020 13:49

I don’t think ehrscape would like that query either as it would object to the duplicate alias - what happens if you make this weight_1 and weight_2

b_b/data[at0002]/events[at0003]/data[at0001]/items[at0004]/value/magnitude as weight_1,
b_c/data[at0002]/events[at0003]/data[at0001]/items[at0004]/value/magnitude as weight_2,

and then do a little post-processing? I might be a little more comfortable with that in any case. Doing a wildcard on a specialisation makes sense, doing it across multiple versions, which just happen to have the same paths, just feels a little more scary (though, of course that is not the case in reality!!).

birger.haarbrandt · 8 February 2020 15:37

I’m quite sure it works in Better (not exactly as done in the example but wildcards should be supported like body_weight.*)

…I might try later to confirm😎

birger.haarbrandt · 8 February 2020 19:08

I can confirm that this actually works in Better:

select a_a
from EHR e
contains COMPOSITION a
contains OBSERVATION a_a[openEHR-EHR-OBSERVATION.body_temperature*]

Dileep_V_S · 9 February 2020 19:11

I feel this would be a good feature in AQL as archetypes are expected to evolve for some time into the future. Over the life of a person, EHR systems would end up using multiple template versions. As I have understood, the biggest strength of OpenEHR is the ability to support evolving clinical data sets.

Since Ian has touched upon the topic of archetype specializations, What are the reason why we should specialize as opposed to creating new ones? Does it bring any advantages in querying?

regards

ian.mcnicoll · 10 February 2020 07:40

@dileep good question about specialisation. Perhaps it would be better to ask in a separate thread in the clini csl channel so we get a wider response?

Dileep_V_S · 10 February 2020 08:07

Thanks Ian. Will do that
Regards

bna · 10 February 2020 11:35

I can confirm it also works in DIPS EHR Store. And if you want to be explicit about the versions you might want to something like this:

select a_a/archetype_details/archetype_id/value as archetype
from EHR e
contains COMPOSITION a
contains OBSERVATION a_a
WHERE a_a/archetype_details/archetype_id/value MATCHES {‘openEHR-EHR-OBSERVATION.body_temperature.v1’, ‘openEHR-EHR-OBSERVATION.body_temperature.v2’}

matijap · 11 February 2020 11:41

In Better Platform, most types need to have an archetype specified, so the last example from @bna would need to be amended with [openEHR-EHR-OBSERVATION.body_temperature*] after observation alias. The WHERE condition would then work as intended.

pablo · 14 February 2020 03:06

This is off topic but related: though is not AQL, in the EHRServer, the Query Builder has a qualifier in the archetype selector “Allow any archetype version”.

That generates a wildcard in the condition/filter, in the archetype ID:

andrazk · 18 March 2020 14:40

Another option within Better Platform would be to use UNION ALL keyword, that merges result-sets of two separate queries.

SELECT f/data[at0002]/events[at0003]/data[at0001]/items[at0004]/value,
c/context/start_time/value
FROM EHR e
CONTAINS COMPOSITION c
CONTAINS OBSERVATION f[openEHR-EHR-OBSERVATION.body_temperature.v1] 

UNION ALL

SELECT f/data[at0002]/events[at0003]/data[at0001]/items[at0004]/value,
c/context/start_time/value
FROM EHR e
CONTAINS COMPOSITION c
CONTAINS OBSERVATION f[openEHR-EHR-OBSERVATION.body_temperature.v2]
ORDER BY 2 DESC

ian.mcnicoll · 18 March 2020 17:05

Thanks @andraz. For others, UNION is not currently part of the official AQL spec but perhaps worth thinking about. I presume you have to carefully align the SELECT statements ?

Can you show any other good examples of where UNION can solve tricky problem?

birger.haarbrandt · 18 March 2020 17:08

Hi Ian,

UNION, MINUS and INTERSECT will be needed for any sensible cohort identification query. Hence, we should add these to the standard sooner than later.

ian.mcnicoll · 18 March 2020 17:10

Can you give an example @birger.haarbrandt (for my education!!)

birger.haarbrandt · 18 March 2020 17:21

If you want to identify patients eligible for a clinical study. Here is a quick example:

With MINUS, we can exclude all patients (EHR IDs) from the result set (those that have a diagnosis of sleep apnea) that was built based on the inclusion criteria inside one query.

Alternatively, we could do these operations outside AQL but then we potentially loose the optimizations provided by the database management systems.

pablo · 18 March 2020 17:31

In the EHRServer we do something like that, it’s called Combined Queries. The idea is, you can have one query to get compositions that match patient_age >= 40, another query for compositions that have diagnosis IN a SNOMED CT Expression (could be multiple queries for multiple comorbilities and allergies), another query for compositions containing some kind of medication, etc.

Then the combined query can mix all of them, and if there is at least one result for each query, the EHR containing those compositions is selected, that means, you have identified patients matching all the queries.

This could be used to identify cohorts for a special prevention program, or for a clinical trial, or just for doing epidemiology and knowing your population. Also is something we use to support CDS rules, since this mixed information is crucial for some rules to send alerts and reminders, also useful to calculate population indicators.

Though, this is not AQL, also the combined queries concept is not in openEHR, is something I designed with this purpose of supporting those things mentioned above.

Here there is an old demo, but works to show the key concepts https://youtu.be/pOMhqc1TZ7A?t=987

andrazk · 19 March 2020 13:03

As you say, this is Better Platform extension. Better (not a company ) sooner than later the AQL spec should be revisited and extended with the most common functions used, to align them between providers.

Anyway, some specifics:
1/ if you use aliases in both queries, those from the first query are used,
2/ you can use ORDER BY in second query and it will be applied to the whole result-set
3/ both queries should SELECT the same number AND same TYPES of data

SELECT y/data[at0001]/events[at0006]/data[at0003]/items[at0004]/value as systolic,
       y/archetype_details/archetype_id/value as archId,
       c/context/start_time/value as time
FROM COMPOSITION c
CONTAINS OBSERVATION y[openEHR-EHR-OBSERVATION.blood_pressure.v1]

UNION ALL

SELECT y/data[at0001]/events[at0006]/data[at0003]/items[at0005]/value as diastolic,
       y/archetype_details/archetype_id/value as archId,
       c/context/start_time/value as time
FROM COMPOSITION c
CONTAINS OBSERVATION y[openEHR-EHR-OBSERVATION.blood_pressure.v2]

ORDER BY time DESC

This query merges blood pressure from two different archetype versions and sorts them out in a single stream of data, ordered by time of creation. But beware, if you observe the second query, it SELECTs diastolic - different path, but same type, so no issues there - but content wise you should be careful.
With latest Better Platform we do support the [openEHR-EHR-OBSERVATION.blood_pressure.v*] notation for this purpose.

UNION ALL comes handy when you would like to have a result-set of a key-value format, for charts containing data from different observations, which fall under separate constraints, different templates used etc. EAsier to manage and establish complex criteria - each query for it’s own content.

Ever tried UNION ALL over three or more queries?

SELECT y/data[at0001]/events[at0006]/data[at0003]/items[at0004]/value as data,
       "systolic" as type,
       y/archetype_details/archetype_id/value as archId,
       c/context/start_time/value as time
FROM COMPOSITION c
CONTAINS OBSERVATION y[openEHR-EHR-OBSERVATION.blood_pressure.v1]

UNION ALL

SELECT y/data[at0001]/events[at0006]/data[at0003]/items[at0005]/value as diastolic,
       "diastolic" as type,
       y/archetype_details/archetype_id/value as archId,
       c/context/start_time/value as time
FROM COMPOSITION c
CONTAINS OBSERVATION y[openEHR-EHR-OBSERVATION.blood_pressure.v2]

UNION ALL

SELECT y/data[at0002]/events[at0003]/data[at0001]/items[at0004]/value as temperature,
        "temperature" as type,
       y/archetype_details/archetype_id/value as archId,
       c/context/start_time/value as time
FROM COMPOSITION c
CONTAINS OBSERVATION y[openEHR-EHR-OBSERVATION.body_temperature.v1]

ORDER BY time DESC