It can; each thing committed in a Contribution is a VERSION containing data, which will be a COMPOSITION (or PARTY-like thing, in the demographic model). Each of those VERSIONs can have its own committer - since each has an independent AUDIT_DETAILS object.
That might also be the optimal approach - it depends on whether you want to consider the nurse activities as more or less self-standing and meaningful - e.g. establish presenting complaint, check some vital signs, meds, takes blood etc - this could easily be committed as a Contribution on its own. Even if the doc were called away and the main visit never happens, all this info is probably useful. The main visit could then generate its own Contribution.
The rule for Contributions is the same for any versioning system: ideally, the contributoin, which is the delta (unit of change) to the information base always leaves the info base (patient record, for us) in a new coherent state. Example: a visit that updates the care plan with a note about changing asthma medication and also updates the medication list with a new Instruction for the new medication should be committed as a single Contribution; if say only the Care Plan change is committed in a Contribution, then the result state post commit is an EHR that has a Care Plan with the new medication, but a meds list with only the old.
In practical reality, I suspect not much computing gets done in openEHR that makes any assumption about proper use of Contributions. In my opinion, systems should be a lot more careful, because Contribution is the unit of rollback.
we indeed use Contribution mainly as the transaction container which will enable a roll-back once something fails. Hence, I don’t think the contribution should be used for representing encounter information or episodes of care, i.e. any clinically relevant data, or even try to group data together. With folders, this is much clearer and closer to what happens in analoge systems (making a shared understanding more likely).
I remember raising this point, but it was not in a written medium I think. I remember asking the transactional semantics of Contribution (specifically atomicity) , as in should a commit fail in case something in the Contribution is invalid, such as one composition out of N compositions failing validation. The answer I got at the time (I think from Tom, or maybe Heath) was that Contribution did not have any transaction semantics specified.
I always thought that it should, and Ocean’s implementation actually follows that approach. So this may be a good opportunity to suggest that we put this into the spec, i.e. clarify the atomicity of a Contribution’s commit operation: atomic, or not?
Tangent comment, I know, but you made a good point and I didn’t want to miss the opportunity.
It really should, and the explanation of Contribution implies this, but doesn’t require it, mainly because commit atomicity is at a DB/file system level, whereas the change control / versioning model described in the specs is at an interface/API level.
However, when interrogating an openEHR EHR through any public API, Contribution should be the reliable unit of commit and rollback. Normally, achieving that means making something equivalent the unit of commit in the concrete back-end - DB transaction, Github like change-set or whatever.
Which was part of the explanation I got at the time, but I omitted that from my post.
I think it’s OK to use the term atomicity without necessarily using its implementation level meaning. I don’t mind inventing something such as ‘should succeed or fail wholly as a single unit’ if we don’t want to imply a lower level concept, but we should clarify the failure scenario for uses of APIs.
So I think Contribution’s definition should require the behaviour we seem to agree on. I’d even say this is actually a good candidate to add to conformance specs…
At least in our implementation in EHRServer, the CONTRIBUTION is atomic, any failure on any COMPOSITION makes the whole CONTRIBUTION to be rejected.
I’m not sure how well specified is that part in our specs, but or my it was the most reasonable behavior, since the result of the CONTRIBUTION commit is for the whole CONTRIBUTION not for the individual items in it. So the result plays an important part in the interpretation of this specific behavior.
Much-needed discussion! I want to contribute by adding a few other options on how this can be done. I feel like the directory approach explained very well by @birger.haarbrandt here is the “official” way to do this. However, I am going to be heavily critiquing the directory approach since I think it has many issues. These issues have also been brought up in more detail here by @pablo and @thomas.beale.
My main issues with the directory approach
There can only be one directory per EHR: So every time you need to update it, you need to retrieve it, update parts that you want, and then post it back. I don’t like doing this since it increases the burden on applications to update the folders correctly. And giving full access to every app to update the root folder is just waiting for things to go wrong. And it’s a nightmare to implement Access Control policies on Directories and Folders based on the content.
If you want to filter an AQL to a specific folder, you can’t do it in AQL directly (yet): You’ll need to get the directory first, run through all the compositions in a tree search, then use a MATCHES clause and include all the composition IDs there. I don’t like this too mostly because the directories can grow extremely large sometimes and having to get all the composition IDs manually only to feed it into an AQL doesn’t feel “well-integrated”. It’ll be easier to just store the directory information in an external database since it’ll at least solve problem 1.
Alternatives that solve both the above problems while still only using existing openEHR APIs
I’m all for the changes proposed here. But until that’s widely implemented and there’s support for executing AQLs on folders, this is what I’m doing:
1. Using other context
Include an archetype in the other_context part of a Composition to record the encounter information - You can probably use an existing archetype to store the Encounter/Visit identifier within this archetype.
It solves problem 1 since visit information is stored in the same transaction as the composition and problem 2 since you can now easily query this using AQL.
2. Somehow stuff the visit identifier in the composition. Somewhere.
Although the above approach works, I wanted to find a solution to this, WITHOUT changing any of our existing templates. That’s when I kinda started misusing the health_care_facility attribute which is part of the Compositions class in the RM. So we’ll do something like (in ECIS flat):
SELECT c/context/health_care_facility FROM COMPOSITION c WHERE c/context/health_care_facility/external_ref/id/value='b9ee9eae-84be-4a4f-9018-2014d8c5341a
I know that I’m probably misusing the intent of the “health_care_facility” attribute in the composition. But it solves both my above-mentioned issues and works without having to change any templates to include special archetypes. And I’ve never really stored useful information in the health_care_facility anyway.
Maybe consider adding another attribute to the COMPOSITION class to uniformly record the visit information? Since it feels much better integrated with the system and the visit identifier is mostly just another context data point.
I welcome suggestions and corrections if I am wrong in taking this approach.
Small clarification: we keep a log of the full CONTRIBUTION in the server even if the commit failed, that way we can keep track of successful and failed commits to check what was wrong with the rejected ones. So the server admin could actually access those failed CONTRIBUTIONS.
This is a design decision I did when I was implementing this part in EHRServer, not part of the openEHR specs.
Hi Sidarth,
I like your first approach with storing the visit ID in other context. However I think it’s not legal to include an archetype there. So I think you’ll have to manually design the compositions in a way that will ensure the paths (including node ids !) are consistent across compositions for reliable querying. Probably think hard about automated testing or similar to find errors at run time, or this can go very wrong. On a side node: composition.context currently is designed only for event compositions and is problematic for persistent compositions (e.g. problem list).
Adding a compostion.meta_data or similar attribute that allows list could solve both problems by e.g. creating a ‘use_archetype’ constraint for a CLUSTER.visit which has a visit_id element. (Current AQL implementations are said to only allow queries on clusters, or you could cut out the cluster middle part and include an element archetype). Issue for the event context for persistent compositions here: [SPECRM-106] - openEHR JIRA
As you already concluded I believe using healthcare facility to store a visit id is misuse. Since the ID is part of party identified to me this is unambiguously the id of the facility (building), not the id of the visit(event). I understand pragmatism is needed in the real world. The risk of misinterpretation for a single systems seems low for the pattern you described.
But since openEHR is intended for interoperability, this will cause real world trouble once you share EHRs with apps that are not controlled by you and don’t know this assumption. (And they shouldn’t need to!).
Slightly off topic I’m also a bit worried about your reluctance to update templates, a minor change like this should be trivial right?
I do feel we may want to support this at RM level. Since it’s such a fundamental usecase most implementers will encounter. And it’s valuable to standardise. And according to reactions from many people, we haven’t solved this one yet.
My first thought would be to add an event_id attribute with DV_IDENTIFIER to composition.context, however this is still problematic for persistent compositions.
@Ohia_George sorry for hijacking your thread, I’m hoping you don’t get overwhelmed by the complexity of our discussion about your straightforward answer. My advices would still be to use folders. But @Sidharth_Ramesh mentioned some important caveats. Let us know if you need further advice, and your experiences in implementing the folders.
Hey Joost,
I agree that something at the RM level would be a great solution.
Regarding my reluctance to updates templates - there are some templates that we don’t have control over. We want to build an “app ecosystem” where each application can have the freedom to model the templates however they like without any restrictions.
A simple example: In India, we have a government program called ABDM that expects FHIR document bundles to be sent for every patient visit to a hospital. We have created an “application” that takes care of this whole integration process. It executes archetype based AQLs to get common data points like chief complaints, diagnosis, medication orders, service orders and multiple observation archetypes and automatically convert them to FHIR resources to send to the government APIs.
So the other applications doesn’t need to reimplement this over and over again and can focus on other specific functionality.
Ideally, we want the ABDM integration application to work with all the other applications as long as they use the right archetypes in their templates and post their compositions to an OpenEHR repository.
Using virtual folders is definitely a workable solution. But when there are multiple applications involved, the problems I mentioned above become a big concern.
What I don’t want is asking every other app to model templates with a specific element to capture the visit id. This will be expensive because there are already many working templates and everyone will need to update them with specific semantics, and also change a lot of application code to reflect these changes.
Pushing it in the context doesn’t require changes to the templates, and can be done at the level of the application, which is relatively quicker and easier.
I think getting the semantics of the “visit” correctly in openEHR is crucial. I agree that the approach we’re taking currently is “hacky” at best. But it works for our situation where changing templates is not in our control, and using folders isn’t practical.
@Ohia_George If you’re just getting started my answer might have provided more questions than answers. I’m sorry!
I think for a use case where you only have your own application using your own OpenEHR data repository, directories are a perfectly fine solution. Being able to query folders by AQL is just a “nice to have” and isn’t really required. You can attain the same functionality using other methods as well.
Well this is potentially true of any attempt to do access control based on content. A fairly common view these days is that viable access control (as opposed to idealistic but probably impossible) is primarily by the record owner (patient) having a way to classify/mark their data as ‘sensitive’, ‘non-sensitive’, and maybe more such classifications. ‘Sensitive’ would require presence of patient to give consent. Emergency situations have a way of over-riding all access control.
We can certainly discuss this, although it was always assumed that that was the purpose of EVENT_CONTEXT.other_context. But if we think we can standardise visit info, more attributes could be added to EVENT_CONTEXT.
In fact you can, but the top-level archetype will either need a slot (ADL1.4) or a direct external ref (ADL2) referring to the other archetype. A new archetype can go anywhere.
That should not be true either - a path in AQL is any path legal in the RM structure.
Yep. No-one is going to expect that…
The problem is: do all institutions have such an identifier for a visit? I don’t think my local GP does. At best the system could construct something like:
or maybe pat-id and doc-id. Internally the various UK GP systems probably do assign a Uid to almost everything, but I doubt if this is exposed in any API or message to the outside world.
In HiGHmed, we use a cluster archetype “case identification” within the other_context of all templates. No problem at all. I get @Sidharth_Ramesh’s point though. Hence, I think folders would actually a bit more elegant.
I think it’s fine to leave the ‘how to get the ID for a visit’ to implementers for now. It would be good though to have a single space where we record ‘a’ visit ID. I’d be fine with recommending uuid if people prefer it. It will prevent people taking (the wrong) meaning from a numbering system.
From an implementer’s perspective, nearly every system I’ve worked with either has its own visit ID or expects to receive one from a parent system.
The problem is typically too many visit IDs, and a disagreement of the semantics of a visit. For a simple example, if a patient is hospitalized, has a radiology appointment, and needs the pharmacy to dispense a contrast agent that’s potentially three different visit IDs from three different systems, and each care document generated in the workflow will only care about a subset of them.
I think having a standardized solution for visit IDs is very important. It’s often the single central identifier for gathering all data relevant to a particular workflow.
I have faced such differences in the semantics of the visit too. Some hospitals take the entire episode as one visit while some have “micro-visits” that consider different transfers and departments as separate visits within the parent visit.
I like @joostholslag idea of including a field in the EVENT_CONTEXT for the visit id. It’ll be great if it can be 0 to multiple dv_identifiers. That is definitely a possible solution.
Regarding what @birger.haarbrandt said about preferring to use folders, I agree it’s elegant if implemented correctly. Especially since it might be more useful when querying visits within visits.
I’m assuming this is the work flow:
Create a composition
Create or update a folder corresponding to the visit id. Update the parent visit if it’s a micro-visit within the same hospital episode.
Query based on the visit id to get all compositions inside a particular folder. This will be super cool especially if the querying can somehow get all the compositions from a parent visit when taking the micro-visits approach.
Though I’d like to have steps 1 and 2 happen together in a transaction rather than separately. Maybe this is where atomic contributions come to the rescue? Although I’m not sure how that’ll work with simplified data formats like ECIS FLAT.
Great to be having this conversation by the way. Let’s standardise how we represent the visit information at least for new implementations on openEHR!
Adding the feature to tag (or more precisely) annotate compositions with different key,value data. One tag is the EpisodeOfCare . By using this we can add any openEHR composition to any episode. This was needed because the legacy system we integrated with didn’t have versioning on such a change
Using folders - adding compositions to folders to make data belonging to the same context being placed in the same folder. As part of this we had to break the specification (at that time) which said that the EHR had only one root folder. This was not useful for our use-cases.
I agree the folder-approach seems to be the most useful and flexible. It also allows a simple way to retroactively include “micro visits” in a broader visit/episode without touching the (possibly already digitally signed) compositions (or the folder) from the “micro visit”.
Clarifying how to filter based on FOLDER in AQL is urgent. Perhaps @bna or somebody else can start a separate discourse thread on that.
Yes this is the tricky part when creating a new composition via API if the New composition-id is created by the server. Then you don’t know what composition id to refer to in the folder-update that you’d like to preform in the same Dingle contribution/transaction.
A possible solution might be to either have the client create the ID (and hope the client uuid generation mechanism avoids collitions) or to specify à New API call to first create/reserve a new unique ID that is then used in both the composition and in the composition reference/pointer of the folder.
In addition to that contribution-basen approach it could be handy to also add a new header or parameter to the “simplified” …/composition REST endpoint that allows you to specify a folder (or a list of folders) that you’d like the server to add this composition to. (The contribution stuff with composition creation and folder update would then be done “under the hood” in the server.)
