# Versioning implementations **Category:** [Implementers (archive)](https://discourse.openehr.org/c/implementers-archive/158) **Created:** 2017-06-21 01:04 UTC **Views:** 3 **Replies:** 29 **URL:** https://discourse.openehr.org/t/versioning-implementations/15484 --- ## Post #1 by @system Hi all, I had this questions in mind for a long time: did someone implemented the distributed versioning of openEHR? The specs define a great distributed versioning mechanism but it is a little trickier to implement. Also there is no clear who will do the work of managing that, and how that structure will be queried. It is very difficult to me to think of an amendment sent to an EHR and that not being available for all the parties looking at the EHR of the patient. In the case of the EHRServer I built, only linear versioning is possible, there is only one latest version for each compo, and queries only get data from latest versions. Just wondering, what do others did for versioning and what policies do you have if you implemented the distributed approach in terms of branching, merging and querying. Thanks! --- ## Post #2 by @system Hi Pablo, I did it a few years ago, just dumped not\-current versions in a slow XML database, because, in normal cases they are never queried, and when they need to be queried, there can always be found a faster solution\. But of course, this was a linear version system\. ExistDB supports distributed versioning on XML out of the box\. And you can also use a normal, not OpenEHR, version system like Git or VCL\. But when looking at how OpenEHR works, is there ever need of merging? Do people edit concurrently same datasets? I think they are they always working on new versions of datasets, there is only one exception, that is the persistent Composition, there could occur merging problems\. But I think, you don't need distributed versioning to handle this, a locking system \(like databases have\) is, I think, good enough\. That is how classic EHR builders handle concurrency\. Bert --- ## Post #3 by @system We have the data structures set up in our database to handle branching and merging\. But we do not yet support it for users and do not currently have plans to build it\. The merge process looks like it could be complicated for users\. Regards, Pieter Bos --- ## Post #4 by @system Hi Bert, see below --- ## Post #5 by @system Merging and keeping the data within the constraints of the archetypes is nearly impossible to do automated. Because, what do you do when person A adds an item to a structure and at the same moment person B adds an item to the same structure, but in the archetype is defined that in that specific structure only one item is allowed. There you have the problem, inconsistent data because of merging. I agree with you that distributed versioning is not feasible, even, sometimes, dangerous. It would be good to remove it from the specs. Bert --- ## Post #6 by @system Hi Bert the case you mention is not versioning control, is concurrence control. Not sure which system will allow two users to insert invalid data on an item when the archetype constraint says it is not valid. If that is an implementation, I don't think it is secure in terms of concurrence. Versioning would be when a uses commits a document that is "complete". IMO incomplete compos should not be final versions, and if one user is working on an incomplete version, no other user can work on that (read-write work). If two users need to read-write incomplete compos, then 2 separate versions are needed and there you have branches. Linear versioning would not allow to create branches, and new versions would not be created until the user that has the current version in read-write mode finishes and commits the completed version. That is the only way to keep it linear, with locks. Not sure about removing the current approach from the specs, but creating a simpler alternative might be of use to enable more and quicker implementations. --- ## Post #7 by @system I must have expressed myself badly. I try again, if I misunderstand you, I am sorry, but that is what I make of your issue, which I think is a valid issue. -------------------------------- It can be concurrency/locking problem to solve when there is no versioning at all or only linear versioning. Then it is simply not allowing to edit a dataset when another has the dataset editing it, there are variants for optimistic and pessimistic locking.. It becomes a merging problem if you allow two instances to create a new version of a dataset, and there is only one dataset to be the result. In that case, the two versions have to be merged. In sourcecode this also sometimes a problem, sometimes hard to do. For example, some lines may only occur once in a Java file. There maybe only one package-line, there maybe only one public class and it must have the same name as the file, etc. Then the version system mostly gives up, and handles the file with the problem to the developer, who mostly immediately starts to curse. I OpenEHR merging conflicts will typically occur in persistent compositions. Two users are concurrently changing a persistent composition dataset. User A post his change, and then user B wants to post it. The version system will not accept it from user B, because the dataset has been changed since User B has opened it. The versioning system must then judge if the change of User B is in conflict with the change of User A. There is a conflict when an automated merge will: 1) overwrite the changes of User A, 2) When the automated merge will be in conflict with an archetype constraint. The best thing the versioning system can do in one of these cases is to handle the change of User A to user B and let him decide what to do. I am pretty sure that 99% of the users of a medical system do not have the talent to solve such a problem. This makes the merging system useless when there is a conflict. And a merging system which can have such a high failure rate is useless. This leaves us to the only version option, that is linear version system, which only one user at a time able to change a dataset. So locking is then needed. Bert --- ## Post #8 by @system Sorry Pablo, I see your full reply first time now, I didn't see it before on my phone, seems my email app was hiding a part of the text\.\. > Versioning would be when a uses commits a document that is "complete"\. IMO incomplete compos should not be final versions, and if one user is working on an incomplete version, no other user can work on that \(read\-write work\)\. If two users need to read\-write incomplete compos, then 2 separate versions are needed and there you have branches\. Linear versioning would not allow to create branches, and new versions would not be created until the user that has the current version in read\-write mode finishes and commits the completed version\. That is the only way to keep it linear, with locks\. We have the same conclusion, but you were first\. Linear version is needed with locks, only one user at a time changing a persistent composition \(or another kind of dataset, but that is unlikely to happen\) > Not sure about removing the current approach from the specs, but creating a simpler alternative might be of use to enable more and quicker implementations\. I think it is the only possible conclusion since a distributed version system cannot be used, because of the inability of users to solve manually merging problems\. Sorry again for writing what you had already written just before that\. Best regards Bert --- ## Post #9 by @system Hi, distributed versioning with branching was included to allow syncing of data gathered about the same patient in different EHR repositories. For most data, syncing the repos is trivial, since it's different data. The classic cases of potential clashes is medication list, problem list, basic clinical demographic data, etc. If a sync was started and two medication lists are found that are forks of a single earlier one, a manual merge will be required. We are only just starting to see the implementation of systems where syncing may be a question, so although there may be adjustments to make to the branched versioning model, I would not be in favour of throwing it out at this point. We are however going to move it to the BASE component and make it a standalone model. - thomas --- ## Post #10 by @Seref I did not realise that this discussion reached the point of suggesting that distributed versioning is taken out from the specs. I have been designing and implementing lots of openEHR data syncing functionality which relies on the distributed versioning specifications. I have heaps of work pending, which will also use that part of the specs. I can tell you that the current specs have worked just fine for me so far and they are up to the same high quality as the rest of the specifications, so they are absolutely usable and useful. The challenges of distributed versioning does not eliminate the need for them, so I cannot agree with the suggestion to remove them. Sure, move them around in the specs all you want, but please keep them. All the best Seref --- ## Post #11 by @system From Thomas comments, it is clear that we have such last two use cases, internal versioning and cross-system versioning / sync / consolidation. Consider people here is talking about their own experience with the specs under the use case they implemented. We can argue that internal versioning is needed in 100% of the implementations while cross-system is a much less implemented case. This doesn't mean that the current specs are not usable and useful in abstract, but we should contextualize the discussion by use case and by the frequency each is implemented. For internal versioning it is clear that distributed versioning spec generate some friction at implementation time. IMO we need to address both use cases trying to minimize friction for new developers. That can improve the quality of the specs without print anything out. Also, I would like to hear more about implementations of both use cases and the challenges implementers had to really validate the idea of addressing both use cases explicitly in the specs. Cheers, Pablo. --- ## Post #12 by @system Naturally I am all for revising the specs (it's what we do ;) and throwing out dead stuff. But one thing I have realised over the years is that many of the scenarios (such as multi-system syncing) we thought of in the 1990s and early 200s are only just coming onto the radar now. Progress is much slower than many of us thought. Consequently, some types of implementation experience gained so far - particularly anything cross-enterprise or regional - is not going to be an indicator to the future. Of course, some kinds of experience, say with using the RM, or ADL 1.4, AQL etc, has been giving us all the feedback that we needed to make the updates we are currently making to the specifications. Probably what we should consider in this case is an update to the Change control spec that describes a variant or guideline for using the model to implement linear versioning, while allowing for later addition of branched versioning when/if needed. - thomas --- ## Post #13 by @system I agree that merging is (normally) only interesting for persistent compositions, that are the only kind of compositions which are candidat for simultaneously editing (branching), and then afterwards merging of the branches is needed. I think, getting rid of the persistent compositions would solve these problems. I don't see objections against medication-lists in normal compositions. Maybe the persistent composition idea was something from the old days to have all medications handy together? If that is so, than we can consider that this way of preordening is not anymore needed because modern systems can quickly find medication-entries, and the extra advantage is that branching and merging is then also not needed anymore. Best regards Bert Verhees --- ## Post #14 by @system We are using persistent compositions a lot in our system. These are compositions with content that lasts and might be updated several times. To make persistent compositions usable we have introduced “scope”. Examples of scopes is episode of care, period of care, ward stay, etc. A persistent compositions contains information where only the latest version holds the correct data. So far we haven’t implemented (no need for) branching in versions. But I know that kind of requirements will come. I think we should keep persistent compositions (and even extend them) and the versioning chapters in the specification. The conformance levels will tell what kind of functions that will be needed in the different profiles. Vennlig hilsen Bjørn Næss Produktansvarlig DIPS ASA Mobil +47 93 43 29 10 --- ## Post #15 by @system I notice the versioning discussion. Is see three different topics; (non-)Persistent, versioning and synchronisation/consolidation. I see no use of non-persistant flags. I see two reasons for versioning. Synchronisation,consolidation is too complex for now. ERS systems document events. Each documented event is equally important to document. What is important now is not later. And vice versa. Always a subjective opinion is documented by the author. **Persistent** Depending on circumstances an event is important or not. In this context I fail to grasp the need to label certain events as persistent and others not. All documented events need to persist in EHR-systems. **Lists re-used data.** Some events warrant to be re-used in context dependent lists: Active medication, Problem list, Previous diagnosis, etc. Each context, HcProvider will need different lists for different purposes. These lists are the result of documented events and persist. Creating lists is an example of re-using data, because list content is derived from pre-existing events/data. These lists are either changing are not-changing over time. **Versioning Lists** Lists can be updated as the result of new events in the patient system and therefor need to be versioned. These are non-technical new versions. They are the result because of changes in the patient system **Versioning events** While documenting events and committing them to the data base sometimes event data needs to be changed, updated, corrected. The same event gets a new technical version, because nothing in the patient system changed but the documentingHcProvider initiated the change. **Synchronisation, merging, syncing** This is a complex topic. Is there an extensive list of examples and requirements? Gerard Gerard Freriks +31 620347088 [gfrer@luna.nl](mailto:gfrer@luna.nl) Kattensingel 20 2801 CA Gouda the Netherlands --- ## Post #16 by @Danny_Nguyen Dear All, Please remove me from this listserv. Best, Danny --- ## Post #17 by @pablo @Bert Persistent records are a well know pattern in ehrs and it's usefulness should not be under question. Of course systems that focus on primary care might not implement them. But for hospital or even regional / national records need a wider view of the patient, persistent shows their value. @Bjorn, what is the relationship between a scope and OPTs/folders? The concepts you mentioned are likely to be modeled with OPTs and folders. It is very interesting that you didn't need branch versioning yet. --- ## Post #18 by @Seref Hi Danny, You can unsubscribe from the same page you've subscribed. [http://www.openehr.org/community/mailinglists](http://www.openehr.org/community/mailinglists) --- ## Post #19 by @system Hi Pablo, two questions Which problem do you solve with persistent records which cannot be solved in another way? Do you agree that persistent records are the only reason to have branching/merging necessary? Thanks Bert --- ## Post #20 by @system well they are likely to be the most common element of an EHR to which branches and merging would be applied\. However they are ubiquitous and are also likely to be extended to things like care plans and so on\. But in principle, branch and merge could happen to anything in the record, e\.g\. for reasons like adding competing translations of clinical notes etc\. \- thomas --- ## Post #21 by @system It is true that care\-plans for a single patient for a single case can be written by more care\-takers simultaneously, but one could argue if they, in that case, belong in the same composition? Wouldn't it be better to have more care\-takers for one patient on a single case have their own compositions, and are grouped by folders outside the compositions? Bert --- ## Post #22 by @system well how a care plan is designed is up to clinical designers\. The usual idea is to have an updatable, evolving plan \(potentially for each major problem\), shared across care providers\. \- thomas --- ## Post #23 by @system On second thoughts, I agree that to facilitate a care\-plan is a good reason to have persistent compositions\. In case if more than one care\-taker is able to edit a single care\-plan in a single composition simultaneously, then branching/merging will become necessary\. However, in that case, I think it is better in that case to lock that particular composition for updates and have it refreshed on the screen like should always be done on database\-based applications after locking\. In that case, only versioning without merging would be necessary\. But like you mentioned before, merging \(without branching\) can become necessary on importing data from another OpenEHR\-based EHR, in that case, persistent compositions can have reasons to be merged\. Thanks very much for the explanation\. I understand that this is a delicate subject which complexity is easily underestimated\. Bert --- ## Post #24 by @system Hi Bert, excellent questions! Adding to Thomas, we can view this from three points of view: technical implementation, clinical modeling, and spec. My previous comments about persistent records are spec related. As Thomas said, _how_ things are done using the spec will depend on modelers, and also implementers. From the spec, I see persistent records as a framework to record everything that is conceptually a Singleton (one instance across the whole patient EHR). Then if that can or can't be modeled as an event record, is a discussion for clinical modelers, but I think that doesn't diminish the need of such a concept on the specs. Versioning and branching is an ortogonal concept to event/persistent records and can be used in any case. The _how_ versioning/branching is used has a lot to do with implementers and that is related to my initial question, and the hunch that maybe other devs like me found branching/merging an friction point (related to complexity) for the most frequent & simple implementations of openEHR, but knowing there are less frequent implementations that make extensive use of branching and merging. From the current answers to this thread, I see the need of a simplified or relaxed versioning model, that maybe is just a constraint over the current version control, allowing only certain versioning patterns, like lineal versioning, and some control mechanisms like versioning lock/release, access to read-only records, etc. These are not changes to the current spec, but additions as specs or as guidelines. What do others think? --- ## Post #25 by @system Pablo, one small remark, a persistent composition is not really a singleton. Not conceptually. A patient can have more care - plans, for example, at different care - institutions or multiple care - takers at a single institution, or have multiple medication-lists. Bert --- ## Post #26 by @system I agree. The singleton is not one persistent compo, but the instance associated with an OPT of a persistent compo. When talking about singleton instances we should define what is considered the "class" :) My mistake. In the case of care plans, different care plans will be defined by different OPTs, same for medication lists if needed and modeled that way (some systems might only need one medication list, one problem list, etc. by EHR). But again, these are all clinical modeling decisions. A bad model might represent different care plans with the same OPT, and of course this breaks the singleton concept, but we are talking about subjectiveness here, so there are no hard rules (call it probabilistic singleton). Best, Pablo. --- ## Post #27 by @system Hi! Shared care-plans, medication lists and allergy lists often need to be updated by several care providers that have different EHR-systms separated by both legal and practical/technical barirers; for example regional vs municipal care in Sweden or private vs public care providers. Today in (usually non-openEHR contexts) that is often done by either: 1. using a separate shared system just for shared care plans etc (which means a degree of double data entry and manual transfer/reentry of things from/to different EHR systems that the patient has records in at different care providers) or 2. forcing everybody to use the EHR system of the "biggest" care provider in the (or that systems shared care planning module). Pretty convenient for the big actor but a mess for the other smaller ones, especially those that have a patient mix from several different "big" providers using the "use my system" attitude... So yes the multi-provider shared records with branch/merge capabilities are certainly needed if we want to avoid #1 and #2 above. And yes merge will be a headache for users when it occurs, but likely less of a headache in total than #1 and #2 above. I believe enough of the subgroup of clinicians dealing often with shared care will become proficient enough to handle merging. The "lock" approach (only allowing one party to change at a time) would only work if things were entered and synced fast and at once after the care event had happened. In reality a doctor may read version 12 of the medication list when seeing the patient on Friday and audio-record an update (that should become version 13), that gets transcribed and recorded in the EHR after the weekend. During the weekend the patient needs to go to another care provider (e.g. emergency) that also looks at version 12 (since that is the latest recorded one) and immediately after the consultation records a version 13. See the merge conflict? In this case it might need to be resolved by the care takers contacting each other to agree on possibly conflicting persciptions. What happens today without openEHRs ability to at least detect a merge conflict is not always good for patient safety. --- ## Post #28 by @system Hi Eric, good story, but one remark, I understand from your story that you favor branching above locking\. Because when you do locking while a composition is in edit mode, you don´ t need branching, and the merging becomes much easier\. In fact, there is no merging when there is no branching When you do locking of a composition in edit mode, every one wanting to edit a composition always works on the latest version\. Locking of a data\-set when taking it into edit mode, is not something very exotic to do\. In fact, it is common practice all over the world\. I never heard of an information system supporting branching, and for good reason\. Branching is used in source\-code files, but even there, it is avoided as much as possible\. All programmers can imagine the trouble that comes to them when at the end of a working day, the version control system refuses to store a change in a file because it was edited simultaneously by someone else who posted it before\. You say, care providers should solve the branching/merging by calling the one who edited a composition simultaneously to find an agree about what to do with the merge\. And what if that other care\-provider has gone home, maybe for a long period, after a week of night shifts, what should one do? Keep the composition in edit\-mode? And causing more trouble and more branches and more merges and more agreements to find, because others need to edit the composition too? And what happens with the people wanting to read the composition, what do they read, the latest posted version \(so all the edit\-states are not accessible?\) I think this is a undesired solution\. I think the locking system, in the end, is more safe also for the patient\. Bert --- ## Post #29 by @system The branching model in the specs is actually not designed to deal with competing editors in the same computing environment (that's taken care of by the optmistic locking rule already in openEHR); it's designed to allow for the possibility of an original resource such as Care Plan or a Medication List originating from some provider enterprise, and being copied to another location and modified there, while still being modified in the original location. In a perfect world this would not happen, because such resources would be true singletons, sitting behind a single-point-of-truth service API that all applications talked to. Well, since the NHS realised they wanted that 10 years ago, they are no closer to it today. I've never seen anything like it in the US either; it might exist inside Kaiser somewhat, and maybe Beth Israel (due to Halamka) but otherwise, no. But forking and modification of resources does happen easily. All the branching model ensures is that the mods made by people at different 'systems' end up on branches specific to those systems, so that if the data are ever copied back to an original location, a safe machine merge can in fact happen, because the branches won't clash - this ensures no data are lost. A true content merge is still potentially required, which will most likely be manual. - thomas --- ## Post #30 by @system So, what happens is that \(for example\) one healthcare\-enterprise has created a care\-plan, which is to use by more care\-providers on different computer\-systems/network/database \(another healthcare\-enterprise\)\. When on the same system/network/database there are things like locking to solve this\. The original enterprise sends a message containing the care\-plan, and the receiving starts integrating it in their own care\-plan\. This happens a lot\. A person was in a hospital, they wrote a care\-plan for when at home again, to do some things\. \(take medication, for example\)\. The care\-provider at home \(the GP, or nurse\) already has a care\-plan, and adds the information of the hospital\. Everything cool\. Sometime later, the patient has to go back to the hospital, here was an unforeseen complication\. The GP/nurse from the home situation, sends their care\-plan to the hospital \(in a message\), so the specialist can see what has been done at home\. Everything cool again, no branching needed\. The message is stored, and some information of the message may be added to the care\-plan of the hospital where the patient has to stay some time\. That is how the things go, in my country\. But even there, to handle this simple information\-transfer is hard enough\. Even too hard\. Hospitals, for example, still do not synchronize medication\-information with the national health network\. I leave the discussion now\. I think, branching and merging might be good in a perfect world, but in common practice, even sharing information in common messages, and coded with a good terminology is not yet reality\. In Germany, for example, they do not use SNOMED\-CT\. Best regards and thanks for your opinions\. Bert --- **Canonical:** https://discourse.openehr.org/t/versioning-implementations/15484 **Original content:** https://discourse.openehr.org/t/versioning-implementations/15484