Socio-technical challenges when the openEHR approach is put to use in Norwegian hospitals

Clinical (archive)
1

Hi everyone!

As some of you may have noticed, a paper called “Evaluating Model-Driven Development for large-scale EHRs through the openEHR approach” (http://www.sciencedirect.com/science/article/pii/S1386505616300247) was recently published by a PhD student at the University of Tromsø. The paper has some pretty direct criticism of the ideal of wide clinical engagement in widely reusable information models, as well as the clear division between the clinical and the technical domain inherent in the openEHR model. I think a lot of the observations detailed in the paper are probably correct, for its limited scope (one Norwegian region and 4 years of observation, half of which was done before the national governance was established). We’ll probably use the paper as a learning point to improve our national governance model, and I’d like to hear any international (and domestic Norwegian for that matter) takes on the implications of the paper.

Kind regards,
Silje Ljosland Bakke

Information Architect, RN

Coordinator, National Editorial Board for Archetypes
National ICT Norway

Co-lead, Clinical Models Program
openEHR Foundation

Tel. +47 40203298

Web: http://arketyper.no / Twitter: @arketyper_no

2

Hi Silje,

Interesting read, thx for pointing to it. I think it's the same problem we have in the data warehouse context: unless there is commitment by the management (chief physicians) to give physicians the space and time during their working hours to participate, things will not happen that fast. Without a proper project structure and resources, you have to rely on the goodwill of the participants. And as we all know, physicians are busy people.

just my 2 cents,

Birger

3

Hi Silje – many thanks for pointing out this. I think it IS a brilliant piece of very good and rigorous research with all its limitations – kudos to the authors.

What this study tells me is MDD methodology itself is not flawed but we as the openEHR community should put much more concerted effort into modelling. It is like having a supercar but not having any fuel to drive it! I have been articulating the need for expanding the editorial capacity and levering much smarter from the interested and capable members of our neat community. As a member of the Management Board I’d like to call everyone to action for making this happen – fast! Especially the Clinical Modelling program leads. We don’t really have any excuse for not doing this.

4

Hi,

> As some of you may have noticed, a paper called “Evaluating Model-
> Driven Development for large-scale EHRs through the openEHR approach”
> (http://www.sciencedirect.com/science/article/pii/S1386505616300247)
> was recently published by a PhD student at the University of Tromsø.
> The paper has some pretty direct criticism of the ideal of wide
> clinical engagement in widely reusable information models, as well as
> the clear division between the clinical and the technical domain
> inherent in the openEHR model.

I've only read the abstract for now and I think it really matches our own observations. Everyone needs to work together to build a system.

I wonder where the idea the paper mentioned comes from that OpenEHR allows a "separation of clinical and technical concerns" and end users should be in control?

End-users (nurses, patients) should not be in control, physicians might have some control but most of this control is with the real users of OpenEHR: the developers.

And nobody ever has build something useful without working closely together.

Does OpenEHR somehow imply nobody has to work together and end-users should be in control?
As far as I'm concerned, OpenEHR is a way to store and exchange data and makes it easier for clinicians to be involved.

In the remainder of this e-mail are more details about our observations and our attempt to handle them. It turned out to be longer than I expected. Sorry about that:

IMHO OpenEHR (or the model) is not something to be seen as an isolated part of a system or the whole system. I feel OpenEHR is just a small part of the system, not the start of an application but more the end of it: a way of storing data just like SQL is.

When developing applications all participants need to cooperate with each other. This includes the technical frontend developers for the UI, the backend developers, the UX experts but also the domain experts and clinicians. If not there is a likely chance the model looks fine in theory but is difficult to implement or vice versa.

For example we tried quite a few different approaches with MedRecord. The first versions all exposed openEHR to the (frontend) developers and basically forced the model and openEHR onto the developers.
This made it difficult to use and difficult to find developers who could quickly start using MedRecord without learning openEHR.
It also caused every application trying to reinvent the wheel for mapping real usecases onto the available model.

Perhaps we could have a clinician spend more time in developing the model (archetypes) and imagining every foreseeable usecase beforehand. This may work for a waterfall model but it does not fit an agile environment very well. Also, when developing a model one should be aware of the technical restrictions of the system implementing the model.

IMHO another part missing in OpenEHR (or at least in most archetypes) is that they lack information on how they should be used. I'm sure a clinician knows exactly how they should be used but the archetype does not contain enough information for a frontend developer to properly use it the way the clinician intended.
I think this is another reason why clinicians and developers need to work more closely together than suggested.

That is why in the latest version of MedRecord we use the requirements from the frontend applications (or rather the usecases) to drive the development of MedRecord.
MedRecord provides APIs for usecases requested by the UI instead of forcing the openEHR way onto the frontend.
While doing that, MedRecord hides all OpenEHR specifics.
OpenEHR is still used to store the data and it is still driven by archetypes, it just isn't visible from the outside.

This way most developers can start using MedRecord without them even knowing OpenEHR or all the clinical details. MedRecord developers will take care of these by mapping the usecase onto available archetypes and work with clinicians to develop new archetypes and document how the API should be used clinically correct.

I like to believe this makes it easier for everybody to work together without frontend developers needing to learn about openEHR and clinicians need learn about all technical details.
But they are still working together.

Another example might be the blood pressure archetype available in the CKM and the blood pressure measurement of Open mHealth.
I think most developers would say they are the same and you can store Open mHealth blood pressure measurements using the archetype available in the CKM.
However, if you look closely Open mHealth and the CKM archetype use different snomed concepts:

Open mHealth uses snomed 271649006 "Systolic blood pressure" while the CKM archetype uses snomed 163030003 "O/E - Systolic BP reading"

Are they the same? I don't know but most developers would assume they are. Perhaps this is a silly example, but it shows clinicians and developers need to work closely together to avoid these details to be overlooked.

Also, most end users aren't interested in modifying the data model or don't have the time to that themselves. They just want something which works and let experienced developers take care of writing the application.

Perhaps that has to do with the kind of end-users we are focusing on. I can imagine in a research setting it would be possible the user (the researcher) might want to add an extra data field to an archetype.

But for our users, for example an user monitoring his blood pressure using his mobile phone, adding an extra field to an archetype is much more complex and probably not even desired.

Cheers,

Ralph van Etten

5

Hi Koray,

> What this study tells me is MDD methodology itself is not flawed but
> we as the openEHR community should put much more concerted effort
> into modelling. It is like having a supercar but not having any fuel
> to drive it!

Sorry but I think you car analogy is flawed with respect to MDD. MDD is not a supercar. It think it is more like this seeing from a MDD perspecitive:

- The end-users (GPs, hospitals) want a supercar (actually they want a bicycle but thats another story :slight_smile: ).
- The vendor (like MedVision) has build a car factory.
- The dealer (clinicians) hired by the end-users order a car from the factory with the options the end-users want.
- OpenEHR is just a part (like an engine) the factory uses to build the car.

MDD by itself is not flawed but at least for us currently isn't a good fit. There are some good reasons (both pro and con) on this website: http://www.theenterprisearchitect.eu/blog/2009/06/25/8-reasons-why-model-driven-development-is-dangerous

Cheers,

Ralph.

6

I would like to congratulate the authors for the amount of effort they’ve put into this paper. It is rarely the case that someone looks at such an important aspect of implementing openEHR with such a level of commitment.

That being said, I think this study lacks a major aspect and it is probably going to become the “archetypes don’t work paper” for the sceptics quite soon even though the authors do not say that.

The missing aspect of this paper is the overall context of model driven development in healthcare and how much openEHR improves on previous and current attempts to apply MDD to healthcare.

If you did not know who Bill Gates is, and if you’d have seen him crying his eyes out on the street saying “I’ve just lost 99% of everything I have”, you’d surely feel sorry for a 60 year old who must be looking at a horrible fate ahead of him. Given the full context, in which his net worth is ~80 billion dollars, you’d hardly be likely to reach to your pockets for some change for a man who is left with only 800 million dollars of wealth.

Without context, local minima becomes global minima, any failure becomes total failure. This study takes openEHR as the representative of MDD in healthcare and provides its observations, but it does not mention how, if any improvements are provided by openEHR over the alternatives. Sure, the authors can’t perform the same study and observe outcomes for all the other alternatives and they’re doing a study on openEHR, but I would expect at least a research question being raised as to whether other methods are subject to same issues or how they compare to archetype driven methodology. (very happy to be corrected if I missed this bit)

I’ve been developing software for a living for 19 years now and I have seen my share of MDD applied to many problems. If you put the alternative, rather generic MDD approaches say, driven by UML, to use in this scenario, what would the results be? I’m going to dare suggest that the clinicians would have a much harder time figuring out how to express their requirements with UML notation.

The paper also seems to assert that a claim of openEHR modelling being usable by every clinician is wrong or partially incorrect but I don’t think openEHR makes such a claim. You could replace all instances of openEHR in this paper and many sentences would read the same for SNOMED CT or HL7 CDA : clinicians and nurses would find developing snomed ct local extensions or ref sets hard, which is expressing clinical concepts using a domain specific language .

I also find the repeated use of the same comment from Dips developer 2 as proof of blurred boundaries between clinical and technical aspects insufficient to make such a claim (sorry dear Dips developer 2 I hope I’m not causing you any trouble here) since the comment is brief and lacking depth.

Despite its findings, the paper actually (ironically) proves that openEHR indeed puts the clinicians in the driver seat, but they just don’t know how to drive, because the technical implementation is good to go as soon as you come up with the models. Very few, if any other alternatives to openEHR can make that claim.

A claim of separation of technical and domain concepts does not imply zero effort for expressing domain concepts . But the paper presents the learning curve and difficulty as proof of MDD in healthcare failing to deliver its promise. openEHR offers a lot of advantages in exchange for investing into clinicians. The idea that clinicians can and should learn how to express their requirements via methods and tools defined for them is the basis of all the great things one can do with computable health. This unusual investment compared to other approaches offers so much value down the line but it is usually trivialized or downright neglected which leads to issues outlined in this paper. At this point, one can see why Koray is pointing at the importance of having more clinical modellers around.

Finally, I don’t understand the pains associated to reaching out to international community as described by the paper, simply because you can get help but you’re not forced to make use of it.

I think this paper just shows that how important it is that governments and healthcare institutions should invest into educating clinical modellers. If the role and its key position for computable health is not recognized and supported as a speciality within the clinical practice, all benefits of openEHR and similar approaches will remain extremely underutilized. (The paper makes the statement that archetype development should be included as part of the effort)

So I would like to conclude by saying that this paper asks the difficult question : “does this really work?” and it provides vital observations for which I’d like to congratulate the authors again. Just providing a bit of a context would make it a lot less amenable to misinterpration, but hopefully we’ll see more research in the future, giving us all a better view of where we are and where we are headed.

Best regards

Seref

7

Hi Seref,

I agree with your comments but I have one question about the following:

Despite its findings, the paper actually (ironically) proves that
openEHR indeed puts the clinicians in the driver seat, but they just
don’t know how to drive, because the technical implementation is good to
go as soon as you come up with the models. Very few, if any other
alternatives to openEHR can make that claim.

What do you feel belongs to the technical implementation?

Because, for the usecases we encounter, having completed the model by writing an archetype does not mean the technical implementation is finished and can be used in an useful manner.

All you have is a way to store data. Finishing a model is just a very small piece in an much bigger puzzle.

Cheers,

Ralph van Etten

8

Hi Ralph,
The capabilities you have at your disposal at the point in which you have the model ready is specific to implementation. Some implementations provide/require less, some provide/require more. But you have template data objects, GDL, storage, XML serialisation, validation (not in this strict order) and more that you can access as features of the platform.

Some implementations may provide partial/full UI generation, some may do more, but the models can be used as input into many downstream scenarios. The focus of openEHR on generic computation on health data lets you turn frequently implemented functionality to services. This is a much better position to be in compared to having an HL7 V2 or V3 content or any other data format which requires a further transform to some internal format which you’ll then use for clinical information system implementation. FHIR also comes close to achieving the same advantages as far as I can see. My point is about the existing and future capability that is possible due to RM and two level modelling.

All the best
Seref

9

My two cents, three roles:
- users-team ( the customers, medical doctors, nurses, the team of people formulating requirements, the payers)
- developers: doing the technical work, building a kernel, doing queries, designing gui's
- domain experts, people with knowledge of protocols, terminologies, standards, know how to use an archetype-editor

There are some shades of gray between the disciplines. :wink:

The problem is that many people try to be smart, but in application-development, there is only one smart person-group, that are the persons representing the users.

10

Hi Bert,

----------------------------------------
OpenEhr in my simplistic view would the be: "user central"

The users communicate functional requirements to the domain experts
(which can create archetypes, know about terminology), etc, and they
communicate GUI-requirements to the developers, so the application gets
hands and feet. In this way, the users-team is the center of knowledge
and end-responsible for the product, and that is alright, because the
users pay for the product..

The characteristic of such a system is that it is just right. The user
keeps sending back results to both sides if they are not satisfying. For
example: "I need to do that, do it, or have a damned good reason to not
do it"
----------------------------------------
Technicians and domain experts should be modest, the user is the King.

Yes, very true. But I would go one step further, technicians and domain experts should work together as a team and not isolated.

Anyway, do you think OpenEHR currently is user central?

It seems to me openEHR currently is very model driven (which is fine if used to communicate requirements) with the idea that after you have written an archetype you're done.

I get the feeling people (or at least the paper) think openEHR is for end-users (GPs, nurses, patients) instead of seeing openEHR as a method for communicating requirements between technicians and domain experts.

Most (end)users have nothing to with OpenEHR and they should not care about it other than that they know they can take their data and it works on any OpenEHR implementation.

Cheers

Ralph van Etten

11

I can only see the abstract for now, but I think the authors seem to have developed the misconception that end-users would somehow be designing applications. openEHR doesn’t try to do that, and it’s the first time I’ve heard anyone suggest it. openEHR just enables domain experts (generally = a small proportion of healthcare professionals, who might also be some kind of system user in some part of the world) to more directly define the information content of the system, in such a way that it can be processed and queried on a semantic level.

The Business Purpose of Archetypes section in the Archetype Technology Overview may help to show why this is useful and necessary (it’s short!).

There are still many other problems to solve such as clinical workflows and user interaction / UX.

I am currently at Intermountain Health in Salt Lake City working with the Activity Based Design (ABD) group that has developed a new architecture that I think has a realistic chance of addressing a) workflow (e.g. typical nursing tasks like cannulation; more complex cooperative workflows that involve shared care) and b) some aspects of UI interaction within workflows. They are just at an early prototype stage, and it has taken nearly 2 years to get to the current architecture (naturally taking into account many previous attempts and experience).

This effort is the first I have seen that has what I think may be the needed theoretical understanding and technical architecture to starting to solve clinical process and (some of) UI/UX. And what does it rely on? Formal clinical models, and it assumes that those models are created by clinical experts. Not only that, it explicitly assumes a ‘template’ concept of the same kind as openEHR’s, in order to construct useful data sets.

It considers these ‘templates’ as the basis of an ‘Activity’ description, which then adds new abilities to blend in some presentation directives, pre- and post-conditions, some workflow elements, cost-related items (e.g. ICD coding) and so on. The innovation here is to consider an Activity a unit of clinical work and to attach these process-related semantics into that level of artefact.

So let’s just reflect on the fact that this research is only now emerging from one of the leading institutions in the world that has historically specialised in workflow and decision support.

openEHR as it is today is just a semantic content + querying platform, and I think we can reasonably say that we have some handle on generating developer-usable artefacts, i.e. things like TDS, TDO etc, but they are all content related. These are starting to be standardised now.

The openEHR of today needs to leverage new work such as ABD (or something like it) to achieve many of the things that the Norwegian paper talks about. The paper seems to be critiquing a somewhat unrealistic set of expectations re: openEHR, although I am sure it has useful lessons.

I’ll provide a proper description of ABD ASAP, which I think will provide our community (particularly those working on clinical workflow, process etc) new ideas on ‘the next layer’ for openEHR.

  • thomas
12

How I see it, which is a very personal view, and very developer central, trying to think politically correct direction user.
We can not live without users, not even in fantasy.
So, we should always try to find users, an we should not start building an application before we found them.

We can create tooling, that is all.

I must also say that I did not read the paper, it is way too expensive to read it.
30 box, it must compete with a good wine, and then I am afraid, the paper will not win.
There is nothing that rings alarm-bells saying I must read it. It are opinions.
I have seen them come and go and I have seen them die and long ago I stopped asking why.

Like Lucky Luke, I sing my "I am a poor and lonesome cowboy"-song.
Independent developer, it is called. It has advantages, play chess with my horse.
http://www.bendav.nl/gif/ebay10/107.jpg

So, enough fun, now a more serious answer.

Model Driven Development, on what level? Domain?
You also have Domain Model Driven Modeling, first you have a RIM, then a DMIM and then a RMIM,
Or first a Reference Model, then CKM, then a template-layer build on Contsys or something else.

And from that we need to go to a nurse wanting to record the blood-pressure..

It has advantages for development, you know what to expect on specific situations, but the problem is that the domain experts go different directions then developers, and the models they create are in some detail terrible from developers point of view. I think this is mutually felt.

But don't throw away the baby with the bathwater, there is always something good in it. That is the point, it is not a black and white question, trying to answer it in that way brings you to the same trouble as the domain model driven modelers bring you.

Do you know, and this does not sound very scientific, but what we see a lot is: models, everywhere.
Trying to capture life. Models for cities, models for the stock market, models for marriage, models for software domains, models for housing.
Compare it with city-planning, as soon as there is an overall plan, it is very often doomed to fail. The strongest models are those which grow naturally, which evolve and evaluate constantly. Which give room for a bicycle-repair shop where no architect would ever plan it.

We can draw some strong lines for the highways to avoid traffic-jam, that is it.
We must learn to respect the micro-level, try not to solve other people's problems, don't be conceited, give problems time to solve themselves.
First step in good software design is let go of the ego

Listen to the nurse, on daily routine, does she/he take temperature before, or after she/he knows the patients-name?
That is quite important to know, how many applications just assume something?

There are models, Always look at them, and copy concepts, always learn from others. Contsys, HISA, smart thinking, but not holy grails.

Keep your tooling/kernel flexible, keep everything flexible, have everything arranged so that flexibility remains possible on every level.
Keep your database flexible, keep your queries flexible, keep your validation flexible

And for that purpose, 2 Level Modeling is very good, an choose a good Reference Model, and why not OpenEHR if you are doing medical applications?

I don't know if this was a useful answer, but still, have a nice day

Bert

13

Hi Bert, in some cases, markets are created after the service or product was developed. In those cases, the only thing we can do is to estimate the number of potential users, because before the product, we have no users. That is the case, for example, of the cellphones and how the market exploded after the solution was developed and improved. Of course, this strategy requires a lot of money. Markets do not develop themselves. Also this doesn’t apply to everything :slight_smile:

14

The role user is a bit blurred, it is not necessary the one who literally is going to use the system, I guess nurses do not have time and education to think about software-ergonomics. The users are, from my opinion, a group of special educated people representing the payers and have a lot of research and communications with the real future users, and of course, the real users will be asked too, but in a later stage.

I understand that this is hard on developing markets, but even then, you must be able to envision the users, and have it used in real situations.
There are so many possible workflows, diseases, priorities, detailed information requirements, authorizations, authentications, depending on cultures, welfare and even institutions, even departments of institutions. I think a Jewish neighborhood has other requirements then the Islamic neighborhood a few streets further or the Hindustan society in the same town. Could also be surprisingly similar in some aspects.

My wife is a GP in a multicultural health-care institution, and they started to use a new GP Information System, one of the best in the Netherlands. Dutch people now say the name. They know which one it is.

She and colleagues were getting a course on how to use it, and during that course, on everything users did not feel good, or did not fit in the workflow of the organization is noted and will be discussed, changes, configurations or new flexibilities, as far as possible in the application adapted special for that installation.

But it is not as flexible as a 2 Level Modeling based system which can change an application in its fundamentals, but also in all layers above that, and also authorization-logic and in the GUI, and maybe one institution wants also Internet-access?

But if you are fixed to a model top-down, f.e., HISA, and you are not willing to negotiate on that, then your system is about the same as an classical EHR system. Then you have a system designed by some wise people, and they do not know the people working in your healthcare institution, and you throw away that what makes 2 Level Modeling an advantage, although, this is my opinion.

Bert

15

Just a thought on the reading of the article

Good article, until I found this sentence: “domain models are now separate from the software (but not the product), and they can be built by non-IT personnel, assuming a tool with a reasonable user interface.”

Making user-interfaces is a profession, not something you learn on a rainy sunday afternoon.

When I read that users should be able to build systems when having the tooling, it gives me doubts about its efficiency.
When I write that users are King, I do not mean that Users must do the work.
Kings don’t work, maybe they have a hobby, but mainly, they drink the wine, they don produce it.

I have spent many hours with non-technical medical doctors who learned programming BASIC and assembler as a hobby in the eighties, they should not interfere, they should not try to understand what Codd has to do with database-systems. We, technicians don’t try to understand diseases, we accept a fairy tale about little monsters inside our body. No problem. We understand that we don’t need to understand. We want to be cured, we trust medical doctors that they do what good is for us.

There are so many ways for non-technical end users to explain what they want to technical staff. It is the result that counts, isn’t it?
And the price is very high, and of course the tax payer is happy to pay. I think that is one of the problems, in normal commercial markets, companies are much more efficient.

We have the situation in the Netherlands that we spent 500 million Euro for a information-exchange-environment for medical data. Not only 500 million, but also we waited 15 years.

Now we have it.
But it does not have, even the most basic, authorization, everyone who has legal access to the system can look at every patients-record.
Not only that, it has no logging accessible for patients, a patient cannot see who looked at his medical records.
The use of the system is against every privacy law in the Netherlands, so there was a court needed to use it, and the court gave its blessing because patients are voluntary in the system (opt-in)

If you would have designed the system from requirements, and gave it to a technical company, together with domain-experts to define a message standard, I think the system would have been ready ten years earlier, for 20% of the costs, and no court needed to approve its use.

I think in medical ICT, the best role of the user in ICT is widely misunderstood. We, technical experts and domain experts should inform users that democracy does not mean that people need to understand how to run a country or build an ICT system.

16

Hi again,

I see from the list lots of people didn't get to read the actual article. I for some reason got a full text download for free, but I can see several of you didn't. Perhaps some sort of time limited offer after publication? I don't think it's easy to judge the article from the abstract, so I hope you can find a copy somewhere else. Perhaps a university library if you have access to one?

Regards,
Silje

17

I asked to author to send the accepted version of the manuscript to the mailing list. She is not on the list, but I got permission to so so. Please see attached.

Jan Talmon

(attachments)

Evaluating Model-Driven Development for large-scale EHRs.pdf (841 KB)

18

Many thanks Jan (and of course to the author),

Clearly this paper has aroused considerable interest/debate. It is
good that the whole community can see it and and discuss in details.

Ian

Dr Ian McNicoll
mobile +44 (0)775 209 7859
office +44 (0)1536 414994
skype: ianmcnicoll
email: ian@freshehr.com
twitter: @ianmcnicoll

Co-Chair, openEHR Foundation ian.mcnicoll@openehr.org
Director, freshEHR Clinical Informatics Ltd.
Director, HANDIHealth CIC
Hon. Senior Research Associate, CHIME, UCL

19

I think my reaction of last weekend must have been confusing.

I mixed up two subjects
- The fantastic ideas about how information and communication and software development should work, and good discussions about that, but also theoretical,
- The reality, which seems we are still stuck in the year 2000, in the Netherlands, but also on many other places, using HL7v2, Edifact, email as message transportation, no fine grained authorizations, and spending lots of money without making much progress. Fore example, now there is a official advise not to use WhatsApp for communicating medical information.

I should not have mixed up these two subjects, although they always mix up at my daily routines.

I am sorry for the confusion I might have caused.

Best regards
Bert Verhees

20

That's great call - many thanks

Cheers,

-koray