CAMSS assessment of openEHR

Technical (archive)
1

Hi guys,

As part of a standards assessment project initiated by the Norwegian Directorate of Health, several health IT standards will be assessed using CAMSS (https://joinup.ec.europa.eu/node/66790). Have anyone done a CAMSS assessment of the openEHR specifications? If so, would it be possible to share the results?

Kind regards,
Silje Ljosland Bakke

Information Architect, RN

Coordinator, National Editorial Board for Archetypes
National ICT Norway

Tel. +47 40203298

Web: http://arketyper.no / Twitter: @arketyper_no

2

Hi Silje,

I had a look at the website - it doesn’t seem as if anyone is using CAMSS much, if we believe this page, which contains CAMSS assessments, none of which are health related. On this page there is a much larger list of ‘recommended’ standards which are all generic (PDF, GIF and so on) - I don’t know what use this is.

This page has a long page of assessment criteria, which seem mostly reasonable to me, but are will probably miss what I would consider the most important criterion for value - longevity. I personally think longevity is ultimately based on semantic scalability (lack of other things will also kill it, like lack of community, implementations etc). See here for my set of criteria.

I also don’t think that compatibility with other standards is necessarily a useful criterion - it depends on whether those other standards are in use and are themselves delivering value or just getting in the way.

But overall the assessment tool seems OK. What I can’t see is any example of where it has been applied to an e-health standard.

  • thomas
3

Hi Thomas, thanks for your reply!

I don’t have any say in which assessment criteria are used, unfortunately. I’ve seen your criteria, and agree they’re more specifically applicable to e-health standards.

I’ll probably need a lot of help answering and justifying the CAMSS questions, especially those regarding the governance of the specifications. I hope it’s okay if I pose specific questions to this list.

Regards,
Silje

4

I have set up a CAMSS wiki page, Silje if you can copy the relevant CAMSS boilerplate in (assessment table etc) and make it look vaguely presentable (make child pages if you need) then I am sure the community can get it populated quickly for you, since it’s clearly of general benefit.

  • thomas
5

I’ve added the CAMSS assessment table to the page. Would appreciate any help in populating the pink cells. J

Regards,
Silje

6

My first go now at CAMSS wiki page,

There appears to be a major snafu with the CAMMS spreadsheet that auto-sets ‘not applicable’ to most of the rows if ‘open specification’ is set as the standard type. This makes some sense for a proprietary specification but none whatsoever for an open specification. @Silje - you may want to seek clarification from the CAMMS authors.

Ian

7

Thanks Ian!

It’s good to see there are so many “yes”-es! However, we’ll need some more info, for instance links to where the “relevant documentation of the development and approval process of the specification is archived and identified”.

We’ll handle the spreadsheet bit. J

Regards,
Silje

8

I have added some links. many of the questions seem inappropriate, not applicable, or virtually impossible to answer within the health domain.

Ian

9

Thanks again Ian! J

Could anyone from Slovenia provide links to documents mandating or recommending openEHR, and procurement documents referring to it? It doesn’t matter if they’re in Slovenian, the main point is that they can be proven to exist.

Regards,
Silje

10

Hi,

The project has now done a preliminary CAMSS assessment of openEHR. It’s identified some issues that I would like some input on:

  1. A.16: “Are the technical specification or standards reviewed using a formal review process with all relevant external stakeholders (e.g. public consultation)?”

· The review process is not described. There is a documented a"change process" (http://www.openehr.org/programs/specification/changeprocess ), but it seems to be for change. Here, however, neither “review” isn’t described any more than that input from members of the “community” is sought for major changes.

  1. A.18: “Is relevant documentation of the development and approval process of the specification archived and identified?”

· Issue and problem tracker available but it’s hard to find/access. Approval process archive not found.

  1. A.23: “Is relevant documentation of the development and approval process of technical specification or standards publicly available (e.g. preliminary results, committee meeting notes)?”

· We’ve been unable to find any minutes from meetings or preliminary results anywhere.

  1. A.26: “Does the maintenance organisation for the technical specification or standard have sufficient finances and resources to be sure of freedom from short- to medium-term threats?”

· Unable to find any info about this.

  1. A.27: “Does the technical specification or standard have a defined policy for version management?”

· The change process has a description about version numbering, but we can’t find anything about handling different versions, compatibility, etc.

  1. A.45: “Are there existing or planned mechanisms to assess conformity of the implementations of the technical specification or standard (e.g. conformity tests, certifications)?”

· Can’t find anything about this on the web pages.

  1. A.48: “Does the technical specification or standard address backward compatibility with previous versions?”

· Can’t find any info about this on the web pages.

Anyone? J

Regards,
Silje

11

there are a number of pages describing governance; see e.g. . Goals described . The left hand menu shows the others. The general process is: See the , i.e. SPECxx. There is a link on the home page (top right corner) going straight to these; also from the specifications governance pages. SEC face to face meeting documentation is published . Community f2f meetings are reported as well, e.g. . Admittedly, these resources should be more clearly linked - we’ll fix that. The Management Board publishes a monthly update on the . I think this may also go via email directly to subscribed members, but I would have to check on that one. What does the question mean? Eveything is versioned in openEHR; the describes the rules of versions assignment to releases. Other than that, we would need to know the specific question to be able to answer better. this is an area of active development in openEHR and now has its own Component. Currently the only proper conformance testing is done by validation of XML data against the published openEHR XML schemas. see the above link to release strategy. - thomas

12

Adding some thoughts below.

4. A.26: “Does the maintenance organisation for the technical
specification or standard have sufficient finances and resources to be sure
of freedom from short- to medium-term threats?”

· Unable to find any info about this.

What does the question mean?

Most SEC members are financed by their "home" organisations where thay are
employed, for example EHR systems providers (vendors/projects) and
healthcare organizations. It is in the interest of those organizations to
stay up to date and also to keep the openEHR specifications updated. Most
work is done online and in teleconferences (cheap). Server resources etc
are financed by the openEHR foundation that has recurring membership
incomes.

So yes, there is enough resources to keep things going at current pace.

5. A.27: “Does the technical specification or standard have a
defined policy for version management?”

· The change process has a description about version numbering, but
we can’t find anything about handling different versions, compatibility,
etc.

Eveything is versioned in openEHR; the release strategy
<http://www.openehr.org/programs/specification/releasestrategy&gt;describes
the rules of versions assignment to releases. Other than that, we would
need to know the specific question to be able to answer better.

Additionally, most openEHR artifacts follow http://semver.org/

7. A.48: “Does the technical specification or standard address
backward compatibility with previous versions?”

· Can’t find any info about this on the web pages.

see the above link to release strategy.

Backwards compatibility for clinical content (already stored EHR data) is
one of openEHRs real strengths. Breaking changes to the RM (technical
reference model) level are uncommon (and migration strategies are usually
provided for any such changes). The more frequent hanges at the AM level
(archetypes and templates) do not break the readability of content based on
previous archetypes etc (since the RM and everything built on the RM stays
the same).

Also the maximal-dataset approach and broad clinical review used for
international archetyping also produces a lower number of surprising
changes in implemented systems due to reduction of the, in other approaches
more commonly occuring, finding of: "Ouch, I didn't consider that less
common usecase when designing my initial datamodel".

Best regards,
Erik Sundvall
Ph.D. Medical Informatics. Information Architect. Tel: +46-72-524 54 55 (or
010-1036252 in Sweden)
Region Östergötland: erik.sundvall@regionostergotland.se (previously lio.se
) http://www.regionostergotland.se/cmit/
Linköping University: erik.sundvall@liu.se, http://www.imt.liu.se/~erisu/

13

I forgot, w.r.t. to versioning of archetypes, this is the specification that applies.

14

Hi Silje,

Thomas and Erik have answered substantial aspects related to specification review.

2. A.18: “Is relevant documentation of the development and approval process of the specification archived and identified?”

The Clinical content review process and governance structures is documented at:

https://openehr.atlassian.net/wiki/display/healthmod/Archetype+authoring%2C+review+and+publication
https://openehr.atlassian.net/wiki/display/healthmod/Content+publication%2C+terminology+binding+and+language+translations
https://openehr.atlassian.net/wiki/display/healthmod/CKM+Governance+Environment

  1. A.23: “Is relevant documentation of the development and approval process of technical specification or standards publicly available (e.g. preliminary results, committee meeting notes)?”

All Clinical content development and approval process is accessed via the international CKM tool at openehr.org/ckm. All reviewer and editorial comments, publication decisions and historical versions are accessible to registered users (free to register).

The Management Board publishes a monthly update on the foundation news list on the web. This is emailed to openEHR Members.

  1. A.26: “Does the maintenance organisation for the technical specification or standard have sufficient finances and resources to be sure of freedom from short- to medium-term threats?”

The Foundation is funded by membership fees from Ordinary members and Industry partners, as well as some sponsorship contributions, and in-kind support from Industry. This seed funding is augmented by considerable voluntary effort by Industry representatives, and clinical reviewers / editors. Some clinical content development is sponsored directly by national health services or industry partners.

  1. A.27: “Does the technical specification or standard have a defined policy for version management?”

The clinical content development process is fully version-controlled using semver.org principles and documented s indicated by Thomas.

  1. A.45: “Are there existing or planned mechanisms to assess conformity of the implementations of the technical specification or standard (e.g. conformity tests, certifications)?”

As Thomas indicated, this is under development for technical artefacts used and produced by openEHR implementers.

Clinical content artefacts are subjected to a rigorous technical test as part of the upload/ maintenance process in the CKM tool.

“this is an area of active development in openEHR and now has its own Component. Currently the only proper conformance testing is done by validation of XML data against the published openEHR XML schemas.”

  1. A.48: “Does the technical specification or standard address backward compatibility with previous versions?”

Clinical content Version compatibility rules and testing are handled specifically in the specifications and are instantiated in the CKM tool when those artefacts are updated, by issuing ‘diff’ and compatibility reports.

Ian
Dr Ian McNicoll
mobile +44 (0)775 209 7859
office +44 (0)1536 414994
skype: ianmcnicoll
email: ian@freshehr.com
twitter: @ianmcnicoll

Co-Chair, openEHR Foundation ian.mcnicoll@openehr.org
Director, freshEHR Clinical Informatics Ltd.
Director, HANDIHealth CIC
Hon. Senior Research Associate, CHIME, UCL

15

Just to add. I sense that there is a real difficulty for those involved in the reviews in understanding anything other than traditional ISO/CEN type standards/specification processes.

Do we have an opportunity to show them how an archetype review process works, or to show how the specifications review process works?

Ian

16

I think they’re only interested in the specs, not archetypes. And I think the point is that it should be possible to learn how the processes work without being shown. J

Regards,
Silje

17

Hi Silje,

As far as I can see the specification process is fully documented at http://openehr.org/programs/specification/ but it is a technical specification and therefore highly detailed.

but in terms of ’ standards’ development, which I thought was the initial remit, surely the clinical modelling layer is just as significant (possibly more so)?

Ian

18

Hi!

Sadly I don’t think it is very often you see goverment initiatives etc actually looking for standardisation at the “clinical” layer in the sense of documentation models that openEHR archetyping covers.

They often look for a technical standard (capable of defining messages and code-value-paitinig etc) plus some set of terminology systems. They often believe that such a combination will solve the interoperability problems efficiently.

I do not think they usually consider the quality of the clinical content or the processes, ecosystems, communities and update mechanisms used to produce those message defintitions or documentation patterns. Neither do they consider how well message definitions match what clinicians actually want to enter in EHR systems and later query for. (They probably think that is the job of each EHR vendor and that mapping-wizards will do the magic of converting from EHR entries to messages and back, as usual.)

Sometimes this applies: https://coiera.com/2015/05/12/a-modest-e-health-proposal-to-government/ I don’t know if that is valid for the Norwegian situation now or not.

Please share good examples of national initiatives actually assesing the clinical qualities of different approaches. That could be userful and interesting to look at.

Best regards,
Erik Sundvall

19

Ditto – the main problem is that while we appreciate the important and complementary roles of clinical terminology, information models and health information exchange specifications (which should normally be created as downstream artefacts from the broader whole of ‘EHR’ content models for the particular use case that they address) the governments and regional authorities don’t seem to get this. Perhaps some get it but find difficult to justify because this is somewhat seen as a rather academic view. I guess it is our responsibility to continue to work on to educate and more importantly create some hard evidence so that they feel more confident in making such decisions. We should also accept the fact that, historically, we have not done a very good job of creating such evidence and be very open and completely transparent to drive that confidence – however I want to believe that things have changed for the better in recent years.