# openehr system validation **Category:** [Technical (archive)](https://discourse.openehr.org/c/technical-archive/156) **Created:** 2007-11-08 16:59 UTC **Views:** 1 **Replies:** 16 **URL:** https://discourse.openehr.org/t/openehr-system-validation/14684 --- ## Post #1 by @Greg_Caulton So regardless of data persistence \(which is an implementation detail and a software engineering choice independent of OpenEhr\) how can we test the compliance of an OpenEHR system? CCHIT developed inter\-operability tests \(I believe\)\. Can we do the same? Can we have say a template that reflects the context of data and have OpenEHR systems demonstrate their display or data entry? Or perhaps a set of data and a set of AQL queries that should all return the same results? Some kind of non\-subjective certification in the long term would be nice\. For now just a 'show you can do this and you are on the right track' would helpful\. Just a thought\. Greg http://www.patientos.org --- ## Post #2 by @Allen_O_Neill Hi All, a bit off\-topic but hopefully its acceptable\! I am researching HCI \(Human Computer Interaction\) in the Healthcare informatics sector and am trying to identify the current state of the art of standards \(official or guidelines\) in user interfaces and usability in the sector\. At the moment my search is quite broad, encompassing anything from standards / recommendations for usability of medical devices at one end through to the Microsoft common user interface / web accessibility standards on the other\. While I am ultimately interested in GUIs, I suspect I will be investigating related areas along the way\. Would anyone on the list here be familiar with the area and be able to point me in the direction of some resources \(online / offline lists, fora, journals etc\), where I could pose a few questions to those familiar with the field and hopefully help focus my research a bit more\. I would greatly appreciate any comment / guidance you may have\! many thanks, Allen --- ## Post #3 by @Allen_O_Neill Hi Juanita, I would respond off list but your address is masked :\) \.\. My study is academic \(I am studying part\-time\) and the end result output will be open\-sourced\. I will be initially keeping my search as wide as possible to get a picture of what is out there, then isolating it down to whatever is of most interest from a HI GUI point of view\. regards, Allen\. Allen O'Neill Health Ireland Partners Ltd\. Registered in Ireland No\. 271290\.Registered Offices: 10 Whitefriars, Aungier Street, Dublin 2, Ireland\. Tel: \+353 \(0\)402 33200 fax: \+353 \(0\)402 33299 --- ## Post #4 by @Tim_Cook2 You might want to contact Dr\. Andre Kushniruk at UVIC\. http://hinf.uvic.ca/people/Andre.htm This has been his major field of study for several years\. Cheers, Tim --- ## Post #5 by @thomas.beale Conformance testing is coming in openEHR\. It needs fleshing out \(interested parties are welcome to contribute\)\. I see the general shape of conformance testing as follows: Content compliance \- the system can receive, store and serve various test items of content; the served versions do not vary from the received versions \- the system corerctly creates content due to a given series of calls to the vEHR API component \(this API will be standardised in the near future\) \- the system corectly creates an EHR Extract in response to a give request\. \- the system correctly receives and integrates a test EHR Extract\. \- the system correctly responds to test queries System integration \- the system correctly receives, converts and serves test messages \(e\.g\. HL7v2\) \- etc for other formats, including CDA, PDF, text, etc \- the system correctly connects to / integrates with IHE, HSSP and/or other recognised infrastructures\. Services \- the system correctly responds when each service function of the EHR and other services are exercised\. Privacy \- various tests to show that the privacy settings of a test user are respected during querying and other access methods Security \- various tests to show that data integrity and signing are working Performace \- various tests to indicate responds to single patient and populatin queries \- volume tests Not all of these will necessarily be standardised by openEHR, but a sufficient number must be so that users of openEHR systems can have confidence in the correctness, safety, privacy and other aspects of systems\. We would expect to group conformance tests into vrious profiles\. \- thomas beale --- ## Post #6 by @Nicholas_Brown You might like to think about the IHE methodology that includes test nodes and annual week long test sessions (connectathons) where a supplier can show that his/her software can interwork correctly with three other supplier systems according to specific IHE specifications. Best wishes Nick Brown --- ## Post #7 by @thomas.beale > You might like to think about the IHE methodology that includes test nodes and annual week long test sessions \(connectathons\) where a supplier can show that his/her software can interwork correctly with three other supplier systems according to specific IHE specifications\. Hi Nick, It will be nice to get to that point\. Actually, some interoperability between the Java project system and the Ocean system was already demonstrated at MedInfo 2007\. \- thomas --- ## Post #8 by @Tim_Cook2 Thanks for addressing this Tom\. I'm not certain how this process should be managed but there should be a formal process put in place \(maybe by the openEHR Foundation\) to mediate the conformance procedures\. > Content compliance > \- the system can receive, store and serve various test items of content; the > served versions do not vary from the received versions > \- the system corerctly creates content due to a given series of calls to the > vEHR API component \(this API will be standardised in the near future\) > \- the system corectly creates an EHR Extract in response to a give request\. > \- the system correctly receives and integrates a test EHR Extract\. > \- the system correctly responds to test queries > I agree completely\. > System integration > \- the system correctly receives, converts and serves test messages \(e\.g\. > HL7v2\) > \- etc for other formats, including CDA, PDF, text, etc > \- the system correctly connects to / integrates with IHE, HSSP and/or other > recognised infrastructures\. > I completely disagree\. Interfacing with other systems \*is\* important\. However, integration compliance outside of openEHR should be tested and confirmed with those agencies\. For example, why would anyone in any African nation care if their system was IHE compliant? > Services > \- the system correctly responds when each service function of the EHR and > other services are exercised\. These certainly need to be specified\. As AQL matures and other web service type accesses become recognized as being technically required\. In fact, does openEHR want to "specify" that SOA is the proper way to approach healthcare information exchange? > Privacy > \- various tests to show that the privacy settings of a test user are respected > during querying and other access methods Agree\. > Security > \- various tests to show that data integrity and signing are working > Agree\. > Performace > \- various tests to indicate responds to single patient and populatin queries > \- volume tests > Totally agree\! > Not all of these will necessarily be standardised by openEHR, but a sufficient > number must be so that users of openEHR systems can have confidence in the > correctness, safety, privacy and other aspects of systems\. > Certainly the core conformance testing is keyed around content validity/reproducibility\. But I agree that privacy and security should be tested\. Of course, openEHR makes this easier by design\. ;\-\) Cheers, Tim --- ## Post #9 by @thomas.beale > > System integration > > \- the system correctly receives, converts and serves test messages \(e\.g\. > > HL7v2\) > > \- etc for other formats, including CDA, PDF, text, etc > > \- the system correctly connects to / integrates with IHE, HSSP and/or other > > recognised infrastructures\. > > > > I completely disagree\. Interfacing with other systems \*is\* important\. > However, integration compliance outside of openEHR should be tested and > confirmed with those agencies\. For example, why would anyone in any > African nation care if their system was IHE compliant? > They probably don't\. But there are soe things that we need to establish, e\.g\.: \- that if content in the form of a PDF is sent to an openEHR system, it is not lost, and it is accessible in the same way for all opeEHR systems \- that if a set of standard HL7v2 messages \(e\.g\. as used in Australia\) are received, they are correctly processed into openEHR content\. Any given system might not provide such functionality, but if it does, there will need to be some conformance criteria\. \- thomas --- ## Post #10 by @Tim_Cook2 > \- that if content in the form of a PDF is sent to an openEHR system, it is not lost, > and it is accessible in the same way for all opeEHR systems Okay\. But isn't this really a matter of the "content" validation that was mentioned before? > \- that if a set of standard HL7v2 messages \(e\.g\. as used in Australia\) are > received, they are correctly processed into openEHR content\. Again, I would like to assert that there is an openEHR way of storing, manipulating and transferring information\. We should not care \(from an openEHR standpoint\) where the information originated\. Only that we can preserve and present it in an "openEHR" way\. We "should" only care about testing those features\. If an application can't handle HL7v2 or a CDA then that is an application interface issue not an openEHR compliance issue\. > Any given system might not provide such functionality, but if it does, there will > need to be some conformance criteria\. > The conformance criteria is clear\. An openEHR system should be able to \*exchange\* any information it contains, as an openEHR extract, upon a request \(this needs to be solidified\) from any other openEHR compliant system\. How and whether that system performs with HL7v2,CDA,CCD, etc\. is outside the scope of openEHR conformance\. Cheers, Tim --- ## Post #11 by @Tim_Cook2 Hmmmmm, re\-thinking \.\.\.\. --- ## Post #12 by @thomas.beale > > > \- that if content in the form of a PDF is sent to an openEHR system, it is not lost, > > and it is accessible in the same way for all opeEHR systems > > Okay\. But isn't this really a matter of the "content" validation that > was mentioned before? We could cover the handling of opaque formats like PDF in that group of tests as well\. > > > > \- that if a set of standard HL7v2 messages \(e\.g\. as used in Australia\) are > > received, they are correctly processed into openEHR content\. > > Again, I would like to assert that there is an openEHR way of storing, > manipulating and transferring information\. We should not care \(from an > openEHR standpoint\) where the information originated\. Only that we can > preserve and present it in an "openEHR" way\. Its not so much where the information originated, it is a question of whether the semantics are preserved and correctly represented\. We can \(and do already\) develop openEHR templates for this purpose\. I think it will be useful to show hat 2 systems that claim to accept let's 5 specific HL7 messages both end up with the sam answer\. The question is about the conversion of the HL7 format into the openEHR format, i\.e\. what are the fine\-grained correspondences\. All this would be an optional or separete conformance profile\. \- thomas --- ## Post #13 by @system <snip> \(I snipped it away because of readability\) > How and whether that system performs with HL7v2,CDA,CCD, etc\. is outside > the scope of openEHR conformance\. >   I must say, I agree with Tim, we must take care that we \(only\) test to openEhr\-specs conformance\. I would like to add some remarks\. There maybe already thoughts about the coming API, can I find some information about that? It is difficult to have this discussion without that information\. Let me explain: There are more platforms in which the openEhr specs can be build, \.NET, Java, Eiffel, also C\+\+ is possible\. There are also many platforms on which openEhr can run\. Linux, Mac and Windows\. There are many interfacing\-possibilities to other services/software\. The good thing about open standards is that all this is posssible\. But now an API, which must be platform\-independent, because there is no platform defined in the specs\. How will that look like, from technical view: I can imagine, this will be a service as in \(http\) SOA/SOAP\-level Maybe I am wrong, so please tell me were to read more about it\. regards Bert Verhees --- ## Post #14 by @system Hi all, Interesting discussions! I think initially we need to focus on full interoperability between "native" openEHR implementations across different platforms. But in the long run, we need to also make sure that all openEHR system have unified way of doing data exchange with external formats. Early on, we have done a lot work just to make sure the tools - parsers, serializers and editors have good interoperability. It's quite challenging but it's possible and the benefits are obvious. I particularly want to stress the need to collaborate on these matters in the open space. As a community, we can create synergies by working on a common code base for core components that are crucial for interoperability. Besides the items proposed by Tom, I would like to comment on a more detailed level. There is an on-going work for a (nearly) platform independent way of testing (validating) archetypes based system. Initially the scope is targeting on the kernel, which is the essential pieces of software in any archetypes based system. Archetypes based data creating, validation and querying are validated using platform independent test fixtures and test case specifications. Test runners are the only bits of platform-specific in the framework but hopefully they can be provided as open source implementations for each major platform. Test fixtures and test cases are generated semi-automatically and can be created once and used by any platform. We intend to submit this work to MIE2008 within a week or so and will likely provide a proof-of-concept implementation within the Java project. If it goes well, we would like to demonstrate it on a software workshop at MIE2008. Cheers, Rong --- ## Post #15 by @thomas.beale > Hi all, > > Interesting discussions\! I think initially we need to focus on full > interoperability between "native" openEHR implementations across different > platforms\. But in the long run, we need to also make sure that all openEHR > system have unified way of doing data exchange with external formats\. > > Early on, we have done a lot work just to make sure the tools \- parsers, > serializers and editors have good interoperability\. It's quite challenging > but it's possible and the benefits are obvious\. I particularly want to > stress the need to collaborate on these matters in the open space\. As a > community, we can create synergies by working on a common code base for core > components that are crucial for interoperability\. > > Besides the items proposed by Tom, I would like to comment on a more > detailed level\. There is an on\-going work for a \(nearly\) platform > independent way of testing \(validating\) archetypes based system\. Initially > the scope is targeting on the kernel, which is the essential pieces of > software in any archetypes based system\. Archetypes based data creating, > validation and querying are validated using platform independent test > fixtures and test case specifications\. Test runners are the only bits of > platform\-specific in the framework but hopefully they can be provided as > open source implementations for each major platform\. Test fixtures and test > cases are generated semi\-automatically and can be created once and used by > any platform\. We intend to submit this work to MIE2008 within a week or so > and will likely provide a proof\-of\-concept implementation within the Java > project\. If it goes well, we would like to demonstrate it on a software > workshop at MIE2008\. > sounds like time for new wiki page ;\-\) \- thomas --- ## Post #16 by @Seref Hi Rong, Interesting discussion indeed. For any standards based project, integration / communication of two different systems includes many hidden costs. It is quite hard to imagine the potential problems that may arise when making two different apps talk to each other, even if they conform to same standards. The problems appear in the technology level, since application A and B might be implementing the standards in very different ways. So, any systems validation work is priceless for it might give a set of best practices for various types of applications. I have previously implemented a solution for connecting a legacy app to a hl7 v2 based system. In theory, everyting should work fine, especially if you think about the relative simplicity of V2. However, when your only option is to use a web services based communication, a legacy app running on oracle forms 4.5 becomes a nightmare. I had to design a seperate java application, which was my only way out, but still I do not feel comfortable about it. This kind of tacid knowledge is priceless, and represents another layer of connectivity, which in my opinion is very important, and can not be obtained without actual experience. Regarding your work, maybe you'd like to take a look at this: [http://www.healthcareitnews.com/story.cms?id=8125](http://www.healthcareitnews.com/story.cms?id=8125) . It does not provide a lot of details, and probably not relevant, but might give some clues for a potential future platform you might consider using All the best Seref Arikan Rong Chen wrote: --- ## Post #17 by @system Dear all, The work by CCHIT in the USA is well known to me. EuroRec (the European Institute for Health Records) co-operates with the CCHIT on quality labeling and certification of EHR-systems. This includes functional and later semantic interoperability aspects. Before mid 2008 the first European project will be finished. This Q-Rec project will include the governance of Archetypes and Templates. In my personal view it is likely that OpenEHR will play a pivotal role in Europe when it comes to testing and certifying semantic interoperability in Europe. EHR-systems, archetypes and Templates. The discussion the last days was very interesting. With regards, Gerard -- -- Gerard Freriks, MD Huigsloterdijk 378 2158 LR Buitenkaag The Netherlands T: +31 252544896 M: +31 620347088 E: [gfrer@luna.nl](mailto:gfrer@luna.nl) Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety. Benjamin Franklin 11 Nov 1755 --- **Canonical:** https://discourse.openehr.org/t/openehr-system-validation/14684 **Original content:** https://discourse.openehr.org/t/openehr-system-validation/14684