So regardless of data persistence (which is an implementation detail
and a software engineering choice independent of OpenEhr) how can we
test the compliance of an OpenEHR system?
CCHIT developed inter-operability tests (I believe).
Can we do the same? Can we have say a template that reflects the
context of data and have OpenEHR systems demonstrate their display or
data entry?
Or perhaps a set of data and a set of AQL queries that should all
return the same results?
Some kind of non-subjective certification in the long term would be nice.
For now just a 'show you can do this and you are on the right track'
would helpful.
I am researching HCI (Human Computer Interaction) in the Healthcare
informatics sector and am trying to identify the current state of the art of
standards (official or guidelines) in user interfaces and usability in the
sector. At the moment my search is quite broad, encompassing anything from
standards / recommendations for usability of medical devices at one end
through to the Microsoft common user interface / web accessibility standards
on the other. While I am ultimately interested in GUIs, I suspect I will be
investigating related areas along the way.
Would anyone on the list here be familiar with the area and be able to point
me in the direction of some resources (online / offline lists, fora,
journals etc), where I could pose a few questions to those familiar with the
field and hopefully help focus my research a bit more.
I would greatly appreciate any comment / guidance you may have!
I would respond off list but your address is masked ..
My study is academic (I am studying part-time) and the end
result output will be open-sourced.
I will be initially keeping my search as wide as possible
to get a picture of what is out there, then isolating it down
to whatever is of most interest from a HI GUI point of view.
Conformance testing is coming in openEHR. It needs fleshing out (interested
parties are welcome to contribute). I see the general shape of conformance
testing as follows:
Content compliance
- the system can receive, store and serve various test items of content; the
served versions do not vary from the received versions
- the system corerctly creates content due to a given series of calls to the
vEHR API component (this API will be standardised in the near future)
- the system corectly creates an EHR Extract in response to a give request.
- the system correctly receives and integrates a test EHR Extract.
- the system correctly responds to test queries
System integration
- the system correctly receives, converts and serves test messages (e.g.
HL7v2)
- etc for other formats, including CDA, PDF, text, etc
- the system correctly connects to / integrates with IHE, HSSP and/or other
recognised infrastructures.
Services
- the system correctly responds when each service function of the EHR and
other services are exercised.
Privacy
- various tests to show that the privacy settings of a test user are respected
during querying and other access methods
Security
- various tests to show that data integrity and signing are working
Performace
- various tests to indicate responds to single patient and populatin queries
- volume tests
Not all of these will necessarily be standardised by openEHR, but a sufficient
number must be so that users of openEHR systems can have confidence in the
correctness, safety, privacy and other aspects of systems.
We would expect to group conformance tests into vrious profiles.
You might like to think about the IHE methodology that includes test nodes and annual week long test sessions (connectathons) where a supplier can show that his/her software can interwork correctly with three other supplier systems according to specific IHE specifications.
You might like to think about the IHE methodology that includes test nodes
and annual week long test sessions (connectathons) where a supplier can
show that his/her software can interwork correctly with three other supplier
systems according to specific IHE specifications.
Hi Nick,
It will be nice to get to that point. Actually, some interoperability between the
Java project system and the Ocean system was already demonstrated at
MedInfo 2007.
Thanks for addressing this Tom. I'm not certain how this process should
be managed but there should be a formal process put in place (maybe by
the openEHR Foundation) to mediate the conformance procedures.
Content compliance
- the system can receive, store and serve various test items of content; the
served versions do not vary from the received versions
- the system corerctly creates content due to a given series of calls to the
vEHR API component (this API will be standardised in the near future)
- the system corectly creates an EHR Extract in response to a give request.
- the system correctly receives and integrates a test EHR Extract.
- the system correctly responds to test queries
I agree completely.
System integration
- the system correctly receives, converts and serves test messages (e.g.
HL7v2)
- etc for other formats, including CDA, PDF, text, etc
- the system correctly connects to / integrates with IHE, HSSP and/or other
recognised infrastructures.
I completely disagree. Interfacing with other systems *is* important.
However, integration compliance outside of openEHR should be tested and
confirmed with those agencies. For example, why would anyone in any
African nation care if their system was IHE compliant?
Services
- the system correctly responds when each service function of the EHR and
other services are exercised.
These certainly need to be specified. As AQL matures and other web
service type accesses become recognized as being technically required.
In fact, does openEHR want to "specify" that SOA is the proper way to
approach healthcare information exchange?
Privacy
- various tests to show that the privacy settings of a test user are respected
during querying and other access methods
Agree.
Security
- various tests to show that data integrity and signing are working
Agree.
Performace
- various tests to indicate responds to single patient and populatin queries
- volume tests
Totally agree!
Not all of these will necessarily be standardised by openEHR, but a sufficient
number must be so that users of openEHR systems can have confidence in the
correctness, safety, privacy and other aspects of systems.
Certainly the core conformance testing is keyed around content
validity/reproducibility. But I agree that privacy and security should
be tested. Of course, openEHR makes this easier by design.
> System integration
> - the system correctly receives, converts and serves test messages (e.g.
> HL7v2)
> - etc for other formats, including CDA, PDF, text, etc
> - the system correctly connects to / integrates with IHE, HSSP and/or other
> recognised infrastructures.
>
I completely disagree. Interfacing with other systems *is* important.
However, integration compliance outside of openEHR should be tested and
confirmed with those agencies. For example, why would anyone in any
African nation care if their system was IHE compliant?
They probably don't. But there are soe things that we need to establish, e.g.:
- that if content in the form of a PDF is sent to an openEHR system, it is not lost,
and it is accessible in the same way for all opeEHR systems
- that if a set of standard HL7v2 messages (e.g. as used in Australia) are
received, they are correctly processed into openEHR content.
Any given system might not provide such functionality, but if it does, there will
need to be some conformance criteria.
- that if content in the form of a PDF is sent to an openEHR system, it is not lost,
and it is accessible in the same way for all opeEHR systems
Okay. But isn't this really a matter of the "content" validation that
was mentioned before?
- that if a set of standard HL7v2 messages (e.g. as used in Australia) are
received, they are correctly processed into openEHR content.
Again, I would like to assert that there is an openEHR way of storing,
manipulating and transferring information. We should not care (from an
openEHR standpoint) where the information originated. Only that we can
preserve and present it in an "openEHR" way.
We "should" only care about testing those features. If an application
can't handle HL7v2 or a CDA then that is an application interface issue
not an openEHR compliance issue.
Any given system might not provide such functionality, but if it does, there will
need to be some conformance criteria.
The conformance criteria is clear. An openEHR system should be able to
*exchange* any information it contains, as an openEHR extract, upon a
request (this needs to be solidified) from any other openEHR compliant
system.
How and whether that system performs with HL7v2,CDA,CCD, etc. is outside
the scope of openEHR conformance.
> - that if content in the form of a PDF is sent to an openEHR system, it is not
lost,
> and it is accessible in the same way for all opeEHR systems
Okay. But isn't this really a matter of the "content" validation that
was mentioned before?
We could cover the handling of opaque formats like PDF in that group of tests
as well.
>
> - that if a set of standard HL7v2 messages (e.g. as used in Australia) are
> received, they are correctly processed into openEHR content.
Again, I would like to assert that there is an openEHR way of storing,
manipulating and transferring information. We should not care (from an
openEHR standpoint) where the information originated. Only that we can
preserve and present it in an "openEHR" way.
Its not so much where the information originated, it is a question of whether the
semantics are preserved and correctly represented. We can (and do already)
develop openEHR templates for this purpose. I think it will be useful to show hat
2 systems that claim to accept let's 5 specific HL7 messages both end up with
the sam answer. The question is about the conversion of the HL7 format into
the openEHR format, i.e. what are the fine-grained correspondences.
All this would be an optional or separete conformance profile.
How and whether that system performs with HL7v2,CDA,CCD, etc. is outside
the scope of openEHR conformance.
I must say, I agree with Tim, we must take care that we (only) test to
openEhr-specs conformance.
I would like to add some remarks. There maybe already thoughts about the
coming API, can I find some information about that? It is difficult to
have this discussion without that information. Let me explain:
There are more platforms in which the openEhr specs can be build, .NET,
Java, Eiffel, also C++ is possible. There are also many platforms on
which openEhr can run. Linux, Mac and Windows.
There are many interfacing-possibilities to other services/software. The
good thing about open standards is that all this is posssible.
But now an API, which must be platform-independent, because there is no
platform defined in the specs. How will that look like, from technical
view: I can imagine, this will be a service as in (http) SOA/SOAP-level
Maybe I am wrong, so please tell me were to read more about it.
Interesting discussions! I think initially we need to focus on full interoperability between “native” openEHR implementations across different platforms. But in the long run, we need to also make sure that all openEHR system have unified way of doing data exchange with external formats.
Early on, we have done a lot work just to make sure the tools - parsers, serializers and editors have good interoperability. It’s quite challenging but it’s possible and the benefits are obvious. I particularly want to stress the need to collaborate on these matters in the open space. As a community, we can create synergies by working on a common code base for core components that are crucial for interoperability.
Besides the items proposed by Tom, I would like to comment on a more detailed level. There is an on-going work for a (nearly) platform independent way of testing (validating) archetypes based system. Initially the scope is targeting on the kernel, which is the essential pieces of software in any archetypes based system. Archetypes based data creating, validation and querying are validated using platform independent test fixtures and test case specifications. Test runners are the only bits of platform-specific in the framework but hopefully they can be provided as open source implementations for each major platform. Test fixtures and test cases are generated semi-automatically and can be created once and used by any platform. We intend to submit this work to MIE2008 within a week or so and will likely provide a proof-of-concept implementation within the Java project. If it goes well, we would like to demonstrate it on a software workshop at MIE2008.
Interesting discussions! I think initially we need to focus on full
interoperability between "native" openEHR implementations across different
platforms. But in the long run, we need to also make sure that all openEHR
system have unified way of doing data exchange with external formats.
Early on, we have done a lot work just to make sure the tools - parsers,
serializers and editors have good interoperability. It's quite challenging
but it's possible and the benefits are obvious. I particularly want to
stress the need to collaborate on these matters in the open space. As a
community, we can create synergies by working on a common code base for
core
components that are crucial for interoperability.
Besides the items proposed by Tom, I would like to comment on a more
detailed level. There is an on-going work for a (nearly) platform
independent way of testing (validating) archetypes based system. Initially
the scope is targeting on the kernel, which is the essential pieces of
software in any archetypes based system. Archetypes based data creating,
validation and querying are validated using platform independent test
fixtures and test case specifications. Test runners are the only bits of
platform-specific in the framework but hopefully they can be provided as
open source implementations for each major platform. Test fixtures and test
cases are generated semi-automatically and can be created once and used by
any platform. We intend to submit this work to MIE2008 within a week or so
and will likely provide a proof-of-concept implementation within the Java
project. If it goes well, we would like to demonstrate it on a software
workshop at MIE2008.
Hi Rong,
Interesting discussion indeed. For any standards based project, integration / communication of two different systems includes many hidden costs. It is quite hard to imagine the potential problems that may arise when making two different apps talk to each other, even if they conform to same standards. The problems appear in the technology level, since application A and B might be implementing the standards in very different ways. So, any systems validation work is priceless for it might give a set of best practices for various types of applications.
I have previously implemented a solution for connecting a legacy app to a hl7 v2 based system. In theory, everyting should work fine, especially if you think about the relative simplicity of V2. However, when your only option is to use a web services based communication, a legacy app running on oracle forms 4.5 becomes a nightmare. I had to design a seperate java application, which was my only way out, but still I do not feel comfortable about it. This kind of tacid knowledge is priceless, and represents another layer of connectivity, which in my opinion is very important, and can not be obtained without actual experience.
Regarding your work, maybe you’d like to take a look at this: http://www.healthcareitnews.com/story.cms?id=8125 . It does not provide a lot of details, and probably not relevant, but might give some clues for a potential future platform you might consider using
The work by CCHIT in the USA is well known to me.
EuroRec (the European Institute for Health Records) co-operates with the CCHIT on quality labeling and certification of EHR-systems.
This includes functional and later semantic interoperability aspects.
Before mid 2008 the first European project will be finished.
This Q-Rec project will include the governance of Archetypes and Templates.
In my personal view it is likely that OpenEHR will play a pivotal role in Europe when it comes to testing and certifying semantic interoperability in Europe.
EHR-systems, archetypes and Templates.
The discussion the last days was very interesting.
With regards,
Gerard
– –
Gerard Freriks, MD
Huigsloterdijk 378
2158 LR Buitenkaag
The Netherlands
..