I have done some thinking on how to document the conformance specifications and framework in some detail, based on some of @pablo 's original ideas. You can see his diagrams in the current (very draft) Conformance Guide, in particular this one on ‘conformance artefacts’.
I propose a new version of this diagram that makes use of the existing ‘block diagram’ (love it or hate it), to try to explain a bit better the relationships between the various existing and needed artefacts.
The top block ‘Conformance Specifications’ is the set of specifications we need to define Conformance testing and enable it to be reported. The Test Execution Report, Conformance Statement and Conformance Certificate aren’t really specifications in the ordinary sense of course - they would be some kind of document template or definition of the minimum sections & fields of such reports.
The main ‘specifications’ we need are:
- The Conformance Guide - working on this now with @pablo and others;
- The Platform Conformance Test Framework - being adapted from EhrBase docs (TODO)
- The Platform Conformance Test Schedule (generic) - NEARLY DONE, initially based on EhrBase
- Executable technology-specific Test Schedule - being adapted from EhrBase Robot files.
This might strike some as too complicated, but I am fairly convinced it is the minimum we can get away with to satisfy contractual as well as engineering needs. Essentially, it boils down to:
- specs of what can be logically tested, with what data sets, and how - the blue blocks on the lower left
- executable test runners for the above - the green block on lower right
- various report / statement artefacts to document test results.
How does the group feel about this approach?