hi,
as most of you know, there are test archetypes at http://www.openehr.org/svn/knowledge/archetypes/dev/adl/test/
We have made some attempts in the past to get these better organised, but have not been very systematic about it. To improve on the current situation, I propose the following that we adopt a regression testing approach. This means having both the input archetype and the expected output in each case. Given that we want these archetypes to be usable by implementers from all languages, we can’t put them into a Java-specific, Eiffel-specific, Python-specific etc test framework - we have to be more catholic than that. After some discussions with Peter Gummer, I propose the following:
-
for each test archetype, we have two files:
-
the archetype file, named using something meaningful in terms of testing, e.g. adl-test-ENTRY.at_code_duplicated_in_ontology.v1.adl
-
an output file, containing a list of expected validity codes, e.g. VCARM, VONC, etc that the compiler should gneerate when encountering this archetype; this file would have the same name except instead of “.adl” or “adls”, some other extension; I suggested “.to” = test output; Peter suggested “.expected”, which is more informative.
-
in your respective test environments, you do the following:
-
read and parse the archetype, collecting the errors generated in your ‘status’ variable or wherever it turns up
-
read the test output file, and try to find its contents in the compiler output
to make this work, we need to agree on exactly what the test output file looks like? I suggest that a first go is comma- or tab-delimited text, with one entry per expected compiler output, e.g. maybe in the table format -
compiler error code
-
whether it is optional or mandatory (note that some compilers like the Eiffel one don’t stop on the first error)
-
other parameters
so it might look like this:
– test output file for adl-test-ENTRY.at_code_duplicated_in_ontology.v1.adl
VCARM M – M = mandatory
VONC O – O = optional
XYZ O aaa bbb ccc – some other output that
Your own test harness will have to know what to do with this of course. For example, in our Eiffel one, we currently have outputs like:
(already attempted; status = ERROR - line 128: Error (VACMC): cannot add C_COMPLEX_OBJECT object with node_id=at0015 to multiply-valued attribute items because cardinality 1..2 does not contain occurrences 0..* of object
[last cADL token = SYM_END_CBLOCK]
(Parse failed) (ARCH_REP_ARCHETYPE.parse_archetype)
which although kind of ugly, does contain the codes, and we would do a has_string() test on the output. The group here may have better ideas on the above.
As a second task, we also should try to regularise the naming and directory structure of the test archetypes.
- thomas beale
