proposal for managing migration to .adls (source form) archetypes

Technical (archive)
1

Dear all,

as some at least have noticed from previous posts, there is the
intention to move toward what we call 'source form' or 'differential'
archetypes, whereby archetypes contain only changes with respect to a
specialisation parent archetype. This move has no effect on the majority
of archetypes (since they are top-level archetypes) but specialised
archetypes are changed from being 'flat-form' archetypes (where
inherited, unchanged elements are repeated in the descendant) to
differential form, where they contain only new or redefined elements
with respect to the parent archetype. Doing this makes specialised
archetypes sustainable to maintain.

So far in the reference parser, we have implemented a fair bit of the
specialisatoin semantics, and are generating .adls (source form) files
from the existing .adl files. However, clearly in the future the aim is
to be authoring from the outset in .adls form. The Archetype Editor has
not had this capability added, but there are places where we want to do
this anyway.

As a result, we have the situation where for some time both .adl and
.adls files will be the authored form of an archetype, and therefore,
both .adls and .ald files may be generated - the former from current
.adl files, and the latter from de novo authoring in .adls (or an XML
equivalent).

My proposal is to add a single flag to the .adl grammar that indicates
whether an archetype, in either format, is 'generated', meaning was it
machine generated from the other form. If this flag is not set, it
means that the file is the original authored form of an archetype. The
actual form of the flag would be in the first line, in the bracketed
part after the 'archetype' keyword. For example, in the archetype at
http://svn.openehr.org/knowledge/archetypes/dev/adl/openehr/ehr/entry/observation/openEHR-EHR-OBSERVATION.ecg.v1draft.html
the first line is:

archetype (adl_version=1.4)

my proposal would be that if it were generated, it would be as follows:

archetype (adl_version=1.4; generated)

in a similar manner to the 'controlled' indicator defined in section
8.3.2 of the ADL 1.4 specification
(http://www.openehr.org/releases/1.0.1/architecture/am/adl.pdf). The
equivalent change would be made in the openEHR archetype XSD to allow
XML archetypes to include this flag.

This allows tools to know which form can safely be discarded, allowing
bugs and development to continue on the tools themselves. To move
forward on this, we would need the agreement of all archetype toolbuilders.

all feedback welcome.

2

Dear Tom,

When you say:

there is the intention to move toward what we call 'source form' or
'differential' archetypes, whereby archetypes contain only changes
with respect to a specialisation parent archetype

how has this intention arisen, and become intended?
Has there been an analysis of the benefits and costs of this approach
that you can point me to? I would like to understand this better.

With best wishes,

Dipak

3

Dipak Kalra wrote:

Dear Tom,

When you say:



there is the intention to move toward what we call 'source form' or
'differential' archetypes, whereby archetypes contain only changes
with respect to a specialisation parent archetype



how has this intention arisen, and become intended?
Has there been an analysis of the benefits and costs of this approach
that you can point me to? I would like to understand this better.

Hi Dipak,

a number of users including the NHS have pointed out (quite rightly) that the current way of representing specialised archetypes is unsustainable, for the technical reason that specialised archetypes contain a copy of the inherited elements from the parent. The effect of this is that, if the parent archetype (think recursively up a chain of specialised archetypes) is changed in some detail, the places where the relevant element is expressed in all child archetypes are now out of sync with the parent. As you can imagine, with any reasonable number of archetypes, especially specialised ones, this cause problems, because the specialised ones will slowly become non-conformant to the parents. In this scheme, each archetype is effectively a fully defined independent entity, and is compiled (i.e. parsed) as such. This is currently how .adl files work.

The way to avoid this is for specialised archetypes to be expressed ‘differentially’, i.e. only overridden and new elments are expressed in a specialised archetype source file. Now when an element is changed in a parent (anywhere in the lineage), the compiler will handle it as follows:

  • all archetypes in the specialisation lineage are used in the compilation of any given archetype

  • if the element is inherited unchanged, nothing special is needed (there may be some validity checks brought into play)

  • if it is overridden in the child, the compiler re-evaluates what is in the child, and may emit an error if the child no longer conforms.

  • if it is defined new in the child, nothing special is needed (there may be some validity checks brought into play)
    In this way, all specialised archetypes in an archetype lineage are always forced to be in sync with the parents up the chain. This is how .adls files work. Note that for a top-level (non-specialised) archetype, there is no difference between these two forms. However, in the first scheme, every specialised archetype is essentially an ‘integration’ of all the effects of inheritance, compressed into one class. This is sometimes known as the ‘inheritance flattened’ form, and some object oriented languages provide tools to show this form. Most importantly it is the form in use at runtime.

The latter scheme above is the same as used in all object oriented languages (to my knowledge at least), or at least all class-based ones (consider any Java, C#, C++, Python, etc class text that inherits from some other class - it does not repeat all the elements from the parent).

The above is not to say that the inheritance-flattened form (i.e. the .adl) is not useful - indeed, as for object oriented languages, it is essential. However, to be reliable, it has to be machine generated by compressing the differential forms down the inheritance lineage. If you download the latest ADL Workbench, you can see graphically how this works in the differential and flat forms.of the node maps, which are colourised to show inherited elements.

In the future proposed, the old kind of archetypes would remain in .adl files, but would be generated from .adls files, which would become the authored source format of archetypes. There needs to be a transition phase in which both are supported as authored source formats by the tools.

These changes will enable the implementation of more powerful specialisation semantics in ADL 1.5 and also template semantics to be completed.

Discussion in the community is of course needed to determine the timeline when these changes are brought into the mainstream of archetype development. However we have already achieved in the latest archetype workbench the ability to silently support both .adl and .adls forms, regardless of which one was used as the authoring format, and also to generate .adl files from .adls files.

One last note: the syntax in a .adls file is the same as in a .adl file, with one exception coming in ADL 1.5. So going down this path does not mean new syntax or new a XSD.

hope this helps,

  • thomas