# trying to flat archetypes

**Category:** [Reference Implementation: Java (archive)](https://discourse.openehr.org/c/reference-implementation-java-archive/154)
**Created:** 2012-05-13 05:48 UTC
**Views:** 7
**Replies:** 20
**URL:** https://discourse.openehr.org/t/trying-to-flat-archetypes/15168

---

## Post #1 by @pablo

Hi Rong,

I've updated my java-ref-impl from SVN: http://www.openehr.org/svn/ref_impl_java
I builded it and generated the jars: xxx-1.0.2-SNAPSHOT.jar

Is the version 1.0.2-SNAPSHOT correct? I remember I builded this a long time ago and was generating the same version.

I have a question about the flattener:
If you see: http://www.openehr.org/svn/ref_impl_java/TRUNK/oet-parser/src/test/resources/archetypes/openEHR-EHR-COMPOSITION.prescription_flattened.v1.adl
It doesn't have the reference to the resolved slots, instead it has at0000 for all those nodes and doesn't resolve the ontology part to include referenced terms.
The suggestion made by Thomas ([http://lists.openehr.org/pipermail/openehr-technical_lists.openehr.org/2012q2/007023.html](http://lists.openehr.org/pipermail/openehr-technical_lists.openehr.org/2012q2/007023.html)) is to replace the nodeId with the archetypeId on the resolved slots, and put the ontology terms of the resolved archetypes in the flattened archetype ontology.

Do you think this should be corrected on the flattener? (I don't know if this is a bug or this is the expected behaviour and the reference to the slots and ontologies are resolved in some other way).

Other question: is this a bug?

I've codedd a simple slot finder and reference loader to test archetype flattening, and it seems the slots I found had no parent (i.e. the reference to the attribute containing the slot is null in the slot):

>> ACTION
org.openehr.am.archetype.constraintmodel.ArchetypeSlot@e53220[
includes=[archetype_id/value matches {/openEHR-EHR-ACTION.medication.v1/}]
excludes=[archetype_id/value matches {/.*/}]
rmTypeName=ACTION
occurrences=org.openehr.rm.support.basic.Interval@3257b8[lower=0,lowerIncluded=true,upper=<null>,upperIncluded=false]
nodeID=<null>
**parent=<null>**
anyAllowed=false
path=/content
hiddenOnForm=false
annotation=<null>
]
>> INSTRUCTION
org.openehr.am.archetype.constraintmodel.ArchetypeSlot@11d2572[
includes=[archetype_id/value matches {/openEHR-EHR-INSTRUCTION.medication.v1/}]
excludes=[archetype_id/value matches {/.*/}]
rmTypeName=INSTRUCTION
occurrences=org.openehr.rm.support.basic.Interval@859a68[lower=0,lowerIncluded=true,upper=<null>,upperIncluded=false]
nodeID=<null>
**parent=<null>**
anyAllowed=false
path=/content
hiddenOnForm=false
annotation=<null>
]

---

## Post #2 by @thomas.beale

I don't have any idea what the current Java flattener does, but we will need to get any/all ADL 1.5 flatteners in sync. I believe the ADL Workbench has the definitive algorithm at the moment, but there may be still details to change.

Anyway, in the near term, I intend to publish the algorithm in a friendly way (the code is of course available in the ref_impl_eiffel SVN, but it's in Eiffel, so we need a language neutral form). I've been cleaning it up a bit, so it should end up being pretty comprehensible... well, for what it is... the full flattening semantics are not trivial.

If I get this out in the next couple of weeks, it would be good if anyone/everyone can have a look, try to find bugs, errors, clean simplifications etc, and also see if the java code can implement it.

When we get to that point we need to develop a bunch of test archetypes for flattening, including with regression. One way to do this is to manually build 'correct' flat archetypes that the compiler should always be able to match by generation. Another way is to implement a second algorithm, which I will also publish, which is the 'flat archetype differencing algorithm', which can reconstitute the child differential from a generated flat. if the latter is done properly then it should always be the case that this latter generated diff form is identical to the original authored diff. While this doesn't theoretically guarantee the correctness of the flattening operation (since both flattening and diffing could be wrong, but in the same way), we can assume it does for practical purposes.

Obviously I don't guarantee that either of these algorithms as I have defined them is perfect, but they appear to work properly on all cases I have tested, so i think they are a good starting point.

- thomas

---

## Post #3 by @pablo

Hi Thomas, it would be great to have the algorithms specified, I can help with Java/Groovy implementation and testing.

I also need some of this functionality for ADL 1.4, but just to resolve slots, not specialization with semantic codes resolution.

We need both 1.4 & 1.5 to put the "test base" together and generate some initial artifacts (I'm working on this right now, I started trying things on 1.4).

AFAIK, seeing the code, the java ref flattener flattens only XML templates, maybe Rong can enlight us on that.

---

## Post #4 by @system

Hi all,

The current flattener in Java project only handles templates in OET
format \(not ADL 1\.5\)\. So, for instance it doesn&#39;t deal with
differential forms of archetypes\.

And Yes, Thomas\! It would definitely be a great step to publish the
algorithms and to build up a set of standard test archetypes\.

/Rong

---

## Post #5 by @pablo

Hi Rong, did you saw my questions about the current java impl version and the possible bug on slot.parent?

Please drop me a line when you have time.

---

## Post #6 by @system

Hi Pablo,

Well, the OET template flattener isn&#39;t quite the same as the ADL 1\.5 flattener\.

/Rong

---

## Post #7 by @pablo

Hi Rong,

I'm just using the adl-parser from the svn trunk with adl 1.4 files, the null problem is with the aom object.

---

## Post #8 by @pablo

Hi Rong, I've found another problem:

groovy.lang.ReadOnlyPropertyException: Cannot set readonly property: nodeID for class: org.openehr.am.archetype.constraintmodel.CComplexObject

It seems that some properties are still read-only, like CComplexObject.nodeID. What I tried to do is what Thomas suggested: put the archetypeId in the nodeID of the resolved slots on the flat archetype.

About the null parent attribute on ArchetypeSlot, I do an ugly workaround passing the parent object to every method of my archetype slot finder walk through.

---

## Post #9 by @pablo

Please forget the previous message, I saw the code and nodeId is writable, there was an error on my side. Sorry for that.

---

## Post #10 by @system

Pablo,

We have relaxed the immutability of AOM objects\. The attribute nodeID
is deprecated, so you should use setNodeId\(\) and getNodeId\(\) instead\.

I just recall that in the Swedish project, we didn&#39;t have any
Instruction/Activities templates so it could be the case that this use
case is not covered by the OET flattener\. Feel free to improve it and
accompany the new functionality with a testcase\.

Not sure about the null parent issue in ArchetypeSlot, can you provide
more details on it?

/Rong

---

## Post #11 by @pablo

Hi Rong, about the null parent on ArchetypeSlot:

I was just loading an ADL 1.4 archetype with slots and printing the slots I found in the definition.

Sample: (this is in Groovy [http://groovy.codehaus.org/](http://groovy.codehaus.org/))

def adlFile = new File( "openEHR-EHR-COMPOSITION.medication_list.v1" ) // See the archetype here: http://www.openehr.org/knowledge/OKM.html#showarchetype_1013.1.286

ADLParser parser = null;
try { parser = new ADLParser( adlFile ) }
catch (IOException e) { print e.message }

Archetype archetype = null;
try { archetype = parser.archetype() }
catch (Exception e) { print e.message }

then when I found a slot:

def findSlots(ArchetypeSlot slot, parent)
{
println "Slot>> " + slot

...
}

It seems that parent and nodeId attributes are null for the slot:

>> org.openehr.am.archetype.constraintmodel.ArchetypeSlot@1690ab[
includes=[archetype_id/value matches {/openEHR-EHR-ACTION.medication.v1/}]
excludes=[archetype_id/value matches {/.*/}]
rmTypeName=ACTION
occurrences=org.openehr.rm.support.basic.Interval@1a85d38[lower=0,lowerIncluded=true,upper=<null>,upperIncluded=false]
**nodeID=<null>**
**parent=<null>**
anyAllowed=false
path=/content
hiddenOnForm=false
annotation=<null>
]

>> org.openehr.am.archetype.constraintmodel.ArchetypeSlot@1860038[
includes=[archetype_id/value matches {/openEHR-EHR-ITEM_TREE\.medication\.v1|openEHR-EHR-ITEM_TREE\.medication-vaccine\.v1/}]
excludes=[archetype_id/value matches {/.*/}]
rmTypeName=ITEM_TREE
occurrences=org.openehr.rm.support.basic.Interval@787c16[lower=1,lowerIncluded=true,upper=1,upperIncluded=true]
**nodeID=<null>**
**parent=<null>**
anyAllowed=false
path=/description
hiddenOnForm=false
annotation=<null>
]

I have one method findSlot for each ArchetypeContraint path, so the walk throw on the archetype.definition is done recursively. E.g.:

def findSlots(CComplexObject c, parent)
def findSlots(CAttribute c, parent)
def findSlots(CPrimitiveObject c, parent)
...

Passing the parent node to the method is the workaround I mentioned, this could be avoided if slot.parent is set to the corresponding CAttribute.

Hope that helps.

---

## Post #12 by @pablo

Hi Rong,

did you take a look at the null parent problem?

---

## Post #13 by @system

Hi Pablo,
I am aware of null parent  in the aom objects\. The thing is before the
aom objects are immutable and during ADL parsing all the objects are
built from bottom up so there is no possibility to set parent when
building the child\. Now immutability is not enforced anymore in aom
but I still question the virtue of have such cyclic references in the
objects\.
There are other ways of retrieving the parent, e\.g\. path based
approach\. so if the use case isn&#39;t sufficient strong, I am bit
reluctant to implement it\. please let me know how you think\.
Regards,
Rong

---

## Post #14 by @pablo

Hi Rong,

Maybe a second pass top-down can set the parent on each node on the parser code.

The problem of the path approach is that if a CObject has no nodeID, then it's path is equal to the CAttribute path.
When I do an archetype.node(pathToAttribute) that contains a child with no nodeID, I get the CObject instead of the CAttribute.
(BTW: I don't know if this is the expected behaviour)

Related to this, I saw that on ACTION archetypes, there are sibling ISM_TRANSITION nodes that doesn't have a nodeID, violating a rule on the specs, but it's getting parsed without problems.

---

## Post #15 by @system

> Hi Rong,
>
> Maybe a second pass top\-down can set the parent on each node on the parser
> code\.

Yes, that&#39;s definitely possible\. I will give it a try to see if this
really affects others\.

> The problem of the path approach is that if a CObject has no nodeID, then
> it&#39;s path is equal to the CAttribute path\.
> When I do an archetype\.node\(pathToAttribute\) that contains a child with no
> nodeID, I get the CObject instead of the CAttribute\.
> \(BTW: I don&#39;t know if this is the expected behaviour\)

> Related to this, I saw that on ACTION archetypes, there are sibling
> ISM\_TRANSITION nodes that doesn&#39;t have a nodeID, violating a rule on the
> specs, but it&#39;s getting parsed without problems\.

Right\. This should be picked up by archetype\-validator after parsing I believe\.

Cheers,
Rong

---

## Post #16 by @thomas.beale

this is probably completely obvious, but just in case... Obviously I don't know the Java code, but the Eiffel and .Net code I think does the same thing: when a node object (e.g. a C_OBJECT of some kind) is created and attached to its parent (a C_ATTRIBUTE, except for the root object), the normal pattern is (in pseudo-code): in CAttribute: addChild (CObject anObjNode) { children.append (anObjNode); anObjNode.setParent (this); } in CObject: setParent (CAttribute anAttrNode) { parent := anAttrNode; } this setParent routine should only be visible to CAttribute There is the same pattern in the other sense, to enable CAttribute to set a parent CComplexObject. - thomas

---

## Post #17 by @pablo

Hi Rong,

Just a related issue:

When flattening archetypes, when resolving slots, I use nodeID with this format: archetypeID::atNNNN.
That's to avoid collisions between node ids in the root archetype and node ids of archetypes referenced by slots.

When I parse the flat archetype I get an error, it seems there is a constraint on nodeID format.

BTW: the flattened ADL was generated by the ADLSerializer.

Encountered "ISM_TRANSITION [openEHR-EHR-ACTION.medication.v1::at0018]" at line 47, column 21.
Was expecting one of:
"}" ...
"c_dv_quantity" ...
<V_TYPE_IDENTIFIER> "<" ...
<V_TYPE_IDENTIFIER> <V_LOCAL_TERM_CODE_REF> ...
<V_TYPE_IDENTIFIER> "occurrences" ...
<V_TYPE_IDENTIFIER> <SYM_MATCHES> ...
"use_node" ...
<SYM_ALLOW_ARCHETYPE> ...
<V_TERMINOLOGY_ID_BLOCK> ...
<V_CODE_PHRASE> ...
"+" ...
"-" ...
<V_INTEGER> ...
"C_DV_ORDINAL" ...
"true" ...
"false" ...
<V_ISO8601_DATE_CONSTRAINT_PATTERN> ...
<V_DATE> ...
"|" ...

...

I changed "::" to "_" but I get another exception: (I don't know if this exception has something to do with the nodeID format or with an ISM_TRANSITION having a nodeID)

Encountered "ISM_TRANSITION [" at line 47, column 21.
Was expecting one of:
"}" ...
"c_dv_quantity" ...
<V_TYPE_IDENTIFIER> "<" ...
<V_TYPE_IDENTIFIER> <V_LOCAL_TERM_CODE_REF> ...
<V_TYPE_IDENTIFIER> "occurrences" ...
<V_TYPE_IDENTIFIER> <SYM_MATCHES> ...
"use_node" ...
<SYM_ALLOW_ARCHETYPE> ...
<V_TERMINOLOGY_ID_BLOCK> ...
<V_CODE_PHRASE> ...
"+" ...
"-" ...
<V_INTEGER> ...
...

---

## Post #18 by @system

> this is probably completely obvious, but just in case\.\.\.
>
> Obviously I don&#39;t know the Java code, but the Eiffel and \.Net code I think
> does the same thing: when a node object \(e\.g\. a C\_OBJECT of some kind\) is
> created and attached to its parent \(a C\_ATTRIBUTE, except for the root
> object\), the normal pattern is \(in pseudo\-code\):
>
> in CAttribute:
>
> addChild \(CObject anObjNode\) \{
> &nbsp;&nbsp;&nbsp;&nbsp;children\.append \(anObjNode\);
> &nbsp;&nbsp;&nbsp;&nbsp;anObjNode\.setParent \(this\);
> \}
>
> in CObject:
>
> setParent \(CAttribute anAttrNode\) \{
> &nbsp;&nbsp;&nbsp;&nbsp;parent := anAttrNode;
> \}
>
> this setParent routine should only be visible to CAttribute
>
> There is the same pattern in the other sense, to enable CAttribute to set a
> parent CComplexObject\.
>
> \- thomas
>

Yes, we have exactly the same logic in the AOM\. Just need to connect
it to the parser\.
/Rong

---

## Post #19 by @system

Hi Pablo,
It&#39;s most likely to do with node\_id format\. The ADL parser on the
trunk doesn&#39;t really support ADL 1\.5 yet\.
Cheers,
Rong

---

## Post #20 by @pablo

Hi Rong,

I was revisiting the specs :)
I'm working with ADL 1.4, trying to implement a suggestion from Thomas: to flat archetype slots preserving the original node ids.
To do so, I need to use the archetype id and the node id to avoid node conflicts, e.g.

**source archetype:**

```
COMPOSITION[at0000] matches {	-- Medication list
  ...
  content cardinality matches {0..*; unordered} matches {
    allow_archetype ACTION occurrences matches {0..*} matches {
      include
        archetype_id/value matches {/openEHR-EHR-ACTION.medication.v1/}
```

**flat archetype:**

```
COMPOSITION[at0000] matches {	-- Medication list
  ...
  content cardinality matches {0..*; unordered} matches {
    ACTION[openEHR-EHR-ACTION.medication.v1.at0000] matches {	-- Medication action
```

The flat ADL file is generated without problems.

When I try to load the flatten archetype, the ADL couldn't be parsed. I've tried several formats:

- openEHR-EHR-ACTION.medication.v1.at0000
- openEHR-EHR-ACTION.medication.v1::at0000
- openEHR-EHR-ACTION.medication.v1_at0000
- openEHR-EHR-ACTION_medication_v1_at0000
- openEHR_EHR_ACTION_medication_v1_at0000

I couldn't find a format constraint in the spects. In the grammar page 105 here: [http://www.openehr.org/releases/1.0.2/architecture/am/adl.pdf](http://www.openehr.org/releases/1.0.2/architecture/am/adl.pdf)
There is a constraint for node ids: [a-zA-Z][a-zA-Z0-9_]* V_IDENTIFIER

(the last format I tried seems to be compliant with this constraint but I still get an exception when parsing pointing to the node with that id).

Maybe there is a different constraint for node ids implemented in the parser?

Thanks a lot!

---

## Post #21 by @pablo

Hi all, I'm coming back to the flattening problem.

I opened an issue for the null parent problem: [https://github.com/openEHR/java-libs/issues/1](https://github.com/openEHR/java-libs/issues/1)
This is not a blocker but makes code more complex since parent should be passed as param to each method that process an archetype node :-/

As for nodeIDs I will go for changing all the nodeIDs on the flattened archetype and generate a file of mappings to the original codes, this is because I can't change nodeID format in AOM java-ref-impl adding something to resolve nodeID collissions. This issue was a blocker on previous tries: [https://code.google.com/p/openehr-archetype-flattener/](https://code.google.com/p/openehr-archetype-flattener/).

New code with successful flattening algorithm will be on github soon.

---


**Canonical:** https://discourse.openehr.org/t/trying-to-flat-archetypes/15168
**Original content:** https://discourse.openehr.org/t/trying-to-flat-archetypes/15168