# Cyclic datatypes: OpenEHR virus **Category:** [Technical (archive)](https://discourse.openehr.org/c/technical-archive/156) **Created:** 2014-05-12 14:07 UTC **Views:** 1 **Replies:** 49 **URL:** https://discourse.openehr.org/t/cyclic-datatypes-openehr-virus/13141 --- ## Post #1 by @system Hi, I found a peculiarity which causes me some trouble\. Not that my trouble is a problem, I can solve that, but not without breaking some rules, and the solutions is quite arbitrarily\. The solution is to check if there is any cyclic recursive going on and break at a certain arbitrary moment\. But it is not a nice solution\. How many times do we see an archetype with an ELEMENT with DV\_TEXT matches \{\*\} in it? So, there are almost no constraints at all on that value\. Condition: The validator always needs to check the parents of a node to find its \(parents\) attributes, because, the parents\-attributes are legal attributes\. So, in a non\-constrained DV\_CODED\_TEXT, the attributes of DV\_TEXT are valid\. In DV\_TEXT a legal attribute is mappings\->TERM\_MAPPING \(because there are no constraints defined\), it is legal to use the mappings\-attribute\. In TERM\_MAPPING, there is attribute: purpose\-> DV\_CODED\_TEXT, DV\_CODED\_TEXT inherits from DV\_TEXT, and there is our cycle\. So it is possible to bring every OpenEHR\-kernel to its knees, and crash the system if this is the case\. This is a situation which can of course be triggered by an evil person\. But mere likely, by an automated feeding system which breaks no rules\. I wonder, shouldn't it be necessary to have something in the Reference Model to avoid this situation? Thanks for any suggestion\. Bert --- ## Post #2 by @Tim_Cook It is an implementation issue, not a modelling issue. --- ## Post #3 by @yampeku Well, isn't that the same case that having nesting Sections or Clusters? I think the way specialization is made in openEHR makes this a similar issue\. However, in your original case I cannot imagine the use case \(or how would work\) the automated feeding system you propose\. I would say that instances have fixed size, so even if you have to process tons of cycles you can assure the process will end \(the instances are finite\)\. But probably I just don't understand well how the supposed malicious system works --- ## Post #4 by @system Of course, like I wrote, you can interrupt the data-entry if you suspect recursion. But then you implement against the rules. --- ## Post #5 by @system Yes, or Cluster which has the ITEM attribute, which allows again a CLUSTER\. But you don't see many times: CLUSTER matches \{\*\}, but you see a lot: DV\_TEXT matches \{\*\} In my original DV\_TEXT case, there isn't really a legitimate use case, it would be a feeding systems with an error\. The problem is, when and how does the receiving kernel automated decides it is an error? In you example, it is even harder for the receiving kernel to decide when to interrupt, because, nesting sometimes can go deep, but as said, that situation will not occur often in archetypes, but it is legal\. Anyway, interrupting is against the rules of the Reference Model\. I found the problem in a test\-case in where I was experimenting how to validate a wildcarded DV\_TEXT according the rules\. It occurs that a 100% good validation of a wildcarded DV\_TEXT is not possible, because then recursion is permitted\. I think this is a shortcoming in the Reference Model\. Bert --- ## Post #6 by @thomas.beale Hi Bert, > Hi, > > I found a peculiarity which causes me some trouble\. Not that my trouble is a problem, I can solve that, but not without breaking some rules, and the solutions is quite arbitrarily\. > The solution is to check if there is any cyclic recursive going on and break at a certain arbitrary moment\. But it is not a nice solution\. > > How many times do we see an archetype with an ELEMENT with DV\_TEXT matches \{\*\} in it? BTW, in ADL 1\.5, this is no longer used, the constraint could just be something like attr matches \{      DV\_TEXT \} if there are no other constraints at all\. The semantics are the same however, so it doesn't change your question\. > So, there are almost no constraints at all on that value\. > > Condition: The validator always needs to check the parents of a node to find its \(parents\) attributes, because, the parents\-attributes are legal attributes\. > So, in a non\-constrained DV\_CODED\_TEXT, the attributes of DV\_TEXT are valid\. the right way to see this is that the validator needs to be using the \(inheritance\-\)flattened form of the node, as is available in the flattened template\. > In DV\_TEXT a legal attribute is mappings\->TERM\_MAPPING \(because there are no constraints defined\), it is legal to use the mappings\-attribute\. > In TERM\_MAPPING, there is attribute: purpose\-> DV\_CODED\_TEXT, DV\_CODED\_TEXT inherits from DV\_TEXT, and there is our cycle\. I don't see the problem here; the DV\_CODED\_TEXT of the TERM\_MAPPING\.purpose is always a different instance from the root DV\_TEXT or DV\_CODED\_TEXT instance\. So how can a loop occur? Are you saying that it could due to buggy software or data? But that's the same possibility for any data processing framework where a type T has a property p of type V with a property q of type T\.\.\. I have to agree with Tim here, this is probably just about implementation quality\. \- thomas --- ## Post #7 by @grahamegrieve there's usually edge cases in validation where you can stack crash if you're not really careful (been there many times). That doesn't mean that the specs are wrong, but if you can clearly describe the case, it might be worth documenting the risks around it Grahame --- ## Post #8 by @ANASTASIOU_A Hello everyone If it is any more help, here is an earlier discussion on cyclic references: http://lists.openehr.org/pipermail/openehr-technical_lists.openehr.org/2012q2/007015.html I think that the ADL 1\.5 modifications took care of this discrepancy at the model level as well\. All the best Athanasios Anastasiou --- ## Post #9 by @system Thomas Beale schreef op 12\-5\-2014 17:25: > I don't see the problem here; the DV\_CODED\_TEXT of the TERM\_MAPPING\.purpose is always a different instance from the root DV\_TEXT or DV\_CODED\_TEXT instance\. So how can a loop occur? What I was doing was writing validation\-rules for: DV\_TEXT matches\{\*\} I am working on the finishing part of software to write validation\-rules automated, archetypes are translated to validation\-rules, and I am doing the last bits, so I came to this which I had saved in a TODO list\. I had a stack\-overflow, and first I thought it was a bug, but after investigating, I found, it was as designed\. For this situation, I had to write a rule for attribute: mappings, which can be used, because there is no constraint\. And I wanted to validate the expression completely, so every possible attribute had to be handled, with occurrence as defined in the XML\-Schema\. Attribute: mappings is optional, so it is allowed\. Every attribute that is not a simple type, but a complex OpenEHR\-type needs to be treated the same way \(recursive\), so in the mappings\-attribute, there is the purpose\-attribute which again can have a mappings\-attribute, which again can have purpose\-attribute, and so on\. A data\-set which would look like that recursive situation would still match inside the archetype\-definition\. Even if this would repeat ten times inside that data\-set, it would still be matching against the archetype\. I admit that the problem is a theoretical one, and I suggested an automated feeding system, which could create that to make it less theoretical\. I have seen systems which go to every detail and every bit, thinkable, automated systems sometimes go very deep\. The problem is, how can validation software distinguish erroneous nesting from legitimate nesting\. I don't think that is possible, so the validating software has to stop at a certain arbitrary level of depth\. At a certain point, the validating software will mark a part of a data\-set as erroneous: "too deeply nested", even if it still fits inside the archetype Then I remembered a teacher from years ago, he said: Don't write perfect software, write feasible software But OK, thank you all for your reply's, I am now convinced that it is not a 100% solvable problem\. best regards Bert --- ## Post #10 by @pablo If the value is not constrained, the validator should return true without continuing checking in cascade\-recursive mode\. For this to work as expected, the data structure should be validated before than the data validation\. The easiest way of validating the structure is serializing the instance to XML and using XSD\. --- ## Post #11 by @system That is the problem, I do not agree, it has to check in cascade because there can be required properties left out, or fantasized properties which make no sense put in\. Every occurring class in a dataset needs, in my opinion, to be validated, if there are no constraints, against the Reference Model\-rules\. By the way, you cannot validate OpenEHR datasets against an archetype by using XSD\. You cannot create XSD's according archetype\-constraints, not even by hand\. I have been there, a few years ago\. Best regards Bert --- ## Post #12 by @system You promote writing feasible software\. I agree that that is the way to go\. Bert --- ## Post #13 by @system Yes, I agree with that, it just happened yesterday that I was wondering if it was something in the specs, but now I agree that specs cannot exclude every danger\. Bert --- ## Post #14 by @thomas.beale > Thomas Beale schreef op 12\-5\-2014 17:25: >> I don't see the problem here; the DV\_CODED\_TEXT of the TERM\_MAPPING\.purpose is always a different instance from the root DV\_TEXT or DV\_CODED\_TEXT instance\. So how can a loop occur? > > What I was doing was writing validation\-rules for: DV\_TEXT matches\{\*\} > I am working on the finishing part of software to write validation\-rules automated, archetypes are translated to validation\-rules, and I am doing the last bits, so I came to this which I had saved in a TODO list\. > I had a stack\-overflow, and first I thought it was a bug, but after investigating, I found, it was as designed\. > > For this situation, I had to write a rule for attribute: mappings, which can be used, because there is no constraint\. > And I wanted to validate the expression completely, so every possible attribute had to be handled, with occurrence as defined in the XML\-Schema\. Attribute: mappings is optional, so it is allowed\. > Every attribute that is not a simple type, but a complex OpenEHR\-type needs to be treated the same way \(recursive\), so in the mappings\-attribute, there is the purpose\-attribute which again can have a mappings\-attribute, which again can have purpose\-attribute, and so on\. > > A data\-set which would look like that recursive situation would still match inside the archetype\-definition\. > Even if this would repeat ten times inside that data\-set, it would still be matching against the archetype\. this bit is true\. Do you have such pathological data? The easiest way I think of dealing with this would be to add a counter for how many times a particular type has been traversed in any tree descent, and stop if it goes over some settable limit like 100\. > I admit that the problem is a theoretical one, and I suggested an automated feeding system, which could create that to make it less theoretical\. > I have seen systems which go to every detail and every bit, thinkable, automated systems sometimes go very deep\. > > The problem is, how can validation software distinguish erroneous nesting from legitimate nesting\. > I don't think that is possible, so the validating software has to stop at a certain arbitrary level of depth\. > At a certain point, the validating software will mark a part of a data\-set as erroneous: "too deeply nested", even if it still fits inside the archetype Agree \- set an arbitrary \(deep\) limit\. > Then I remembered a teacher from years ago, he said: Don't write perfect software, write feasible software > > But OK, thank you all for your reply's, I am now convinced that it is not a 100% solvable problem\. well it's solvable in a heuristic / ad hoc way\.\.\. \- thomas --- ## Post #15 by @yampeku Just curious, are you using schematron for validation? because you can always define recursive rules to deal with such cases --- ## Post #16 by @system > Just curious, are you using schematron for validation? because you can > always define recursive rules to deal with such cases I did not know that\. I will look into it\. Thanks for the tip Bert --- ## Post #17 by @system In an open system, or a SOA\-environment, or when selling it, one does not always have control over data being offered to a system\. Software\-system lives in an ecosystem of software\. That is why nesting needs to be controlled, I have seen systems which, because of a bug dive into a recursive loop, and spit out data and feed it to another system\. If it is an automated process, and it does in a few separated threads, your OpenEHR kernel can crash, and if it runs in a operating system with a notorious reputation regarding process\-management, even that can crash\. I have seen server systems which did not even allow to login, because there was no processor\-time left to handle the login\-process\. Thinking about precautions regarding malformed\-data\-entry in software is a good thing\. Bert --- ## Post #18 by @thomas.beale Agree with all this\. But you can't code for every possibility in existence, you still have to pick and choose, according to the feasibility criterion ;\-\) --- ## Post #19 by @pablo Hi Bert, I'll clarify because what you interpreted is not what I tried to say, but we're on the same page. > Date: Tue, 13 May 2014 08:47:35 +0200 > From: bert.verhees@rosa.nl > To: openehr-technical@lists.openehr.org > Subject: Re: Cyclic datatypes: OpenEHR virus > > > If the value is not constrained, the validator should return true without continuing checking in cascade-recursive mode. For this to work as expected, the data structure should be validated before than the data validation. The easiest way of validating the structure is serializing the instance to XML and using XSD. > > That is the problem, I do not agree, it has to check in cascade because > there can be required properties left out, or fantasized properties > which make no sense put in. Every occurring class in a dataset needs, in > my opinion, to be validated, if there are no constraints, against the > Reference Model-rules. What I meant with "structure validation" is to validate against the IM (i.e. syntactic validation), when I say "data validation" I mean to validate against archetypes (i.e. semantic validation). If the constraint over a node is "not constrained at all", so there are not required values defined by the archetype, but, there might be required values defined by the information model. > > By the way, you cannot validate OpenEHR datasets against an archetype by > using XSD. You cannot create XSD's according archetype-constraints, not > even by hand. I have been there, a few years ago. > The information model can be validated with the XSDs, because the XSDs define the IM constraints. The XSDs are not to validate against archetypes (totally agree with you), is the IM validation that validates the structure and some required fields (by the IM!). Once you receive a well formed structure (should be valid against the IM) you can validate it against archetypes. If you already checked the instance against the IM and is valid, you'll have all the required values (required by the IM), then when validating data (this is the archetype validation!) and you find a {*} constraint, why the validator should need to continue traversing the instance? Hope that helps (or at least make sense :) Kind regards, Pablo. --- ## Post #20 by @system why the validator should need to continue traversing the instance? Hi Pablo, because in the attributes are often also complex OpenEhr datatypes, so the validator needs to check these complex data types in the attributes too, and those datatypes again can have complex datatypes. In case of this example: Dv_Text matches {*} you'll need to check everything, every structure, until you reach the leaf nodes, which, in this example can be anything. Only then, you can be sure that the data set is OpenEhr compliant. The thing is that a DvText can have the attribute: mappings and then can find a the attribute: purpose, of type DvCodedText, which again can have an attribute: mappings, which can again have an attribute: purpose, etc. So, the occurrence of the leafnode can be far away, and still be compliant with the statement: DvText matches {*}, and a 100% compliant validator will need to follow al these steps. Of course this is not a normal situation, but it can happen. As said, we cannot always control incoming data sets. There maybe buggy software in the ecosystem where a kernel runs. To be safe and with feasibility in mind, a validator would need to stop validating, at some arbitrary point, although there is no error. So a validator which follow the rules for 100% is dangerous! it can crash a system. That was my point. You are right in your statement, that when a part of an archetype is wildcarded, the XSD is the place where to find the validation rules. Best regards Bert --- ## Post #21 by @pablo Hi Bert, why the validator should need to continue traversing the instance? Hi Pablo, because in the attributes are often also complex OpenEhr datatypes, so the validator needs to check these complex data types in the attributes too, and those datatypes again can have complex datatypes. In case of this example: Dv_Text matches {*} you'll need to check everything, every structure, until you reach the leaf nodes, which, in this example can be anything. Only then, you can be sure that the data set is OpenEhr compliant. That was my point :) The validation that needs to reach leaf nodes is not the archetype validation, but the IM structure validation. That has nothing to do with the open constraint {*} in the archetype. In fact, that validation can be done completely without considering the archetype. What I said about using the XSD is just one way of implementation, you can do that by code also. The thing is that a DvText can have the attribute: mappings and then can find a the attribute: purpose, of type DvCodedText, which again can have an attribute: mappings, which can again have an attribute: purpose, etc. I got it ;) So, the occurrence of the leafnode can be far away, and still be compliant with the statement: DvText matches {*}, and a 100% compliant validator will need to follow al these steps. Of course this is not a normal situation, but it can happen. As said, we cannot always control incoming data sets. There maybe buggy software in the ecosystem where a kernel runs. That really depends on implementation. Let say the system doesn't control the input, so you can receive anything, for example binary data where you expect a dv_quantity. In that case, what I proposed implicitly is to have a 2 phase validator, 1st syntactic (against the IM, yes we need to reach leaf nodes here!), 2nd semantic (IMO we can prune the validator if we reach stuff like {*}). If the 1st phase returns invalid, there's no need to execute the 2nd. If you execute the second, you'll never reach an infinite recursion because of pruning. Sorry, maybe I can't explain myself clearly, is difficult to show the on email. Maybe others can validate or deny this. To be safe and with feasibility in mind, a validator would need to stop validating, at some arbitrary point, although there is no error. So a validator which follow the rules for 100% is dangerous! it can crash a system. Having two phase validators, I don't know if there's any case that you didn't cover 100% and might get valid from invalid data or cover 100% and end with stack overflow. Finding a counter case would be enough to invalid my proposal :) That was my point. You are right in your statement, that when a part of an archetype is wildcarded, the XSD is the place where to find the validation rules. Maybe the problem is trying to validate against the archetype at first and then validate the IM. I think it should be IM 1st and AM 2nd. But of course, I may overlooked some pathological case and this might not work on 100% of the cases. Another thing that might be helpful is not to use archetypes directly, use OPTs. I learned that in the hard way. OPTs can contain the whole structure and constraints of specific compositions. So if someone specifies DV_TEXT in the OPT, my interpretation is they don't need a DV_CODED_TEXT there. Also, an OPT is all in one file, while with archetypes you have to deal with slots (argghhhh). In fact, right now I'm changing all my systems adding OPT support. Simpler to validate, simpler to query. Cheers, Pablo. Best regards Bert --- ## Post #22 by @system This, I do not agree, an archetype constraints the structure of a dataset, and it constraints the contents of a leafnode. The archetype constraints, IMHO, which class-attributes are to be used, and what/where they will lead to. Of course the archetype is modeled inside the boundaries of the reference model (possible expressed in XSD), so that is always something that has to fit. If there are no constraints in an archetype (wildcarded) then everything, which is possible in the reference model, is legal in a dataset. There is a lot of validation done by the libraries you probably use. For example, if you import XML, the Xerces, or other libraries check the XML if it contains no errors. If it is a character-stream, libraries check for illegal binary-codes, many many checks are done inside libraries. And before and after you libraries, the operating system does a lot of checking too, for example if you do nothing illegal with memory, and you have virus-scanners which sit on your network-stack. I guess half of the code on a computer, on all levels, does nothing but checking for errors. The problem with this situation is that nothing illegal happens, there is no error in the dataset, the problem can occur inside a dataset, which is fully compliant with the Reference Model. I was thinking about something like that, but I could not imagine a pattern which would handle this. Maybe you can give us some pointers. As I set, the dataset which can cause the problem is fully OpenEHR compliant, there are no invalid data involved, in that case, it would be easy to handle. I don't know why you call it pathological, as if it has to do with humans, something with freaks who want to crash systems, it can surely be the case. But more likely is a system which has a bug, and creates a faulty dataset. A "pathological" dataset can be the result of a buggy system. And you know Murphy: If a problem CAN occur, it WILL occur. This is not conforming the specs. If in an archetype a DV_TEXT is defined, a DV_CODED_TEXT is legal in the dataset. Inheritance-rules are valid in datasets. I see that you dislike slots, it is another discussion, but in my view, slots are the best way to make a data-definition flexible, extensible. But lets not distract from the original discussion theme. Maybe you can work this point out in a new thread. I would welcome discussion about slots. Best regards Bert --- ## Post #23 by @thomas.beale Both of these statements are correct I guess, but I would suggest that what Pablo is saying is usable at a practical level. We could in theory (and I remember thinking about doing this 10 eras ago) add an invariant to the DV_TEXT class of the form: context DV_TEXT mappings.for_all (m: TERM_MAPPING | m.purpose /= self) This invariant would be expressible in e.g. OCL and maybe even Schematron, and could be evaluated completely independently from the archetype level validation. Admittedly these kind of checks are not routinely built into programming languages or XML, so you have to go to some effort to implement them. But it is certainly doable, and I would say this kind of check would belong in a small library of data validation checks that are executed in various passes over the data, with the archetype checks coming later than more basic checks. I would recommend a 2 or 3 pass validator. It's tempting to try to do everything in one pass, and it may be more efficient, but it's much harder to get the logic right. - thomas --- ## Post #24 by @Tim_Cook issue\. Especially in openEHR or anywhere that you are using a DSL and there are not a existing tools to choose from that have been tested across thousands of use cases\. The implementation programming language, operating system and platform will have an impact on your decisions about allowed recursion depth\. Take a look at a Google search for "patterns for data validation"\. --- ## Post #25 by @system I have a one\-phase validator, and because an archetype is strictly hierarchical, it is easy going from the top down to the leafnodes, and at every CAttribute or CObject, validate the constraints\. I have no problem with this, except for some things, like the one we are discussing, which I solved by using a recursion\-counter, which starts counting as soon an CComplexObject has no attributes in the AOM \(then it is wildcarded\) But that is an arbitrary\-solution\. It works, but it gives an unpleasant feeling because, in fact, it is breaking in the logic\. So, just for learning\. To get rid of that unpleasant feeling, which phases would you distinguish in validating a dataset? --- ## Post #26 by @system > Precisely\. This is why I said before that it is an implementation level issue\. Especially in openEHR or anywhere that you are using a DSL and there are not a existing tools to choose from that have been tested across thousands of use cases\. I guess, you wanted to say: "there are existing tools to choose from that have been tested across thousands of use cases" \(without "not a"\) In that case, I would be interested in an example use case, one of the thousands, and according tools, I will be happy to learn from you\. > The implementation programming language, operating system and platform will have an impact on your decisions about allowed recursion depth\. So you advice to stop recursion on an arbitrary point? In that case, we have the same strategy, as I already have written a few times last days\. > Take a look at a Google search for "patterns for data validation"\. Sorry Tim, that is a too easy answer for a man of your qualities\. That is not a useful advice, everyone, even children know you can google something\. Why do you bother typing this? Bert --- ## Post #27 by @Tim_Cook It is not an answer to anything\. It is an illustration of how varied the approaches can be based on the implementation situation\. --- ## Post #28 by @system I tried googling it, of course\. I always try to find an answer by myself, before discussing it on a mailinglist\. I got 6 million hits to the question you proposed, and the first 30 were not very useful\. I would like to see an approach to one specific problem I discussed under this subject\-line\. And not that approach that I already proposed/explained myself a few times last few days ago\. Can you help me there? Thanks, Bert --- ## Post #29 by @Tim_Cook I cannot\. --- ## Post #30 by @system Dear Tim, I don't think that MLHIM has a good base, I explained to you why, in an open discussion\. More then one time\. There is no need to be angry at me, for that, it is just my opinion\. Don't take it so serious\. It is not important what I think about MLHIM\. So please stop breaking in subjects I start, with the purpose to give it a bullshit distraction\. It hinders me, for me, these discussions are serious matters, I do this for living\. I don't walk in your way\. I never do that\. Please don't walk in my way\. Just ignore me, is that so hard? Try again\. Best regards Bert Verhees --- ## Post #31 by @Tim_Cook Well, excuse the \#$% out of me\. I didn't know this was Bert's Q&A list\. I thought it was a discussion list for openEHR related technical issues\. My comments were certainly appropriate for the topic of data validation\. I did not mention MLHIM or even XML\. I frankly do not care what you think about MLHIM\. Though you are free to express your opinions\. I think that \*I\* am not the angry one here\. Kind Regards, Tim --- ## Post #32 by @pablo Just for fun: [http://en.wikipedia.org/wiki/Pathological_(mathematics)](http://en.wikipedia.org/wiki/Pathological_(mathematics)) > Maybe the problem is trying to validate against the archetype at first and then validate the IM. I think it should be IM 1st and AM 2nd. But of course, I may overlooked some pathological case and this might not work on 100% of the cases. I don't know why you call it pathological, as if it has to do with humans, something with freaks who want to crash systems, it can surely be the case. But more likely is a system which has a bug, and creates a faulty dataset. A "pathological" dataset can be the result of a buggy system. --- ## Post #33 by @system Specially this section: "awareness of pathological inputs is important as they can be exploited to mount a on a computer system" --- ## Post #34 by @system I have a one-phase validator, and because an archetype is strictly hierarchical, it is easy going from the top down to the leafnodes, and at every CAttribute or CObject, validate the constraints. I have no problem with this, except for some things, like the one we are discussing, which I solved by using a recursion-counter, which starts counting as soon an CComplexObject has no attributes in the AOM (then it is wildcarded) But that is an arbitrary-solution. It works, but it gives an unpleasant feeling because, in fact, it is breaking in the logic. So, just for learning. To get rid of that unpleasant feeling, which phases would you distinguish in validating a dataset? --- ## Post #35 by @system Sorry, sent by accident (stupid iPad) ---------- Doorgestuurde bericht ---------- --- ## Post #36 by @pablo I mentioned the phases, several times, in my previous messages :) Maybe Thomas can break that up into more phases. --- ## Post #37 by @thomas.beale I think Pablo has summarised some useful things: --- ## Post #38 by @system On 16-05-14 09:16, Thomas Beale wrote: I agree that there is no standard answer, and there will never be one. There will always be technical progress. That keeps us, developers, at work. ;-) I wish more people would discuss their technical working out of the standards, so we can learn. ---- I am not familiar with the term "OPT". I assume, this is opt-out. As Pablo gave an example, if some has a DV_TEXT in an archetype, he does not want a DV_CODED_TEXT at that point. I agree partly with this. But only if the DV_TEXT is not wildcarded. If it has an attribute mentioned in the archetype: DV_TEXT matches { value matches {*} }. Then there can come nothing but a value-attribute which is a constrained string, in this case, without constraints. (and if applicable, other required attributes also) But if in the archetype is DV_TEXT matches {*} then every attribute is allowed to use, and it is allowed to derive inheritors from DV_TEXT, which is DV_CODED_TEXT. I had this discussion some years ago, at that time you agreed that inheritance is allowed, according to the standard. Divergence from the theoretical standard on technical/practical reasons makes code in the context of that standard less flexible and less extensible and once diverged code leads to more divergence. It maybe can even lead to another standard on new premiss, of which I know an example. But as they say in Fawlty Towers: Never mention the war ;-) If I misunderstood the term OPTs, please forgive me, it was not with rhetorical intention. ------ The RM-validation pass is very easy in code. Just check it against the RM-XSD and let it roll. Maybe we are not aware, because everything happens in a library. There may be a performance issue. Will it be more efficient to check before, what you are checking anyway afterwards? Also you do not detect all pathological structures, because the one I mentioned is perfectly legal in the RM, when DV_TEXT matches {*} is in the archetype. That makes this example dangerous, it is not possible to detect by a basic RM-check. But you can find other problems, and have a quick way out. That is true. The punishment is for data-sets which are valid, they need more processor-time to get accepted. ------ I don't understand what is meant with "archetype (OPT) pass", so I cannot comment on that. ----- The one pass situation: the logical path through an archetype is very hierarchical and very easy. It is a kind of classical visitor-pattern which is followed, but, in my case, without unnecessary formalities. I have rewritten the AOM validation-interpretation three times, every time to create an XML-validation. First for XML-Schema, then for RelaxNG, both combined with Schematron. XML-schema and RelaxNG have shortcomings which are trouble in relation to the features of the AOM/ADL. Some developers have written workarounds for that. But that is divergence of the standards. Now I have rewritten it for schematron-only. But the base-structure remained the same, just the simple one-pass validation. Schematron seems to be the best way, for now. The asserts are sorted to the context, so all tests for a specific node will be done in one group. This avoids, at validation-time, repeated retrieval of nodes, by the XML-interpreter. By the way, I have a basic RM-check in my validator, I have converted the XSD's to Schematron, which is not hard to do. I use it to check for existence of required attributes, which are not in the archetype, and check for valid inheritance. But this basic RM validator runs in the same one-pass validation. Thanks. Bert --- ## Post #39 by @system For those who don't know Fawlty Towers https://www.youtube.com/watch?v=yfl6Lu3xQW0 --- ## Post #40 by @thomas.beale OPT = Operational Template \- it's the fully compiled version of a template\. See the Template Designer <http://www.openehr.org/downloads/modellingtools> for this \- it generates them\. Or else you can just do a fully flattened template in the ADL 1\.5 workbench\. I can provide details on this if you need\. \- thomas [details="(attachments)"] ![ocean\_full\_small.jpg|92x41](upload://omlkPxIt2jb2NPEgfRNrw9aCGOK.jpeg) ![btn\_liprofile\_blue\_80x15.png|80x15](upload://wvuEw46mMYRYzHqv86Wkoxe35S4.png) [/details] --- ## Post #41 by @system Yes, please do. I will read it. Until now, I ignored ADL1.5 Life is difficult enough without it ;-) But I think, time has come to read about it. Bert --- ## Post #42 by @Tim_Cook Tom gave a good intro to ADL 1.5 on the webinar Tuesday. Maybe Tom knows if they recorded it and posted to YouTube? --- ## Post #43 by @thomas.beale I don't see a link yet, but in any case, it was pretty rough, so I might do one myself and post it on Youtube. The time has come... I have re-organised the ADL wiki pages into a . See following announcement. - thomas --- ## Post #44 by @Tim_Cook Great\. May I suggest Google Hangouts? It makes it dead simple to post to YouTube\. --- ## Post #45 by @thomas.beale Tim, do you mean: run a google hang-out and capture that and post it? - thomas --- ## Post #46 by @Tim_Cook Exactly. One of the options when you start a Hangout is to enable live recording to YouTube. You can then go back and edit it if you wish. Otherwise it is streamed and recorded on YouTube. You fill in the metadata and enable publication. That is how all of the HEalthcare IT Live episodes were recorded. [https://www.youtube.com/playlist?list=PL5BDmBjSV7CsBYbzNBw-D03WEqSJcWxbP](https://www.youtube.com/playlist?list=PL5BDmBjSV7CsBYbzNBw-D03WEqSJcWxbP) As well as tutorials [https://www.youtube.com/playlist?list=PL5BDmBjSV7CvJbk68kWtywDjH25hQ6Ltg](https://www.youtube.com/playlist?list=PL5BDmBjSV7CvJbk68kWtywDjH25hQ6Ltg) --- ## Post #47 by @thomas.beale Tim, I see I have catching up to do - these are great! Do you have any kind of 'best-of' or compressed version capturing gems of wisdom? (Not that I have a problem with watching the long form, just wondering). Is there a website / wiki with an index of topics covered, questions asked or anything like that? Great work. You're the DJ for e-health ;-) - thomas --- ## Post #48 by @Tim_Cook I guess I should create some kind of index. Your the second person this week to ask where to start. For the Healthcare IT Live! videos; they are all numbered with the guests name in the title and are listed in the playlist link above. The tutorials, really was supposed to be a series about knowledge modelling but I didn't attract any openEHR or HL7 guests for it. Has a numbered line by line purpose of a MLHIM CCD. Also one video on using the CCD Generator (out of date but still pertinent). There are several other videos, in playlists by category, on the MLHIM Channel [https://www.youtube.com/user/MLHIMdotORG](https://www.youtube.com/user/MLHIMdotORG) HTH, Tim --- ## Post #49 by @thomas.beale well maybe we can take the idea forward with more participants. I had originally thought this effort was centred around only MLHIM (which we need to know more about anyway), but your approach has clearly been much more ecumenical. I'm only sorry that I did not realise earlier. But... let's think about what we can do in the future. There is a lot of useful resource here. Yep, there is a lot of good resource here. I suspect the two things that would be useful for people trying to catch up would be: --- ## Post #50 by @Tim_Cook > > well maybe we can take the idea forward with more participants\. I had > originally thought this effort was centred around only MLHIM \(which we need > to know more about anyway\), but your approach has clearly been much more > ecumenical\. I'm only sorry that I did not realise earlier\. But\.\.\. let's > think about what we can do in the future\. There is a lot of useful resource > here\. > Thanks\. We could restart the Modelling Q&A\. The Healthcare IT Live\! shows took a lot of work and planning each week\. I do not have the time right now\. It would be great to get some domain experts involved such as Bill Hersh and other clinical informaticians\. Maybe one day we will see these universities including knowledge modelling concepts in the informatics courses\. > There are several other videos, in playlists by category, on the MLHIM > Channel https://www.youtube.com/user/MLHIMdotORG > > Yep, there is a lot of good resource here\. I suspect the two things that > would be useful for people trying to catch up would be: > >    \- an MLHIM 'learning centre' page that summarises tutorial and other >    learning oriented material \(indexes to actual technical specs etc already >    exist obviously\) >    \- a more general multi\-level health modelling resources page\. Maybe we >    could host a ring of pages across more than one wiki / website\. > > I know it would take some time, but I would really like to be able to zoom > down some index and discover that Fred Trotter was talking about x, y and > z, that topic abc was talked about by Stan Huff, Grahame Grieve and whoever > else\. You get the idea\. > I still have the show notes from all of the HIT Live\! shows\. The notes are the planned questions and general topics\. I'll see if I can get time to put together an index/overview page\. Cheers, Tim --- **Canonical:** https://discourse.openehr.org/t/cyclic-datatypes-openehr-virus/13141 **Original content:** https://discourse.openehr.org/t/cyclic-datatypes-openehr-virus/13141