As we have started to get into the detail of tidying-up the AQL specification and related issues like datatype ordering and comparison rules, there has been a pretty lively discussion here about the best way of documenting the required behaviour of AQL in the context of the openEHR Reference models.
I think we now have to make a decision so that we can make tangible progress. Broadly speaking two approaches have been suggested, and debated.
Define and document the rules and behaviours directly in terms of openEHR classes , inside the current specifications e.g as part of the UML , or indeed defined directly in BMM, using somewhat abstract modelling language
Define and document the rules and behaviours as a separate part of the specification, working much more directly and in human-language terms along the lines that @Seref started here but referring, of course to the existing specifications. I think of this is a document profiling the generic AQL specification for use with openEHR RM - perhaps not the correct use of âprofileâ but we know that we have to give explicit advice about how AQL should work in openEHR CDRs.
I very strongly favour option 2, and it was my reading of the discussion that this was also the strong preference from the current CDR implementers. The sense I got was that while everyone understood some theoretical benefits from option (1) that none of the current CDR implementations do make use of BMM or intend to do so.
There are, I believe, also disadvantages in making the specifications even more difficult for those looking on from outside. Whilst it may not be important for newbies to understand the AQL spec, we should at least try to make it accessible for those coming into our world. BMM may well be where we head (particularly in the tooling space) but it is not what most CDR implementers seem to need right now.
I think we need to decide - can we get some kind of consensus / decision here?
I agree, with the remark that 2.) does not exclude the 1.), so that it can come at a later stage, and might be still beneficial for some of the community.
There still seems to be some confusion here. Including openEHR RM specific semantics in the definition of AQL (i.e. the spec) is something we absolutely should not do.
This is not predicated in any way upon using BMM. BMM is just one meta-model method to implement AQL so that it its semantics are correct. There are other ways, including hard-wired libs etc. None of this should be visible in the AQL spec.
I accept that there should be an rm neutral expression of aql. But we also badly need to pin down how aql should behave in the context of openehr. The latter cannot be derived automatically from the former.
The only point at issue is how we document expected behaviour for aql in the context of openehr ref models. You can assume that the separate model neutral aql spec will exist separately. Two documents required.
Still donât really see the problem, at least not if we are talking about how âCONTAINSâ works. Because all that is needed is that an AQL processor has some way (doesnât even have to be specified, for now) of determining which relationships in an underlying information model are logically âcompositionâ. Nothing needs to be mentioned about openEHR anywhere at specification level.
That is not the message that was coming from the implementers that responded (most of them). There is sufficient wiggle-room in the high-level AQL spec that they see the need to be explicit about how e.g CONTAINS works in openEHR - Seref has already pointed out that EHR CONTAINS COMPOSITION is quite different (practically) from COMPOSITION CONTAINS and there are many other scenarios when the correct (or simply agreed) behaviours cannot be reliably devined from a generic AQL spec - just look at Serefâs teasers. The responses have all been different from experienced implementers and no-one has really argued that the alternative viewpoints are actually wrong in principle. So the behaviour is legitimately open to interpretation but people have an appetite to reduce or eliminate that variation (for openEHR). The generic AQL spec stands, but from my reading of the discussion it is not sufficient to direct the consistent behaviour that implementers want to agree.
With respect to CONTAINS, Iâm afraid this is in error. There is no need for any reference to any particular RM for AQL semantics to be specified. What can be generically specified is that CONTAINS can be used both over direct (by-value) model associations and also over indirect (by-id) model associations, where logical containment is defined in a model. For the latter case, an AQL processor would have to have a method provided of resolving an id to a target object. None of this is specific to openEHR RM; it can exist in any model, and is routine in modelling.
The teasers are something else - they show that people have different ideas of what the current projection semantics are, because a) they are not yet formally defined and b) they are not necessarily intuitive such that everyone who does the thought experiment gets the same answer. What the semantics should actually be is a question, and @Seref has thought more about this than anyone else in his PhD work. I expect the final semantics we define will be based on that work or at least heavily informed by it. But that has nothing to do with the containment question, or openEHR RM in particular.
The spec certainly needs a lot of work on the semantics to be done. It doesnât need to refer to any particular RM to achieve this.
Iâm afraid that is not the message I am getting from the implementers( I may be wrong).
I asked the question and have been told repeatedly that they want to frame further documentation directly in the context of the openEHR RM. That is the work that Seref suggested at the top of the other thread, and was supported by all of those who have ,or are intending to, implement AQL on openEHR. I am not aware of anyone currently implementing AQL on other models.
We are wasting time now IMO. @bna@Seref@matijap - please express a preference of the options above so we can actually do this work in whatever mode is preferred. Or restate the options if I am not expressing these correctly.
My position has been clear from the beginning but happy to repeat:
I think AQL is a query language specific to openEHR RM, with a potential, but not necessary extension to openEHR demographics.
Its behaviour should be defined in reference to RM types and structure implied by those types. I donât want to use a meta model for this definition, but instead describe the expected behaviour using RM types (EHR, COMPOSITION, âŚ)
I gave my reasons for my position many times, so I wonât do that again.
Again, there is no need for any reference from AQL to any particular reference model. Thatâs not an opinion, itâs a formal fact. Iâm not sure why this is not clear. The idea of logical containment being realised in different ways (direct reference, id-reference) is absolutely standard in IT, itâs not specific to any particular model. There is no reason we would not make AQL work for demographics or any other model. The semantics are identical.
If current implementers want to only make it work for the openEHR RM, thatâs fine - I might do the same practically. But thatâs an implementation question, not a specification question.
I donât know how to make this any clearer. AQL (or EQL, as it was when we published it in 2007) was always intended as a general language. Thereâs no need to make otherwise now.
âIf current implementers want to only make it work for the openEHR RM, thatâs fine - I might do the same practically. But thatâs an implementation question, not a specification question.â
It is an openEHR specification question - we need to have this to be able to do conformance testing and cross-vendor querying. If people see it as part of ITS, Iâm happy but we need it, in the openEHR specs.
To perform conformance testing of AQL implementations, we need to know that the AQL implem knows how to correctly process the CONTAINS statement w.r.t. an underlying model - any model. Along with a lot of other semantics implied by Serefâs teasers - none of which are specific to any model.
No doubt for the moment, we will specify actual conformance test sets and results using openEHR RM. But later on we will use openEHR demographics. And later, something else, probably TP queries and structures. Specific tests & regression results can certainly be specific to a model; the spec of the language semantics they are testing cannot.
That doesnât make sense to me. There is nothing special about EHR or COMPOSITION. What we have to do is specify AQL containment semantics (which is what we are specifically dicsussing here) in terms of the kinds of relationships they may apply to, viz: direct-reference and indirect-reference. Indirect references need a resolution mechanism, but that can be assumed to be there in the data access layer anyway, since it is always needed to make the system function properly anyway. There is no need to mention any particular model or particular classes from that model (you might use them as examples, which is another thing entirely).
Which may be true in theory (I am not qualified to comment) but what I am hearing very consistently from implemneters is that there is no appetite to do it that way. They want to work on a common set of rules based on known experience with how AQL can be and has been applied to the openEHR RM, not to work at an abstract âhow any modelâ should work layer.
These are the people who have ot make this work, I really think we need to start listening to them. Ultimately openEHR will not survive without their support in kind and in fees.
We can carry on this argument indefinitely but most of us have real projects and customers who expect us to deliver on the promise of vendor-neutral querying.
I have not seen any statements to the effect that AQL should be specified in terms of one particular information model, other than by @Seref . I doubt very much whether that is the consensus. If it were, Iâm afraid it doesnât make it any more correct. It also would complicate, not simplify the AQL specification. I have no idea why anyone would want to substitute complexity and unclarity for the opposite.
Doing things properly, clearly and simply is the aim we should always have - this is what makes implementation easier, and reduces long-term maintenance costs. Itâs why archetypes work. Half-baked hacking we can leave to SDOs and other orgs who donât want to do their homework - with that approach you get the huge compendium of impossible-to-integrate special case junk like this.
Doing AQL properly is not in any way a theoretical consideration, itâs how normal, real-world engineering is done.
Doing things properly in a shared platform development is really the only option. Hacking just leads to non-reusability, fragility, lack of maintainability and is contrary to commercial interests, let alone the interests of correctness (i.e. clinical safety).
We need a good (i.e. understandable and strict) human-readable specification so that most questions like the ones that @bna provides a constant stream (and now he revealed why ) can be answered simply by pointing out a sentence or paragraph in the specification (that can hopefully be interpreted only in one way).
Current use of AQL is limited to query EHR RM based data. I agree with @matijap that we need some clarifications in text which covers the use-cases that customers or clients will face. If we some time in the future will do more work on DEMOGRAPHICS or TASKPLANNING then we may add text to clarify such use-cases. I think @sebastian.iancu will provide some good use-cases for DEMOGRAPHICS and we will eager to learn about their experiences.
I have asked Birger for his views from an ehrBase perspective , as he is not a SEC member, and has said much the same thing.
No-one is hacking
Doing AQL properly is not in any way a theoretical consideration, itâs how normal, real-world engineering is done.
but it is not how all of the current successful implementations have been done, and I think you need to be careful of suggesting that the current CDRs are not doing ânormal, real-world engineeringâ. They may not be doing it the way you think it might imagine it could or should be done âproperlyâ but that is not how the current CDR vendors have successfully implemented âreal-world engineeringâ, or how they wish to go forward.
Doing things properly in a shared platform development is really the only option. Hacking just leads to non-reusability, fragility, lack of maintainability and is contrary to commercial interests, let alone the interests of correctness (i.e. clinical safety).
@Thomas I think you need to choose your words more carefully. You can justifiably accuse me of having a hacker mindset but not others here, who have real-world successful deployments and a desire to push forward, just not necessarily in the manner you have proposed.
I have not said any such thing. On the contrary, they are all quite obviously doing real-world engineering, with all the compromises that entails. I am after all, an actual engineer, I know engineering when I see it.
What I am talking about is how we (in the SEC) perform the specification work properly. Hacking is what we observe in the SDO world, and in the majority of application development in IT in general. Everyone knows that most software is bad. Most standards (in e-health at least) are bad too.
All I am saying is that we should not start âhackingâ at the specification level. If we donât do specifications properly, we are done. We get incomprehensible junk. The industry is already drowning in that (I get paid to study it). Iâm not interested in creating more.
I am not accusing anyone of hacking, on the contrary, the openEHR community has been characterised by strategic, long-term thinking, and has embraced good engineering. I am just saying that we (collectively) should not start down that course.
On the question whether there is a clear consensus that the following is true:
we should make the AQL specification specifically dependent upon the openEHR RM, rather than being general (as it now is);
we should do this even though it is perfectly possible to specify it generically;
if we change the RM, then it is OK for the AQL spec to break (the logical consequence of the above);
if we expect to want similar querying on e.g. demographics, then we are up for writing another query language specification;
we intend that no general-purpose AQL processor be implementable.
I have not heard anything like a clear consensus, or even clear discussion on the above from implementers. That is surely yet to come.
Leaving your view of my suggestions aside, people who took the time to respond to last few weeksâ discussions deserve more respect than that.
But then again, is there a point?
Iâm standing somewhere in the middle ground here, actually.
I agree with @thomas.beale (and therefore disagree with @Seref) that AQL is a generic language with generic execution rules that can be applied over any model. I agree that we should not add arbitrary openEHR-RM-related execution rules to it, like âCONTAINS INSTRUCTION should behave in a slightly different manner than CONTAINS OBSERVATIONâ just because that would be convenient in one special case.
However, where we could and indeed should (and will) define such exceptions (like âdata from VERSIONs whose lifecycle state is âincompleteâ is not returned unless the v/lifecycle_state attribute is directly referenced in any part of the queryâ), is a document that describes behaviour of AQL when invoked in CDRs over openEHR RM-based data. I think such a document must exist if we donât want to lose user base due to even-steeper-than-need-be learning curve due to (over)abstractness of the standard. I will even argue that composing such a document, while formally unnecessary and incomplete, has a priority over the abstract AQL specification.
Like I stated earlier, I understand Thomasâ argument that if you have properly formally specified model (RM) and query language (AQL), they can be specified independently and everything will just snap together. The problem is, it wonât: weâll find a lot of behaviours we wonât like for practical reasons, and then weâll patch up one or the other spec. Iâd prefer a bottom-up approach where we clarify border-cases of AQL on openEHR RM first, and then infer the rules of one and the other after the fact. And there will be some leftover rules that we will not want to include in either, and that will be âImplementation guidelines for AQL in openEHR CDRâ or something like that â the rule with incomplete versions might fall into this category.
) can be answered simply by pointing out a sentence or paragraph in the specification (that can hopefully be interpreted only in one way).