# AQL (and related) documentation **Category:** [AQL](https://discourse.openehr.org/c/aql/43) **Created:** 2020-03-03 10:33 UTC **Views:** 1089 **Replies:** 41 **URL:** https://discourse.openehr.org/t/aql-and-related-documentation/402 --- ## Post #1 by @ian.mcnicoll As we have started to get into the detail of tidying-up the AQL specification and related issues like datatype ordering and comparison rules, there has been a pretty lively discussion [here]( https://discourse.openehr.org/t/aql-formal-definition-of-from-clause/322/63) about the best way of documenting the required behaviour of AQL **in the context of the openEHR Reference models**. I think we now have to make a decision so that we can make tangible progress. Broadly speaking two approaches have been suggested, and debated. 1. Define and document the rules and behaviours directly in terms of openEHR classes , inside the current specifications e.g as part of the UML , or indeed defined directly in BMM, using somewhat abstract modelling language 2. Define and document the rules and behaviours as a separate part of the specification, working much more directly and in human-language terms along the lines that @Seref started [here](https://discourse.openehr.org/t/aql-formal-definition-of-from-clause/322?u=ian.mcnicoll) but referring, of course to the existing specifications. I think of this is a document profiling the generic AQL specification for use with openEHR RM - perhaps not the correct use of 'profile' but we know that we have to give explicit advice about how AQL should work in openEHR CDRs. I very strongly favour option 2, and it was my reading of the discussion that this was also the strong preference from the current CDR implementers. The sense I got was that while everyone understood some theoretical benefits from option (1) that none of the current CDR implementations do make use of BMM or intend to do so. There are, I believe, also disadvantages in making the specifications even more difficult for those looking on from outside. Whilst it may not be important for newbies to understand the AQL spec, we should at least try to make it accessible for those coming into our world. BMM may well be where we head (particularly in the tooling space) but it is not what most CDR implementers seem to need right now. I think we need to decide - can we get some kind of consensus / decision here? --- ## Post #2 by @sebastian.iancu I agree, with the remark that 2.) does not exclude the 1.), so that it can come at a later stage, and might be still beneficial for some of the community. --- ## Post #3 by @thomas.beale There still seems to be some confusion here. **Including openEHR RM specific semantics in the definition of AQL (i.e. the spec) is something we absolutely should not do**. This is not predicated in any way upon using BMM. BMM is just one meta-model method to *implement* AQL so that it its semantics are correct. There are other ways, including hard-wired libs etc. None of this should be visible in the AQL spec. --- ## Post #4 by @ian.mcnicoll I accept that there should be an rm neutral expression of aql. But we also badly need to pin down how aql should behave in the context of openehr. The latter cannot be derived automatically from the former. The only point at issue is how we document expected behaviour for aql in the context of openehr ref models. You can assume that the separate model neutral aql spec will exist separately. Two documents required. --- ## Post #5 by @thomas.beale Still don't really see the problem, at least not if we are talking about how 'CONTAINS' works. Because all that is needed is that an AQL processor has *some way* (doesn't even have to be specified, for now) of determining which relationships in an underlying information model are logically 'composition'. Nothing needs to be mentioned about openEHR anywhere at specification level. --- ## Post #6 by @ian.mcnicoll That is not the message that was coming from the implementers that responded (most of them). There is sufficient wiggle-room in the high-level AQL spec that they see the need to be explicit about how e.g CONTAINS works in openEHR - Seref has already pointed out that EHR CONTAINS COMPOSITION is quite different (practically) from COMPOSITION CONTAINS and there are many other scenarios when the correct (or simply agreed) behaviours cannot be reliably devined from a generic AQL spec - just look at Seref's teasers. The responses have all been different from experienced implementers and no-one has really argued that the alternative viewpoints are actually wrong in principle. So the behaviour is legitimately open to interpretation but people have an appetite to reduce or eliminate that variation (for openEHR). The generic AQL spec stands, but from my reading of the discussion it is not sufficient to direct the consistent behaviour that implementers want to agree. --- ## Post #7 by @thomas.beale [quote="ian.mcnicoll, post:6, topic:402"] There is sufficient wiggle-room in the high-level AQL spec that they see the need to be explicit about how e.g CONTAINS works in openEHR [/quote] With respect to CONTAINS, I'm afraid this is in error. There is no need for any reference to any particular RM for AQL semantics to be specified. What can be *generically specified* is that CONTAINS can be used both over direct (by-value) model associations and also over indirect (by-id) model associations, where logical containment is defined in a model. For the latter case, an AQL processor would have to have a method provided of resolving an id to a target object. None of this is specific to openEHR RM; it can exist in any model, and is routine in modelling. The teasers are something else - they show that people have different ideas of what the current projection semantics are, because a) they are not yet formally defined and b) they are not necessarily intuitive such that everyone who does the thought experiment gets the same answer. What the semantics should actually be is a question, and @Seref has thought more about this than anyone else in his PhD work. I expect the final semantics we define will be based on that work or at least heavily informed by it. But that has nothing to do with the containment question, or openEHR RM in particular. The spec certainly needs a lot of work on the semantics to be done. It doesn't need to refer to any particular RM to achieve this. --- ## Post #8 by @ian.mcnicoll [quote="thomas.beale, post:7, topic:402"] The spec certainly needs a lot of work on the semantics to be done. It doesn’t need to refer to any particular RM to achieve this. [/quote] I'm afraid that is not the message I am getting from the implementers( I may be wrong). I asked the question and have been told repeatedly that they want to frame further documentation directly in the context of the openEHR RM. That is the work that Seref suggested at the top of the other thread, and was supported by all of those who have ,or are intending to, implement AQL on openEHR. I am not aware of anyone currently implementing AQL on other models. We are wasting time now IMO. @bna @Seref @matijap - please express a preference of the options above so we can actually do this work in whatever mode is preferred. Or restate the options if I am not expressing these correctly. --- ## Post #9 by @Seref My position has been clear from the beginning but happy to repeat: I think AQL is a query language specific to openEHR RM, with a potential, but not necessary extension to openEHR demographics. Its behaviour should be defined in reference to RM types and structure implied by those types. I don't want to use a meta model for this definition, but instead describe the expected behaviour using RM types (EHR, COMPOSITION, ...) I gave my reasons for my position many times, so I won't do that again. --- ## Post #10 by @thomas.beale [quote="ian.mcnicoll, post:8, topic:402"] I’m afraid that is not the message I am getting from the implementers( I may be wrong). [/quote] Again, there is no need for any reference from AQL to any particular reference model. That's not an opinion, it's a formal fact. I'm not sure why this is not clear. The idea of logical containment being realised in different ways (direct reference, id-reference) is absolutely standard in IT, it's not specific to any particular model. There is no reason we would not make AQL work for demographics or any other model. The semantics are identical. If current implementers want to only make it work for the openEHR RM, that's fine - I might do the same practically. But that's an implementation question, not a *specification* question. I don't know how to make this any clearer. AQL (or EQL, as it was when we published it in 2007) was always intended as a general language. There's no need to make otherwise now. --- ## Post #11 by @ian.mcnicoll "If current implementers want to only make it work for the openEHR RM, that’s fine - I might do the same practically. But that’s an implementation question, not a *specification* question." It is an **openEHR specification** question - we need to have this to be able to do conformance testing and cross-vendor querying. If people see it as part of ITS, I'm happy but we need it, in the openEHR specs. --- ## Post #12 by @thomas.beale To perform conformance testing of AQL implementations, we need to know that the AQL implem knows how to correctly process the CONTAINS statement w.r.t. an underlying model - *any model*. Along with a lot of other semantics implied by Seref's teasers - *none of which are specific to any model*. No doubt for the moment, we will specify actual conformance test sets and results using openEHR RM. But later on we will use openEHR demographics. And later, something else, probably TP queries and structures. Specific tests & regression results can certainly be specific to a model; the spec of the language semantics they are testing cannot. --- ## Post #13 by @thomas.beale [quote="Seref, post:9, topic:402"] Its behaviour should be defined in reference to RM types and structure implied by those types. I don’t want to use a meta model for this definition, but instead describe the expected behaviour using RM types (EHR, COMPOSITION, …) [/quote] That doesn't make sense to me. There is nothing special about EHR or COMPOSITION. What we have to do is specify AQL containment semantics (which is what we are specifically dicsussing here) in terms of the *kinds* of relationships they may apply to, viz: direct-reference and indirect-reference. Indirect references need a resolution mechanism, but that can be assumed to be there in the data access layer anyway, since it is always needed to make the system function properly anyway. There is no need to mention any particular model or particular classes from that model (you might use them as examples, which is another thing entirely). --- ## Post #14 by @ian.mcnicoll [quote="thomas.beale, post:12, topic:402"] To perform conformance testing of AQL implementations, we need to know that the AQL implem knows how to correctly process the CONTAINS statement w.r.t. an underlying model - *any model* . Along with a lot of other semantics implied by Seref’s teasers - *none of which are specific to any model* . [/quote] Which may be true in theory (I am not qualified to comment) but what I am hearing very consistently from implemneters is that there is no appetite to do it that way. They want to work on a common set of rules based on known experience with how AQL can be and has been applied to the openEHR RM, not to work at an abstract 'how any model' should work layer. These are the people who have ot make this work, I really think we need to start listening to them. Ultimately openEHR will not survive without their support in kind and in fees. We can carry on this argument indefinitely but most of us have real projects and customers who expect us to deliver on the promise of vendor-neutral querying. --- ## Post #15 by @thomas.beale [quote="ian.mcnicoll, post:14, topic:402"] Which may be true in theory (I am not qualified to comment) but what I am hearing very consistently from implemneters is that there is no appetite to do it that way. [/quote] I have not seen any statements to the effect that AQL should be specified in terms of one particular information model, other than by @Seref . I doubt very much whether that is the consensus. If it were, I'm afraid it doesn't make it any more correct. It also would complicate, not simplify the AQL specification. I have no idea why anyone would want to substitute complexity and unclarity for the opposite. Doing things properly, clearly and simply is the aim we should always have - this is what makes implementation *easier*, and reduces long-term maintenance costs. It's why archetypes work. Half-baked hacking we can leave to SDOs and other orgs who don't want to do their homework - with that approach you get the huge compendium of impossible-to-integrate special case junk [like this](https://www.healthit.gov/isa/). Doing AQL properly is not in any way a theoretical consideration, it's how normal, real-world engineering is done. Doing things properly in a shared platform development is really the only option. Hacking just leads to non-reusability, fragility, lack of maintainability and is contrary to commercial interests, let alone the interests of correctness (i.e. clinical safety). --- ## Post #16 by @ian.mcnicoll [quote="thomas.beale, post:15, topic:402"] I have not seen any statements to the effect that AQL should be specified in terms of one particular information model, other than by @Seref . I doubt very much whether that is the consensus. [/quote] From the other thread @Matija > We need a good (i.e. understandable and strict) human-readable specification so that most questions like the ones that @bna provides a constant stream (and now he revealed why :slight_smile: ) can be answered simply by pointing out a sentence or paragraph in the specification (that can hopefully be interpreted only in one way). @bna > Current use of AQL is limited to query EHR RM based data. I agree with @matijap that we need some clarifications in text which covers the use-cases that customers or clients will face. If we some time in the future will do more work on DEMOGRAPHICS or TASKPLANNING then we may add text to clarify such use-cases. I think @sebastian.iancu will provide some good use-cases for DEMOGRAPHICS and we will eager to learn about their experiences. I have asked Birger for his views from an ehrBase perspective , as he is not a SEC member, and has said much the same thing. No-one is hacking [quote="thomas.beale, post:15, topic:402"] Doing things properly in a shared platform development is really the only option. Hacking just leads to non-reusability, fragility, lack of maintainability and is contrary to commercial interests, let alone the interests of correctness (i.e. clinical safety). [/quote] > Doing AQL properly is not in any way a theoretical consideration, it’s how normal, real-world engineering is done. but it is not how all of the current successful implementations have been done, and I think you need to be careful of suggesting that the current CDRs are not doing 'normal, real-world engineering'. They may not be doing it the way you think it might imagine it could or should be done 'properly' but that is not how the current CDR vendors have successfully implemented 'real-world engineering', or how they wish to go forward. > Doing things properly in a shared platform development is really the only option. Hacking just leads to non-reusability, fragility, lack of maintainability and is contrary to commercial interests, let alone the interests of correctness (i.e. clinical safety). @Thomas I think you need to choose your words more carefully. You can justifiably accuse me of having a hacker mindset but not others here, who have real-world successful deployments and a desire to push forward, just not necessarily in the manner you have proposed. --- ## Post #17 by @thomas.beale [quote="ian.mcnicoll, post:16, topic:402"] I think you need to be careful of suggesting that the current CDRs are not doing ‘normal, real-world engineering’. [/quote] I have not said any such thing. On the contrary, they are all quite obviously doing real-world engineering, with all the compromises that entails. I am after all, an actual engineer, I know engineering when I see it. What I am talking about is how *we (in the SEC) perform the specification work properly*. Hacking is what we observe in the SDO world, and in the majority of application development in IT in general. Everyone knows that most software is bad. Most standards (in e-health at least) are bad too. All I am saying is that we should not start 'hacking' at the specification level. If we don't do specifications properly, we are done. We get incomprehensible junk. The industry is already drowning in that (I get paid to study it). I'm not interested in creating more. I am not accusing anyone of hacking, on the contrary, the openEHR community has been characterised by strategic, long-term thinking, and has embraced good engineering. I am just saying that we (collectively) should not start down that course. On the question whether there is a clear consensus that the following is true: * we should make the AQL specification specifically dependent upon the openEHR RM, rather than being general (as it now is); * we should do this even though it is perfectly possible to specify it generically; * if we change the RM, then it is OK for the AQL spec to break (the logical consequence of the above); * if we expect to want similar querying on e.g. demographics, then we are up for writing another query language specification; * we intend that no general-purpose AQL processor be implementable. I have not heard anything like a clear consensus, or even clear discussion on the above from implementers. That is surely yet to come. --- ## Post #18 by @Seref [quote="thomas.beale, post:17, topic:402"] I have not heard anything like a clear consensus, or even clear discussion on the above from implementers [/quote] Leaving your view of my suggestions aside, people who took the time to respond to last few weeks' discussions deserve more respect than that. But then again, is there a point? [quote="thomas.beale, post:15, topic:402"] whether that is the consensus. If it were, I’m afraid it doesn’t make it any more correct [/quote] --- ## Post #19 by @matijap I'm standing somewhere in the middle ground here, actually. I agree with @thomas.beale (and therefore disagree with @Seref) that AQL is a generic language with generic execution rules that can be applied over any model. I agree that we should not add arbitrary openEHR-RM-related execution rules to it, like "CONTAINS INSTRUCTION should behave in a slightly different manner than CONTAINS OBSERVATION" just because that would be convenient in one special case. However, where we could and indeed _should_ (and will) define such exceptions (like "data from VERSIONs whose lifecycle state is 'incomplete' is not returned unless the v/lifecycle_state attribute is directly referenced in any part of the query"), is a document that describes behaviour of AQL when invoked in CDRs over openEHR RM-based data. I think such a document must exist if we don't want to lose user base due to even-steeper-than-need-be learning curve due to (over)abstractness of the standard. I will even argue that composing such a document, while formally unnecessary and incomplete, has a priority over the abstract AQL specification. Like I stated earlier, I understand Thomas' argument that if you have properly formally specified model (RM) and query language (AQL), they can be specified independently and everything will just snap together. The problem is, it won't: we'll find a lot of behaviours we won't like for practical reasons, and then we'll patch up one or the other spec. I'd prefer a bottom-up approach where we clarify border-cases of AQL on openEHR RM first, and then infer the rules of one and the other after the fact. And there will be some leftover rules that we will not want to include in either, and that will be "Implementation guidelines for AQL in openEHR CDR" or something like that -- the rule with incomplete versions might fall into this category. --- ## Post #20 by @Seref [quote="matijap, post:19, topic:402"] I agree that we should not add arbitrary openEHR-RM-related execution rules to it, like “CONTAINS INSTRUCTION should behave in a slightly different manner than CONTAINS OBSERVATION” [/quote] this was never suggested. just for the record. --- ## Post #21 by @thomas.beale [quote="matijap, post:19, topic:402"] (like “data from VERSIONs whose lifecycle state is ‘incomplete’ is not returned unless the v/lifecycle_state attribute is directly referenced in any part of the query”), [/quote] well, things like this we have discussed before, but they belong in the realm of semantic interpretations of the data, not the mechanical act of getting it out of the persistent store. So further filtering or other smart behaviours based on the content / values has to be in a layer above, which today no-one has, and for which we have not even a draft spec. From @ian.mcnicoll's point of view, these kinds of things are burning clinical safety priorities (I agree); it's just that you can't achieve all levels of semantics in the one layer of functionality. Now, you could achieve some (all?) of these kinds of things with rules, e.g. a rule that states something about draft VERSIONs. What AQL would then need is a *generalised notion of such rules*, so that any such rule could be written. These rules can be simple to start with. But it would be wrong (again) to embed in the spec statements like 'if the data model of the query is openEHR RM, and the data item is inside an instance of VERSION class that has draft=true, then don't return the object' or whatever. Instead, the AQL spec should have a general capability to post process (or maybe inline process) some rules that might knock out certain query results if a general pattern is matched (e.g. the content was inside some container considered 'incomplete' or 'draft'). The notion of draft or incomplete is a general one; how openEHR RM or any other model marks its content as draft or incomplete is a model-specific thing. So somewhere in an the query processing process, a 'remove draft data' filter is activated, and it will have to look up the appropriate rule that that do this for the data at hand. These sorts of things are somewhat sophisticated, but never have to be know by query authors, only query engine implementers (probably < 10 people in the whole world). For authors, an engine that knows how to remove or include 'draft' info on a switch just does so magically. [quote="matijap, post:19, topic:402"] I’d prefer a bottom-up approach where we clarify border-cases of AQL on openEHR RM first, and then infer the rules of one and the other after the fact. [/quote] Right. As we infer the general rules / semantics, we can put them into the AQL spec. While we don't know what the general case is, we don't put anything (certainly not specific references to particular models or classes). In general, I think any shared specification should only be the general case 'rules' (structures, whatever) that have been inferred by study of specific systems and contemplation of known theory. In other words, normal scientific method. A specification (i.e. a language, model etc) is just today's version of a general theory about some topic. So the bottom up approach here is that the AQL spec is anaemic on a lot of semantics, but that with the experiences to date, we can slowly start determining the general formulation of various features (CONTAINS, projections, draft content inclusion ...) and when we agree on them, we add them in. Before we have them, the community is still experimenting, and we don't know what the general formulation that can be put into the spec is. Again, normal experimental science approach. --- ## Post #22 by @matijap Like @sebastian.iancu said, the two approaches are not exclusive. However, as (a representative of) an implementor, I see much more value in a document that is a slight mix of query language and RM: definitely I'd like to see openEHR RM classes in the examples in the AQL spec, and most probably also in clarification sentences, for example while CONTAINS would be defined in abstract terms, there should also be a clarification that an example of indirect reference is between EHR and COMPOSITION, and an example of direct reference is between COMPOSITION and SECTION. This requires just a note on top stating that most (all?) examples and clarifications refer to the openEHR RM. As for the special processing cases (like the one with version lifecycle) I'm open for discussion, but for simplicity's sake I'd just put them into another section in the same document, and also put a note in the lifecycle_state attribute description about the querying exception (yes, AQL would be referenced informally in the model specification). --- ## Post #23 by @thomas.beale [quote="matijap, post:22, topic:402"] CONTAINS would be defined in abstract terms, there should also be a clarification that an example of indirect reference is between EHR and COMPOSITION, and an example of direct reference is between COMPOSITION and SECTION. This requires just a note on top stating that most (all?) examples and clarifications refer to the openEHR RM. [/quote] Right. generic definition; concrete examples (that are familiar). [quote="matijap, post:22, topic:402"] put a note in the lifecycle_state attribute description about the querying exception (yes, AQL would be referenced informally in the model specification). [/quote] Agree here, after all, that is part of the semantics we want for the 'incomplete' lifecycle state - that it somehow affect querying. To be completely correct though it should not mention AQL, but just querying methods in general. That way, it correctly defines not just what happens in openEHR systems that use AQL, but also ones that do it a different way (eg. Code24, EHRserver). --- ## Post #24 by @matijap [quote="Seref, post:20, topic:402"] this was never suggested. just for the record. [/quote] The exact notion about CONTAINS was not; I made it up as a radical example. The fact about specially handling some attributes (like by default hiding incomplete versions, or hiding non-PARTY_SELF content) was discussed and if any of those are agreed upon, there's a question of where to describe them. --- ## Post #25 by @Seref I apologise, I misunderstood the nature of your disagreement with me. To further clarify things, I do not object to AQL's generic nature. As I said under the FROM semantics discussion: [quote="Seref, post:12, topic:322"] I’ve seen this point made before and I meant to respond then. AQL has the potential to be generic query language to query any underlying model but I’m in favour of defining it strictly based on openEHR terms, based on openEHR EHRs, data types, structures etc [/quote] I merely attempted to follow an approach that considers various realities of software development, but it failed for reasons which I personally find extremely disappointing. I'm OK as long as I can clarify my position with you and other vendors, that's all. --- ## Post #26 by @rong.chen Sorry to jump in at a late stage, and I won't comment on specifics since we don't implement AQL/CDR ourselves. However some of the discussions here do echo what's happened/happening in decision/process support space namely the TP and EL specifications, and now the brand new specs for DS/M. I feel we as industry members/implementers of given areas (CDR/CDS etc) are not fully involved/consulted enough before new specifications took place. As an open standards based CDS product supplier, we are very sensitive to any significant changes to GDL or related/emerging CDS design specifications in the openEHR space. These changes could raise lots of questions from our existing/potential customers especially from large national bodies/regional authorities. And it’s hard to exaggerate the burden (and business risks) potentially caused by rapid changing specs. I think other vendors share the same concern, which was noted on 8th of Jan, 2020 at our first industry members meeting: > we need to improve our efforts of governance of specifications. Where to go from here, I really think it’s important to consult key implementers/suppliers of a given technology areas before any major changes of specs in that relevant space. We need to be very careful to introduce new specs especially if there is considerable overlapping with existing ones. The SEC’s decisions undertake certain directions (change/introduce new design) must be motivated by clear business cases. --- ## Post #27 by @bna [quote="ian.mcnicoll, post:1, topic:402"] Define and document the rules and behaviours as a separate part of the specification, working much more directly and in human-language terms along the lines that @Seref started [here](https://discourse.openehr.org/t/aql-formal-definition-of-from-clause/322) but referring, of course to the existing specifications. I think of this is a document profiling the generic AQL specification for use with openEHR RM - perhaps not the correct use of ‘profile’ but we know that we have to give explicit advice about how AQL should work in openEHR CDRs. [/quote] There is IMHO the only way to make the needed progress related to AQL now. We need, as a group, to work out the semantic understanding of what we want to achieve with AQL queries. This is absolutely needed to make AQL portable. I have been the product owner for our openEHR solutions in DIPS AS since we started our openEHR journey in 2010. Our most important assets when developing our solutions is the narrative descriptions in the specifications. The narratives defines the intention and context for the implementation. This is the most important message to the developers. For every developer I have met I find they have no problem doing the technical implementation. We have not used the BMM formalism when developing the solutions. It has, so far, not been requested by any developer. The hardest part implementing openEHR is to understand e-health and how to apply a specification into the underlying persistence layer and development language used by the vendor. This is, as far as we can see, work that has to be done carefully by the development team and can not be done "automagically" based on abstract models. I assume the story might be different for modelling tools. For those you might have to adjust to difference reference models and need a abstract model as the basis for your implementation. In such use-cases the BMM might be a perfect match. We've also seen it's usage as a way to test our implementations with the different RM versions. Handling data has to be dealt with carefully with a deep understanding of your persistence model. As you all know we use Apache SOLR as our core engine for indexing the data. The last weeks we've re-implemented the core usage of SOLR. We've used lots of hours working with the details in the underlying Lucene index and the core parts of SOLR. We are cooperating with one of the committers in the Apache SOLR group on this work. We have a developer with who is Doctor in computer science leading the development. He is one of the brightest minds I have met. My intention with this writing is to explain how much effort we have to put in the details on the persistence layer go implement a CDR. I don't know how we could use any more formalism in the specification to guide or help us with this. Back to AQL: It's a great to have such a portable query language. Kudos to the people who invented it! Right now we have a few vendors with CDR's supporting AQL. I know the queries are not directly portable. This is related to some of the semantics in the openEHR data (and archetypes). Most important is to work out a shared understanding of how to map hierarchical data into AQL, and then we have the datatype and equal/order . Similar to this we've seen the need for functions within AQL. Most needed was the INSTRUCTION/ACTIVITY/ACTION related operators which DIPS and Better both support. We as a SEQ group have to work on this topics and discuss the semantics. The outcome of these discussions might be written into some formal definition. But the semantic agreement has to come first! --- ## Post #28 by @Seref I thought long about leaving this without a response. I decided that you need hear a particular point of view, mine, in this case, since I think I earned the right to express that. I find your overconfidence in your engineering skills troubling Tom. You seem to be disregarding concerns from implementers too easily, mine to be specific in this case, from a perfectionist, idealist point of view. I really did make an effort to emphasise that there is a trade off here and we may gradually improve things, but you're displaying a repeating pattern of simply dismissing implementation and adoption related concerns, in addition to trivialising all implementation work, despite not always necessarily having actual experience or expertise in particulars of it. Members of this group, including myself, are aware of many of the principles you come across as lecturing, despite not every one of them being engineers. We are not always in a position to adopt those principles fully and openEHR SEC needs to accommodate that fact. I'm not sure how SEC can function as a committee without you, given your position, developing a habit of considering pragmatic concerns and making an effort to strike some balance in specification work. I am truly sorry having to express all the above, but regardless of your opinions of me, I deserve the right to hear some convincing responses to my questions and concerns, both as a SEC member and as a member of openEHR community, without condescension and with a level of courtesy that you've always been receiving but failing to give at times. I hope we'll see some indication of you considering what I'm saying here in your future correspondences. I do not expect or wish to receive a response, this is all for your consideration, not written with a wish to discuss. --- ## Post #29 by @thomas.beale Well I won't respond to the main criticism, but I want to make it clear that I am not at all dismissing implementation concerns or working from a theoretical or perfectionist point of view. The only thing I have said in this entire thread is that we should observe a basic, orthodox dictum in IT, which is not to create dependency or close coupling where it can be avoided and doesn't appear to be needed. The cost of doing that in AQL compared to the cost of making it openEHR RM specific and dependent I would guess to be slightly more in the short term, but greatly lower in the long term. For me the specifications and archetypes aren't really the important thing, they are interim artefacts that establish explicit agreements on shared semantics of something or other that will then have some useful results in real systems that function in places of care. I'm really only interested in that final results. If I had had a theoretical point of view on all of this, I would have gone and turned it into a PhD. Instead I have just worked on it in the industry context, to try and make an effect in the real world. As part of that, I have worked as closely as possible with vendor companies (including Ocean, which I started in 2000 with MD colleagues and left only a few years ago). Anyway, I'm certainly sorry if any of my posts here seemed discourteous, they weren't meant to be, and I apologise if anyone found them otherwise. I always assume this group is up for technical discussions and debate, and for me, it's never, ever personal. I'm sorry if anyone else thought it was. Anyway, if people want to tell me it is better for me to go, no problem, I won't take offence at that either. Maybe it is. --- ## Post #30 by @sebastian.iancu Pff, ... what a thread!! Come on guys, we are here to resolve problems not to create new ones. How many hours we already burned on this topic? I'm reiterating one of my initial post, where I tried to say that, from my point of view both Thomas and Seref are right, both solutions are needed and are complementary, it is just a matter of prioritizing and investing in right resource. **Instead of focusing on our differences, on what makes us apart, let's focus on what we have in common, how can we use all our multi-diverse experience and competency to create better specs.** And certainly we need **all of us** on board! --- ## Post #31 by @ian.mcnicoll [quote="thomas.beale, post:29, topic:402"] Anyway, if people want to tell me it is better for me to go, no problem, I won’t take offence at that either. Maybe it is. [/quote] Thomas, I can assure you that this is **absolutely not the case**. Every single member of this community, me included, holds you in the highest regard, for the seminal contribution you made to openEHR, for the ongoing drive and energy that you continue to show, and as a true personal friend. @Sebastian I understand your frustration. We can all get passionate and forget ourselves at times but there are some very important points of principle in here (strongly felt) about how we proceed. I actually think most people agree with you that there is room for two types of documentation but a clear preference for putting almost all of our collective energy into some kind if openEHR RM focussed AQL documentation 1) an ongoing tidy-up of the 'core AQL' specification which should remain model-neutral, other than with some openEHR examples. 2) a more practical 'openEHR AQL' document , perhaps phrased as guidance, and in ITS if that helps, that very practically works through some of the issues that Seref, Matija and Bjorn have highlighted, and will allow us to make rapid progress on AQL openEHR conformance discussion , and can inform (1) over time. I'd like to make a start on this with Bjorn to chew over his datatype comparison/ordering questions (which from my POV have significant clinical safety/sense aspect) and perhaps then fold in Seref's original 'FROM' issues comments at the top of this thread I;m open to further suggestions about how and where this is done but my preference would be to carry on as before with Topics in discourse, at least for a bit. How does that sound for moving forward? Ian --- ## Post #32 by @sebastian.iancu @ian.mcnicoll, I'm fine with your proposals, so far. I have nothing against it. One more remark (still have to make sure people are not going to omit it) is that AQL should ("must" IMO) also work with demographics, so please consider this in the specs, and in the examples for 1.1 release. --- ## Post #33 by @ian.mcnicoll Absolutely though I suspect most CDR vendors may want to use that as a facade to support population queries e.g on a few simple aspects like Dob (age) , sex, gender , locality without actually having a real openEHR demographics engine sitting behind that , as you do. --- ## Post #34 by @bna [quote="thomas.beale, post:29, topic:402"] Anyway, if people want to tell me it is better for me to go, no problem, I won’t take offence at that either. Maybe it is. [/quote] Please don't. Since 2010 I have worked with openEHR. We have been a team of a few core members reading the specifications over and over again. For each year implementing improved solutions. What we found most appealing was all the details explained so well about how to build a structured clinical data repository. So please forgive me if I told you this was not a huge gift for e-health. At the same time. In our company there have only been a few who are able to read the spec and apply it to implementations. That's ok. I think we also need an abstract formalism to make sustainable platforms. I am so proud to be part of the openEHR community. All the great people who will use their time to read this post and to think about the same problems that I face in my day-to-day life. We, as a community, are unique. We have expertice in different areas and we need to embrace this. Coming in to openEHR in 2010 I thought the community was bigger. I thought there was lots of vendors delivering real systems based on openEHR. I was wrong. There was only one; Oceans. And there was this "new kid in block"- Marand. We met Thomas, Ian and a few guys form Marand at the MIE conference in Oslo. It was a high fame factor for us to meet you guys. And we where so impressed about the way you talked, the specifications you had and the platform Marand had implemented. Since then we've implemented three generations of openEHR software without the need to convert data. That's fantastic. And it is a lot of kudos for all the good you all have done with the specifications. We need your bright mind. That's for sure. We also need good guidelines for developers to cover all the dirty details. And this is an area where I am not sure if we can handle by formalism. This is the area where we have to talk about dirty details which might never be formalized. I intend to work with e-health and openEHR for the rest of my career. I hope you all can bear with me, and I hope to be part of a community where we can argue loudly and still be friends. I want to be part of a vibrant community with a shared vision. I think openEHR is the only option in the world. I hope and think we can continue the work we are doing and in the future get even more people involved in the work. I wish you all a great weekend. PS! I am using the weekend to work on the COVID-19 application based on openEHR. It is a great example and opportunity to demonstrate for the whole world what we have made together based on the heritage from some really great minds. --- ## Post #35 by @sebastian.iancu Well, I'm not pushing anybody towards supporting demographics if their system actually does not implement it (or support it) already. It is just that once is going to be used more than us it should be clear how the query should look like. It is perhaps also a marketing/sales statement, that openEHR has also Demographics and it could be used in AQL without any problem, and that is more powerful (and complex) than PERSON/PRACTITIONER from FHIR... Therefore it would be nice not to have anymore in text any statement like "AQL is to query medical data" or "to query CDR (clinical)", etc. (shortly saying), because that implies that its purpose is to query EHRs. --- ## Post #36 by @sebastian.iancu :smiling_face_with_three_hearts: owww, such a warm message ... let me hug you! :hugs: --- ## Post #37 by @bna [quote="sebastian.iancu, post:36, topic:402"] let me hug you! [/quote] You might when the COVID-19 period is over... haha :smile: But thanks Sebastian. I appreciate your feedback! --- ## Post #38 by @ian.mcnicoll I completely agree - we need to show AQL does demographics in a vendor-neutral manner and it makes sense to use the DEMOGRAPHICS RM as a faced for that, especially as the key queryable content will be archetyped in CLUSTERS and not directly tied to RM attributes/ classes. So for instance, we can use the FHIR-aligned demographic archetypes as the nominal 'target' for population style queries, again even if what is actually under the hood is neither FHIR or openEHR DEMOGRAPHICS. You will want to, and are able to dig deeper into supporting DEMOGRAPHICS but this will probably not be necessary or doable for others. That's fine /Mixed economy always wins. --- ## Post #39 by @thomas.beale [quote="sebastian.iancu, post:35, topic:402"] It is perhaps also a marketing/sales statement, that openEHR has also Demographics and it could be used in AQL without any problem, and that is more powerful (and complex) than PERSON/PRACTITIONER from FHIR… Therefore it would be nice not to have anymore in text any statement like “AQL is to query medical data” or “to query CDR (clinical)”, etc. (shortly saying), because that implies that its purpose is to query EHRs. [/quote] I can't remember whether I published my FHIR demographics analysis here, but [just in case, it's here](https://wolandscat.net/2019/09/11/fixes-for-fhir-the-admin-resources/). --- ## Post #40 by @yampeku Wow, somehow I missed this thread! I for one, have been using every part of the specifications for other RM since forever... except AQL. AQL has always been a little out of our scope because, well, we used other query languages for querying and validating data. I have to say that archetypes work with whatever model you can think of. I think we have made lots of proposals in the past to ensure that more or less everything works regardless of the model (e.g. CDA having uppercase attributes and ADL not really supporting that), and I'm happy to say that in that regard every part of the model is quite robust. I'm sure more things will be needed to make it completely model agnostic, but we have, as a community, always found the best solution (or a good compromise) to get things working. For AQL, I thing the 2 most explicit issues we always have had are actually the ones being discussed these days: 1) What does AQL return? 2) The things that AQL assumes as known by the engine (standard dependent). This include quite a lot of things that BMM de facto solves (what is my node id attribute for this model? which are the valid parent classes?etc. etc.) After all these years of being the "weird nitpicky Spanish guys" we have only received love from this community. So this is to say that we love you all as much :grin: --- ## Post #41 by @thomas.beale [quote="yampeku, post:40, topic:402"] What does AQL return? [/quote] Theoretically, the [RESULT_SET structure you see here](https://specifications.openehr.org/releases/SM/latest/openehr_platform.html#_query_package) (reverse-engineered from REST API and previous informal definition found on the wiki). Probably this small model should be moved to a chapter inside the AQL spec? --- ## Post #42 by @yampeku probably, as they are also a generic model that could be used with any RM --- **Canonical:** https://discourse.openehr.org/t/aql-and-related-documentation/402 **Original content:** https://discourse.openehr.org/t/aql-and-related-documentation/402