Agreeing on optional user interface hints in templates

thomas.beale · 4 March 2022 17:03

We can certainly find some examples where the logic of whether a data hierarchy should only be present based on the value of another element. Classic example:

items
    +--- tobacco user: Y/N
    +--- tobacco use details
               +--- amount per week
               +--- cessation attempts
               |          +--- xxxx
               +--- yyy

rules
    /items[tobacco user] = 'Y' implies exists /items[tobacco use details]

If we consider the above rule a semantic property of the data independent of whether it is on the screen, in a message, document, survey or whatever other form, agreed on by the archetype experts, then it belongs within models - either the archetype or somewhere similar. (@erik.sundvall 's case)

However we can undoubtedly find examples where the same structural conditionality is use / context specific, and not applicable to all instantiations of the template as data. We may well need a UI level of mechanism to accommodate this. (@pablo 's case)

So it just means that not everything that looks like a certain kind of business rule for data or UI is necessarily one or the other.

pablo · 5 March 2022 20:38

Hi @thomas.beale I agree if there is any data set related rule, it should be in the template itself, or in AOM in general, though each requirement need thorough analysis to know what mechanism should be used to satisfy them, if it’s a data cross-node constraint, or if it’s something that will affect the user workflow or the GUI.

A separate but related topic is how to define and name those IF-THEN-ELSE rules for the data sets. I know we have the assertion language plus other new constructs.

Let’s say certain part of the structure defined by a template, which is optional, should be prohibited depending on a value in another part of the same template. I think we can use the AOM itself to be dynamically changed when these conditions are true, for instance:

IF value_at(path) IN [a,b,c] THEN node_at(path).occurrences = [0…0]

So instead of defining something new, we use the current RM functions and attributes to change their values at runtime. It’s like a RM compatible scripting language.

Same could be done if the optional node should be mandatory depending on the value(s) of another node in the template.

Of course all values should come from COMPOSITION instances, so the “script” can only be evaluated using a given COMPOSITION + TEMPLATE.

The language you described above doesn’t seem to use the RM and seems like OCL (which we use to represent RM invariants in the specs).

About naming, what I don’t like is something like the mentioned “enableWhen”, which kind of implies something related with GUI because of the “enable” part. In a data structure the nodes are not enabled or disabled, but are required, optional or prohibited. So naming is important to avoid misunderstandings with the scope of these rules: someone might want to use this mechanism to represent workflow or GUI rules when that is not the purpose of the rules. Of course, it is also difficult to discourage this usage when people have tools that allow them to do that and use template how they want instead of how they are designed to be used for.

thomas.beale · 6 March 2022 15:52

we already have this - we just don’t use exactly that syntax. In the syntax of today, it is :

/path/to/value matches {constraint} implies exists /path/to/other/node

In the newer syntax (future), it would be:

assert name_1 matches {constraint} implies defined(name_2)

or:

assert name_1 matches {constraint} implies name_2 matches {constraint}

where name matches {constraint} will often be name = value for the common simple cases.

In the above, name_1 and name_2 have bindings to paths in a separate place.

We also don’t need special functions like value_at() or node_at() because predicates like events[any_event].items[systolic_bp] can be used, where [any_event]is shorthand for [@archetype_node_id = '#systolic_bp'] and #systolic_bp is a future form of [id23] in ADL2.

This is really just dot-syntax mixed with Xpath-style predicates. Some other more advanced things will be possible using lambdas, as described here.

NB: this new syntax is still being refined; I’m mentioning it here to show a proposed direction, not to say it will be in the next version of ADL. Next versions would potentially support path bindings (see here - look for ‘symbol_bindings’), but we are working with @pieterbos and others to agree on how far to go at each ADL release.

It’s not RM-specific, if that’s what you mean - it’s makes no assumptions about functions that might be available on specific classes in specific models - the way of finding items in a sub-tree or testing existence etc is completely generic, and doesn’t change when you use it with another RM. So it’s more like OCL in that way, true.

I agree with this.

pablo · 6 March 2022 17:22

I mean that the expression language could use the RM API (functions from the RM classes) instead of defining a whole new language without reusing the infrastructure we already have, which at implementation time could accelerate things.

thomas.beale · 7 March 2022 08:36

Well it could, but then you are relying on functions from each RM to provide general semantics, e.g. finding nodes at paths. Each RM could easily have different functions for doing that. Worse, most RMs are not likely to have the complete set of functions defined that would be needed.

We do however rely on the openEHR Base and Foundation types, which are not openEHR-specific.

For example use of lambdas to find matching sub-parts:

    class Book {
        title: String
        pub_date: Date
    }

    book_list: List<Book>

    book_list [(b:Book) {b.title.contains("Quixote")}]

Well, the Expression Language is based on very common / standard concepts, grammar etc, and the various parts are mostly cherry-picked from existing languages, e.g. Xpath, for path-related predicates.

We do have some of the infrastructure needed for an Expression Language, in:

the simple and incomplete one in ADL2
the very simple expression part of AQL
the expression meta-model in GDL
the small expression language in Task Planning.

The main aim of a revised EL is to bring the semantics of all these together, or as close as possible, even if surface syntaxes might differ (as appears inevitable in AQL). This will support:

upgraded BMM so that all invariants and functional parts can be represented;
smart decision logic like you see here;
more interesting possibilities in AQL.

The intermediate form we are currently working on for ADL 2.3 will do a fair bit.

erik.sundvall · 7 March 2022 11:15

Hi!

I have no strong opinions regarding the naming, I just used FHIR’s name to point to the use-cases and design pattern.

What we have an immediate need for in the multi-vendor project is a vendor netrual possibility to express if/then conditionals like the ones you both have discussed above (e.g like Thomas’ “tobacco user Y/N”).

It looks like the ADL implies+exist rules or future expression language with assert+implies might do the job if I understand you correctly above.

One of the remaining questions:

Are there suitable (template?) authoring tools that can be used to explore the ADL rules approach?

If not, could we (for practical test purposes) make some annotations using present tools with the rule syntax in the annotation data somehow? The template(s) with annotations could then easily be automatically post-processed and convert the annotations into for example either:

“proper” ADL rules and insert into the template or
current vendor specific if/then rules in openEHR-based form engines (like Betters’s and Cambio’s)

Currently the national project is a “proof of concept”, so it can be used to test the general mechanisms and no essential harm is done if a future official openEHR syntax would change later.

pablo · 8 March 2022 16:06

I understand your point, but I think any implementation, for any RM will need to have some node location by path function, isn’t it?

In other words, any reference to a node in a RM instance, openEHR or any other RM, in a expression language, needs to have an underlying API that will resolve the reference (path or another locator) to the right node from the instance, so I’m wondering if we still need to write that API for ANY RM so the expression language can be evaluated, why not making the expression language more openEHR friendly to reduce the burden for openEHR implementers if it will be the same work for other RM implementers?

Note the language is only the syntax part, the complex part is the API that implements the language and executes/checks things then an expression is evaluated against an RM instance, and the syntax will indicate which operations are needed in the API, so we are still imposing the semantics of those operations for any RM, with an unavoidable node_at(path or locator) operation needed at the API.

It just feels we are introducing a new language for every new requirement…

erik.sundvall · 8 March 2022 17:04

Tom loves DSLs (and parsers?).

But seriously isn’t the ADL rule language (“matches” + “implies exists” above) already defined in spec and also implemented in some solutions.

Does anybody have opinions on using helper annotations in templates for this functinality in tools that do not yet support rules? (that can then be post processed as described in post #10 above)

items
    +--- tobacco user: Y/N <-- ANNOTATION POSITION A
    +--- tobacco use details <-- ANNOTATION POSITION B
               +--- amount per week
               +--- cessation attempts
               |          +--- xxxx
               +--- yyy

FOR RULE CAPABLE TOOLS:

rules
    /items[tobacco user] = 'Y' implies exists /items[tobacco use details]

ANNOTAIONS on one of the above marked nodes FOR RULE INCAPABLE TOOLS

Perhaps using a “THIS” macro for current node so that a rule annotation on the marked nodes can be written as either of the following table rows

Annotation position	Annotation Key	Annotation value
A	rule	THIS = ‘Y’ implies exists /items[tobacco use details]
B	rule	/items[tobacco user] = ‘Y’ implies exists THIS

Alternative solutions are welcome.

thomas.beale · 8 March 2022 21:38

There’s not usually an API - the statements are normally parsed to an augmented AST (abstract syntax tree). That’s what the AOM, BMM and similar models are in fact - special ASTs. Once you have the AST you validate anything that couldn’t be validated at a syntax level, which is usually typing, semantic coherence etc. Then you can execute validated statements / expressions by traversing the AST and performing the computation. You can certainly build that as a class-level API, but it’s not the kind of service API you’d expose through something like REST.

In fact, the operation to get the value at a path within the wouldn’t be handled by any of this, because archetype paths will be mapped to symbolic variables. The runtime system has to work out what data paths an archetype path mentioned in a binding possibly refers to (e.g. it might be 50 systolic BP values, or just one).

To resolve an archetype path against data you could potentially rely on some API that talks to data, and contains path-resolving functions.

But apart from resolving bindings, the language doesn’t rely on an API as such to be executable at run time.

I hope not - we are just trying to upgrade and consolidate a single expression language with a fairly standard syntax and semantics, as per the draft spec.

pablo · 8 March 2022 21:40

I don’t know about ADL2.

matches is for sure there, then in AOM we have existence in the constraint model, but not sure if implies exists is something already defined.

A related question would be if this is the only rule form we will have or if there are going to be more complex rules, just to check how syntax and underlying API would look like.

pablo · 8 March 2022 21:44

With API I’m referring to the classes and functions that implement the logic, once parsed from the syntax in the AST. The ASW is acutally mapped to something that can be instantiated, invoked, and that client code can use the results for such invoked methods/functions. So I guess it is what is binded by the lexer from the parsed DSL if I didn’t mess the terms

thomas.beale · 8 March 2022 22:10

I think this is a reasonable short-term solution for non-rule-supporting systems. I need to think a bit on the details. It would still be better to get to a syntax that didn’t have archetype paths inline (these greatly complicate the semantics) and used bindings for them instead, as per this version of the older Expression Language.

thomas.beale · 8 March 2022 22:32

Pretty much everything turns into calls to functions that are either defined on objects of the RM in question, or underlying primitive types available in the language of the execution engine implementation. For example, traversing a piece of AST representing the expression systolic + 0.33 * (systolic - diastolic) will resolve to calls to Real.add() etc. Normally you will replace that with calls to the built-in types.

You can guess how it could be executed from the default generate AST:

The BMM AST is a bit smarter and allows more interesting things to be achieved, mainly to do with validation.

Calls to external functions should also be supported, e.g. to enable function that computes e.g. BMI or pregnancy risk based on certain variables.

In my proposed future EL, decision tables (aka if/then chains and case statements) will also be supported e.g.@

Result := case ipi_raw_score in
        =======================================
        |0..1|  : #ipi_low_risk,
        |2|     : #ipi_intermediate_low_risk,
        |3|     : #ipi_intermediate_high_risk,
        |4..5|  : #ipi_high_risk
        =======================================
    ;

and its default AST:

Also other nice structures:

score := Result.add (
        ---------------------------------------------
        basic.gender = #male                 ? 1 : 0,
        age_score                                   ,
        has_congestive_heart_failure         ? 1 : 0,
        has_hypertension                     ? 1 : 0,
        has_stroke_TIA_thromboembolism       ? 2 : 0,
        has_vascular_disease                 ? 1 : 0,
        has_diabetes                         ? 1 : 0
        ---------------------------------------------
    )
    ;

I’m getting ahead of the original question here of course

erik.sundvall · 9 March 2022 07:32

Good suggestion, it also matches tha approach in Cambio’s form editor (see screenshot below) where each node used in a form is assigned a short default id (based on the template’s node name) that can be changed manually.

Just as in ADL Cambio’s rules (called actions) are collected in a single place (not attached to each node).

Edited at 18:00, adding info aboout Better:
In Better’s form editor the rules are anchored to a specific node but can read and modify any node (see “Method code equals Auscultation” in screenshot below), not just the node it is anchored to, so I guess ther rules are actually ass free-standing as in Cambio’s solution

For template tools that do not handle rules (yet) we could then perhaps add an “id” annotation that will through post-processing create a symbol binding based on the path of the annotated node.

Edited: added the (namespace)prefix ‘a.’ to the examples below to reduce risk of name clashes and to make it work better in Ocean’s template designer tool. The letter ‘a’ being the first in the word ‘automation’

Example based on the structrue from post 13:

items
    +--- tobacco user: Y/N <-- ANNOTATION POSITION A
    +--- tobacco use details <-- ANNOTATION POSITION B
               +--- amount per week
               +--- cessation attempts
               |          +--- xxxx
               +--- yyy

Annotation position	Annotation Key	Annotation value
A	a.id	tobacco_user
B	a.rule	tobacco_user = ‘Y’ implies exists THIS

Or if choosing to manually giving an ID to both nodes, more like below. Below an alternative rule version using the ADL2 syntax desctibed in [Archetype Definition Language 2 (ADL2)](https://section 7.11.2.2. (Value-dependent Existence)) and a cambio inspired one is also provided

Annotation position	Annotation Key	Annotation value
A	a.id	tobacco_user
B	a.id	tobacco_details
root of template?	a.rule.adl	tobacco_user = ‘Y’ implies exists tobacco_details
root of template?	a.rule.adl2	check tobacco_user = ‘Y’ implies defined (tobacco_details)
root of template?	a.rule.cambio-style	tobacco_user == ‘Y’ ASSIGN tobacco_details.hidden = false OTHERWISE tobacco_details.hidden = true
root of template?	a.rule.better-style	tobacco_user = ‘Y’ THEN tobacco_details show OTHERWISE tobacco_details hide

In the Cambio/Better-style rows above, the CAPITALIZED words correspond to the rule section headers in the rule editors, see screenshots above. I skipped the first (constant) “WHEN” in order to sugest shorter syntax.

thomas.beale · 9 March 2022 11:07

Just to go a bit further on this: the reason it complicates things is that the mapping between archetype paths and data paths is in general 1:N, due to multiplicities in runtime data. The other thing to remember is that expression languages (at least typical ones) work with data, i.e. real values, not ‘models’.

So a symbolic variable like heart_rate mentioned in an expression has to be populated with an actual heart rate value at runtime. What will this be? Well the binding shows at what archetype path it can be found in openEHR data (nice!) but we still have the problem of dealing with there being e.g. 50 heart rate values over time, all at that same archetype path. Which one(s) does the expression execution engine use? In some cases, there is an implied ‘for all’ operator (as @pieterbos pointed out a long time ago), but that is not universal. That means the EL engine potentially has to execute an expression 50 times or do something else that is not at all obvious from the expression itself. The expression itself is clear however; the challenge lies in how it will be applied to data. I believe we want to separate that complication from the syntax and semantics of expressions, so that expressions are just like they are in other languages.

I won’t bore everyone with further details, hopefully this gives a clue as to why bindings are important. For the masochistic, a long discussion here.

thomas.beale · 9 March 2022 11:09

Something like the second suggestion in that post pretty closely simulates the bindings + symbolic approach I have advocated. It means that the rules are readable, which I think helps a lot.

pieterbos · 9 March 2022 11:28

A for_all and perhaps an exists block, rather than a for_all just in a statement, in which one can define variables as pointing to a path, would result in a small modification to the current grammar resulting in much more easy to read rules. Then rules could be written so that paths appear only in binding to variables, and assertions can be written only with variables, and nearly all variables could point to single values instead of lists or other collections. Without any complex ‘operators have to be defined on lists as well as single values’ logic, as we currently have.

However, if data binding is made separate from the language, and you cannot use a path anywhere else in the language, this will complicate these issues very much - then you cannot easily express ‘I want these rules to apply to this event in this observation only, even if the event occurs multiple times, but I still want to reference other data in the observation outside of the single event’.

if data binding is separate, but sub-paths are still possible, and a for_all block is added, then the problem is also easily solved.

thomas.beale · 9 March 2022 13:29

Wouldn’t you just create a more specific binding for that particular event, to a more specific variable, e.g. ‘Apgar_heartrate_5_mins’ or similar, when you might already have ‘Apgar_heartrate’? Or you are talking about apply these rules in the sub-tree at this path? For the latter I think you would define a binding to the container object (HISTORY or whatever) under which you want a for_all or there_exists to execute.

It’s an interesting question generally as to whether the binding needs to be separate from the language. I have made the assumption that it does because the bindings in GDL, Task Planning and Archetypes are all completely different.

I am more inclined to this approach - because with only symbolic vars in the EL, EL texts are least is readable and maintainable. This goes back to our discussion last year.

thomas.beale · 9 March 2022 14:42

I had another thought on this - I just realised you probably thought that I propose no path access method at all within a revised EL. The approach I propose is more like Xpath. Firstly it is access to nodes below some root object within data, i.e. instance level paths. In Xpath, as we all know, this is done with predicates in []. And predicates can be (nearly) any expression to get to the intended sub-node, including path-patterns, but also other expressions, e.g. the simple ones like [1] (the first item). If we treat an inline archetype path as a path pattern predicate we then achieve what you want (which I agree with of course - but the semantics are now clearer - they are as in Xpath - if the path pattern matches 10 real paths in the current data instance, then that’s what you get.

The new proposed EL has this kind of functionality, see here (but no path example there yet).

The paths by the way would have ‘.’ separators rather than ‘/’, indicating formal RM structure access.

erik.sundvall · 25 March 2022 17:11

A side note: In Ocean Template designer 3.1 (from 2015?) you can add configuration of annotation types (in subcatefories called sets), and say what elements they apply to, and optionally what the allowed values are. As default on installation the Categories “Notes” and “GUI directives” are included, see screenshot.

Does anybody remember what the “Group” annotation was intended for?

Also, when trying to load templates with “plain” annotation names (like “fhir_mapping”) the tool gives an error. So the annotation names have to be changed to be in a prefix.suffix format. (I assume that is to cater for the annotation sets.

Prefixing the experimental annotations discussed in this thread may be a good thing anyway.