# Evaluation Archetypes and assessment protocol
**Category:** [Clinical (archive)](https://discourse.openehr.org/c/clinical-archive/153)
**Created:** 2007-10-30 10:17 UTC
**Views:** 8
**Replies:** 13
**URL:** https://discourse.openehr.org/t/evaluation-archetypes-and-assessment-protocol/14689
---
## Post #1 by @Stef_Verlinden1
Dear all,
From the lengthy and educative discussions on this mail list, the last couple of weeks, the following occurred to me:
- observations are to do with gathering evidence
- an evaluation is an activity of comparing data to a knowledge base (and draw a conclusion based on that comparison (SV))
(Both excellent definitions provided by Thomas Beale)
The other thing that occurred to me is that the ‘assessment’ of data quality is an evaluation too, based on a knowledge base, which knowledge is summarized in a protocol. As one might recall, I’ve been struggling were to put the data quality protocol and the outcome of the evaluation. This ‘insight’ makes it much easier from my point of view, I can just ad an evaluation archetype data quality.
Over thinking this I wondered if we should give these knowledge bases/ assessment protocols a firmer position in the evaluation archetypes
To explain this I would like to start with a question: what’s the purpose of an EHR?
I don’t know if there is a better definition but from my point of view it’s something like: to capture all healthcare related data of a patient/citizen in such a manner that his/her medical situation is represented as good and as complete as possible and that the (relevant) data is (re-) usable by all health care providers (and ultimately the agents) involved in a certain process.
My interest is especially in the (re-) usability of available data and evaluations, since this is essential to improve the quality and efficacy of our healthcare system. On top of that, if the data is of good quality, some (parts of) processes can be fully automated which could even further increase quality and efficacy.
If we look at the re-usability of data (as put down in observations), this is provided for in the current openEHR EHR. One can use this system to capture and share ‘evidence’.
This shared ‘evidence’ can then be (re-) used by any healthcare provider involved in the process to asses it to the desired protocol (either in ones head of written down in paper) to evaluate it. Everybody re-evaluates the available data every time in order to draw his/ her ‘own’ conclusion. This is a perfect valid way to do it, albeit cumbersome.
If we look at the re-usability of evaluations, we still miss two things.
I’ll enlarge the ‘problem’ a little in order to make it, but the point is that trust and re-usability are essential components for a, not only functional but, widely accepted and used EHR system
Firstly: In order to be able to re-use an evaluation one needs to know against which protocol this conclusion is drawn. Only if this protocol aligns with or is similar to the protocol used locally, one can consider re-using the evaluation.
To give an example: the diagnosis criteria for rheumatoid arthritis (RA) are not ‘engraved in stone’. It varies over time and it varies locally. So in one situation RA is diagnosed if X joints are inflamed for Y weeks but in another situation the diagnoses is only given if at least one ‘large’ joint is involved and/or if Z joint are inflamed for Q weeks (I made this up, so don’t hold this against me:-) ). So if one doesn’t know which protocol is used to evaluate it, the diagnose RA doesn’t mean too much.
Secondly: In order to obtain a reliable conclusion one needs to use ‘evidence’ of ‘good quality’. Even if the protocol is known, an evaluation won’t be re-useable if the data quality of the underlying ‘evidence’ is poor/ unknown. Who would trust an evaluation if it’s possibly based on false assumptions?
Similarly as with the evaluation of a medical situation, data quality has to be assessed against a (locally accepted) protocol. This assessment leads to a conclusion (f.i. good/poor/undetermined data quality). Also here, that conclusion isn’t worth much if it isn’t known against which protocol the assessment is made.
I’ve looked into the current evaluation archetypes and although there is some room to reference to additional information (f.i. ‘reference, useful information about this condition’ in the EVALUATION.problem-diagnosis AT), there is not a dedicated place for the protocol used to draw a certain conclusion.
So my suggestion is to add a field (”protocol used for evaluation”) to all the evaluation archetypes, in order to stimulate the re-usability of, not only observations but also, evaluations.
Cheers,
Stef
PS Needless to say that I’m not a big fan of knowledge bases or protocols, which are only present in the head of a healthcare provider
---
## Post #2 by @Stef_Verlinden1
I just saw that in the EVALUATION.problem-diagnosis AT their is a data field 'Diagnostic criteria'. I was expecting this in the protocol section, so I overlooked it.
This is exactly what I'm looking for, except that it would be (in my opinion) better to refer to the list of criteria (or protocol) instead of listing all these criteria here in the evaluation AT everytime a diagnoses is made. Does that make sense?
Cheers,
Stef
---
## Post #3 by @system
Hi Stefan
It is worth noting that the CARE_ENTRY class - which is the immediate ancestor of the OBSERVATION, EVALUATION, ACTION, INSTRUCTION classes has an attribute called GuidelineID - which is a reference to the guideline or protocol used. I believe this is what you are looking for - but it does require identification of the Protocol for you to use this.
Cheers, Sam
Stef Verlinden wrote:
[details="(attachments)"]
[/details]
---
## Post #4 by @system
Hi Stefan
Diagnositic criteria could appear in the protocol - I hadn't thought of that. What do others think. An example is in Diabetes - Fasting Blood Sugar > 7.8 mmol/L
Cheers, Sam
Stef Verlinden wrote:
[details="(attachments)"]
[/details]
---
## Post #5 by @Beatriz_de_Faria_Le1
I also think that the diagnostic criteria should be in the protocol\.
Makes sense for me\.
Beatriz
Sam Heard wrote:
---
## Post #6 by @Stef_Verlinden1
Hi Sam,
First of all congratulations and thanks to everybody who contributed to the new website. It's really a great improvement.
> Hi Stefan
>
> It is worth noting that the CARE_ENTRY class - which is the immediate ancestor of the OBSERVATION, EVALUATION, ACTION, INSTRUCTION classes has an attribute called GuidelineID - which is a reference to the guideline or protocol used. I believe this is what you are looking for - but it does require identification of the Protocol for you to use this.
I think this could indeed be what I'm looking for, but have some additional questions. I guess my main question is a bit similar to those in the device discussion. Let me see if I can explain it. It might well be that this is more a question for the technical list. But since I'm not a technical person, let me try to ask this in 'lay mans' terms.
I think that we all agree that in evaluation archetypes such as 'diagnoses' there should be a list of criteria (the guideline or protocol) which are used to reach that diagnosis. My point/ question is: should one enter (by hand or by the application that is used) these criteria everytime a diagnose is made, or can we create a separate guideline/ protocol archetype to which we can refer.
If one chooses the first option one should fill out the criteria each and every time, when writing down a diagnosis. Of course this can be automated by the application that is used. Thing is that the knowledge will be stored locally at application level. My feeling is that your suggestion to use the GuidelineID is in conjunction this option. Point is that if I refer to an external guidelines, data in that guideline isn't 'archetyped' and can't be used for computational purposes. So I have to translated those into an archetype.
Alternatively one refers to to actual version of the guideline/protocol which is stored through a separate guideline/protocol archetype. By doing so guidelines/protocols can be be stored/ monitored/ approved centrally. Furthermore they can be (re-)used by others and queried for, etc.
I can imagine that one would like to query for all diagnosis X which comply to the guidelines Y and Z
The similarity with the device discussion that one could enter every time all the details of a device (type, brand, certified. service date. calibration date, etc. etc.) in the appropriate archetype or, similarly as above, refer to an device archetype.
Another thought, although I'm not sure whether that's valid is that in case of the first option those evaluation (such as diagnosis) and observation (such as blood pressure) archetypes would become rather 'bulky' if we have to include all these variables as well. By referring to a separate archetype one could safe a lot of 'space'.
Hope this explanation helps and I'm curious to hear your viewpoint.
Cheers,
Stef
---
## Post #7 by @Arild_Faxvaag1
The problem with diagnostic criteria are that they often not exist\.
What we have in rheumatology are \_classification\_ criteria: http://
tinyurl\.com/b57yd Most classification criteria in my domain have
been developed with the purpose of answering research hypotheses
\(hypotheses on the class level \- assumptions on patients as a group\)\.
A given patient with arthritis may at a given time fulfill the
classification criteria for rheumatoid arthritis or he may not\. It is
useful to know whether a patient fulfills the classification
criteria, but it is not so that all patients who don't fulfill the
classification criteria do not have the disease \(the disease may be
in development\)\.
In my opinion, the diagnosis is a justified true belief \- a
hypothesis about which \(instance of a\) disease inheres in the
individual patient\. It is the product of the activity of medical
problem solving, it is coupled to the person who makes the statement
and it's main "function" is to justify subsequent decision making and
therapeutic actions\.
Wouldn't it be nice if we were able to move this discussion to the
new openEHR wiki?
regards
Arild Faxvaag
---
## Post #8 by @system
> If one chooses the first option one should fill out the criteria each and every time, when writing down a diagnosis. Of course this can be automated by the application that is used. Thing is that the knowledge will be stored locally at application level. My feeling is that your suggestion to use the GuidelineID is in conjunction this option. Point is that if I refer to an external guidelines, data in that guideline isn't 'archetyped' and can't be used for computational purposes. So I have to translated those into an archetype.
Guidelines can be computerized as well. There are published guidelines representation formats, e.g. GLIF for instance and their execution engines. Being rules based and decision trees like, personally I think they are not best modelled as archetypes since archetype model are designed to represent more 'static' clinical content model. In applications, EHRs and guidelines should work hand-in-hand: 1) guidelines need to query EHRs through standardized EHR representation format and query language; 2) EHRs need record what guidelines are used etc.
Cheers,
Rong
---
## Post #9 by @Stef_Verlinden1
> If one chooses the first option one should fill out the criteria
> each and every time, when writing down a diagnosis\. Of course this
> can be automated by the application that is used\. Thing is that the
> knowledge will be stored locally at application level\. My feeling
> is that your suggestion to use the GuidelineID is in conjunction
> this option\. Point is that if I refer to an external guidelines,
> data in that guideline isn't 'archetyped' and can't be used for
> computational purposes\. So I have to translated those into an
> archetype\.
>
> Guidelines can be computerized as well\. There are published
> guidelines representation formats, e\.g\. GLIF for instance and their
> execution engines\. Being rules based and decision trees like,
> personally I think they are not best modelled as archetypes since
> archetype model are designed to represent more 'static' clinical
> content model\. In applications, EHRs and guidelines should work
> hand\-in\-hand: 1\) guidelines need to query EHRs through standardized
> EHR representation format and query language; 2\) EHRs need record
> what guidelines are used etc\.
Thanks, this makes sense to me and thanks for pointing out GLIF\. I
wasn't aware of that organisation\.
Let me check if I understand it: 1\) in order to be able to compute
a diagnoses against a guideline/ protocol \(with the aid of a decison
tree/ decision \(support\) system\) the criteria of this guideline/
protocol should be stored in a format that is 'readable' by a rule
based/execution engine \(such as GLIF\)\. So it becomes possible to
compute that if symptoms A, B and C are present, that according to
guideline X, diagnose Y is present\.
2\) In order to be able to establish the 'quality of a given
diagnoses' the reference to the guideline\(s\)/protocol\(s\) used should
be stored in the AT \(preferably in the protocol section\)\. So if
guideline X is used by a qualified person to come to diagnosis Y,
this could be re\-usable information \(for a person who adapted
guideline X\)\), while if guideline Z was used one would like to do
some additional testing \(for a person who rejected or is unfamiliair
with guideline Z\)
Is the assumption that, the GuidelineID in hte CARE\_ENTRY class
\(that Sam mentioned\) could be used to link \(if wanted\) the reference
to the actual computable criteria, correct?
Cheers,
Stef
---
## Post #10 by @system
Hi Rong, Sam and others,
---
## Post #11 by @system
> Hi Rong, Sam and others,
>
> **From:** [openehr-clinical-bounces@openehr.org](mailto:openehr-clinical-bounces@openehr.org) [mailto:[openehr-clinical-bounces@openehr.org](mailto:openehr-clinical-bounces@openehr.org)] **On Behalf Of** Rong Chen
> **Sent:** Wednesday, November 07, 2007 10:34 PM
> **To:** For openEHR clinical discussions
> **Subject:** Re: Evaluation Archetypes and assessment protocol
>
> > If one chooses the first option one should fill out the criteria each and every time, when writing down a diagnosis. Of course this can be automated by the application that is used. Thing is that the knowledge will be stored locally at application level. My feeling is that your suggestion to use the GuidelineID is in conjunction this option. Point is that if I refer to an external guidelines, data in that guideline isn't 'archetyped' and can't be used for computational purposes. So I have to translated those into an archetype.
>
> Guidelines can be computerized as well. There are published guidelines representation formats, e.g. GLIF for instance and their execution engines. Being rules based and decision trees like, personally I think they are not best modelled as archetypes since archetype model are designed to represent more 'static' clinical content model. In applications, EHRs and guidelines should work hand-in-hand: 1) guidelines need to query EHRs through standardized EHR representation format and query language; 2) EHRs need record what guidelines are used etc.
>
> __*[Chunlan Ma]*__
>
> __*I'm personally very interested in how clinical guidelines are used within an openEHR-based EHR system even though I'm new in this field. In my understanding, the*__ **guidelineId** *in CARE_ENTRY is used as a reference to an external clinical guideline; the* **protocol** *attribute in CARE_ENTRY is used as local guideline/protocol definition. Please correct me if this is wrong.*
>
> __*Because the type of*__ **protocol** __*attribute is*__ **ITEM_STRUCTURE**__*, it means the local protocol/guideline definition is archetyepable. Archetypes are very good to record facts, but don't know how it can be used to record rule criteria, such as when HbA1c is greater than 9%, do such.*__
Hi Chunlan and others,
The limitation is from the underlying information model not the archetype formalism. The archetype formalism is very generic so it can be used to constrain any object model. My thinking was limited to EHR model when I made the statement that guidelines are not best modelled as archetypes. If the underlying model is a good object model for representing guidelines, of course it's possible to model guidelines as archetypes.
Back to protocol attribute, quoted from the EHR Information Model, "This may take the form of references to guidelines, including manually followed and executable; knowledge references such as a paper in Medline; clinical reasons within a larger care process ". It seems to be intended as a reference to a guideline instead of the guideline itself fully archetyped in-line.
/Rong
---
## Post #12 by @ian.mcnicoll
Hi all,
As things currently stand I would agree with Rong that guidelines are not best modelled in archetypes. There are already a plethora of candidate formalisms for computable guidelines and their requirement goes well beyond what the current OpenEHR Ref model provides.
However, I have a current interest in adapting formal guidelines so that they can be used as semi-structured data entry templates i.e some of the content of the guideline can be used to populate the EHR - in this case it may well be appropriate to embed archetypes within guidelines e.g a guideline which contains some suggested instructions and actions such as drug therapies or referrals which can act as triggers in the EHR.
Ian
---
## Post #13 by @system
Hi Chunlan and others,
The limitation is from the underlying information model not the archetype formalism. The archetype formalism is very generic so it can be used to constrain any object model. My thinking was limited to EHR model when I made the statement that guidelines are not best modelled as archetypes. If the underlying model is a good object model for representing guidelines, of course it's possible to model guidelines as archetypes.
Back to protocol attribute, quoted from the EHR Information Model, "*This may take the form of references to guidelines, including manually followed and executable; knowledge references such as a paper in Medline; clinical reasons within a larger care process* ". It seems to be intended as a reference to a guideline instead of the guideline itself fully archetyped in-line.
__*[Chunlan Ma]*__
__*I don’t think my question was clear enough, but thanks that you interpreted it correctly. I understand that most time, protocol is used for recording how an assessment was done*__. *Yeah, my original question should be whether it’s possible to use protocol attribute to record local clinical guideline criteria/pathway. The reason why this question came out of my head is that I was concerned about merely a guideline_id is not enough information about how an diagnosis was made for an evaluation. Well, if it can be used as a reference of guideline which would provide more information than a guideline_id. Thanks!*
__*Cheers,*__
__*Chunlan*__
/Rong
---
## Post #14 by @system
Hi all,
As things currently stand I would agree with Rong that guidelines are not best modelled in archetypes. There are already a plethora of candidate formalisms for computable guidelines and their requirement goes well beyond what the current OpenEHR Ref model provides.
However, I have a current interest in adapting formal guidelines so that they can be used as semi-structured data entry templates i.e some of the content of the guideline can be used to populate the EHR - in this case it may well be appropriate to embed archetypes within guidelines e.g a guideline which contains some suggested instructions and actions such as drug therapies or referrals which can act as triggers in the EHR.
__*[Chunlan Ma]*__
__*I’m glad that someone pointed this. I’ve been thinking this for a while. My original thought is a mapping between the formal guidelines criteria and archetypes. Then archetype query statement can be generated automatically when the data need to be populated from EHR. I think our thoughts have some similarities.*__
__*Cheers,*__
__*Chunlan*__
Ian
---
**Canonical:** https://discourse.openehr.org/t/evaluation-archetypes-and-assessment-protocol/14689
**Original content:** https://discourse.openehr.org/t/evaluation-archetypes-and-assessment-protocol/14689