# Semi structured narrative data
**URL:** https://discourse.openehr.org/t/semi-structured-narrative-data/2007
**Category:** Apps
**Created:** [30 October 2021 15:44 UTC](https://discourse.openehr.org/t/semi-structured-narrative-data/2007 "2021-10-30T15:44:06Z")
**Posts on this page:** 20
**Page:** 1
### Author:  [@joostholslag](https://discourse.openehr.org/u/joostholslag)
#### Post date: [30 October 2021 15:44 UTC](https://discourse.openehr.org/t/semi-structured-narrative-data/2007/1 "2021-10-30T15:44:07Z")
A lot of our clinical data is in narrative reports form. Let’s say for EVALUATION.clinical\_synopsis.synopsis there’s a DV\_TEXT that contains a paragraph of text where some words in an individual sentence could be mapped to e.g. SNOMED.
e.g. :The patient has shown signs of dysuria([http://snomedct.info/id/](http://snomedct.info/id/) 49650001\>) warranting a urinary sediment to exclude a UTI ([http://snomedct.info/id/68566005](http://snomedct.info/id/68566005)\>). "
Currently it’s only possible to map the entire DV\_TEXT to a code system like SNOMED. The deprecated DV\_PARAGRAPH offered the option to ‘concat’ multiple DV\_TEXTs so that you could have separate DV\_TEXTs like (syntax probably incorrect) :
1. DV\_TEXT.value = “The patient has shown signs of”
2. DV\_TEXT.value = “dysuria”
DV\_TEXT.mapping: ([http://snomedct.info/id/49650001](http://snomedct.info/id/49650001)\>)
3. DV\_TEXT.value = “warranting a urinary sediment to exclude a”
4. DV\_TEXT.value = “UTI”
DV\_TEXT.mapping: “([http://snomedct.info/id/68566005](http://snomedct.info/id/68566005)\>)”
The use would be to offer the user possibly relevant similar reports or information from the EHR using SNOMED to find synonyms. Not to query based on definitive data e.g. “has UTI”
I understood DV\_PARAGRAPH was deprecated in favour of markdown. But markdown doesn’t support term mappings. So what is the current advice how to achieve this. And do other implementors share this problem?
I know the proper openEHR way would be to to do fully structured data where the `entry` is the smallest unit of information. So the dysuria should be a CLUSTER.symptom\_sign and the UTI possibly a EVALUATION.differential\_diagnoses. But I’m struggling to imagine a user interface that facilitates recording all narrative information in such a strictly structured way, without driving clinicians crazy.
(The EVALUATION.clinical\_synopsis states recording (semi) structured data as misuse btw. So unless you know what your doing, don’t just implement what I described above.)
---
### Author:  [@ian.mcnicoll](https://discourse.openehr.org/u/ian.mcnicoll)
#### Post date: [30 October 2021 16:00 UTC](https://discourse.openehr.org/t/semi-structured-narrative-data/2007/2 "2021-10-30T16:00:18Z")
My MSc was on precisely that topic!!
> **[Supporting Narrative-Based Medicine in GP Systems | PDF | Evidence Based...](https://www.scribd.com/document/50351864/Supporting-Narrative-based-medicine-in-GP-systems)**
>
> Scribd is the world's largest social reading and publishing site.
or at a Dropbox link [Dropbox - 50351864-Supporting-Narrative-based-medicine-in-GP-systems.pdf - Simplify your life](https://www.dropbox.com/s/b9mr9fse3zxpvf5/50351864-Supporting-Narrative-based-medicine-in-GP-systems.pdf?dl=0Skip) to page 57!!
I thin this would require something like Markdown but with custom markup. CDA had a go at something similar but I think you need go well beyond just embedding Snomed terms to embedding links to all sorts of Structured entries ( e.g prescriptions.
---
### Author:  [@borut.jures](https://discourse.openehr.org/u/borut.jures)
#### Post date: [30 October 2021 17:02 UTC](https://discourse.openehr.org/t/semi-structured-narrative-data/2007/3 "2021-10-30T17:02:15Z")
_I’m new to SNOMED (and many other things here) so I might be completely wrong._
Would it be possible to use SNOMED to check against everything a clinician is typing? Computers and SNOMED APIs are fast enough to do this without a clinician noticing any delays.
If a partialy typed word maps to something in SNOMED, a full term would be offered as auto-suggestion (like when we write on a phone). A clinical can select the offered term or continue writing.
A similar checks could be done to other structured data repositories.
A similar solutions probably already exist for popular markdown web components (I’ve seen some but would have to search again). They would need to be adapted to use SNOMED instead of whathever they are using for their example.
* * *
Edit: And one day we will be able to replace “typed word” with “spoken word”.
* * *
Edit #2: Of course the system would still have to put everything into a DV\_TEXT or some other mappings but this is “trivial” after the clinician selected/indicated what he/she wants to record. My suggestion is only about the data entry type that wouldn’t drive the clinicians crazy.
---
### Author:  [@joostholslag](https://discourse.openehr.org/u/joostholslag)
#### Post date: [30 October 2021 18:01 UTC](https://discourse.openehr.org/t/semi-structured-narrative-data/2007/4 "2021-10-30T18:01:28Z")
> [@borut.jures](#):
>
> A similar solutions probably already exist for popular markdown web components (I’ve seen some but would have to search again). They would need to be adapted to use SNOMED instead of whathever they are using for their example.
Hi Borut, thanks for chipping in:D you’re completely right, systems like what you describe exist. e.g. [https://twitter.com/mwardle/status/1452920467799609345?s=20](https://twitter.com/mwardle/status/1452920467799609345?s=20)
What I’m looking for is how to store that code mapping data in an openEHR system. Preferably in a standardised way. Compared to [TERM\_MAPPING](https://specifications.openehr.org/releases/RM/latest/data_types.html#_term_mapping_class) markdown only offers the inline text (`.value`) the link (`code_string`) and a text “title”, not a `match`, `purpose`, `terminology_id` nor `preffered_term`.
So apart from that it would be a 3rd/4th way of doing mappings, it also has fewer features, and is non standardised. It’s debatable wether a term\_mapping/binding should be openEHR specific.
@edit2 If it’s trivial could you give an example syntax of what a DV\_TEXT instance would look like, please?
---
### Author:  [@joostholslag](https://discourse.openehr.org/u/joostholslag)
#### Post date: [30 October 2021 18:04 UTC](https://discourse.openehr.org/t/semi-structured-narrative-data/2007/5 "2021-10-30T18:04:15Z")
Hi Ian,
Really curious. But do I really have to pay to view the doc? What would Markdown with custom markup look like? See my comments on Boruts post why I’m not too enthousiastic about the idea.
I fully agree it should go well beyond SNOMED. I’d especially like us to solve linking to other openEHR datapoints in the EHR.
edit: Should it be possible to record a link in a DV\_TEXT that is validated against [Ehr\_scheme](https://specifications.openehr.org/releases/RM/latest/data_types.html#_definitions)?
---
### Author:  [@borut.jures](https://discourse.openehr.org/u/borut.jures)
#### Post date: [30 October 2021 18:18 UTC](https://discourse.openehr.org/t/semi-structured-narrative-data/2007/6 "2021-10-30T18:18:07Z")
> [@joostholslag](#):
>
> If it’s trivial could you give an example syntax of what a DV\_TEXT instance would look like, please?
The DV\_PARAGRAPH example could be the end result of what the clinicians enter. But I’m not qualified to discuss the DV\_ types (I have read about them only once so far - I’ll know more after my 3rd or Nth reading).
My comment was only for the part about using markdown for data entry. Since the markdown field is free-form text, it needs to be converted into DV\_TEXT or other structures.
It looked to me that you would like markdown to work the same way as DV\_PARAGRAPH. My suggestion is to **separate** data entry (using free form text with auto-suggestions) and create DV\_TEXT or other structures in the background. I would expect clinicians aren’t too happy to “build” these structures during their data entry. They shouldn’t be bothered with that if we can perform the transformation in the background.
---
### Author:  [@joostholslag](https://discourse.openehr.org/u/joostholslag)
#### Post date: [30 October 2021 18:28 UTC](https://discourse.openehr.org/t/semi-structured-narrative-data/2007/7 "2021-10-30T18:28:06Z")
> [@borut.jures](#):
>
> My comment was only for the part about using markdown for data entry. Since the markdown field is free-form text, it needs to be converted into DV\_TEXT or other structures.
>
> It looked to me that you would like markdown to work the same way as DV\_PARAGRAPH. My suggestion is to **separate** data entry (using free form text with auto-suggestions) and create DV\_TEXT or other structures in the background. I would expect clinicians aren’t too happy to “build” these structures during their data entry. They shouldn’t be bothered with that if we can perform the transformation in the background.
A DV\_TEXT.value can contain the text in markdown format, if DV\_TEXT.formattign is set to `markdown`. That’s not the issue. The issue is how to then query based on snomed codes, and the other context info, like what kind of match it is (exact, narrower, broader etc), how to recognize it’s snomed (not loinc), the purpose of the mapping, the preferred term displayed etc. And this with the openEHR premise of a specified data format where different client apps can natively interact with different backend CDRs.
I agree you want to seperate partly data entry from storage. You probably want user confirmation on the mapping, and may want to suggest a fully structured form based on the text input. And you definately do not want to render separate text fields for the 4 data parts I previously described.
---
### Author:  [@ian.mcnicoll](https://discourse.openehr.org/u/ian.mcnicoll)
#### Post date: [1 November 2021 11:17 UTC](https://discourse.openehr.org/t/semi-structured-narrative-data/2007/8 "2021-11-01T11:17:07Z")
I think the only way to do this is to embed links to the relevant Entries, which contain the SNOMED codes, in the usual way.
Apart from anything else how do you know ‘Essential Hypertension’ in the narrative (with an embeddedSNOMED code) , is a diagnosis, and not a family history , or a reason for encounter.
The embed of the codes is the least of the problems!!
---
### Author:  [@joostholslag](https://discourse.openehr.org/u/joostholslag)
#### Post date: [1 November 2021 12:43 UTC](https://discourse.openehr.org/t/semi-structured-narrative-data/2007/9 "2021-11-01T12:43:04Z")
> [@ian.mcnicoll](#):
>
> Apart from anything else how do you know ‘Essential Hypertension’ in the narrative (with an embeddedSNOMED code) , is a diagnosis, and not a family history , or a reason for encounter.
If it’s just about showing related information, that the user then skims through, it’s ok not to know it’s one or the other. Both may be relevant. I’m not proposing decision logic here, only presenting information to the user.
> The embed of the codes is the least of the problems!!
So how would you do it?
---
### Author:  [@ian.mcnicoll](https://discourse.openehr.org/u/ian.mcnicoll)
#### Post date: [1 November 2021 12:52 UTC](https://discourse.openehr.org/t/semi-structured-narrative-data/2007/10 "2021-11-01T12:52:24Z")
> [@joostholslag](#):
>
> it’s ok not to know it’s one or the other.
I don’t understand why you are capturing SNOMED CT codes then. I’m not understanding clearly!!
---
### Author:  [@joostholslag](https://discourse.openehr.org/u/joostholslag)
#### Post date: [1 November 2021 13:23 UTC](https://discourse.openehr.org/t/semi-structured-narrative-data/2007/11 "2021-11-01T13:23:08Z")
Given that most data is currently free text narrative data: progress notes/clinical synopsis etc. It’s a good feature to search through those. e.g. give me anything related to “urinary tract infection”. SNOMED is really useful to query on synonyms: UTI, Bacterial urinary infection etc. etc.
If words in the free text are classified by the original author using SNOMED codes that could would make the results a lot more precise instead of string matching. So you will want to record the SNOMED code with/in the text.
The search usecase I describe is a lot less specific and quicker compared to the specific AQL queries in openEHR. Where the query should be of very high sensitivity and specificity to be fit for e.g. decision support. And thus take lot’s of time and effort to get precisely right.
Maybe this screenshot from our demo app (in Dutch) will help.
---
### Author:  [@ian.mcnicoll](https://discourse.openehr.org/u/ian.mcnicoll)
#### Post date: [1 November 2021 14:12 UTC](https://discourse.openehr.org/t/semi-structured-narrative-data/2007/12 "2021-11-01T14:12:35Z")
In that case, why not just put some kind of markdown link in there?
I think he may have a [UTI](https://snomed.info/12345678)
---
### Author:  [@joostholslag](https://discourse.openehr.org/u/joostholslag)
#### Post date: [1 November 2021 15:38 UTC](https://discourse.openehr.org/t/semi-structured-narrative-data/2007/13 "2021-11-01T15:38:40Z")
Hhaha yes, glad you understood the Dutch.
Yes putting it in markdown can work, but it has several downsides, see my post above: [Semi structured narrative data - #7 by joostholslag](https://discourse.openehr.org/t/semi-structured-narrative-data/2007/7)
---
### Author:  [@thomas.beale](https://discourse.openehr.org/u/thomas.beale)
#### Post date: [2 November 2021 17:11 UTC](https://discourse.openehr.org/t/semi-structured-narrative-data/2007/14 "2021-11-02T17:11:34Z")
This post highlights one of the challenges with moving away from fully structured/atomised text representation (the old DV\_PARAGRAPH approach) to narrative-oriented representation (the DV\_TEXT markdown approach).
The way to think about these two representations is that DV\_PARAGRAPH is something like a post-parse AST (abstract syntax tree) representation, i.e. the kind of in-memory structured object tree that results from parsing some text into pieces.
The markdown representation is a pre-parse representation.
Narrative is nice (or at least acceptable) for authoring and reading, but bad for computing; that’s why you parse it.
Structure is great for computing, but annoying for authoring and reading.
Given that we have adopted a markdown narrative representation for lumps of text larger than a single atom - i.e. paragraphs, or ‘lines of text’ etc, we need a way to represent terminology mappings.
This could be done as links, as @ian.mcnicoll suggested below - the question is links to what? SNOMED Uris like `http://snomed.info/sct/id/1234567890` can be constructed, but they won’t function like URLs, i.e. they are not really links.
Another approach would be to write them inline, to be processed by another layer, i.e. they don’t constitute markdown as such. To achieve this, some means similar to markdown linking `[]` has to be used to establish which words the coding applies to.
This might be something like the following:
```auto
"^dysuria^[snomedct::49650001] warranting a urinary sediment to exclude a ^UTI^[snomedct::68566005]"
```
If the text you want is exactly the same as the preferred term, we could do:
```auto
"[snomedct::49650001|dysuria|] warranting a ..."
```
But the term for 68566005 is “urinary tract infection”, not “UTI”.
The above is not super-readable, so some better choice of syntax might be possible:
```auto
"/dysuria/[snomedct::49650001] warranting a urinary sediment to exclude a /urinary traction infection/[snomedct::68566005]"
```
I used ‘urinary traction infection’ as the text to emphasise the fact that the delimiters (here, `//`) are needed to indicate the text the mapping applies to.
I’m sure someone else can do better, but what we aim to do here with this kind of solution is:
- define (yet another) openEHR micro-syntax that allows terminology mappings to be represented in structured plain text that is not markdown (i.e. won’t do something weird when seen by a markdown renderer) but can be reliably parsed into a structured form.
Edit: I tested the string above in the [CommonMark tester page](https://spec.commonmark.org/dingus/); it comes through OK.
---
### Author:  [@joostholslag](https://discourse.openehr.org/u/joostholslag)
#### Post date: [2 November 2021 17:22 UTC](https://discourse.openehr.org/t/semi-structured-narrative-data/2007/15 "2021-11-02T17:22:26Z")
Thank you for your elaborate response. I also like that you proposed a syntax. It makes it easier to discuss.
What I don’t like is that it’s yet another micro syntax and yet another way of dealing with mappings in openEHR. And, as stated before that it’s less feature rich than TERM\_MAPPINGs. But mostly that it will require a custom parser for DV\_TEXT data instead of just treating it as markdown. Besides the coding work (we’ll have to do some anyways) it gives uglyness, if e.g. you sync the text to a non-openEHR system. You’ll have to clean out the openEHR micro syntax. Or what if markdown introduces a different function for the openEHR unique syntax/
The thought that just came to mind, could we do it the other way around, instead of annotating part of the text with a mapping, could we add an attribute to the TERM\_MAPPING that’s part of the DV\_TEXt that records what part of the text the mapping refers to?
---
### Author:  [@thomas.beale](https://discourse.openehr.org/u/thomas.beale)
#### Post date: [2 November 2021 18:06 UTC](https://discourse.openehr.org/t/semi-structured-narrative-data/2007/16 "2021-11-02T18:06:04Z")
> [@joostholslag](#):
>
> What I don’t like is that it’s yet another micro syntax and yet another way of dealing with mappings in openEHR
Welcome to the (horrible) world of markdown 😉
> [@joostholslag](#):
>
> could we add an attribute to the TERM\_MAPPING that’s part of the DV\_TEXt that records what part of the text the mapping refers to?
Theoretically yes. You’d have a single DV\_TEXT for the sentence
“dysuria warranting a urinary sediment to exclude a UTI” with TERM\_MAPPINGS carrying some sort of position data like:
mappings:
- [1]
- charpos = |1…7|
- target = [snomedct::49650001]
- [2]
- charpos = |51…53|
- target = [snomedct::68566005]
That would not be super-reliable, since people would get mixed up on whether the character positions referred to the number of characters in the output, or in the input (potentially full of other markdown text like links etc).
Another possibility might be to adopt a kind of referencing/citation approach. E.g.
“dysuria[1] warranting a urinary sediment to exclude a UTI[2]”
The numbers [1] and [2] refer to the 1st and 2nd items in the `mappings` list. We assume that when there is a single word to have a mapped term, we just do “word[n]”. If there are more words, then we need some delims, e.g. “dysuria[1] warranting a urinary sediment to exclude a (urinary traction infection)[2]”
We’d have to mess around to figure out which kind of brackets or other delimiters would work best, but this is at least readable, and would not even break the RM.
---
### Author:  [@ian.mcnicoll](https://discourse.openehr.org/u/ian.mcnicoll)
#### Post date: [2 November 2021 18:49 UTC](https://discourse.openehr.org/t/semi-structured-narrative-data/2007/17 "2021-11-02T18:49:44Z")
I’ll be honest and say right way that capturing these snomed codes like this without context, and just as context-free mappings, is probably a very bad idea!!.
But (if you insist!!), why not just use something like my simple URI type of idea, and just lob the list of SNOMED codes into mappings against he whole DV\_TEXT element. I don’t think you really need character positions , the codes themselves align the correct positions. And if there are duplicate SNOMED codes - well so what!!
---
### Author:  [@joostholslag](https://discourse.openehr.org/u/joostholslag)
#### Post date: [2 November 2021 18:52 UTC](https://discourse.openehr.org/t/semi-structured-narrative-data/2007/18 "2021-11-02T18:52:21Z")
Yes, my first thought was something like the `charpos` as you described. But I agree it can get mixed up pretty easily. Maybe it could be solvable in an acceptable way by implementers, since it doesn’t have to be super reliable, for today’s usecase. But I also like the other possibility, since mappings are conceptually quite close to citations, and it reuses the TERM\_MAPPINGS. I agree it’s very well readable. Would there be a way not to break the markdown syntax: not just valid, but also something that makes sense without openEHR. There is the footnote syntax `(`[^1]` )` but it doesn’t have delimiters [afai can tell](https://www.markdownguide.org/extended-syntax/). And we still have to go from footnote to TERM\_MAPPING. We could put a uri in the footnote that reference the term mapping.
But how about putting an Ehr\_scheme uri in the url part of the markdown link syntax?
e.g.
`"[dysuria](ehr://system_id/ehr_id/top_level_structure_locator/path_inside_top_level_structure|mapping1) warranting a [urinary sediment to exclude a [urinary tract infection] (ehr://system_id/ehr_id/top_level_structure_locator/path_inside_top_level_structure|mapping2)"`
---
### Author:  [@joostholslag](https://discourse.openehr.org/u/joostholslag)
#### Post date: [2 November 2021 18:57 UTC](https://discourse.openehr.org/t/semi-structured-narrative-data/2007/19 "2021-11-02T18:57:49Z")
Why do you think it’s a bad idea? Whole of SNOMED is without context, right? The value of the feature is clear right, would you solve it another way?
You could be right about not needing character positions. It indeed is mostly about the report as a whole that should be highlighted in a query. But you probably still would want to highlight the matching characters in the text. But that could be done by matching again when rendering the result of the query, may not be necessary to store the characters the original match relates to. But I struggle to accept that such a simple problem can not be solved well 🙄
I’m curious for the view of others, maybe @bna has an opinion?
---
### Author:  [@ian.mcnicoll](https://discourse.openehr.org/u/ian.mcnicoll)
#### Post date: [2 November 2021 19:09 UTC](https://discourse.openehr.org/t/semi-structured-narrative-data/2007/20 "2021-11-02T19:09:26Z")
dysuria warranting a urinary sediment to exclude a urinary tract infection
Bad idea - it is very, very easy for the wrong SNOMED codes to be picked up - this is a good example because actually this is arguably not a diagnosis at all, but an indication for an investigation.
Or better example
[Painless haematuria](openehr\_mapping://snomed.info/sct/ 197938001) but warrants a urinary sediment to exclude a urinary tract infection
[Next page](https://discourse.openehr.org/t/semi-structured-narrative-data/2007.md?page=2)