Simplified JSON data formats for openEHR

For those interested, here is a wiki page on which we are refining so-called simplified (JSON) data formats for representing EHR data when talking through openEHR REST APIs, and maybe in other contexts.

This is likely to be relevant if you use EhrScape, EhrBase, EtherCIS among other products.

1 Like

Great, does this include both template and archetype in one json?

The JSON ā€˜templatesā€™ referred to here are close-to-runtime data instances (i.e. just need the multiple fields expanded and values filled out) of an OPT, but in this ā€˜compactā€™ JSON format which programmers seem to prefer, and which is definitely more efficient than standard JSON.

To get an idea of what this kind of JSON looks like, see the Simplified Data Template example (the Lab one) in this draft spec.

That example has programmer-friendly paths, not just compressed data values, and was originally developed by Better (i.e. Marand).

Just to be clear, the wiki page being talked about here is just trying to reach agreement on ā€˜compactā€™ JSON form(s) the leaf data items. Itā€™s a work in progressā€¦

In the stringified datatype format for DV_CODED_TEXT, I wonder if we need to distinguish to use casesā€¦

  1. Where the rubric/value (text) is intended to be read-only /advisory i.e the original intent of

local::at00034 | Large cuff |

or just

at00034 | Large cuff |

as Better allow in AQL and is very useful in other documentation /visualisation

  1. Where the text/value/rubric is intended to be written into the data

e.g the Ocean Temple Designer shortcut for default values

local::at00034:: Large cuff

Iā€™m not clear that we need do both variants - perhaps the context of use makes it clear??

I had the need to look into this because weā€™d like to transition from the better API to the openEHR API. Currently we can only use the canonical model for compositions and as we work with partners that donā€™t need to know openEHR this is a barrier.

In my experience if you want to simplify, you will need to get rid of archetype idā€™s for the simplified model. I like the FLAT model as proposed here (flath , directory path stile JSON), however we should also offer a structured JSON for s simplified model. We need this because most JavaScript Frameworks nowadays can easily access structured JSON paths, which makes it simpler to work with.

I would very much like to help out with developing this idea (although I lack the in-depth technical knowledge). Is there a timeline for developing this further?

Gr,

Wouter

We have created a draft spec here.

Your input - review / suggestions would be most welcome.

The adoption of the Better FLAT format and web template by EhrBase (with some assistance from Better, I understand), gives us a clear path, to make use of this as a de-facto V0 of the draft spec Thomas pointed to.

My current dvice to clients is to use the openEHR REST API for all of the resources other than /composition - where they should use the ehrscape / ethercis endpoints which support the FLAT formats. In due course, Iā€™m sure everyone will move to the formal V1 spec of the Simplified Data Formats but we can anticipate this will take a little time, and may well introduce breaking changes.

Note that EhrBase currently only offers FLAT not STRUCTURED but based on work down by Rob Tweed of Ripple, I am increasingly confident that there is a standard algorithm that will switch between the 2, without reference back to the web template or .opt.

2 Likes

Can we here continue the thread regarding Simplified/Serial JSON formats and associated REST call headers and content-types/mimetypes) with input from some recent openEHR SEC discussions Feb 22 and March 8, to make it easier to include more people (in addition to the SEC)?

@birger.haarbrandt and I discussed this issue briefly (March 3) and he mentioned that we now have open sourced solutions from both Better and EHRbase that can be used as a reference to point to from specs. There is also a set of conformance tests from Better that they both use. Some related links follow (more details will likely be added later)ā€¦

Documentation and/or Source code from EHRbase:

Source code from Better:

Documents to check for possible change points in, e.g. add references to the open source implementations/tests, update references/examples of formats and REST APIs:

Possible REST spec change points

  • General info in API Overview mime types etc. The application/openehr.wt.flat+json as already suggested likely works, also maybe the application/openehr.wt.structured+json should be added. (Any possible later version of the format could be called ā€¦wt2ā€¦ etc.)
  • The /composition call in the EHR API needs a way to add the name (ID) of the template, a suggestion is to use a HTTP header named openEHR-TEMPLATE_ID (named after the corresponding class and prefixed as other openEHR-specific headers in the REST API) or alternatively openEHR-LOCATABLE.archetype_details.template_id We might also want to add request/response examples.
  • Weā€™ll need to add a way to download ā€œweb templateā€ representations of Operational Templates, likely under the definitions API endpoint /definition/template/adl1.4/{template_id} in the REST spec. Would application/openehr.wt+json work? The thing returned would be the same Web template representation format you get from EHRscape API Explorer see image below to navigate to the right spot. EHRbase uses the same format (documentation including pedagogical example) and it is very useful for creating user interface code/tools. You can also download templates in this format from the Archetype Designer GUI
  • Suggested by @ian.mcnicoll below: IF any of the open source implementations contains an example implementation, then /definition/template/adl1.4/{template_id}/example could be added with behaviour corresponding to /rest/v1/template/{templateId}/example in the EHRscape EHR API (Maybe needs to formally go into in version 1.1.* of the REST spec since it is a new endpoint?)
  • If .../example is added, then maybe also a call corresponding to the EHRscape /rest/v1/template/{templateId}/fields should be considered?

(This is a ā€œwikiā€ post, so others than the original author can edit it.)

2 Likes

The 3 pressing issues IMO are agreeingā€¦

  1. How to work with the FLAT compositions via the openEHR REST API -agree additional Content-types, agree how to carry extra parameters like templateID and commitername.

  2. Add Get WebTemplate from CDR - as per Erikā€™s last requirement ? an Accept header?

  3. Add Get ā€˜Example compositionā€™ calls as part of /definition - @pablo has been doing something on this for xml and the Better /template/examples really unlocks the struggle that people have to understand what a composition object should look like.

1 Like

Corrections, additions and other small changes on Content-Type and content negotiation (in API Overview) or extra headers (in EHR Api) can be done easily as part of upcoming Release 1.0.2, as long as that is the actual reality on how the real implementation works (which I believe it is).

Similarly, to add an extra header and examples under Definitions API might be ok. But I just wonder if:

  1. the WebTemplate is conceptually, from the perspective of a RESTful resources, the same as an OPT-Template or not ? ā€¦otherwise we should add a new definition type (i.e. /definition/webtemplate/v1/{template_id})
  2. if it is another format for an OPT, agnostic of the ADL language being used, then we should have /definition/template/webtemplate/{template_id}
  3. if is just another representation of an OPT1.4, then perhaps only an extra header on /definition/ad1.4/{template_id} would work, and is doable for Release 1.0.2

Which one from the above you think is more applicable?

About an extra /example as a subresource of template is a good idea, but I would do that as part of Release 1.1.0

1 Like

It is directly derived from the .opt but re-formulates the content (not based on AOM) to be much easier to parse (from a third-party dev POV) and also adds the shortened path segments that the FLAT/STRUCTURED composition formats use.

I think it is close enough to justify just an alternative Content-Type in line with the discussions around adding support for FLAT compositions

Content-Type: application/openehr.wt+json

THis would work nicely with the existing xml .opt : Content-Type: application/openehr.wt+json
or any future JSONified .opt (pure AOM) Content-Type: application/json and be very consistent with the proposed approach to /composition

3 Likes

There might be one piece missing to make openEHR web application serving easierā€¦
Do we need to recommend (or standardize) a way to store/retrieve versioned additional resources commonly needed for serving UIs, for example form definitions (to be retrieved by form renderer components), images used as UI elements etc?

One way could be to recommend a path and pattern similar to the GET adl2 template adl2 template: /definition/template/adl2/{template_id}/{version_pattern}, so something like /definition/ui/{id-of-the-component}/{version_pattern} (and return latest version when version pattern is omitted).

Another (likely wiser) suggestion would be to not reinvent the wheel and instead recommend using something already standardized for this purpose like WebDAV (or a subset of it) that is already supported by many (web) server and client products (and even naively by many operating systems). If a WebDAV recommendation is considered a good idea we could include that in the 1.0.2 release. If so, should we also recommend a URI mount point for the openEHR-related webdav-repository like /definition/dav/ or not?

P.S.
If comparing to Betterā€™s EhrScape we are talking of storing things like the ā€¦/form API call does: image

Hi Erik,

the question is if the should be the scope of the openEHR standard. From my perspective, the layer above the platform should be more losely coupled to EHR specification. Would be great to get some opinions on this first before going this route. We need to make sure that besides all the standardization we still have enough room for quick innovation and degrees of freedom.

Birger

1 Like

Yes, I am not at all sure we should say/recommend anything about accessing resources/files with content that is not covered by any openEHR standardization. Only asking.

(Looking deeper into for example WebDAV, the way to access a specific older version might not be super-obvious to the average developer.)

I think the general idea of having a standard way to reference UI artefacts associated with semantic entities is worth thinking about. In Task Planning, tasks can refer to ā€˜form idsā€™ and/or ā€˜template idsā€™. The former is the one an application really wants; for now, itā€™s just an open string field.

Good discussions, but letā€™s focus now on simplified format here, UI related things can be discussed on a separate thread.

To amend REST specs, I created [SPECITS-57] Update info about simplified format - openEHR JIRA and [SPECITS-58] Add support for /example sub-resource - openEHR JIRA

3 Likes

It would be good to have these simplified formats as my experience with the better formats is very good I am happy to see this moving in that direction.

In the get template call we might consider adding a header for template language so you get the template in a certain language.

It would be perhaps a good idea to work on providing a JSON schema (under /definition) variant based on the webtemplate most information should be available in either the webtemplate or the opt.

1 Like

Great suggestions @wouterzanen. Weā€™ll have to discuss with implementers (ping @birger.haarbrandt and @matijap) what goes into first vs next version. We want to avoid delays of a first version.

For language choice, do you think the standard http header Accept-Language would work for requests, with a Content-Language header in the response if requested language(s) was(were) available - and returning a 406 (Not Acceptable) error code if the template was not available in (any of) the requested language(s)?

So a common use case will be that one specific language is requested and youā€™ll either get a response with it or a 406 error if not available. And if you donā€™t send any Accept-Language header in your request you get some kind of default or all languages?

Should the above language featureā€¦

  • ā€¦only be for the calls /definition/template/adl1.4/{template_id} and the /definition/template/adl1.4/{template_id}/example (and /rest/v1/template/{templateId}/fields if added)ā€¦
  • ā€¦or also apply to (some of) the .../composition calls in the EHR API?

@wouterzanen, regarding your last suggestion about JSON schema, I guess you mean a schema defining the rules to validate instances (EHR content) of a particular (web-)templateā€™s flat or structured data instance form? There is a mime-type application/schema+json for JSON schema but in order to separate flat, structured and future other formats without adding extra API calls we could use http Accept headers like application/openehr.wt.flat.schema+json and application/openehr.wt.structured.schema+json in requests to the the definitions API endpoint /definition/template/adl1.4/{template_id}.

Would that cover the needs?

I am still confused about which format will be considered the ā€œdefaultā€ simplified JSON format.
I see in the wiki that the ā€œcompact formatā€ is preferred, but Iā€™ve never come across any CDR offering that uses this. I think having a string only representation of the value like ā€œ[snomed_ct(3.1)::3415004|cyanosis|]ā€ has itā€™s advantages, but for most use cases itā€™s easier to work with Betterā€™s flat format.

Also, is the ā€œPreferred compact formatā€ flattened or structured?

Also, is it possible to use flattened paths, while keeping the value as structured json like so?

{
"path/obtained/by/concatenating/ids": {
  "|code": "238",
  "|value": "other care",
  "|terminology": "openehr"
  }
}

We definitely need to work more on the documentation for these simplified formats. Deciding which one will be the default will help a lot.

3 Likes

Trying to clear out things:

  • the wiki page you indicated is not a spec, just a SEC workspace we have working on the flat-format subject, so a placeholders for thoughts, etc
  • the format specs are in DEVELOPMENT state, and located here: Simplified Data Template (SDT)
  • these specs are also referred from latest (upcoming Release 1.0.2) of REST API specs: Overview # alternative-data-formats

Nevertheless, documentation might not be sufficient or not clear enough - therefore weā€™re happy to receive feedback.

4 Likes