JSON Schema and OpenAPI: current state, and how to progress

I agree.

With “optional” I had modelers (and my use case) in mind.

ADL is not optional for CKM and CDR builders.

In practice you need to be able to do both. Tools do most of the work for sure, but no tool will ever be perfect. Search and replace for changing a word throughout an archetype, for example. Or transforming an archetype from one class to another. AFAIK no current tool does either of those.

As for doing this in a generic language like JSON… Please, no! :scream: We’ve seen what operational templates look like, whether in XML, JSON or whatever. ADL is heaven compared to that :innocent:

3 Likes

Thank you for the great insight into the life of a clinical modeler.

Has nobody asked what modelers need to do their work?

It is clear to me that ADL is useful to you. Even with additional features added to the tools, ADL+text editor is the most flexible format. However mentioned search and replace using the semantics would be a great feature.

They say the user is king/queen. And when a queen says :scream: about JSON, we should stop pushing it :flushed:

Nobody is pushing developers to use a programming language they don’t like. Developers shouldn’t push the modelers either.

p.s.
I probably don’t know all the needed background information why some prefer JSON over ADL. I’m new at openEHR and unencumbered about the past. I just see the future :innocent:

1 Like

Good news from GitHub: “decided to go further with current JSON schema (generated with Archie from BMM)

Can this make few of us that like BMM happy and everybody who prefers JSON schema to have the format they like? :smiling_face_with_three_hearts:

1 Like

Hi Borut, I’m late to the party…

At implementation level and runtime, you can use any format you like, as long as you can transform back and forth to/from ADL or OPT, which means: the format you use your be semantically equivalent to the standard ones. So if JSON, XML, YAML, CSV or whatever simplifies your work at the technology level, USE IT!

Though, what you think might simplify your work today, might make if difficult in the future when things get complicated, like modeling a complex EMR with openEHR.

If you are new to openEHR, maybe you want to solve problems you don’t really have or that are already solved elsewhere. You just need the experience to go through the learning process, understand the specs, the tech available, and how to apply both to tour specific business requirements. This might take a while, so even if you are not sure of what you are doing, it’s all time invested in learning. Try, make mistakes, ask and learn. It could be a painful but enjoyable journey! (I’ve been there and made many mistakes, but learned a lot).

That is just my 2 cents :slight_smile:

All the best and welcome!

2 Likes

I needed some time to realise you’re right. I think you summed up the arguments for domain specific languages nicely by order of priority. But I also still feel openEHR DSLs are a big barrier to entry, from a learning and an implementation perspective. So I would still like us to make ‘more’ effort to do work based on standards. And I think there is more room for compromise on the requirements than we currently allow. But maybe that compromise is making more effort to support standards, I’m thinking of:

  • OpenAPI export of flattened templates
  • deprecating Odin for json
  • sharing libraries (not just archie Java)
  • creating subsets of standards instead of inventing everything ourselfselves (expression language, task planning etc)

But this is ultimately not my domain, so it’s up to the engineers to decide on these issues. But there’s a catch 22 to here: engineers able to build the stuff described are so well versed in openEHR they don’t benefit from the simplifications.

If you try hard enough you could use JSON instead of BMM (ODIN). But it is not the same experience editing a JSON compared to editing a BMM file. The JSON would be too complex for a human to edit. That is why DSLs are used. To simplify writing/editing. To not lose focus with formatting while trying to write your thoughts before you get distracted by another thought. To stay in the flow for as long as possible.

What if openEHR specifications web site had to be written in pure HTML since it is easier for others?
Let the author pick his tools for writing specifications.

BMM format (ODIN) is not that hard for anyone willing to spend a few days learning/programming. There are ANTLR parsers in almost any language (and ANTLR is a “standard”).

Why hasn’t somebody converted BMMs into JSON (e.g. OpenAPI) if that is THE answer? Using Archie solves the issue of having to know how to parse BMMs. It shouldn’t take more than a few days.

I propose somebody uses Archie and existing BMM files to prepare an initial proposal for the specifications in JSON. Then both formats may be compared and a decision can be made which one to use in the future.

Be careful with engineers :wink:
It is easy to convince non-engineers something is hard if we don’t feel like doing it. Or that it would take a loooong time to do it. Can you tell when they are telling you the facts? In medicine you ask for a second opinion to be sure.

1 Like

Well… replace json with json5 or yaml, and it is a direct and simple transformation. These file formats are very closely related, and BMM can just as well be expressed in another object serialisation format, for which proper tools are available in every editor. So, I do not agree with that part.

Writing and parsing BMM in json and yaml is already supported out of the box in Archie, without any changes to Archie.

2 Likes

This is my main point - Archie has a good BMM parser. Somebody should use it to convert them to a proposed replacement for BMM files.

The result would be JSON (or whatever format is selected) versions of the existing BMM files. Then the new format is ready to be considered.

p.s.
I’ll be happy to use the new format if it includes everything that is in the BMM files.

Currently UML is the primary source of the specifications. BMMs are generated from UML. Not much editing is required. Same would be true for the new format if UML remains the primary source.

But if the new format becomes the primary source, then it makes the difference how easy the editing is. Compared to ADL2, the BMM is verbose but JSON is even more verbose.

I believe that JSON is for computers. DSLs like ADL2/BMM are for humans.

JSON should be a result of a conversion from DSL not the primary source.

To me the question is not replacing BMM/ODIN but providing alternative formats generated from them so that the specifications may be used in formats that the developers are more familiar with.

1 Like

That is coming (Expression Language); but there will always be a JSON / YAML / xyz equivalent. As long as we make all of this bullet-proof, everyone gets what they want :wink:

That is my personal thinking as well, but some people like JSON etc, and to be fair, YAML is pretty acceptable in certain editors.

Yes, computationally it does have to be a serialisation of an in-memory meta-model, which is the BMM in this case.

I was writing an OPT2 to JSON converter. Since some sections in ADL2 are using ODIN grammar I added 4 toJson() methods to the ODIN classes in my ODIN parser. This returned JSON for the ODIN sections in ADL2.

Then I thought if those 4 toJson() methods will also convert BMM files to JSON. Yes, they do!

On the next run of my generator I saved JSON versions of all the BMM files:

{
  "bmm_version": "2.3",
  "rm_publisher": "openehr",
  "rm_release": "1.0.4",
  "model_name": "EHR",
  "schema_name": "rm_ehr",
  "schema_revision": "1.0.4.1",
  "schema_lifecycle_state": "stable",
  "schema_description": "openEHR Release 1.0.4 EHR schema",
  "schema_author": "Thomas Beale <thomas.beale@openehr.org>",
  "includes": {
    "1": {
      "id": "openehr_rm_structures_1.0.4"
    }
  },
  "packages": {
    "org.openehr.rm.ehr": {
      "name": "org.openehr.rm.ehr",
      "classes": [
        "EHR",
        "EHR_ACCESS",
        "EHR_STATUS",
        "ACCESS_CONTROL_SETTINGS"
      ]
    },
    "org.openehr.rm.composition": {
      "name": "org.openehr.rm.composition",
      "classes": [
        "COMPOSITION",
        "EVENT_CONTEXT"
      ],
      "packages": {
        "content": {
          "name": "content",
          "classes": "CONTENT_ITEM",
          "packages": {
            "navigation": {
              "name": "navigation",
              "classes": "SECTION"
            },
            "entry": {
              "name": "entry",
              "classes": [
                "ENTRY",
                "CARE_ENTRY",
                "ADMIN_ENTRY",
                "OBSERVATION",
                "EVALUATION",
                "INSTRUCTION",
                "ACTION",
                "ACTIVITY",
                "ISM_TRANSITION",
                "INSTRUCTION_DETAILS"
              ]
            },
            "integration": {
              "name": "integration",
              "classes": "GENERIC_ENTRY"
            }
          }
        }
      }
    }
  },
  "class_definitions": {
    "EHR": {
      "name": "EHR",
      "properties": {
        "system_id": {
          "name": "system_id",
          "type": "HIER_OBJECT_ID",
          "is_mandatory": true
        },
        "ehr_id": {
          "name": "ehr_id",
          "type": "HIER_OBJECT_ID",
          "is_mandatory": true
        },
        "time_created": {
          "name": "time_created",
          "type": "DV_DATE_TIME",
          "is_mandatory": true
        },
        "ehr_access": {
          "name": "ehr_access",
          "type": "OBJECT_REF",
          "is_mandatory": true
        },
        "ehr_status": {
          "name": "ehr_status",
          "type": "OBJECT_REF",
          "is_mandatory": true
        },
        "directory": {
          "name": "directory",
          "type": "OBJECT_REF"
        },
        "compositions": {
          "name": "compositions",
          "type_def": {
            "container_type": "List",
            "type": "OBJECT_REF"
          },
          "cardinality": []
        },
        "contributions": {
          "name": "contributions",
          "type_def": {
            "container_type": "List",
            "type": "OBJECT_REF"
          },
          "cardinality": [],
          "is_mandatory": true
        },
        "most_recent_composition": {
          "name": "most_recent_composition",
          "type": "COMPOSITION",
          "is_computed": true
        }
      }
    },
    "EHR_ACCESS": {
      "name": "EHR_ACCESS",
      "ancestors": "LOCATABLE",
      "properties": {
        "settings": {
          "name": "settings",
          "type": "ACCESS_CONTROL_SETTINGS"
        }
      }
    },
    "ACCESS_CONTROL_SETTINGS": {
      "name": "ACCESS_CONTROL_SETTINGS",
      "ancestors": "Any",
      "is_abstract": true
    },
    "EHR_STATUS": {
      "name": "EHR_STATUS",
      "ancestors": "LOCATABLE",
      "properties": {
        "subject": {
          "name": "subject",
          "type": "PARTY_SELF",
          "is_mandatory": true
        },
        "is_queryable": {
          "name": "is_queryable",
          "type": "Boolean",
          "is_mandatory": true
        },
        "is_modifiable": {
          "name": "is_modifiable",
          "type": "Boolean",
          "is_mandatory": true
        },
        "other_details": {
          "name": "other_details",
          "type": "ITEM_STRUCTURE"
        }
      }
    },
    "COMPOSITION": {
      "name": "COMPOSITION",
      "ancestors": "LOCATABLE",
      "properties": {
        "language": {
          "name": "language",
          "type": "CODE_PHRASE",
          "is_mandatory": true
        },
        "territory": {
          "name": "territory",
          "type": "CODE_PHRASE",
          "is_mandatory": true
        },
        "category": {
          "name": "category",
          "type": "DV_CODED_TEXT",
          "is_mandatory": true
        },
        "composer": {
          "name": "composer",
          "type": "PARTY_PROXY",
          "is_mandatory": true
        },
        "context": {
          "name": "context",
          "type": "EVENT_CONTEXT"
        },
        "content": {
          "name": "content",
          "type_def": {
            "container_type": "List",
            "type": "CONTENT_ITEM"
          },
          "cardinality": []
        }
      }
    },
    "EVENT_CONTEXT": {
      "name": "EVENT_CONTEXT",
      "ancestors": "PATHABLE",
      "properties": {
        "health_care_facility": {
          "name": "health_care_facility",
          "type": "PARTY_IDENTIFIED"
        },
        "start_time": {
          "name": "start_time",
          "type": "DV_DATE_TIME",
          "is_mandatory": true
        },
        "end_time": {
          "name": "end_time",
          "type": "DV_DATE_TIME"
        },
        "participations": {
          "name": "participations",
          "type_def": {
            "container_type": "List",
            "type": "PARTICIPATION"
          },
          "cardinality": []
        },
        "location": {
          "name": "location",
          "type": "String"
        },
        "setting": {
          "name": "setting",
          "type": "DV_CODED_TEXT",
          "is_mandatory": true
        },
        "other_context": {
          "name": "other_context",
          "type": "ITEM_STRUCTURE"
        }
      }
    },
    "CONTENT_ITEM": {
      "name": "CONTENT_ITEM",
      "ancestors": "LOCATABLE",
      "is_abstract": true
    },
    "SECTION": {
      "name": "SECTION",
      "ancestors": "CONTENT_ITEM",
      "properties": {
        "items": {
          "name": "items",
          "type_def": {
            "container_type": "List",
            "type": "CONTENT_ITEM"
          },
          "cardinality": []
        }
      }
    },
    "ENTRY": {
      "name": "ENTRY",
      "is_abstract": true,
      "ancestors": "CONTENT_ITEM",
      "properties": {
        "language": {
          "name": "language",
          "type": "CODE_PHRASE",
          "is_mandatory": true,
          "is_im_infrastructure": true
        },
        "encoding": {
          "name": "encoding",
          "type": "CODE_PHRASE",
          "is_mandatory": true,
          "is_im_infrastructure": true
        },
        "subject": {
          "name": "subject",
          "type": "PARTY_PROXY",
          "is_mandatory": true
        },
        "provider": {
          "name": "provider",
          "type": "PARTY_PROXY"
        },
        "other_participations": {
          "name": "other_participations",
          "type_def": {
            "container_type": "List",
            "type": "PARTICIPATION"
          },
          "cardinality": []
        },
        "workflow_id": {
          "name": "workflow_id",
          "type": "OBJECT_REF",
          "is_im_runtime": true
        }
      }
    },
    "ADMIN_ENTRY": {
      "name": "ADMIN_ENTRY",
      "ancestors": "ENTRY",
      "properties": {
        "data": {
          "name": "data",
          "type": "ITEM_STRUCTURE",
          "is_mandatory": true
        }
      }
    },
    "CARE_ENTRY": {
      "name": "CARE_ENTRY",
      "is_abstract": true,
      "documentation": "Abstract ENTRY subtype corresponding to any type of ENTRY in the clinical care cycle.",
      "ancestors": "ENTRY",
      "properties": {
        "protocol": {
          "name": "protocol",
          "type": "ITEM_STRUCTURE"
        },
        "guideline_id": {
          "name": "guideline_id",
          "type": "OBJECT_REF",
          "is_im_runtime": true
        }
      }
    },
    "OBSERVATION": {
      "name": "OBSERVATION",
      "documentation": "ENTRY subtype used to represent observation information in time, as either a single or multiple samples.",
      "ancestors": "CARE_ENTRY",
      "properties": {
        "data": {
          "name": "data",
          "documentation": "Data of the observation, in the form of a HISTORY of EVENTs.",
          "is_mandatory": true,
          "type_def": {
            "root_type": "HISTORY",
            "generic_parameters": "ITEM_STRUCTURE"
          }
        },
        "state": {
          "name": "state",
          "type_def": {
            "root_type": "HISTORY",
            "generic_parameters": "ITEM_STRUCTURE"
          }
        }
      }
    },
    "EVALUATION": {
      "name": "EVALUATION",
      "ancestors": "CARE_ENTRY",
      "properties": {
        "data": {
          "name": "data",
          "type": "ITEM_STRUCTURE",
          "is_mandatory": true
        }
      }
    },
    "INSTRUCTION": {
      "name": "INSTRUCTION",
      "ancestors": "CARE_ENTRY",
      "properties": {
        "narrative": {
          "name": "narrative",
          "type": "DV_TEXT",
          "is_mandatory": true
        },
        "expiry_time": {
          "name": "expiry_time",
          "type": "DV_DATE_TIME"
        },
        "wf_definition": {
          "name": "wf_definition",
          "type": "DV_PARSABLE",
          "is_im_runtime": true
        },
        "activities": {
          "name": "activities",
          "type_def": {
            "container_type": "List",
            "type": "ACTIVITY"
          },
          "cardinality": []
        }
      }
    },
    "ACTIVITY": {
      "name": "ACTIVITY",
      "ancestors": "LOCATABLE",
      "properties": {
        "description": {
          "name": "description",
          "type": "ITEM_STRUCTURE",
          "is_mandatory": true
        },
        "timing": {
          "name": "timing",
          "type": "DV_PARSABLE"
        },
        "action_archetype_id": {
          "name": "action_archetype_id",
          "type": "String",
          "is_mandatory": true
        }
      }
    },
    "ACTION": {
      "name": "ACTION",
      "ancestors": "CARE_ENTRY",
      "properties": {
        "time": {
          "name": "time",
          "type": "DV_DATE_TIME",
          "is_mandatory": true,
          "is_im_runtime": true
        },
        "description": {
          "name": "description",
          "type": "ITEM_STRUCTURE",
          "is_mandatory": true
        },
        "ism_transition": {
          "name": "ism_transition",
          "type": "ISM_TRANSITION",
          "is_mandatory": true
        },
        "instruction_details": {
          "name": "instruction_details",
          "type": "INSTRUCTION_DETAILS"
        }
      }
    },
    "INSTRUCTION_DETAILS": {
      "name": "INSTRUCTION_DETAILS",
      "ancestors": "PATHABLE",
      "properties": {
        "instruction_id": {
          "name": "instruction_id",
          "type": "LOCATABLE_REF",
          "is_mandatory": true,
          "is_im_runtime": true
        },
        "wf_details": {
          "name": "wf_details",
          "type": "ITEM_STRUCTURE",
          "is_im_runtime": true
        },
        "activity_id": {
          "name": "activity_id",
          "type": "String",
          "is_mandatory": true,
          "is_im_runtime": true
        }
      }
    },
    "ISM_TRANSITION": {
      "name": "ISM_TRANSITION",
      "ancestors": "PATHABLE",
      "properties": {
        "current_state": {
          "name": "current_state",
          "type": "DV_CODED_TEXT",
          "is_mandatory": true
        },
        "transition": {
          "name": "transition",
          "type": "DV_CODED_TEXT"
        },
        "careflow_step": {
          "name": "careflow_step",
          "type": "DV_CODED_TEXT"
        },
        "reason": {
          "name": "reason",
          "type_def": {
            "container_type": "List",
            "type": "DV_TEXT"
          },
          "cardinality": []
        }
      }
    },
    "GENERIC_ENTRY": {
      "name": "GENERIC_ENTRY",
      "ancestors": "CONTENT_ITEM",
      "properties": {
        "data": {
          "name": "data",
          "type": "ITEM_TREE",
          "is_mandatory": true
        }
      }
    }
  }
}

And then I re-read what Pieter wrote:

It took me 10 days to understand what Pieter tried to say. He was probably referring to the specifications-ITS-JSON. It contains JSON Schema versions of the BMM files.


I still like the ANTLR4 grammars and generated parsers. A person would go crazy writing a grammar in JSON. But at least if somebody wants to use JSON instead of BMM as their starting point they can. And they could do it a long time ago using Archie.

Is there anyone who uses BMM JSON files as computable specifications for openEHR?

I guess everybody is using JSON-schema to validate data, or perhaps to generate instance models compliant with RM. The BMM derived JSON does not contain all specifications, like full textual descriptions, inheritance info, neither those invariants. Nevertheless they are useful and some are doing experiments JSON Schema (openAPI) for various use cases.

1 Like

@pieterbos seen this: native support for swift code generation from OpenAPI: Meet Swift OpenAPI Generator - WWDC23 - Videos - Apple Developer

Hi every body,
I need your help please to convert general Json file to openEHR template. it is possible?
I want to store the PGHD data from Fitrockr application.

I have a sample from its Data
Many thanks for your guidance

Hi,

Welcome to openEHR discourse.
I am not sure about the level of experience with openEHR technology and specifications, but based on your question I would assume you want to quickly solve a problem without diving too much into openEHR.

However, I would suggest you to first identify your data-models you’ll need (I see there in your dictionary some demographic and some health information), by choosing right archetypes and templates. For some data you might choose other standards (FHIR?).
Once that step is done, you will need to work with a CDR, perhaps EHRbase. If you define (upload) there the right Template, which is in fact the schema of your data, then you’ll be able to use their REST API to publish data.
Try to follow some presentations about this stack, for instance you can find on youtube @Sidharth_Ramesh presentations which might be helpuful.

Dear Sebastian,

Thank you so much for your kind message and guidance. I truly appreciate it. I would like to request the data in a JSON file format, as it is the type of data which is mentioned in the data dictionary. Based on your guidance, I found that I should follow these steps(Please correct me):

  1. Analyze the data dictionary and design any Archetypes and templates that are needed. From my analysis, I have found 7 templates in this data: 1) Stress Analysis, 2) Validation Info, 3) Person Activity, 4) Clinical Trial, 5) Sleep Info, 6) Epidemiologic Information, and 7) Demographic/Person/User Information.

  2. I will need to assemble a developer team to set up the EHRbase platform and develop APIs to implement databases based on the designed Templates.

  3. The developer team will then need to design the necessary API to receive data from the JSON file and store it in the designed CDM that it has developed based on Templates

Regarding your comment about using other standards such as FIRE for some data, do you think it is possible and correct to design a template for ‘other data’, without applying other standards like FHIR.

Thank you once again for your assistance, and I apologize for any basic questions I may have asked.

Best regards,
Somayeh

That might be right, but I cannot evaluate truly; better ask clinical modelers :wink:

If you can handle containers, EHRbase can de deployed using Docker; you might find easily here on discourse people already having experience on that.

Yes, that’s a way of doing it.

That is also possible, there are use-case where demographic data was modelled with archetypes and stored in the CDR. A better alternative would be to store in the Demographic Repository based on demographic archetypes and templates - not widely supported yet, but perhaps nowadays possible - ask @pablo and CaboLabs/Atomik.

Question is not that basic actually, and no need to apologize anyways - everybody was at some point new on openEHR.

2 Likes

Thanks a lot for comprehensive respond.

  1. You are professional and clinical modeler also :pray: :star_struck:
  2. Sure, I have to promote my knowledge about apply docker and containers about EHRbased using
  3. My issue was about some data which don’t set up in demographic or clinical categories. I thought that may be it is possible to design a archetype or template as a ‘other’ name to put in some unstructured and fragment information
  4. Thank you. It is your kindness

Yes that is possible - usually you do that by having an administrative entry composition see this chapter on Entry Package and fig.20.

1 Like