Converting ADL to JSON

Hi,

First of all I would like to say Hello since this is my first post on the mailing list.

Second, my questions:

I’m trying to do a implementation of OpenEHR in Java with the persistence layer in JSON.

I’ve searched the web but all the implementations/libraries I find use XML to store and the openEHR java-libs to process the ADL files.

My application would try to follow the REST/JSON API in https://www.ehrscape.com/ and since the persistence is also made in JSON, I think it’s an expensive overhead to convert from JSON->XML-> AOM → validate → XML->JSON-> database.

If I follow the java-libs way but use JSON at both ends ( storage and REST API ) I would have to convert back to XML or AOM to validate the structure with the java-libs.

I’m I missing something here or is just a bad idea to do persistence in JSON?

Hope my doubts actually make sense and you can give me some guidance.

Best,

Duarte Ferreira

Posting in right thread :slight_smile:
I have made a proof-of-concept of a JSON-Schema generation from ADL in
order to validate JSON instances. I have also read about several
projects to generate other artifacts from JSON-Schema.

Hi,

If I follow the java-libs way but use JSON at both ends ( storage
and REST API ) I would have to convert back to XML or AOM to validate
the structure with the java-libs.

For MedRecord we are converting the archetypes (ADL files) into JSON
schemas (and Java bean like classes) for our JSON API.
These schemas are used by clients to generate those JSON documents and
for validation of the data.

You can find an example here:

http://mrdia.dev.medvision360.org/mrdia/schemas/objects/nl_medrecord_dia_model_ehr_composition_bodymassindex_v1_BodyMassIndexComposition.json

This is the JSON schema for a COMPOSITION containing a BMI OBSERVATION.

More APIs can be found here:
http://mrdia.dev.medvision360.org/mrdia/apidocs/
Make sure you clicked the 'show advanced APIs' link because by default
all openEHR related APIs are hidden.

Currently we do not support everything which is possible in ADL, just
enough to support our use cases.

In theory such a JSON schema should be sufficient to validate your data,
however, it seems there are archetype constructions possible which can
not be converted into a JSON schema. (we only use archetype
constructions which are convertible into JSON schemas)

> I’m I missing something here or is just a bad idea to do persistence
> in JSON?

Depending on how you persist the data, you might run into performance issues. But it depends on what backend you are planning to use.

And, as I said before, I don't think every possible archetype construction can be expressed as a JSON schema.

Ralph van Etten
MedVision360

Ralph, what I do not understand is why you have JSON, it is an efficient transport mechanism, but it does not have a well developed eco-system.

You say it yourself that you cannot support everything which could occur, so why not stick simply to XML, which is very rich, has more ways of validation, has different schema-techniques, which has very matured query-languages.

What is your advantage of using JSON?

Bert

does tools like this of any help?
https://github.com/Zedenem/XMLParser

Hi Bert,

> What is your advantage of using JSON?

For us, the main reason to use JSON is that it is easy to use for UI developers and all modern browsers can parse it without extras.
Currently most HATEOAS REST APIs seem to be using JSON so why not also use it? It would lower the threshold for a random UI developer to get started. XML always seems to be a bit of a challenge for browser based UIs or their developers...

But we are using JSON only as a transport mechanism. We are not persisting the data as JSON and we are not validating the data against JSON schemas (although it helps detecting errors in the data early) or do queries on the JSON data.

About the JSON eco system, I agree XML has more tools and stuff but I think JSON is also getting there. Although still draft specs, there are JSON schemas, JSON xpath etc.

Regarding XML to store data, I thought it also was not possible to express all possible archetype constructions using standard XML schemas?

Ralph.

That is right, you cannot use XML Schema’s to validate conform archetype-constraints. So Schematron can be used for that purpose. But you can use XML Schema’s for other purpose. You can read it back on this list, only a few days ago. JSON has quite a few shortcomings. It is only useful in well-known unambiguous structures, because in structures with more possible types, you don’t know the type. For example if you have an element with a number in it or an element with a text in it, JSON cannot differentiate between a text or a number when the text is a number that should be treated as text, this is a weakness, so the other side has to know what is there. In the OpenEHR or CEN context it is easy to formulate examples where JSON is not safe. Suppose 1960, this can be a date in yyyy-XX-XX-pattern, or it can be your roomnumber in an office-complex or a number. You need type information to parse this example safe. Suppose the archetype gives no clue (this is legal ADL): items cardinality matches {0..} matches{ ELEMENT[at0010] occurrences matches {} matches{ value{*} } } XML can give, and in ambiguous situations, it should give, type-information, JSON does not, as far as I know. JSON also has no way to express the attributes. In XML this would look like this: <items archetype_node_id=“at0010”]> 1960 JSON is useful and efficient, but it is not that rich or trustworthy that you can use it for everything.

JSON has quite a few shortcomings. It is only useful in well-known
unambiguous structures, because in structures with more possible types,
you don't know the type.

For example if you have an element with a number in it or an element
with a text in it, JSON cannot differentiate between a text or a number
when the text is a number that should be treated as text,
this is a weakness, so the other side has to know what is there.

Yes, but this is the same as with XML.

<something>
   <value>5432</value>
</something>

and

{ "something": { "value": 5432 } }

Both are meaningless without each side knowing what it is through a schema.

Suppose 1960, this can be a date in yyyy-XX-XX-pattern, or it can be
your roomnumber in an office-complex or a number.
You need type information to parse this example safe.

Suppose the archetype gives no clue (this is legal ADL):

items cardinality matches {0..*} matches{
     ELEMENT[at0010] occurrences matches {*} matches{
         value{*}
     }
}

Why would you want to use such a construction?
I know it is possible, but is it a good idea to use it?
If you want clean, type safe interfaces, you should not use these kind of anything-is-possible data fields.

XML can give, and in ambiguous situations, it should give,
type-information, JSON does not, as far as I know. JSON also has no way
to express the attributes.

In XML this would look like this:

<items archetype_node_id="at0010"]>
     <value xs:type="DV_DATE">
         <value>1960</value>
     </value>
</items>

Nothing prevents you from adding the same type information to the JSON document:

"items": [
   { "archetype_node_id": "at0010",
     "value": {
       "type": "DV_DATE",
       "value": "1960"
     }
   }
]

and depending on the JSON serializer you're using they might have standardized it.

But the better way might be to have an accompanying JSON schema which tells you what that field is. IMHO an JSON schema does not need to be a static document which can match any possible document.
Why not have a JSON schema describing just the document you are interested in? So if the field is a date in this document, the JSON schema says its a date.

But the best way is to avoid such ambiguities and not allow it to happen.

JSON is useful and efficient, but it is not that rich or trustworthy
that you can use it for everything.

Just a JSON file alone is indeed worthless. The same as just an XML file is worthless.
If you want a rich and trustworthy XML file, you'll need an XML schema, just as with JSON.
Just an XML file and just an JSON file convey exactly the same information. Perhaps XML has standardized more but you can do the same in JSON.

I am not saying JSON is better or XML is better. They both have their pros and cons.
But in our case, as a transport mechanism to exchange data between browser based UIs, JSON won. And by using JSON schemas and avoiding unambiguous constructions, I think it is a clean, typesafe, trustworthy way of communicating.

Ralph.

JSON-Schema lacks a native way for type checking. The day it finally
supports that you can overcome most of these problems.

What do you mean by 'native way for type checking' ?

Ralph.

Not having to rely on defining an additional 'xsi:type' attribute in
order to know
the original type

Of course you can add all kind of attributes and other information to Json, but then you are just mimicking XML, and you could as well use XML in the first place.

Of course it is better to avoid ambiguous archetypes, but the thing with two level modeling is that you, as developer, should not create archetypes, you should only process them, and Json in the current state, with a few tricks could do the job, but XML can do it without any tricks, just the standard as is, has enough in it to support the full OpenEhr Xperience.

I still have not seen any advantage of using XML, except for some well/narrow defined transport mechanism.

That was my point, archetype data relies heavily in typing, which is
something JSON-Schema still has to work out.

I see there are still reasons to use ODIN (dADL)...

something = < -- parses as an integer
     value = <542>
>

something = < -- parses as a Real
     value = <542.0>
>

something = < -- parses as a String
     value = <"542">
>

+ lists of any primitive type, Intervals of comparable types and Lists of Intervals.

I have to wonder why these other formats didn't learn the lesson of type-based primitive value syntax. And if you want more, you just do ...

something = < -- parses as a Real
     value = (MY_STRANGE_TYPE) <542>
>

- thomas

Why does the whole XML vs JSON matter so much? They both are formats for exchanging data, they both are not perfect.

You asked why we are using JSON and not XML and I gave you the answer:
The main advantage of using JSON and not allowing ambiguous archetypes is to make it easier and less confusing for front end developers and that is a huge advantage IMHO.

Sure, you can't do everything which is in the openEHR standard, but then again, because something is in a standard it not necessarily means its a good idea to use it.

Ralph.

How is this different from XML?

In Berts example he also used an extra attribute to know the original type:

<items archetype_node_id="at0010"]>
     <value xs:type="DV_DATE">
          <value>1960</value>
      </value>
</items>

Ralph.

Yeah, but xsi:type is a XML special attribute with specific meaning in XML instances. You can use it, but you are using an XML construct and probably means that there is or will be a correct way of expressing this in JSON-schema.

JSON-Schema lacks a native way for type checking. The day it finally
supports that you can overcome most of these problems.

What do you mean by ‘native way for type checking’ ?

Not having to rely on defining an additional ‘xsi:type’ attribute in
order to know
the original type

How is this different from XML?

In Berts example he also used an extra attribute to know the original type:

<items archetype_node_id=“at0010”]>

1960

Ralph.

Hi Ralph, me too I use Json to communicatie with clients, but in another way, so that there can be no ambiguity. It is not in object representation.

My idea is when you use Json to construct object-representations, like one can do in XML, then you will find that Json is not rich enough to express all requirements of OpenEhr. You can add them, the datatypes, the archetype_node_id’s, the array indicator for fixed order, but then I don’t see much difference between XML and Json.

But on the other hand, if your clientsoftware is happy to parse Json object representations, and the businessmodel in which your software will be adapted, is happy with some limitations on archetype-use regarding to ambiguity, then it sure is innovative, and can as result have surprising new possibilities.

Thanks for discussing this.

Bert

This e-mail message is intended exclusively for the addressee(s). Please inform us immediately if you are not the addressee.