ADLParser exception :-/

[ANASTASIOU_A1 (ANASTASIOU A.)]

Hello all

I have set up a simple project where i am using the following code to
read and parse an archetype:

File ADLFile=new file("/blah
blah/openehr-demographic-person.person.draft.adl");

ADLParser parser=new ADLParser(ADLFile);
Archetype MyArchetype=parser.parse();

When i try to run this, i get:

Exception in thread "main"
se.acode.openehr.parser.ParseException:Encountered "[
\"ac0001\"]=<http://free.terminology.org?terminology_id=acme;synonym_of=
[163020007]" at line XX, column XX

It then goes on to tell me what it was expecting (which is exactly what
is inside the ADL file) followed by a stack trace.

I tried changing the file to read the location archetype but now i get an:
Exception in thread "main" se.acode.openehr.parser.TokenMgrError:Lexical
error at line 12, column34, Encountered: "h" (104), after : ""

When i am accessing the files with the ADL workbench it is read properly
and passes all the tests.

Any ideas regarding this?

All the best
Athanasios Anastasiou

[system]

Hello all

I have set up a simple project where i am using the following code to
read and parse an archetype:

File ADLFile=new file("/blah
blah/openehr-demographic-person.person.draft.adl");

It looks there is an error in your ADL and the workbench is
(unattentially) permissive at that point, or there is an error in the
parser.

To find out, you better send the ADL-file which causes the problem,
maybe some expert here can find what is wrong.

I must regrettely say that I do not have the time to help, I work with
the wrong kernel, but you can find people helpful if you provide them
with enough information and make it easy to help. (Maybe I can take a
quick look.)

regards
Bert Verhees

[ANASTASIOU_A1 (ANASTASIOU A.)]

Dear Bert

Thank you for your email and help.

I have an up to date working directory which i checked out 2-3 days ago
(at least the ADL files have not changed) and i have not modified the
ADL file in anyway.

The file i am using is:
/archetypes/dev/adl/openehr/demographic/openehr-demographic-person.person.draft.adl

And the parsing exception seems to be thrown when the parser reaches the
  following line:

[Within Person.identities.purpose]
CODED_TEXT matches {
  code matches {[ac0001]} --legal name
}

I have also tried saving the file (under a different name!) as simple
ANSI (in case the file was saved accidentally in an encoding that would
add "illegal" characters for the parser) but no joy still.

Can anyone working with the latest Java kernel version send me their
demographics ADL files to try once again?

I don't think this is a problem with the code but anyway, i will ask: Do
you do any special initialisation to the parser prior to using it? For
example, does it need terminal symbols to be specified explicitly? Does
it need to know the encoding? etc.

Looking forward to hearing from you
Athanasios Anastasiou

Bert Verhees wrote:

[thomas.beale]

Athanasios Anastasiou wrote:

Dear Bert

Thank you for your email and help.

I have an up to date working directory which i checked out 2-3 days ago
(at least the ADL files have not changed) and i have not modified the
ADL file in anyway.

The file i am using is:
/archetypes/dev/adl/openehr/demographic/openehr-demographic-person.person.draft.adl

And the parsing exception seems to be thrown when the parser reaches the
  following line:

[Within Person.identities.purpose]
CODED_TEXT matches {
  code matches {[ac0001]} --legal name
}

I have also tried saving the file (under a different name!) as simple
ANSI (in case the file was saved accidentally in an encoding that would
add "illegal" characters for the parser) but no joy still.

Can anyone working with the latest Java kernel version send me their
demographics ADL files to try once again?

I don't think this is a problem with the code but anyway, i will ask: Do
you do any special initialisation to the parser prior to using it? For
example, does it need terminal symbols to be specified explicitly? Does
it need to know the encoding? etc.

Looking forward to hearing from you
Athanasios Anastasiou

There is nothing wrong with the archetype you mention as far as I can
see (although I can see some things that should be fixed); the reference
parser passes it fine. I think this must be a problem in the Java
parser. Rong - what is the status of the Java parser with respect to the
openEHR archetypes, and the tests that pas & fail in the workbench?

- thomas

[Andrew_Patterson]

Exception in thread "main"
se.acode.openehr.parser.ParseException:Encountered "[
\"ac0001\"]=<http://free.terminology.org?terminology_id=acme;synonym_of=
[163020007]" at line XX, column XX

Athanasios, the left square bracket is invalid in a http URL as per
the RFC 1738 spec (used in the reference Java parser I believe).
The correct form to use is synonym_of=%5B163020007%5D

As the ac0001 constraint is used within Person.identities.purpose
I'm guessing this is where it does a parse of the URI and
fails. Try removing the square brackets (I do not have the Java
parser running so I can't check it myself)

Andrew

[Andrew_Patterson]

see (although I can see some things that should be fixed); the reference
parser passes it fine. I think this must be a problem in the Java
parser. Rong - what is the status of the Java parser with respect to the

I should note that it passes the reference parser because it uses the
much less correct definition of

[a-z]+:\/\/[^> ]*

to define URI's (which basically allows everything until the next > or space
is encountered)

Andrew

[thomas.beale]

Andrew Patterson wrote:

Exception in thread "main"
se.acode.openehr.parser.ParseException:Encountered "[
\"ac0001\"]=<http://free.terminology.org?terminology_id=acme;synonym_of=
[163020007]" at line XX, column XX
    
Athanasios, the left square bracket is invalid in a http URL as per
the RFC 1738 spec (used in the reference Java parser I believe).
The correct form to use is synonym_of=%5B163020007%5D

As the ac0001 constraint is used within Person.identities.purpose
I'm guessing this is where it does a parse of the URI and
fails. Try removing the square brackets (I do not have the Java
parser running so I can't check it myself)
  

I think the easiest fix here is just to change the ADL archetypes to be
legal URLs by removing the [] - they are not needed in these URLs anyway
- they were there simply because we use the [] form elsewhere in the
archetype.

I will still have to fix the lax URL matching pattern.

- thomas

[ANASTASIOU_A1 (ANASTASIOU A.)]

Hello Andrew

Yes, that was it! Thank you very much.

I simply assumed that the fact that the file passes the test in the
workbench meant it was syntactically sound and the error was to be found
elsewhere.

All the best
Athanasios Anastasiou

Andrew Patterson wrote:

[system]

Exception in thread “main”
se.acode.openehr.parser.ParseException:Encountered “[
"ac0001"]=<MailScanner has detected a possible fraud attempt from “free.terminology.org” claiming to be http://free.terminology.org?terminology_id=acme;synonym_of=
[163020007]” at line XX, column XX

Athanasios, the left square bracket is invalid in a http URL as per
the RFC 1738 spec (used in the reference Java parser I believe).
The correct form to use is synonym_of=%5B163020007%5D

As the ac0001 constraint is used within Person.identities.purpose
I’m guessing this is where it does a parse of the URI and
fails. Try removing the square brackets (I do not have the Java
parser running so I can’t check it myself)

Andrew is right, both left square and right square are not allowed in the search part of the http URL according to RFC1738. Just remove those “”, the adl should parse. (and this is a known issue on the openEHR archetypes

Thomas, just a quick run of the latest java ADL parser on the latest openEHR archetypes turns out 37 failed in parsing out of total 82 archetypes. Most of failures are due to invalid c_domain_type and missing attributes in the description part. Things could be done to improve the parsing, e.g. inserting a empty string for a missing attribute, but on the other hand by doing that we would open a door for “illegal” archetypes. Maybe we could explore the idea on setting a flag in the parser to indicate the level of tolarance of ill formatted ADLs.

Anyway we have to improve the interoperability between the parsers. (perhpas we can work on this together when we meet in Germany)

Rong

[thomas.beale]

Rong Chen wrote:

Andrew is right, both left square and right square are not allowed in
the search part of the http URL according to RFC1738. Just remove
those "[]", the adl should parse. (and this is a known issue on the
openEHR archetypes

well, I started knowing about it 10 minutes ago;-) I have uploaded
changed versions of the demographic archetypes to the openEHR
repository. I will fix the ADL workbench today sometime, and the change
will be in the next version (inside a week from now).

Thomas, just a quick run of the latest java ADL parser on the latest
openEHR archetypes turns out 37 failed in parsing out of total 82
archetypes. Most of failures are due to invalid c_domain_type

what are the problems with c_domain_type?

and missing attributes in the description part. Things could be done
to improve the parsing, e.g. inserting a empty string for a missing
attribute, but on the other hand by doing that we would open a door
for "illegal" archetypes. Maybe we could explore the idea on setting a
flag in the parser to indicate the level of tolarance of ill formatted
ADLs.

yes, we should do that.

- thomas

[thomas.beale]

Andrew Patterson wrote:

Exception in thread "main"
se.acode.openehr.parser.ParseException:Encountered "[
\"ac0001\"]=<http://free.terminology.org?terminology_id=acme;synonym_of=
[163020007]" at line XX, column XX
    
Athanasios, the left square bracket is invalid in a http URL as per
the RFC 1738 spec (used in the reference Java parser I believe).
The correct form to use is synonym_of=%5B163020007%5D
  

I remember now why we allowed [] in URLs - according to the RFC, most
bracket characters are included in the "unsafe" group simply on the
basis that mail agents and gateways "sometimes modify such characters".
I don't know why [] or {} are included there to be frank - I have never
seen any such agent do anything with these.

I think the more likely danger is that interoperable texts like ADL
archetypes containing URLs will be passed by one parser and failed by
another due to more or less strict interpretation of the RFC

here is the relevant part:

Characters can be unsafe for a number of reasons. The space
   character is unsafe because significant spaces may disappear and
   insignificant spaces may be introduced when URLs are transcribed or
   typeset or subjected to the treatment of word-processing programs.
   The characters "<" and ">" are unsafe because they are used as the
   delimiters around URLs in free text; the quote mark (""") is used to
   delimit URLs in some systems. The character "#" is unsafe and should
   always be encoded because it is used in World Wide Web and in other
   systems to delimit a URL from a fragment/anchor identifier that might
   follow it. The character "%" is unsafe because it is used for
   encodings of other characters. Other characters are unsafe because
   gateways and other transport agents are known to sometimes modify
   such characters. These characters are "{", "}", "|", "\", "^", "~",
   "[", "]", and "`".

I will change the ADL reference parser up to include all these characters.

- thomas

[system]

Rong Chen wrote:

Andrew is right, both left square and right square are not allowed in
the search part of the http URL according to RFC1738. Just remove
those “”, the adl should parse. (and this is a known issue on the
openEHR archetypes
well, I started knowing about it 10 minutes ago;-) I have uploaded
changed versions of the demographic archetypes to the openEHR
repository. I will fix the ADL workbench today sometime, and the change
will be in the next version (inside a week from now).

Thomas, just a quick run of the latest java ADL parser on the latest
openEHR archetypes turns out 37 failed in parsing out of total 82
archetypes. Most of failures are due to invalid c_domain_type
what are the problems with c_domain_type?

missing property or list of c_dv_quantity:
openEHR-EHR-EVALUATION.goal.v1.adl, line 56
openEHR-EHR-EVALUATION.problem-diagnosis-histological.v1.adl , line 60
openEHR-EHR-EVALUATION.problem.v1.adl, line 121
openEHR-EHR-ITEM_TREE.medication-formulation.v1.adl, line 48
openEHR-EHR-ITEM_TREE.medication-vaccine.v1.adl, line 41
openEHR-EHR-ITEM_TREE.medication.v1.adl, line 39
openEHR-EHR-OBSERVATION.blood_pressure.v1.adl, line 62, 59
openEHR-EHR-OBSERVATION.laboratory-thyroid.v1.adl, line 102, 108, 114

mising unit of c_dv_quantity_item:
openEHR-EHR-OBSERVATION.blood_film.v1.adl , 274

/Rong

[thomas.beale]

Rong Chen wrote:

    Rong Chen wrote:
    >
    >
    >
    > Andrew is right, both left square and right square are not
    allowed in
    > the search part of the http URL according to RFC1738. Just remove
    > those "[]", the adl should parse. (and this is a known issue on the
    > openEHR archetypes
    well, I started knowing about it 10 minutes ago;-) I have uploaded
    changed versions of the demographic archetypes to the openEHR
    repository. I will fix the ADL workbench today sometime, and the
    change
    will be in the next version (inside a week from now).
    >
    > Thomas, just a quick run of the latest java ADL parser on the latest
    > openEHR archetypes turns out 37 failed in parsing out of total 82
    > archetypes. Most of failures are due to invalid c_domain_type
    what are the problems with c_domain_type?

missing property or list of c_dv_quantity:
openEHR-EHR-EVALUATION.goal.v1.adl, line 56
openEHR-EHR-EVALUATION.problem-diagnosis-histological.v1.adl , line 60
openEHR-EHR-EVALUATION.problem.v1.adl, line 121
openEHR-EHR-ITEM_TREE.medication-formulation.v1.adl, line 48
openEHR-EHR-ITEM_TREE.medication-vaccine.v1.adl, line 41
openEHR-EHR-ITEM_TREE.medication.v1.adl, line 39
openEHR-EHR-OBSERVATION.blood_pressure.v1.adl, line 62, 59
openEHR-EHR-OBSERVATION.laboratory-thyroid.v1.adl, line 102, 108, 114

Hm....some of these have

C_QUANTITY <>

which according to the specification is illegal, but the reference
parser doesn't complain. There is a semantic question here: if
C_QUANTITY is used to provide a custom constraint on QUANTITY; currently
the specification for C_QUANTITY has the invariants:

    Overall_validity: list /= Void or property /= Void
    Items_valid: list /= Void implies not list.is_empty

This means that at least property is there, or else the list part is
there. Which seems to make sense; but on the other hand, a C_QUANTITY
with no attributes set still says "the object here must be a QUANTITY,
but otherwise can do what it likes (i.e. what the reference model
allows)". Now the problem is that in an archetype editor will need to be
able to create C_QUANTITYs acording to what the user does - the user
might sometimes set the property only, other times set property, values
and units, or other times just units. But the user might just want to
say "this is a QUANTITY" and nothing else; now the editor could deal
with this by detecting it as a special case, and creating a node:

some_attr matches {
    QUANTITY matches {*}
}

but it seems just as reasonable to allow it to create a C_QUANTITY node,
as it would have (usually) done for the other cases, but with no further
restrictions, which still is a valid constraint. This would mean
relaxing the invariants to:

    Items_valid: list /= Void implies not list.is_empty

We should also probably force property to exist if units are used, and
also units to exist if magnitude is specified, which would mean changing
the invariants in C_QUANTITY_ITEM.

I see this as a good opportunity to revisit the semantics that were
originally defined - what is happening now is that with more
implementation and testing, we are hitting boundary cases (e.g. QUANTITY
matches {*}); we need to solve these in a sensible way, and the
specifications and implementations become stronger.

further thoughts?

- thomas beale

[system]

Thomas Beale wrote:

Rong Chen wrote:
  

    Rong Chen wrote:
    >
    >
    >
    > Andrew is right, both left square and right square are not
    allowed in
    > the search part of the http URL according to RFC1738. Just remove
    > those "[]", the adl should parse. (and this is a known issue on the
    > openEHR archetypes
    well, I started knowing about it 10 minutes ago;-) I have uploaded
    changed versions of the demographic archetypes to the openEHR
    repository. I will fix the ADL workbench today sometime, and the
    change
    will be in the next version (inside a week from now).
    >
    > Thomas, just a quick run of the latest java ADL parser on the latest
    > openEHR archetypes turns out 37 failed in parsing out of total 82
    > archetypes. Most of failures are due to invalid c_domain_type
    what are the problems with c_domain_type?

missing property or list of c_dv_quantity:
openEHR-EHR-EVALUATION.goal.v1.adl, line 56
openEHR-EHR-EVALUATION.problem-diagnosis-histological.v1.adl , line 60
openEHR-EHR-EVALUATION.problem.v1.adl, line 121
openEHR-EHR-ITEM_TREE.medication-formulation.v1.adl, line 48
openEHR-EHR-ITEM_TREE.medication-vaccine.v1.adl, line 41
openEHR-EHR-ITEM_TREE.medication.v1.adl, line 39
openEHR-EHR-OBSERVATION.blood_pressure.v1.adl, line 62, 59
openEHR-EHR-OBSERVATION.laboratory-thyroid.v1.adl, line 102, 108, 114

Hm....some of these have

C_QUANTITY <>

which according to the specification is illegal, but the reference
parser doesn't complain. There is a semantic question here: if
C_QUANTITY is used to provide a custom constraint on QUANTITY; currently
the specification for C_QUANTITY has the invariants:

    Overall_validity: list /= Void or property /= Void
    Items_valid: list /= Void implies not list.is_empty

This means that at least property is there, or else the list part is
there. Which seems to make sense; but on the other hand, a C_QUANTITY
with no attributes set still says "the object here must be a QUANTITY,
but otherwise can do what it likes (i.e. what the reference model
allows)". Now the problem is that in an archetype editor will need to be
able to create C_QUANTITYs acording to what the user does - the user
might sometimes set the property only, other times set property, values
and units, or other times just units. But the user might just want to
say "this is a QUANTITY" and nothing else; now the editor could deal
with this by detecting it as a special case, and creating a node:

some_attr matches {
    QUANTITY matches {*}
}
  

It seems to me that this should be handled as C_COMPLEX_OBJECT with
any_allowed = true. Use C_QUANTITY just to constrain the type of the
node is bit overkill.

/ Rong

[thomas.beale]

Rong Chen wrote:

Thomas Beale wrote:

some_attr matches {
    QUANTITY matches {*}
}

It seems to me that this should be handled as C_COMPLEX_OBJECT with
any_allowed = true. Use C_QUANTITY just to constrain the type of the
node is bit overkill.

that was my initial reaction as well. But the more I think about it, the more I think:

• there’s nothing special about a C_QUANTITY - it’s just a way of constraining QUANTITYs

• there is nothing logically invalid about a C_QUANITTY that simply constraints the type - why shouldn’t C_QUANTITY be able to constrain all the possible variants of QUANTITY, including the one where only the type is stated?

• why complicate software (e.g. an archetype editor) that is already going to use C_QUANTITY most or all of the time, by forcing it to specifically check if the user is not specifying anything more than the type, and then reverting to the standard ADL for that, but otherwise using C_QUANTITY?
  We can take the same line with C_CODED_TEXT, and C_ORDINAL, making everything more flexible. Don’t forget, these C_XXX classes are not models of an XXX, they are models of possible constraints on an XXX.

• thomas

[system]

Agreed.

What I like about this approach is that it unifies all possible constraints on QUANTITYs eliminating the need to use C_COMPLEX_OBJECT. This will reduce the complexity of the RM object creation logic since essentially all leaf node constraints can be covered by either C_DOMAIN_TYPEs and C_PRIMITIVE_TYPEs.

/Rong

[Mattias_Forss1 (Mattias Forss)]

Hi,

I’m glad that you guys have started this discussion and are working together in making the parsers more compatible with each other. I will gladly follow the progress on further developments…

Cheers,

Mattias

2006/10/5, Rong Chen <rong.acode@gmail.com>:

[thomas.beale]

Rong Chen wrote:

Agreed.

What I like about this approach is that it unifies all possible
constraints on QUANTITYs eliminating the need to use C_COMPLEX_OBJECT.
This will reduce the complexity of the RM object creation logic since
essentially all leaf node constraints can be covered by either
C_DOMAIN_TYPEs and C_PRIMITIVE_TYPEs.

I think based on this it is reasonable of me to raise a CR to make the
appropriate changes to the openEHR Archetype Profile (this is the model
containing C_QUANTITY, C_ORDINAL, C_CODED_TEXT). I will alert this group
when I have done that so objections can be raised.

- thomas

[thomas.beale]

Thomas Beale wrote:

Rong Chen wrote:
  

Agreed.

What I like about this approach is that it unifies all possible
constraints on QUANTITYs eliminating the need to use C_COMPLEX_OBJECT.
This will reduce the complexity of the RM object creation logic since
essentially all leaf node constraints can be covered by either
C_DOMAIN_TYPEs and C_PRIMITIVE_TYPEs.

I think based on this it is reasonable of me to raise a CR to make the
appropriate changes to the openEHR Archetype Profile (this is the model
containing C_QUANTITY, C_ORDINAL, C_CODED_TEXT). I will alert this group
when I have done that so objections can be raised.

- thomas

The CR is
http://coruscant.chime.ucl.ac.uk:8200/openEHR_Collector/projects/specifications/CR/224

I will put the changed openEHR profile up in a few days.

- thomas

[Mattias_Forss1 (Mattias Forss)]

Agreed.

What I like about this approach is that it unifies all possible constraints
on QUANTITYs eliminating the need to use C_COMPLEX_OBJECT. This will reduce
the complexity of the RM object creation logic since essentially all leaf
node constraints can be covered by either C_DOMAIN_TYPEs and
C_PRIMITIVE_TYPEs.

There are leaf node constraints that won't be covered by either of
these, for example MULTIMEDIA. How would the RM object creation logic
complexity be simplified when the openEHR specs say that there should
be three ways to express constraints on the RM.

1) Standard cADL constraint e.g. defining_code matches { CODE_PHRASE ... }
2) Customised cADL constraint e.g. defining_code matches { [local::at0001] }
3) dADL constraint which represents an instance of an domain type constraint

So depending on how the constraint is expressed, different object
creation methods must be used...

Personally, I think it's stupid to use more than one way to express
constraints, but in order to allow compatibility with EN13606
archetypes (which may have constraints expressed in standard cADL) the
complexity of RM object creation cannot be reduced.

Mattias