Redux: Circular Imports

Implementers (archive)
1

Hi All,

This is different than the issues brought up by Rong in the Java
implementation.

I brought this up before when reading the specs. But now as my code has
matured my fears are confirmed.

Using Python:

In an interface I define attributes of a class. In addition to some
meta-data I define the 'type' of that attribute.

Code is here;
http://www.openehr.org/wsvn/ref_impl_python/TRUNK/oship/src/oship/openehr/am/archetype/?rev=0&sc=0

For example in IArchetype (the Archetype interface) I define ontology as
an ArchetypeOntology type. Now Archetype of course is supposed to
implement IArchetype. IArchetypeOntology defines parentArchetype as an
Archetype type. This creates a circular dependency when the interfaces
try to create a datatype based on a class that depends on itself.

The same thing happens with DvMultimedia.thumbnail being of the
DvMultimedia type.

Other languages may handle these issues differently but I'll be
surprised if the same issue isn't raised in the Ruby implementation???

I have tried various delayed import approaches but they do not solve
this problem.

My solution for Archetype.parentArchetype is to make it an ArchetypeId
type.

My solution for DvMultimedia.thumbnail is to create a Thumnbnail class
that mirrors DvMultimedia without the thumnbnail attribute.

All thoughts, comments, suggestions are welcome.

Cheers,
Tim

2

Sorry, this should have said make it an ObjectRef type that points to the parent Archetype.

3

Just one more note here for my personal pet peeve in this area.

If you do not break up your archetyped data in some non-hierarchical
persistence layer then you don't need this attribute anyway. :slight_smile:

--Tim

4

Hi Tim,

Tim, you're reminding me of the old days when I programmed in Delphi, which
also had this interesting ... errr ... "language feature". In Delphi's case,
the root cause of the problem was that Delphi was a single-pass compiler;
this helped make compilation blindingly fast, but also crippled the language
in this respect. The standard work-around in Delphi was to put the two
classes or interfaces in the same module, and possibly to declare a "forward
reference" to one of the interfaces within that module.

In Python's case, I imagine that the root cause of the problem would be that
it's an interpreted language. Is this why circular imports cause trouble in
Python?

Can you fix the problem by putting the two interfaces in the same module?

A quick google for python "circular import" came up with a couple of other
possible solutions:

1. Qualify the reference to the class name:
http://mail.python.org/pipermail/python-list/2004-February/250192.html

2. Delay the import:
http://www.velocityreviews.com/forums/t335136-circular-import.html

Does one of these work for you?

- Peter

P.S. Your posts to the mailing lists are always interesting, Tim, but those
of us using Microsoft Outlook (and its successor Windows Live Mail) have to
go to considerable effort to read them, because the messages appear empty
with the content as an attachment. I realise that this is because of a
defect in those applications, but you are one of only two people in all the
various lists that I subscribe to who send emails in this form, so I'm
certainly not going to all the trouble of moving to a new email client. I
know of one person who simply deletes your posts because of this issue. (For
the benefit of people using these email clients, I've manually pasted your
message in full below.)

5

Those of us using good mail clients like thunderbird have no such problems… :slight_smile:

Peter Gummer wrote:

6

Tim Cook wrote:

Hi All,

This is different than the issues brought up by Rong in the Java
implementation.

I brought this up before when reading the specs. But now as my code has
matured my fears are confirmed.

Using Python:

In an interface I define attributes of a class. In addition to some
meta-data I define the 'type' of that attribute.

Code is here;
http://www.openehr.org/wsvn/ref_impl_python/TRUNK/oship/src/oship/openehr/am/archetype/?rev=0&sc=0

For example in IArchetype (the Archetype interface) I define ontology as
an ArchetypeOntology type. Now Archetype of course is supposed to
implement IArchetype. IArchetypeOntology defines parentArchetype as an
Archetype type. This creates a circular dependency when the interfaces
try to create a datatype based on a class that depends on itself.
  
Tim,

just to be clear - I believe the problem here is that you are having
trouble serialising to and deserialising from a database (Zope in your
case) from in-memory objects because some objects have circular
references? If so, we need to be clear (or at least I do;-) about a few
things:

1. some attributes don't need to be persisted at all. generally parent
attributes do not need to be persisted, although they can. If they are
not, you reconstruct the parent link on deserialisation. Your
implementation may not even need the parent link but most do.

2 the difference between circular references in the object model, and
circular references in instances graphs. When a circular reference
occurs in an instance graph, you have to deal with it when serialising
to any medium - and this is done by doing a 2-phase 'mark & store'
traversal on the in-memory objects. This enables any complexity of
circular links in an isntance graph to be correctly stored.

3. a self-reference in the class model, such as for DV_MULTIMEDIA
doesn't necessarily mean a circular reference in an instance graph (and
it doesn't in this case) - it just means there are 2 or more instances
of the same class with references between. This should cause no problems
at all for serialise / deserialise.

The main reason for parent references by the way is for path processing
code to traverse back up and down the object trees. In my Eiffel
implementation of the ADL parser this is used ubiquitously. There are
other approaches of course. Anyway, it won't hurt your openEHR data not
to have these references!

- thomas beale

7

Thomas Beale wrote:

Tim Cook wrote:

For example in IArchetype (the Archetype interface) I define ontology as
an ArchetypeOntology type. Now Archetype of course is supposed to
implement IArchetype. IArchetypeOntology defines parentArchetype as an
Archetype type. This creates a circular dependency when the interfaces
try to create a datatype based on a class that depends on itself.

2 the difference between circular references in the object model, and
circular references in instances graphs. ...

3. a self-reference in the class model, such as for DV_MULTIMEDIA
doesn't necessarily mean a circular reference in an instance graph (and
it doesn't in this case) ...

Now I need to be clear about something :wink:

I'm 99% certain, Thomas, that Tim is talking about circular references in
the class model, not in the instance graph.

Eiffel seamlessly lets classes reference each other, but some languages
can't cope with it as easily. First thing is, Tim has done the mind-body
split thing, writing an interface for each class. That's fine. But his
problem is this:

Archetype
  --- implements --->
    IArchetype:
      ontology: IArchetypeOntology
        IArchetypeOntology:
          parentArchetype: Archetype

So he has Archetype depends on IArchetype depends on IArchetypOntology
depends on Archetype depends on IArchetype depends on ... ad infinitum.

Apparently it gives the Python interpreter an apoplexy when it tries to
resolve references that it has discovered yet. But as I mentioned in my
earlier email, a bit of googling suggests that Python does have ways of
working around this.

Let's see what Tim thinks.

- Peter

8

Hi Tim,

Tim, you're reminding me of the old days when I programmed in Delphi,

Ahhh yes. The good old days of Clipper. :slight_smile:

In Python's case, I imagine that the root cause of the problem would be that
it's an interpreted language. Is this why circular imports cause trouble in
Python?

Well, first of all Python doesn't have the concept of Interfaces in the
language. This is added by another library that abuses the 'class'
statement to create them. More on this issue in followup emails.

Can you fix the problem by putting the two interfaces in the same module?

Nope. In fact I started with this approach and it was MUCH worse.

Does one of these work for you?

Nope. They aren't the same problem.

- Peter

P.S. Your posts to the mailing lists are always interesting, Tim, but those
of us using Microsoft Outlook (and its successor Windows Live Mail) have to
go to considerable effort to read them, because the messages appear empty
with the content as an attachment. I realise that this is because of a
defect in those applications, but you are one of only two people in all the
various lists that I subscribe to who send emails in this form, so I'm
certainly not going to all the trouble of moving to a new email client. I
know of one person who simply deletes your posts because of this issue. (For
the benefit of people using these email clients, I've manually pasted your
message in full below.)

Thanks for this reminder Peter. The problem is manifested in the way
that Outlook Express and it's successor(s) handle MIME types. I
digitally sign all of my outgoing email. I think it is the right thing
to do and if the world ever catches on then we could eliminate SPAM with
simple filters that dump all unsigned/unauthenticated email. However, I
realize that I am in a 1% minority and I try to remember to subvert this
issue by creating an attachment to my emails. This (somehow) causes
those email clients to correctly display the text (see my attached
graphic). I often forget to attach something and without it, that causes
certain clients to display my text as an attachment.

I have also had several corporate email systems reject my emails because
of the signature. They seem to think it's a virus. Go figure; a more
'certain' email is rejected due to sociological and technological
ignorance. I remain steadfast though and will hopefully remember to
add the attachment so people don't have to reach for the delete
button.

Though I am sure many people go into the ignore mode upon receiving
email from me. :slight_smile:

Cheers,
Tim

(attachments)

Displayemail.gif

9

        Now I need to be clear about something :wink:
        
        I'm 99% certain, Thomas, that Tim is talking about circular
        references in
        the class model, not in the instance graph.

True.

        Apparently it gives the Python interpreter an apoplexy when it
        tries to resolve references that it has discovered yet. But as
        I mentioned in my earlier email, a bit of googling suggests
        that Python does have ways of working around this.
        
        Let's see what Tim thinks.
        
Let's see if I can add some context and explain my complaint a bit
better.

An Archetype instance is a container and it has 6 items in it. Some are
required, some optional. Some of those are containers as well. You can
visualize this by thinking of a directory tree:

Archetype
  adl_version
  archetype-id
  uid
  concept
  parent_archetype_id
  definition
  ontology
  invariants
  revision_history (inherited)

Sticking with the ontology issue:

Archetype
  adl_version
  archetype-id
  uid
  concept
  parent_archetype_id
  definition
  ontology
    terminologies_available
    specialisation_depth
    term_codes
    constraint_codes
    term_attribute_names
    parent_archetype_id
  invariants
  revision_history (inherited)

The issue here is the REQUIRED attribute parent_archetype_id. The
description is; Archetype which owns this ontology. Intuitively (to me
anyway) this would indicate that I would assign the OID of the parent
Archetype to this attribute. However, the specifications call for the
attribute to be of the type 'Archetype' and it can't be just any
archetype, it is supposed to be the Archetype containing this ontology.
so I would have a repeating tree like this:

Archetype
  adl_version
  archetype-id
  uid
  concept
  parent_archetype_id
  definition
  ontology
    terminologies_available
    specialisation_depth
    term_codes
    constraint_codes
    term_attribute_names
    parent_archetype_id
      Archetype
                  adl_version
                  archetype-id
                  uid
                  concept
                  parent_archetype_id
                  definition
                  ontology
                    terminologies_available
                    specialisation_depth
                    term_codes
                    constraint_codes
                    term_attribute_names
                    parent_archetype_id
                      ... (repeat infinitely)
                  invariants
                  revision_history (inherited)
  invariants
  revision_history (inherited)

So my suggestion is that ARCHETYPE.ontology.parent_archetype_id be of a
'type' that represents the OID of the parent and NOT the parent itself.

Cheers,
Tim

(attachments)

Displayemail.gif