persistence

[Karsten_Hilbert]

IMO the answer is easy.

Eventually, a good API crystallizes from several
(attempted/undertaken) *implementations* of a specification.

Implementation means work. Nitty-gritty, non-theoretical
work. Day-job bores.

There's two kinds of (useful) people (and pray, both are
needed) in IT: Those that figure out *how* to do something
right (Thomas Beale and his clan). And those that *do* it
(Bert and friends). The former, by "design", need more
visible communication, it seems to me. The latter "just"
need to get down and DO it. The (volunteer/OSS) doers are a
minority in my observation.

If one wants an implementation to happen one needs to start
one (and preferably open source it). From there one can go
back and define a sensible API between specs and
implementation which will not fall down with the next
implementation.

My 2 Cents,

Karsten

[system]

If one wants an implementation to happen one needs to start
one (and preferably open source it). From there one can go
back and define a sensible API between specs and
implementation which will not fall down with the next
implementation.
  

Exactly that is what I am doing, and to not reinvent wheels
unnecessarily, I am calling others to help do it.
Because, I am already figuring out how to do it, others can share and
use my experience, and hopefully, I can learn from others.

What I don't need is a discussion on *IF* we should build an API, but I
need a discussion on *HOW* to build an API.

It seems to me that this forum is the right place to find people which
eventually want to do this. If not here, where then should I look?
It would be a sad situation if there is worldwide no one to find, who
want to build on a *complete* Openehr implementation on open source base.

I do not want to say anything bad about Rong and friends, in this,
because, I cannot stress enough, he (they) have done a very good and
necessary job. But it is not finished, and that is what is needed to be
done.
But there are some problems with using the ref_impl_java as it is to a
persistence-API, the problems I found will reflect automatically, and
need to be solved. I solved them, but I don't know if it is the best
way, and what is more, I cannot use the code from ref_impl_java, because
I have a fork, and must always rewrite. For this I want a solution, and
I think, when building an API, this solution will come automatically,
and find its way to the kernel. So there will be something in it for
others (they get a API to persistence), and something for me, and also
others, some problems will be solved.

But I am not going to do the first step, because, there is no one who is
wanting to help, and an Open Source project from one person is small, it
can stay in my house. Then publishing is of no use to me.

Bert

[Karsten_Hilbert]

Exactly that is what I am doing, and to not reinvent wheels
unnecessarily,

Good !

I am calling others to help do it.

Makes sense.

Because, I am already figuring out how to do it, others can share and
use my experience,

No they (currently) cannot (easily in an OSS way) because
...

But I am not going to do the first step, because, there is no one who is
wanting to help, and an Open Source project from one person is small, it
can stay in my house. Then publishing is of no use to me.

... they do not have access to your experience (in code).

Note that I am NOT saying you *have* to share the code.

Karsten

[system]

Karsten Hilbert schreef:

Exactly that is what I am doing, and to not reinvent wheels
unnecessarily,
    

Good !

I am calling others to help do it.
    

Makes sense.

Because, I am already figuring out how to do it, others can share and
use my experience,
    
No they (currently) cannot (easily in an OSS way) because
...

But I am not going to do the first step, because, there is no one who is
wanting to help, and an Open Source project from one person is small, it
can stay in my house. Then publishing is of no use to me.
    
... they do not have access to your experience (in code).

Note that I am NOT saying you *have* to share the code.
  

What I need is a strongly expressed intention to do something in a way I
suggest.
Serious people, not , say, pbublish your thing, and we will see. That is
not good enough.

If we have a few people which want to join, and they seriously have
thought about in what way to join, then we discussi further how to proceed.

And I am not suggesting that I have found the holy grail, or that I am
looking for it, it is, like you said before, there is a job to do.

Bert

[Tim_Cook2 (Tim Cook)]

Hi All,

I have watched this thread with great interest. My first question is;
Why?

Why is there such an interest in developing a specific persistence layer
API for openEHR? I think that this is an area where we should encourage
many implementations to follow their own ideas and then we can discover
which works best for this type of information. Some may believe that
the Node + Path approach on the wiki is right, others may select an XML
database like eXist. My Python implementation will use a native object
database.

The *important* thing is that they all be able to respond to EHR
Queries. IMHO the thing that we need to examine and discuss is the
completeness and correctness of EQL.

So maybe what Bert is advocating is discussion of a persistence model
for his Java implementation?

Cheers,
Tim

[system]

Tim Cook schreef:

Hi All,

I have watched this thread with great interest. My first question is;
Why?

Why is there such an interest in developing a specific persistence layer
API for openEHR? I think that this is an area where we should encourage
many implementations to follow their own ideas and then we can discover
which works best for this type of information. Some may believe that
the Node + Path approach on the wiki is right, others may select an XML
database like eXist. My Python implementation will use a native object
database.

The *important* thing is that they all be able to respond to EHR
Queries. IMHO the thing that we need to examine and discuss is the
completeness and correctness of EQL.

So maybe what Bert is advocating is discussion of a persistence model
for his Java implementation?
  

Yes this is true, I am advocating a persistence layer-API that can
connect to A java-implementation.
Not *MY* model, but *A* model, in fact, a transparent layer.
Why?

Sorry to repeat myself.

The most important reason (amongst others) is that a defined API would
speed up the development of more implementations, also other
architectures could benefit from this. At this moment, there is hardly a
market for OpenEhr, a market needs products, a market with only a few
products does not invite people to come and look.
An well defined API would facilitate other developers, would facilitate
a growing market and demand for OpenEhr.

(Many standard-definers like to mention implementors of their standard
to prove the importance of their standard.
The more people that use it, implement it, sell it, buy it, love it or
even hate it -> ->The more people will have heard about it, have an
opportunity to pay attention to the advocates, generate business,
implement it, decide for it.)

In my opinion, a good, transparant API supports many
database-architectures, object/relational, just (XML)files.... whatever
It hides the database-technique for the upper layers.
(This is a often used definition for the term API, like WIndows, it does
not need to know which hardware specs you follow, as long as your driver
connects to the API Windows has for this purpose)
In OpenEhr, in my humble opinion, this means, a persistence-API enables
the kernel objects to save their selves, it does not tell them how to
save their selves, below the API can be database-related code, or again
another abstraction-layer, that is not important.
On the upper side homes the kernel, how can this connect to an API, are
there any modifications needed to connect, which modifications are best,
etc....
Do the kernel objects save their selves, or is there a visitor-pattern
which enables them to save, or another pattern, related to
archetype-parsing-structs, Discuss what is best, that is the purpose of
defining an API and build it. An API-definition van possibly
dictate/demand some contructs to be done in the kernel itself.

There is also need for another API, that is the services-API, this is
needed for other software to connect to the openehr system.
Maybe this is also not considered as being an essential part of OpenEhr
(as Thomas put it), but it is also needed, and I am sure many will
benefit from it if this will be discussed in public.

There is more to say about this, but the danger is big that I will be
repeating myself even more without any result. Repeating is not a
problem, that is a normal process in advocating an idea, so now it is
time to stop doing, some people may get bored, and no one seems to catch on.

Bert

[system]

Tim Cook schreef:

Hi All,

I have watched this thread with great interest. My first question is;
Why?

Why is there such an interest in developing a specific persistence layer
API for openEHR? I think that this is an area where we should encourage
many implementations to follow their own ideas and then we can discover
which works best for this type of information. Some may believe that
the Node + Path approach on the wiki is right, others may select an XML
database like eXist. My Python implementation will use a native object
database.

The *important* thing is that they all be able to respond to EHR
Queries. IMHO the thing that we need to examine and discuss is the
completeness and correctness of EQL.
  

Excuse me forgot about this.
As Karsten told yesterdays, there are two kind of people, those who
define and those who build.
There is also a category who does both.

But what you suggest discussing the completeness of EQL is a good thing
to do, but that is defining, not building.
Defining EQL does not interfere with defining an API.

But is has, in my opinion nothing to do with that. Maybe when the EQL
definition is ready to use (I don't see much of discussion about this
subject, but maybe the EQL discussion is a private discussion?), than
again a way to implement it must be found,a nd maybe this changes the API.

But that is a normal process in software-development, it is called
innovation

Bert

[system]

And the last thing I want to say about this.

You changed the subject of the discussion, I do not blame you (it is
called hijacking of subject), but it says something.
It says that you think that the discussion I was involved should change
to another discussion

http://en.wikipedia.org/wiki/Thread_hijacking

Now, I go on with my work, thanks for your attention

[thomas.beale]

Bert Verhees wrote:

The most important reason (amongst others) is that a defined API would
speed up the development of more implementations, also other
architectures could benefit from this. At this moment, there is hardly a
market for OpenEhr, a market needs products, a market with only a few
products does not invite people to come and look.
An well defined API would facilitate other developers, would facilitate
a growing market and demand for OpenEhr.
  

Bert,
we need to be careful with what we say here - there is certainly a
market for openEHR - it is called the EHR/EMR/GP desktop market, and it
is worth many hundreds of £m. But markets are for products, so we have
to talk about 'products based on openEHR', which can be many and
diverse. Clearly there are not that many around yet - it is the early
part of the productisation cycle, just like with any other new technology.

In my view the APIs that are of the most use right now are the vEHR and
the EHR service model. Ocean will publish theirs in the coming weeks,
and others are encouraged to do the same, in order to work towards a
standardised EHR interface for openEHR. This is of more importance than
the current standards activities in my view, since the result will have
running software behind it and will be known to work. It will also have
been designed rather than put together by committee.

- thomas beale

[thomas.beale]

Bert Verhees wrote:

But is has, in my opinion nothing to do with that. Maybe when the EQL
definition is ready to use (I don't see much of discussion about this
subject, but maybe the EQL discussion is a private discussion?), than
again a way to implement it must be found,a nd maybe this changes the API.

But that is a normal process in software-development, it is called
innovation

no, it is not a private discussion. EQL, now called AQL (archetype query
language) will also have a draft specification soon on openEHR.org. Just
a matter of time to put the documentation together....

- thomas

[system]

Thomas Beale schreef:

Bert Verhees wrote:
  

But is has, in my opinion nothing to do with that. Maybe when the EQL
definition is ready to use (I don't see much of discussion about this
subject, but maybe the EQL discussion is a private discussion?), than
again a way to implement it must be found,a nd maybe this changes the API.

But that is a normal process in software-development, it is called
innovation

no, it is not a private discussion. EQL, now called AQL (archetype query
language) will also have a draft specification soon on openEHR.org. Just
a matter of time to put the documentation together....
  

Thanks, I am very interested.
Will this be a part of a following version of OpenEhr?
(this is good to know in the context of conformance)

Bert

[Tim_Cook2 (Tim Cook)]

I 'added to' the subject line because I was introducing a new subject.

If you are offended then I am sorry but I think it's inappropriate to
have conversations drift off topic and keep the same subject line. It
makes the archives less useful. IMHO.

You probably should have changed this one to: Hijacking

Cheers,

Tim

[system]

Thomas Beale schreef:

Bert Verhees wrote:
  

The most important reason (amongst others) is that a defined API would
speed up the development of more implementations, also other
architectures could benefit from this. At this moment, there is hardly a
market for OpenEhr, a market needs products, a market with only a few
products does not invite people to come and look.
An well defined API would facilitate other developers, would facilitate
a growing market and demand for OpenEhr.
  

Bert,
we need to be careful with what we say here - there is certainly a
market for openEHR - it is called the EHR/EMR/GP desktop market, and it
is worth many hundreds of £m. But markets are for products, so we have
to talk about 'products based on openEHR', which can be many and
diverse. Clearly there are not that many around yet - it is the early
part of the productisation cycle, just like with any other new technology.
  

I agree there is a market-potential, and the only thing I want to
achieve is to facilitate it, f.e. with an persistence-API
It wonders me that a simple call for people to do this (with or without
me) causes opposition.
There was someone else, he started this thread, who was asking for this,
and a few weeks ago, another thread, there was someone to, and I have
got a few emails outside the discussion list, asking for this.

It is something that lives, but does not come together. Maybe later, in
a year or so.

In my view the APIs that are of the most use right now are the vEHR and
the EHR service model. Ocean will publish theirs in the coming weeks,
and others are encouraged to do the same, in order to work towards a
standardised EHR interface for openEHR. This is of more importance than
the current standards activities in my view, since the result will have
running software behind it and will be known to work. It will also have
been designed rather than put together by committee.
  

This is very good news, and is, just as the persistence-API is, very
good to boost the market-potential.

Bert

[system]

Tim Cook schreef:

I 'added to' the subject line because I was introducing a new subject.

If you are offended then I am sorry but I think it's inappropriate to
have conversations drift off topic and keep the same subject line. It
makes the archives less useful. IMHO.
  

I remembered that I talked about the persistence-API exactly one year
ago, I was working on it than, and ran against some problems, and came
to the idea that we should work together to solve them, because we could
get a unified solution.
But the discussion slipped of to other subjects then which had no follow
up, but the momentum of my discussion was lost, I never got an answer or
an opportunity to discuss these problems further.

The situation looked a lot like this one.

You probably should have changed this one to: Hijacking
  

You are right, and not only change it, but start a new one, because, it
are also the messageID's which mess up.

Doesn't matter for now

Bert

[thomas.beale]

Bert Verhees wrote:

Thomas Beale schreef:
  

Thanks, I am very interested.
Will this be a part of a following version of OpenEhr?
(this is good to know in the context of conformance)

*yes it will - it will appear in the next minor release as a draft
document (like the existing draft documents). It will become a non-draft
specification in a subsequent release like 1.1, according to what makes
sense for the community.

- thomas

[thomas.beale]

Bert Verhees wrote:

What I need is a strongly expressed intention to do something in a way I
suggest.
Serious people, not , say, pbublish your thing, and we will see. That is
not good enough.

If we have a few people which want to join, and they seriously have
thought about in what way to join, then we discussi further how to proceed.

*I would think the problem here is that no-one knows at this stage what
you are asking them to consider. Is it:
- just an idea (i.e. why don't we write a new persistence layer together)
- an implementation that you have running or partly running, but you
need more effort on it

Probably you need to clarify the concrete basis of this discussion from
your point of view.

- thomas

[system]

Thomas Beale schreef:

Bert Verhees wrote:
  

What I need is a strongly expressed intention to do something in a way I
suggest.
Serious people, not , say, pbublish your thing, and we will see. That is
not good enough.

If we have a few people which want to join, and they seriously have
thought about in what way to join, then we discussi further how to proceed.

*I would think the problem here is that no-one knows at this stage what
you are asking them to consider. Is it:
- just an idea (i.e. why don't we write a new persistence layer together)
- an implementation that you have running or partly running, but you
need more effort on it

Probably you need to clarify the concrete basis of this discussion from
your point of view.
  

The fact is, I have a persistence layer running, can be optimized on
some places (working on that), but is does it job. I don't need others
to help me.

(please forgive me for repeating myself, but maybe I did not say it clear)
It is not that I want people to write together a persistence layer, I
am thinking about an API to a persistence layer, an abstraction layer to
which the kernel and the persistence layer can fit to. One could then
write an example persistence layer to fit on to that, but that would not
be the point.

I think there are some problems with the Java-kernel and getting it
fitted into an API (on the top-side), because of some reasons, and I
hope, that when a persistence API layer is build, these problems will
show up, and will reflect on further Java-kernel development.

Because, the java-kernel from Rong is a good piece of work, very usable
as a code-base, but it is handicapped, and it stays that way, because it
is not noticed because it is not used (as far as I know). For me, I have
to rewrite all the time, because, as the kernel is know, I have to work
on a fork.

The most important problems are its immutability forced in code, not
having setters (at least protected, everywhere) and, there are some
algorithms in some setters, which can be avoided, and also, can throw
exceptions. This is wrong I think, because you don't want exceptions at
places a persistence layer uses to fill up objects, this is against ACID
(you have a nice transaction below, and without knowing, a part of the
transaction is disturbed on a higher level, and in code, you need to
roll back that parts of the database-transaction which did fine.). There
are also problems with the fact that Java cannot do multiple
inheritance, which leads to redundant code.
So all together quite some things eed to be solved when building a
persistence layer.

There may be some discussion on some points, I can be wrong. Possible.
We had these discussions a year ago, but suddenly we were discussing
AspectJ and other things by people who had not to deal with consequences
of what they were suggesting. Anyway, none of the problems I mentioned
then, came close to a solution, a very frustrating experience. That is
why I now came to the conclusion, that writing a persistence layer would
help solving these problems.

Not for me, I solved them, but it is a waste of time having to work on
forked code.
And what is more, if there is an open source implementation, meant to be
a product, then more people will help solve problems, I think this will
lead to an elegant product.

About this is why I do all this trouble. Maybe, a motivation can also be
on open source base, others can help me find other reasons why a
persistence layer-API would be a good idea.

Please note the word *API*, a lot of people confuse it with the
persistence layer itself.

bert

[Tim_Cook2 (Tim Cook)]

Hi Bert,

Hmmmm, I find this a little confusing. In your many posts you are
calling for people to help develop a persistence API. Yet here you are
saying that you have one and do not need help.

Maybe some clarification will help. Is there a place where you have
made your Java implementation open source and available to others for
assessment?

Regards,
Tim

[system]

Tim Cook schreef:

  

Thomas Beale schreef:
    

Probably you need to clarify the concrete basis of this discussion from
your point of view.
  

The fact is, I have a persistence layer running, can be optimized on
some places (working on that), but is does it job. I don't need others
to help me.
    
Hi Bert,

Hmmmm, I find this a little confusing. In your many posts you are
calling for people to help develop a persistence API. Yet here you are
saying that you have one and do not need help.

Maybe some clarification will help. Is there a place where you have
made your Java implementation open source and available to others for
assessment?
  

(I do not, at this moment want to publish my code. I worked two years on
my code, I keep it for myself until there will be a good reason to
publish it)
First, we can discuss an API, maybe my code this not fit at all in the
results of this discussion.
I am not important, nor is my code important, the *plan* is important,
and that I explained a few times.

Do we need the code from other participants before we can discuss a good
way to implement the ideas behind it?
I can explain how I did things, for assessment-purposes, if you like,
better is, explain it for progress-purposes. That is, maybe important,
that is part of the discussion. Others can have others ideas or like
what I have done, or have on some parts the idea that it can be done on
a better way, that is discussion. That is what I call for.
But first we need people who want to discuss, A discussion on my own is
a lonely thing.

I want with others to build an API, with or without me (I am not
important), so problems that will show up during doing this will reflect
to the Java-kernel. ("the way is important" (Buddhist saying)).
Also I hope this project then will facilitate others to build their
products, so the market potential of Openehr is boosted because of more
products coming to that market.
If needed and wanted, I will be happy to share my experience, and
work/code, but there must be some reason to do so. Maybe my code does
not fit at all in what others are going to do, why should I publish it then?

Open Source in my opinion means, working together, not one does the job,
others look.
This last statement is not personally pointing to someone special.
Open Source, can be, in my opinion, develop a plan together, an
architecture, and build it together, so others of that community have
ways to analyze and judge the progress on criteria they made up together
before. If others have other opinions on this, please say.

I discussed about one/two month ago with Rong, on this very same
mailinglist.
He agreed an open source API would be a good idea, but the discussion
stopped without coming to further plans.

Every month or few months, we see emails appear from people the ask
where to start, or how to do persistence.
They always get an answer. From some of these people we never hear
again, because, I can only guess, maybe, they give up.
That is not good, we need implementers, we must facilitate them, make it
easy to step in, making openehr become something that will be important
worldwide.

Now I stop writing about this, for the moment, I believe I explained
every aspect a few times in a few days.

Thanks for your attention
Bert

[Karsten_Hilbert]

> Maybe some clarification will help. Is there a place where you have
> made your Java implementation open source and available to others for
> assessment?
>
(I do not, at this moment want to publish my code. I worked two years on
my code, I keep it for myself until there will be a good reason to
publish it)

That is your very right but some people think it is
unfortunate.

Do we need the code from other participants before we can discuss a good
way to implement the ideas behind it?

We don't need it but it usually helps. Additionally an
existing implementation may draw the interest of more
people.

to the Java-kernel. ("the way is important" (Buddhist saying)).

Well, not really. The Way IS, that's all.

Every month or few months, we see emails appear from people the ask
where to start, or how to do persistence.
They always get an answer. From some of these people we never hear
again, because, I can only guess, maybe, they give up.

Wouldn't it be helpful to tell them:

Well, here's Rong's work (URL) and here's Bert's more
complete code based upon Rong's.

With that they would be halfway there. Of course, the
existing code may not be the One True Way. But then there is
no One True Way (there's only the Way, you might argue). So
they will fork/develop further/rewrite and the community
would go from two to >two implementations. The more
implementations the easier it may get to define a well-done
API. That's one approach, anyway.

Karsten