Bosphorus Web Services and an open source communication layer for openEHR implementers

Dear members of the openEHR community,

We announced Project Bosphorus from UCL, CHIME at the end of 2010 and
have been preparing an update on progress, including access to source
code and a web services implementation, for testing purposes . Recent
questions on the openEHR lists are relevant to the progress made and
the following is therefore posted as an interim update, indicating the
open source materials that we expect to release shortly, as a further
component of the Opereffa platform, that has been under development
here since 2008.

The Bosphorus project was initiated in order to improve Java
implementations of openEHR, by providing direct access to the Eiffel
code base, which has been made available as open source software by
Ocean Informatics.

Over the past year, we have been working on a new method for
connecting Java technology to the Eiffel code base. For simple
applications, and in a technically simple scenario, such connectivity
is not hard to construct. However, we have realized that the outcomes
we wish to achieve must address more complex and demanding
requirements.

It is quite hard to imagine a successful openEHR implementation today
which does not have a service-based design. Web-based approaches are
strongly dominating everything else, and realizing their advantages in
openEHR has required detailed exploration of a number of architectural
and technological design choices. Scalability, performance and
technology neutral access to underlying functionality are key
considerations for any modern architecture.

This study led us to develop a software layer which would go beyond
out of the box integration options for Java and Eiffel. Especially in
relation to scalability and stability requirements, it proved
extremely hard to develop a satisfactory solution with available out
of the box options for these two technologies. Therefore, we went on
to develop a custom communication channel, using two high performance,
open source frameworks: ZeroMQ from iMatix Corporation and Protocol
Buffers from Google. We had earlier adopted a service-focused design
for our related research on a new openEHR-based data analysis
framework, and for this the new Bosphorus connectivity layer needed to
be exposed to other software components, as a web service.

We are excited by the potential of the evolving Bosphorus
architecture, since it allows us to expose the very mature and capable
open source Eiffel code base as a Java web service, with excellent
characteristics in terms of scalability and performance. We believe
the design will prove an important new approach to system
implementation, as we start to pull key low level openEHR
functionality into web services. Components, such as archetype parsers
and new functionality required to exploit ADL 1.5, have proven to be
good early candidates for such web services.

By slowly moving to provide key openEHR functionalities as web
services, we are aiming to lower current barriers to openEHR
implementation, very significantly. To demonstrate the outcomes
achieved to date, using this approach, we will be deploying a test web
service under the Opereffa Studio Project, using Bosphorus, as
previously announced. This test web-service,will provide the
functionality of an archetype parser, with outputs in the form of
openEHR XML schema compatible XML, and JSON.

We are currently finalising our first full implementation and will be
providing further details and access to the web service, shortly.

We would appreciate feedback regarding the approach and, later on,
regarding the use of the test web service. This is an open source
project, and source code will be available with the release.

We would like to thank Thomas Beale for his excellent open source
Eiffel code, and his support and feedback during the development of
Bosphorus.

Seref Arikan and David Ingram UCL, CHIME

Wonderful news Seref!

I think we should take time to discuss some time soon in order to
match efforts and reduce overlap. :slight_smile:

I believe many things can complement each other nicely e.g. in a
partly REST-based open source framework aimed for web-based
approaches.

// Erik

Thanks Erik,
The current implementation is using RestEasy from Jboss. I am a little
bit skeptic when it comes to philosophy of REST, but at least the
architecture allows extension in that direction.
My experience shows me that there is usually no rest for me...

Regards
Seref

Hi Seref,

Thanks for update. Look forward to the release of the project.

Cheers,
Rong

Dear All

The Board is interested in using tools for collaborative online working and
keeping these coordinated. We already have Jira and Confluence from
Atlassian. These are working well as far as I am aware.

I would like to propose that we consider going to third parties for:

Source code repository
  - Bitbucket from Atlassian (Mercurial)
      - ? other options

Agile development
  - Pivot Tracker (I have applied for 4 accounts for free - openEHR
Specifications, openEHR Software, openEHR Clinical Models, openEHR
Localisation) Projects would then be created under each of these by the
Members if we chose to use this tool.
  - ? other options

Please discuss these issues. I have asked a couple of the less technical
people to review Pivot Tracker to consider its role in tasks other than
software development.

We will obviously be guided totally by the lists on these matters - the
point being that we really want to use one tool for one purpose across
different programs if possible or appropriate. Even if the tool turns out
not to be the best one at times, the uniformity will be very important going
forward.

Cheers, Sam

Dr Sam Heard
Chief Executive Officer
Director, openEHR Foundation
Senior Visiting Research Fellow, University College London

214 Victoria Avenue
Chatswood, NSW, 2067
Phone: +61 2 9415 4994
Mobile: +61 4 1783 8808
21 Chester Cres
London E8 2PH
Phone: +44 20 7249 7085
Mobile: +44 75 7549 7937

Well, maybe you should consider real open source tools.

does git http://git-scm.com/
have any place in the discussion?

As a version control/repository system it has the advantage
that it's designed to combat bi-trot - while being
nicely distributive.

github - https://github.com/plans
The git repository service for many open-source project
(including linux) is _not_ free (as in beer) though.

Hi Tim,

Can you give some examples of good open-source tools in this area?

Ian

Dr Ian McNicoll
office +44 (0)1536 414 994
fax +44 (0)1536 516317
mobile +44 (0)775 209 7859
skype ianmcnicoll
ian.mcnicoll@oceaninformatics.com

Clinical Modelling Consultant, Ocean Informatics, UK
openEHR Clinical Knowledge Editor www.openehr.org/knowledge
Honorary Senior Research Associate, CHIME, UCL
BCS Primary Health Care www.phcsg.org

For openEHR, Atlassian hosted solution JiraStudio (not open source) may be worth considering since it solves the problem of physical hosting without (in theory) causing much disruption, since all the tools are the same - Confluence, Jira (particularly) and SVN.

Atlassian bitbucket (completely separate from Atlassian mainstream hosted tools) uses Mercurial, a better DVCS than SVN, but its issue tracking etc is minimal.

For the price of more disruption, Github would be one place to go, and it is probably the best DVCS there is (it was designed based on the BitKeeper solution we used to use in openEHR). How good the project tracking tools are I don’t know, but they are claimed to be good. The main thing that is needed is integrated DVCS, project / issue tracking (with configurable workflows, security etc), wiki, mailing lists and continuous build server.

Whether having everything open source is fundamentally important is debatable - in principle it is nicer, but I am more interested in getting work done efficiently, not battling tools that are in early development (certainly my experience with most free issue tracking systems - maybe the Git one is better).

  • thomas
(attachments)

OceanInformaticsl.JPG

I agree with you Thomas,

Whether these tools are open source or just free as in beer (for openEHR) - doesn’t matter too much…for me it is far more important that the tool does its job.
If there are open source tools that really do the job - fine…very fine indeed, but if not, I for one, do not want to use tools just because they are open source if we can have better ones that are just free.

Not sure where this discussion stems from, but I am reasonably happy with the current Jira, Confluence and SVN approach and do not think that changing this is a huge priority.
(It’s not like there isn’t anything on the foundation’s priority list at the moment :slight_smile: )
But I may have missed the reasoning why openEHR’s current tooling is not sufficient in the first place?

Sebastian

I agree with you both: we need to get things done and find reliable tools up to the task.

Many opensource projects use cloud based services, and don’t need/try to make everything open source at the infrastructure level.

Jira is great for issue reporting and bug tracking. http://www.atlassian.com/software/jira/
Nabble is great for mailing lists. http://www.nabble.com/ (one thing that bothers me is the 40KB limit of the openEHR lists emails)
SVN or Git area great for version control.

The 40kB limit is indeed a bit on the low side.
But it would not be so bad if the mailing list would immediately reject
your post and notify you.
Then you can shorten (usually just delete part of the email trail) and
send again - This is a bit annoying, but handable.
But the way it is currently, you don't know what is happening to your
email..and may not even realise that it is still in limbo...until you
get a rejection email a day later.
Even if you realise that it hasn't been delivered so far, you don't know
why and so you wait with resending it - often so long that the email
thread has moved on so much that you actually hope that your email will
be rejected :slight_smile:

Sebastian

My two cents.

I wouldn’t trust software which is not open source. You cannot judge if it does its job well, especcially in long lasting use, like f.e. a database, how do you know if it still works if your table reach the million records and the table is 127 fields wide with on the second field a VARCHAR (127), I have seen many strange things in software products.
What happens if a vendor gives up, or isn’t interested anymore in the specific product, or is bankrupt
What happens if a vendor decides to break backwards compatibility, or he is just clumsy?

An opensource product mostly is produced in a community, or at least, watched closely by a community. If that is not the case, than it is the same as a closed source, because a normal user cannot study or maintain the code, and is depending on the good will of the developers. Open Source controlled by a single person or company is almost the same as closed source.

But good open source software products-developing, like from the Apache community, is very trustworthy, more trustworthy than a closed source vendor will ever will be able to be.
It is also a good indicator, if a community looses interest, the open source product will be less trustworthy, its future becomes at stake. It is a public indicator, everyone can see it happen.

This is often not the case with closed source vendors. Even a good financial position is no guarantee. Someone can buy the company, just to for the purpose to kill the products.
For me, I always use open source software, always, never, never, never I use closed source software. Not because I read the source, I almost never do. But because open source causes other circumstances than closed source.
And it feels safe, as long as there is a community, the product will continue to exist, and will be maintained

There are some guidelines for starting open source projects. For example, develop it in closed environment, and if the product is more or less mature, than start community-building and open source the product.
The concept of everyone starting from scratch and build without central directions a good product does not work.

It is how ever possible to invite people or companies at a more or less matured product.
It has advantages. It forces the developer to write good code, because he is planning to show it to the world. People coming into the product later on are skilled and serious developers. Their motivation is tested, because they had to read thousands of lines of code, to get the feeling. Often, when parties are hooking up to a matured product which is open sourced, that are companies, which have advantage when that product in stable way is on the market.

Bert

Hi all,

I am quite happy with the current infrastructure - mailing lists,
issue tracking and SVN repositories. One thing I am missing for the
java project is a build server, something like Apach Continuum that
can check out the latest code, compile, run all the testcases, and
publish reports and successful builds somewhere.

Sebastian, It seems we have a manual step in the process of
rejecting(or allowing) large mail. At least that's the case with the
java list that I am monitoring. It will be convenient if the mail is
just bounced automatically if it's too large. Perhaps there is a
setting somewhere we could use with the current mailing list software.

Cheers,
Rong

BTW I wasn’t trying to make any ideological statements, I have just fought with Bugzilla and some other bug tracking solutions in the past whose names I forget now, and they just got in the way. Likewise, in the days that CVS was the only open source versioning system, it was just not a choice from my point of view - the commercial products were miles ahead if you wanted proper change sets and so on (Torvalds more or less still says that SVN is pretty hopeless, and in some ways he is right, depending on what your requirements are). This has all changed as we now know. And I for one don’t want to go back to fighting tools!

In terms of need - one need we do have is to get off the UCL servers where all the infrastructure has to be maintained by hand and get on the cloud where it is maintained and backed up server-side.

  • thomas

The 40kb limit was one of the sysadmin rules at UCL, and I happen to
agree with it (obviously, it could have been 50 or 100 or whatever, but
they use 40). I know it is sometimes annoying but it does prevent
massive attachments. Some years ago I was on probably 6 HL7 lists (they
have about 40) and there was not only no limit to size, but continual
cross-posting, with the result that 3Mb attachments were regularly sent,
and received 4 times! I for one don't want to go there again.... i think
today there is always an option for putting up some large file and just
emailing the link to it.

If people want to up the actual current limit of 40kb to 70, 100 or so,
that is certainly doable, but I am not sure what problem it is solving.

- thomas

This would be great!

It would be nice to know if an email is sent or not with an instant notification, instead of waiting a random amount of time to receive a notice (if the notice ever comes :).

The real problem though is not the actual size of the limit , but the manual interaction that is required and the time delay until rejection.
If the list could be changed to automatic rejection this would solve the problem .
Sebastian

Hi Bert,

I have participated in a lot of open source projects, for a long time, that make use of software in the cloud as infrastructure.

These projects are “long lasting” and because the infrastructure is in the cloud, we as users doesn’t have to bother with backward compatibility issues.

Cheers,
Pablo.

I wouldn’t trust software which is not open source. You cannot judge if it does its job well, especcially in long lasting use, like f.e. a database, how do you know if it still works if your table reach the million records and the table is 127 fields wide with on the second field a VARCHAR (127), I have seen many strange things in software products.
What happens if a vendor gives up, or isn’t interested anymore in the specific product, or is bankrupt
What happens if a vendor decides to break backwards compatibility, or he is just clumsy?