openEHR prototype

Hi Robert

Thank you for that input - I like your attitude and approach. People like Seref are deeply into implementation and find it a bit frustrating to start at the beginning. I am not sure that anyone has a working implementation and a non-sql database although I know Seref looked into it in detail.

So I believe it is a new area. The principles are available in the openEHR specifications and Erik Sundvall may have some knowledge of who is the go to person in this area.

Extending the openEHR XML schema to fulfill the requirements for a NoSQL approach might be necessary and I am personally very interested in how you decide what the indexes might be. I would suggest that the indexing should be reactive rather than set at design time - so any query repeated can be indexed for the values that are sought.

Anyone had any experience?

Cheers, Sam

Hi

Yes Seref I read this links, maybe i didn’t understand them well, I will review.

Thank you for your reply Sam. I thought in using XML because I think that I read somewhere that XML or dALD are the standard for communication in openEHR. And if it is, I think that is not good to use different languages to communicate between different openEHR systems. Using different languages would force other openEHR systems to understand each language, having different methods for each repository that needed to communicate with.

Regards,
José Silva

Seref

okay, I understand about the ACID support and the pros and cons of nosql databases. And I’m assuming that you favor a relational database. So are you saving the openEHR as a blob or are you normalizing the data into the tables. If storing as a blob (although I don’t think this is a good way to go) then, what format is the data in (XML, JSON, Text, etc)

To me, the reason why a nosql database is attractive is that the data does not need to be normalized into an rdbs, and so it is more like true object persistance. This is nice because it decreases the complexity of the app and allows one to focus on the app it self and not how to store the data.

If you could explain more about how you are using a relational database or point me to a resource, I would appreciate that.

Thanks,

Rob

Robert,
I actually do not prefer them, but due to historical and various other domain specific reasons, I kinda got stuck with them.
Thomas has actually provided the fundamentals of blob based persistence in the wiki: http://www.openehr.org/wiki/pages/viewpage.action?pageId=786487
In the past, I’ve implemented a persistence model where a key value representation for each data commit was used in RDMS system in
Opereffa. Opereffa now belongs to Charing Systems, and if you google for it, you can still access the source code that includes the db design. The problem with that was that it would not scale to large data, which I need to process for machine learning, mathematical modelling that I’m doing.

Currently I’m using a different method with RDMSs, but it is unpublished work at the moment, so I can’t share it.

Personally the biggest problem I’m seeing with relational dbs is the difficulty of scaling out for writes. There is a lot one can do for read scaling, but write scaling with a relational db is really hard, and so far nothing I’ve seen comes close to Oracle RAC, which is not the cheapest piece of software you can buy.

If you’re not going to build a system that must support heavy writes with immediate consistency, RDMS is not a bad option.
Most relational dbs can offer very impressive performance. but you have to invest into them. That is, you’ve got to know your db layer product really, really well. In the past, I’ve managed to cut query times from 3.5 minutes to 20 ms in the past, but that takes serious work.

I keep going back to what I was trying to explain in the beginning, choices for db layer is very much dependent on many key factors. The better one defines the requirements, the clearer the choice for db technology becomes.

Cheers
Seref

Hi guys,

About Jose's concerns I think the better way to understand openehr persistence complexities ia trying things out. I agree with Seref that this take a long time, specially for generic solutions. For specific solutions like persisting data defined by a small set of archetypes could be easily done in a short time.

About relational vs nosql, for a real solution, if scalability is taken into account, a mixed solution of relational for inserts and updates and nosql for querying would be a good solution because writes on disk should assured and that is not granted by many nosql solutions. For a small set of users relational with some level of normalization would work ok. Take a look at the Open EHRGen Framework as an example.

Ing. Pablo Pazos www.cabolabs.com

Is scalability really an important argument for everyone?

I mean, a doctor only needs access to his direct patients in, say, half a year. He never looks at any other patients in his system.
How many patients are that, 5000, 10000? Maybe even 20000. The largest scale is hospital scale, but even in hospitals are situations with distributed systems desirable or even fact.

For National Healthservices, you don't need central systems with millions of patients, but you need a good message-system, and an index-server, which is not a medical information system.

For disease-control, epidemiological warming systems, or for medical research purpose, you don't need to have access to all patients on a single cluster, it is enough to make smart use of semantic webs and/or eventually, distributed queries.

Even in hospitals, in the Netherlands, hospitals are markets for specialist, which work on their own account or in small businessgroups for hospitals and also inside the hospital-buildings.
Often they have their own information systems, and often with bad messaging. That needs improvement. The hospital itself also offers services, and has information systems for that. For example, financial accounting, medication, and beds and nurses.

Only academic hospitals in the Netherlands have specialist in service for the hospital, and they have central information systems, and there are advantages to that, but there are more ways.

When you have 10 million patients on one machine-cluster, is the same as having 10000 patients on 1000 machines. Most of these patients live on more systems, at the GP, dentist, local hospital, insurance, etc. So, those 10 million patients live on perhaps 5000 machines, machines with systems with different requirements.
This is the situation in most countries, and this will not change in most countries. There is a lot of opposition of several groups against central machines, not only privacy concerns, but also how will the software-companies make money if the government hijacks their market? In the liberal market situation most governments will not chose for central systems. At least not in coming decades. Unless North Korea wins the war, of course.

Health related arguments are thus not the only arguments which decide how the Health-information landscape will look.

Those 1000 machines are cheap, not depending hard on the internet, software can be available on sharp markets, competition in features and price, the government can guard quality rules.

Most of us are not building a single National Healthservices cluster with all patients centralized stored. Realize that and life becomes more simplier, you can concentrate on other, more important things.

If you cannot let go the dream of being the one who delivers the National Healthservices cluster, make you storage-layer transparant, so with it can be exchanged with not much effort. You should do that anyway, at every system-designer school you learn that.
Keep in mind that the other software layers in your system do not need to know on which kind of database they run.

This all is only about scalability for databases/systems. We do need scalability on other subjects, like in the discussion between Tom and Tim on identification on archetyping, and identification of patients, etc.

So even when you are designing a small system, you need to remember that it must be possible to safely share information with thousands of systems. That is not scalability in database, but in logical design.

Have a good day,
Bert Verhees

For networks of hospitals with centralized systems (like public hospitals here in Uruguay) an insurance+healthcare companies that are buying clinics and hospitals every day (as it happen in the US and many other countries), scalability is a must.

For the mentioned cases 1. the are more users every day, 2. new data services are needed every day, and when performance limits are reached, scaling is the only solution.

Ing. Pablo Pazos www.cabolabs.com

Hi Pablo,

do you have some examples of processes that require the immediate availability at the terminal of employee A of data entered/obtained at the terminal of employee B in these largish organisations?

It is a solution, another solution is distributing.

Distributing is the way how the upcoming National Health Service network in the Netherlands is designed.
Only a centralized index-service which knows where information about a patient is, and a message system, which retrieves the information when needed.
This solution was to avoid a central point of failure, but more to comfort the software companies, which would lose their markets if there would come a centralized system.
A centralized system is also innovation killing. Because the market then becomes in hand of one commercial party.

But most of all, no-one working in health needs direct access to hundred thousands or millions of patient-records.

Bert

In the centralized environment I mentioned, distribution is not an option. The concept here is: there still exists a lot of centralization, we cannot change that, so we need to provide solutions under that scenario too. In Latin America centralization is the rule, not the exception.

My message is to give Jose some input to help him on his project, I think we need to discuss about that here and create other threads to discuss solutions in real environments, considering different scenarios. Please consider Jose’s project is a PROTOTYPE, and he needs to try things out.

He Roger, I don’t fully understand your question, what kind of data are we talking about? terminals A and B are on the same organzation? what type of organization are we talking about? a clinic, a hospital, a goverment agency?

There are different sources of data, different kinds of data and different accessing requirements, and different organization structures (federated, associated, acquisitions).

Also, one thing is what exists now and another is what we should do to improve that (our vision/ideal, here I’m talking about that ideal, not reality. Our reality will make your bones shake…).

I understand, in that case you need mega-databases.
Good luck with it.

I am happy to live where the market is divided, which gives newcomers a way to enter.

Bert

Bob, this is where it is useful to ‘know something about the data’. In openEHR land (and it’s the same for 13606, CDA, CCR, anything similar), your base lump of committed information is something like a ‘document’. In openEHR we don’t think of them like that, but the granularity is the same as for those document standards. In openEHR, the container is the COMPOSITION. So you know you are going to commit one or more COMPOSITIONs at a time; you know also that the contents of those COMPOSITIONs are archetyped. So that enables you to store them as blobs with a smart index based on archetype ids and paths. Then you have two practical choices: Store the blobs in technology-specific binary form, e.g. java objects, binary data infosets, whatever. This will be optimised for your implementation. As long as you can convert this to an interoperable format (like the published openEHR XSD; JSON in the future) this is a good way to go. OR.. the second choice is you actually create the blobs directly from an interoperable form like XML or JSON. This won’t be optimised for your internal system computation, but it will make your internal software a bit easier to write, and it will make it easier to export the data in an acceptable standard format. For high capacity systems, the first choice is the most likely. this is why the only realistic way to go with RDBMS is blob storage; you can make the RDBMS use its brute force as a blob manager. Even better to make them variable size blobs e.g. as . - thomas beale

Exactly, that is the case here, but we need more than good luck :stuck_out_tongue_winking_eye:

Ing. Pablo Pazos www.cabolabs.com

Exactly, that is the case here, but we need more than good luck :stuck_out_tongue_winking_eye:

OK, much wisdom too.

regards
Bert

The Tao gave birth to machine language.
Machine language gave birth to the assembler.
The assembler gave birth to the compiler.
Now there are ten thousand languages.
Each language has its purpose, however humble.
Each language expresses the Yin and Yang of software.
Each language has its place within the Tao.
But do not program in COBOL if you can avoid it.

http://www.canonical.org/~kragen/tao-of-programming.html#book1

Hi Bert,

Scalability had been a great concern in the last decades, but cloud
computing has
succeeded to disguise it. We can purchase computer resources by reasonable cost
on demand. Even an EHR system for small clinics needs to discuss to
use such cloud
system for sustainability, because severe disaster can easily break
intra-hospital/clinic
system. (It was just proved by the earthquake and tsunami, 2 years ago)

However, this is a very good for us, openEHR developers. We can build
from 10 to billions
patient system by same logical information model on cloud system as
you mentioned.
My suggestion to build a prototype is to consider to use such cloud system.

Shinji

My suggestion to build a prototype is to consider to use such cloud system

I agree with you, Shinji, cloud computing is the way you can scale up systems. It is in fact what I suggested, but the term "cloud" did not come to mind.

When you have a distributed OpenEHR system, deployed by an organized group, they can easily exchange information in a OpenEHR way, and even in a disaster (when you need your system more then ever) or a less catastrophic network failure, still all the islands in the cloud remain having their own databases.

In the Netherlands we had last week three times a major network breakdown in banking systems. Two concerned only one bank, but the largest bank, ING, and one concerned iDeal, which is a Dutch service for all banks for Internet money transfer. People could not get to their money, and in the third breakdown webshops lost millions, there was no money transfer at all. Two breakdowns were because of software problems, the third was because of the largest DDOS attack in Dutch history. And because this was in the news, they showed us that this happens many times, worldwide. In the USA, there was a network outtage concerning the Bank of America, also last week.

This illustrates how important it is not to be dependent on networks. Distribution of systems is the way to scale up systems. And an other advantage of distribution is that it is not necessary for whole regions to switch to OpenEHR. The organizations which rather wait can be served by an acceptable message system.

I think this is the way I can also agree with the participants in that other parallel discussion.

Thanks for your suggestions. Concerning prototyping distributed/cloud OpenEHR, it will be very interesting to work on that.

Bert

Maybe it is the iPad, but I receive my messages twice, if you also have this, please excuse me.

Bert

I also happen to agree that a realistic view of the world would be as Bert has described below. Some countries have a different background and want centralised systems. But the reality is that the vast majority of health transactions occur locally, and will never be relevant outside that situation / location. Even care in the community, which requires localised communication of a care team and some local clinic is essentially still 'local'.

Central storage of patient data in a large country at least is not a /needed /approach - it just involves absurd expense and fails to deliver the main functions (e.g. UK). For small countries (sub 5m people) it's more attractive, but still might not match the reality on the ground.

However, there are places that do want e-health computing hubs with 10,000,000 patients and more, for whatever reason. Plus the need to do secondary studies on millions of patients. So the picture gets muddy, and the notion of 'scalability' starts to extend in ways that a rational analysis wouldn't expect.

- thomas

Just food for thought: is scalability only a function of data size? What happens if your solution is performing adequately for all reads/writes you need up until a point and you need to access your repository with a completely different query?
Maybe robustness is a better word for it in this case, but scaling across data volume is not the same as scaling across data access patterns even if the data size is kept constant.