It’s difficult to assess these claims without detailed context on the tests and metrics.
In the end, what I assume you are seeing here is that the DB execution takes the same amount of time as, or less than, the processing part on EHRbase. These methods, of course, could use some optimizations, but when it comes to optimizing a few ms, it’s pretty low on our priority list.
I’m glad that EHRbase is still the reference point for all the openEHR-based CDRs spawning right now with the help of AI-driven coding, but it’s a bit overwhelming right now to follow up on these threads. We’ll do our best while still providing updates for EHRbase.
I know somebody downloaded teh datasets a couple of monts ago, but it was slow and could down sometimes. Have not checked recently.
Maybe we should put the datasets on a torrent site somewhere, any suggestions? The Pirate bay originated in Sweden but EHRs there (although anonymized) might not be good for scientific reputation…
Risking a side track here (we can move it to other topic if it grows too much)…
@borut.jures & @rubentalstra and other CDR implementors if you would happen to experiment with several different databases regarding perfomance, features, possible execution environments etc, then perhaps you could also as one of the tests consider IndexedDB that is built into almost every browser and also provide a version of your CDR code transpiled into WASM or something else that can run in a normal browser.
IF you do, then it is very easy to experiement with and demo a CDR in a browser without backend. Perfornmance might not be stellar but it opens up people’s minds to offline distributed solutions. Combining a reasonably efficient local offline CDR with browser-runnable UI based on e.g. Ehrtslib or similar, makes for a neat offline/distributed EHR solution experimentation that is still compatible with standard server based CDRs.
Distributed local first EHRs are also nice fallback alternatives during disaster recovery (I know a region that back in the days had a backup alternative of sending taxis with CD/DVD-copies of EHR data to the regional hospitals in the case of really bad outages - think infrastructure disasters.) Exploring this path may also get people to appreciate the beauty of the distributed features built in to openEHR specifications (see figure 8 in Common Information Model) by @thomas.beale et.al. over a decade ago, but not always requested in procurements and thus not always fully implemented in commercial solutions. I think people used to avoid it partly because they worried about clinicians having to worry about resolving merge conflicts of EHR content, but nowadays som AI-assistant may make that less mind-boggling and time consuming - I certainly thanks to AI fear the drudgery of merge conflicts in source code less than I used to…
Some of this may also resonate with @marcusbaw’s ideas, right? Even though you might want to avoid databases completely for some use cases where they are mostly bloat if i have understood you right Marcus - but we could perhaps consider indexdDB to be more of an indexed object/file storage that you actually are allowed to write to in a browser (as opposed to local filesystem) .
I checked if IndexedDB has the features I used for ArcEHR.
As an object store, IndexedDB is a good fit for the RM data.
It has IDBObjectStore: createIndex() method with options.multiEntry which would allow indexing paths inside the RM data and omitting the “nodes” table. I’m concerned there is no efficient way to filter across multiple fields simultaneously.
IndexedDB has its limits. We could also consider “SQLite WASM” as an alternative which would offer the same DB in the browser and on the server if the solution requires it.
With SQLite we would get full SQL queries, joins, subqueries. Transpiling AQL to SQL is also a solved problem. I suspect that SQLite indexes are more performant and flexible.
How about framing this as in supporting offline mobile use? MiniCDR which coukd well use Marcus filebased approach or a lightweight browserdb. There is definitely a need for this.
Yes offline mobile use (ambulance etc) and sync when online again is one of the main use cases I was thinking of.
Sidetrack:
Another use case is offline (cut off from main servers) at hospitals when network or server hall is down/hacked. You would likely like to have a Contribuition Builder that you update via real CRDT/OT (via P2P or dynamic Leader-Replica mode) and sync between clients. That would likely call for an android+iOS app to get more permissions than the browser sandbox, and then we can of course use any language and database that the phone OS allows.
Browsers (e.g. on laptop) could of course conenct to a nearby phone or interconnected cluster of phones with EHR servers. After the incident or when you go home for the day the security guys might want you to wipe data form your phone, even if encrypted.
.