@pablo It would, but how? I can only access EHRServer but didn’t want to test it since it was superseded by better/faster Atomic which I cannot download.
Testing Atomic over the internet wouldn’t produce results comparable to locally run tests.
This testing began to ensure I can be confident in the performance of a CDR based on a graph database. It is not a sophisticated test 
Thanks for sharing all this! Would be interesting to see numbers for Better too.
It would, but they have nothing to gain and a lot to lose by making the performance public (in case they are not the top performer).
@borut.jures if you can provide a script I can run locally, I can give you the numbers on my machine, it’s not super fast but will give an indicator. I also have a big server and I might get to run your benchmark there, but it’ll be after a client finishes with their tests.
I predict lower inbound performance because of all the data transformation and validations we do, though tolerable. I’m interested on seeing the query ones since we added partial support for AQL (some queries might not run but will run eventually).
@pablo You liked the post #8 with the script 
Please also run EHRbase and FerroEHR so that we can calculate the relative speed of your HW.
yep though in need “something I can run”, I’m not sure how to run that 
@pablo I created another ZIP file with the package.json. It is a TypeScript “project”.
Unzip and cd into the benchmark folder. Run npm install.
Edit the atomicBaseUrl and atomicAuth in src/openehr-rest-api.ts.
Uncomment first two lines in src/benchmark/benchmark-ehr-api.ts to run the write part of the test:
await new InitCdr(atomicClient).runInitCdr()
await new BenchmarkApi(atomicClient).runEhrApiCreateBenchmark()
Then restart your server to clean any caches and put the first two lines in comments and uncomment the third line to run the read test:
await new BenchmarkApi(atomicClient).runEhrApiReadBenchmark()
You can run the tests with: npx tsx src/benchmark/benchmark-ehr-api.ts.
benchmark.zip (24.3 KB)
Perhaps someone will develop a better benchmark script.
I’ve added code for Better in case they want to join us 
Good stuff. @borut.jures Are you using Basic Auth?
Yes. I opted for the simplest possible auth so that it doesn’t interfere with running the tests.
Cool, just to know how to configure the auth
I just love that this partly turned into a performance race thread 
@rubentalstra perhaps you could also try running something like https://arcadedb.com/ not only postgres under the Rust based hood so that you also can use graph based querying (or some non-JVM-based DB competitor if you don’t like JVMs)
Competition and cooperation between projects is healthy on the community/market and if you find similar good multimodel solutions in the Rust ecosystem we at Karolinska and others will of course have a look.
Perhaps some design thougts from Client Challenge could be used to combine solutions/projects too in a more modular way, but the fastest network call is the one that can be avoided so naive network based containerization lil´kely won’t be performant.
I think several actors will realize that they need to move further up the value stack now when formal specifications combined with AI make CDR construction achieavable. In the AI age a closed source implementation of the basic standards based stack is just asking for open competition
I added AQL to my CDR and included it in the tests:
| CDR | Write | Read | AQL |
|---|---|---|---|
| EHRbase | 9,14 ms | 2741,87 μs | 54,68 ms |
| FerroEHR | 5,69 ms | 2073,70 μs | timeout * |
| ArcEHR | 13,48 ms | 273,01 μs | 31,66 ms |
My write times increased because data is prepared for AQL queries. Most additional time is spent by the database creating and compacting indexes.
@rubentalstra * FerroEHR 4.1.1 times out on AQL: POST /query/aql failed with 408 Request Timeout: {"error":"Request Timeout","message":"the request exceeded the maximum execution time and was aborted" I see you are doing work on AQL at the moment. I’ll update your result when your changes are completed.
The most interesting AQL query statistics for production use is for stored queries (that any smart system will have pre-translated to DB-native upon storage). So perhaps you want separate tests/times listed for ad hoc and stored queries.
Regarding performance in general you might also be interested in comparing to procurement (mininmum and optional "extra evaluation point"levels) in e.g. the procurement coordinated by Karolinska (that got inspiration for the performance from a Catalonian procurement).
https://openehr.atlassian.net/wiki/spaces/resources/pages/416514052/Procurement+of+openEHR-related+systems+and+services links to an excel file called Karolinska-2024-A1C1-Appendix 2…
The AQL-to-SQL transpilation takes surprisingly little time compared to the SQL execution time. Stored queries will be faster if reading them from the database (=disk) takes less time than the AQL-to-SQL transpilation (=memory). I will test both scenarios as you suggested.
Remember I didn’t intend to turn this into full-blown benchmarking. My main goal was to avoid developing something slower than other CDRs.
It would be interesting to test the future Catalyst CDR inside Karolinska in a controlled environment. If you decide to build Catalyst on top of ArcadeDB, you should get results similar to mine and compare them with other CDRs you have access to.
I’m also exploring an idea to implement the same architecture with PostgreSQL. This would allow for:
The AQL-to-SQL transpilation takes surprisingly little time compared to the SQL execution time.
Its not like any vendor ever reported this is a problem
Its just some gossip people assumed.
We have tools that eliminate the need for assumptions. Engineers shouldn’t assume.
I used the profiler to measure the actual time spent:
I would assume that implementations also use the stored query functionality of the underlying DB they have when the openEHR stored query API is used, that was kindo of a major thought behind introducing it in Applying representational state transfer (REST) architecture to archetype-based electronic health record systems That part could be the major part of performance gain in optimizing databases as compared to sending SQL etc every time instead of just some parameters of a parametric query.
Also note that the architecture in that paper suggests possibilies for pluggable/swappable databases, so that the rest of the implementation stays when you just swap query translator and database.
I added the percentage of time spent preparing an execution plan for the SQL (0,67%). This is 0,2 ms per AQL query.
It would be great if somebody from EHRbase would publish their data so that we can stop assuming things. Maybe I’ll run the profiler myself
I profiled how EHRbase executes AQL queries, revealing an opposite distribution of time spent preparing the SQL versus executing it:
It seems to me that EHRbase converts any ad-hoc AQL query into a stored query and reuses it for subsequent API calls using the same ad-hoc AQL (AqlQueryRequest.prepareNamed()). This process uses 33% when executed. If I’m correct about this, then the percentages are:
This is an unexpected result. Maybe there is something about the gossip @SevKohler 
These methods in AqlQueryServiceImp.query() seem to be clear candidates for optimization:
AqlSqlLayer.buildAslRootQuery()AqlQueryParsingPostProcessor.afterParseAql()aqlQueryRepository.prepareQuery()