# openEHR API implementation in Rust **Category:** [Implementation](https://discourse.openehr.org/c/implem/39) **Created:** 2023-01-24 22:40 UTC **Views:** 1694 **Replies:** 44 **URL:** https://discourse.openehr.org/t/openehr-api-implementation-in-rust/3489 --- ## Post #1 by @pintariching Hi there! I want to implement an API from scratch, to be an alternative to EHRBase. I've already started some work here: [ehrust](https://github.com/pintariching/ehrust/) (not sure about the name). I'm reading the documentation mainly from [here](https://specifications.openehr.org/releases/ITS-REST/latest/ehr.html). Are there any other good sources I should look up? I'm wondering if I should make it as similar as possible as EHRBase? I'm not entirely about the exact specifications of the responses and how to structure the database. --- ## Post #2 by @sebastian.iancu It is indeed the right link of generic API specifications, release 1.0.3. If you would want to do something less generic and do 1:1 with EhrBase, you have to look it up in their repo / docker. About openEHR API specs, I would add two things: - that there is besides others also a generated Rust codebase, in a development branch in the repo. - will start soon working on v2 of of the specs from your link above, so consider this aspect in your plan. --- ## Post #3 by @pintariching One thing I wasn't entirely sure was, that EHRBase uses headers a lot to return minimal data and I was looking through the specifications if there's anything about that, but I guess there isn't? About the generated Rust codebase, could you please link it, I'm not having any luck finding it. Can you say roughly when the V2 spec is going to be released? Are there going to be massive breaking changes that I should worry about and postpone my project? --- ## Post #4 by @pablo Please note it is not only an API, you'll need to implement the persistent model, the data management flows (versioning), the query processing, etc. --- ## Post #5 by @pintariching Yes I'm aware of that. I thought starting at the API and seeing what it needs and uses and then implementing that. There's only so much I can learn in a given time :sweat_smile: --- ## Post #6 by @pablo It's a very interesting are to work with, specially n a new technology stack, though it's painful, been there, done that with EHRServer some years ago, and it gave me knowledge I wouldn't get anywhere else. The advice I would give to my past self is: design your data model thinking in the query mechanism you will implement as the first thing. The API is the easier part but requires the rest of the components to be in place. Best of luck! --- ## Post #7 by @pintariching Okay so start from back to front, thank you! I still have to read up on the AQL and by data model, you mean how I'll structure things in the database like tables and such? --- ## Post #8 by @thomas.beale You should also have a look at the [Platform Services specification](https://specifications.openehr.org/releases/SM/latest/openehr_platform.html) - that gives the abstract semantics; then you can separate REST specific semantics (HTTP headers and the like) cleanly. --- ## Post #9 by @ian.mcnicoll I suspect @Pablo was indirectly suggesting the opposite!! openEHR is designed to work at a logical level from a client perspective, essentially hiding the database layer from client apps and services. My strong suggestion (to everyone) is to get your head around the service layer and how openEHR works logically, compositions, aql, templates etc by using an existing CDR and the standard openEHR API. There will be lots of opportunity to build some middleware type code in Rust as a convenience wrapper, and to understand how templates can be used to drive other artefact generation - see Pablo's work. CDR building is serious engineering and database design / optimisation / indexing is a critical part of that and non-trivial. I'm sure you have the capacity to do it but get very comfortable with the openEHR paradigm before you go there. By all means delve into the OSS projects like EhrBase and Pablo's cloud server to see how they have tackled the challenge but I'd really hesitate to putting real effort into this until you are up and running using an exiting CDR. --- ## Post #10 by @pintariching One thing I'm unsure about is, that archetypes use a lot of class inheritance that Rust doesn't have. I'm wondering how I'll solve this problem. --- ## Post #11 by @ian.mcnicoll In practice, you are not working with individual archetype but rather templates, and in the xml/json representations that we use in the REST APIs , a lot of these inheritances are flattened out. However beyond my 'clinical hacker' territory to comment further. --- ## Post #12 by @thomas.beale [quote="pintariching, post:10, topic:3489, full:true"] One thing I’m unsure about is, that archetypes use a lot of class inheritance that Rust doesn’t have. I’m wondering how I’ll solve this problem. [/quote] Do you mean the archetype meta-model, i.e. what we call [Archetype Object Model (AOM2)](https://specifications.openehr.org/releases/AM/latest/AOM2.html)? Or archetypes themselves - which are instances of that model. Since you are implementing, I would think you mean AOM2. You can see what it looks like in Java by looking at [Archie](https://github.com/openEHR/archie#readme). @pieterbos and other implementers of Archie will show you around. Since Rust doesn't have inheritance per se, I would imagine that you will need to use traits and so on. There would be common Rust community tricks for achieving what inheritance achieves. Aside: I have been contemplating what other new languages might be interesting for openEHR, as have many others. I'm currently messing around with Kotlin (so are others, more seriously). @borut.jures is using Dart. Java, C#, and PHP seem to be default languages for implementers. These languages all have inheritance. I am not 100% if Rust is a really good fit for meta-models (which tend to contain inheritance), although it seems to me it should be really good at APIs. And in API design, you can flatten out most if not all inheritance to concrete types. Maybe that will help? --- ## Post #13 by @pintariching [quote="thomas.beale, post:12, topic:3489, full:true"] Or archetypes themselves - which are instances of that model. Since you are implementing, I would think you mean AOM2. You can see what it looks like in Java by looking at [Archie](https://github.com/openEHR/archie#readme). @pieterbos and other implementers of Archie will show you around. [/quote] Yes I meant the AOM2. I still have to read further into it but the Archie repo will come in very handy to get a better idea how it all looks like thank you! [quote="thomas.beale, post:12, topic:3489, full:true"] I am not 100% if Rust is a really good fit for meta-models (which tend to contain inheritance), although it seems to me it should be really good at APIs. And in API design, you can flatten out most if not all inheritance to concrete types. Maybe that will help? [/quote] There is a few ways to get around inheritance, as you mentioned traits are one way, another potential way is to use Enums, duplicate fields or simply have a base field that represents the parent class. --- ## Post #14 by @borut.jures I agree with @thomas.beale about the lack of inheritance in Rust. Java has polymorphism. Dart doesn't so I used single inheritance with `implements`: ```Dart class Template extends AuthoredArchetype implements Serializable { class AuthoredArchetype extends Archetype implements AuthoredResource, Serializable { class Archetype implements Serializable { abstract class AuthoredResource extends Object implements Serializable { ``` Archie (Java) implements this as: ```Java public class Template extends AuthoredArchetype { public class AuthoredArchetype extends Archetype { public class Archetype extends AuthoredResource { public abstract class AuthoredResource extends ArchetypeModelObject { public abstract class ArchetypeModelObject extends OpenEHRBase implements Serializable, Cloneable { ``` @pintariching Can you please write how Rust would handle this (so that Thomas and I learn a bit about Rust :blush:). This example doesn't use polymorphism. --- I personally like Dart but I wonder if TypeScript would be a better option :thinking: At least to help web developers work with openEHR without learning it (TypeScript could also be used for the backend but there are many other options on the server-side). (Dart already has a Dart -> JavaScript compiler which I used to create a [JavaScript SDK for openEHR](https://neoehr.com/openehr/sdk-java-script)) @pintariching I have to suggest that you **generate** your AM/RM classes in Rust. There are different release versions of AM/RM and your app/server must work with many different versions (you cannot control which version another hospital is using to send data to your openEHR CDR). openEHR uses BMM files to specify the AM/RM models. They can be used by your generator to write 100s of required classes for you :wink: --- ## Post #15 by @ian.mcnicoll Im going to make bold, and probably unpopular statement here. Walk away from AOM2 /ADL: (for now) or listen to @borut.jures. As an run-time implementer you really do not need to understand this stuff. It is for tooling designers and even if you want to get into tooling / utility writing , there are much easier starting points. Get to understand how the REST API works, how to hook it up to an app/ querying etc. By all means build some libs/utilities in Rust. Get it working. Understand how new or updated template manifest them selves in new data structures. Understand where you might have to flatten out some inheritance to work in Rust and have a look in the underlying RM to see how/if that might be improved. By all means dip into archetypes and templates from the perspective of design-time tooling/utility support but start with the .opt operational template format or even better the simpler Web template format. Basically these are 'compiled/ aggregated constraint definitions. Ducks for cover!! --- ## Post #16 by @borut.jures @ian.mcnicoll Where were you with the same advice when I popped-up here :wink: However @pintariching idea probably includes the CDR. He/she/they used `API` but it looks like a CDR was meant (based on): [quote="pintariching, post:1, topic:3489"] ...to be an alternative to EHRBase. [/quote] @pintariching Can you share more about your project so that we don't confuse you too much? (but listen to Ian regardless of what your project is :blush:) --- ## Post #17 by @pintariching [quote="borut.jures, post:14, topic:3489, full:true"] @pintariching Can you please write how Rust would handle this (so that Thomas and I learn a bit about Rust :blush:). This example doesn't use polymorphism. [/quote] I would have to look what each class extends. If it just extends with functions then In Rust I'd maybe write it like this: ```rust struct AuthoredRespource; trait Archetype {} trait AuthoredArchetype {} impl Archetype for AuthoredResource {} impl AuthoredArchetype for AuthoredResource {} ``` and so on. However if there are fields added, then I'd probably do it like: ```rust struct AuthoredResource; struct Archetype { resource: AuthoredResource, } struct AuthoredArchetype { resource: AuthoredResource } ``` And so on. I'm unsure if this is the right order on what inherits what so excuse me for it :sweat_smile: --- ## Post #18 by @pintariching [quote="borut.jures, post:16, topic:3489, full:true"] @pintariching Can you share more about your project so that we don't confuse you too much? (but listen to Ian regardless of what your project is :blush:) [/quote] Okay honestly I wasn't entirely sure what I was getting myself into but now it's becoming more clear. I want to implement a CDR like EHRBase, so mostly the API, the Query things and such. For all of this it's clear to me that I firstly need to be able to parse a Template and be able to validate it, which brings me to all of the AOM/ADL things. However I didn't realize I could auto generate classes like @borut.jures mentioned which is going to be a major help in development at least until I understand how it completely under the hood. --- ## Post #19 by @borut.jures You could skip most of the ADL if you decide to use JSON OPT2. There is Nedap's Visual Studio Code extension to convert ADSL into JSON: https://discourse.openehr.org/t/adl-and-aql-visual-studio-code-extension/1085 --- ## Post #20 by @ian.mcnicoll "For all of this it’s clear to me that I firstly need to be able to parse a Template and be able to validate it, which brings me to all of the AOM/ADL things" To store data in the CDR, no you don't!! You just send the instance data json blob, and tell it the name of the template to validate against (already uploaded and registered). the CDR will do the validation and give you validation errors. If you want to do client-side validation, you do want to be able to parse the template, for this (at least for staters) I would strongly suggest you look at the Web template produced by both ehrBase and Better. THis is derived from the .opt which is pure AOM-based XML but is quite hard to get your head around. Even I can work out Web templates!! - here is a Typescript [Asciidoc generator] (https://github.com/freshehr/wt2doc) based on parsing a Web template (forked from work by @bna . That is the fastest way to get to grips with the constraints involved, and expressed partly by the underlying RM but then with further constraints at archetype-level. Lots of opportunity to add value and experiment with tooling an utilities Once that makes sense by all means get into .opt syntax/parsing/code generation as various including @Neo4j have done. This is really clever and powerful technology but it does have a number of concepts that challenge almost everyone at first and its's easy to be overwhelmed at the start . I always urge people to start at the client layer and work deeper into back-end stuff, even if that is the end goal. The 2-level modelling paradigm and the way that archetypes graft onto the RM semantically is particularly mind-bending at first. Great questions and keep asking!! --- ## Post #21 by @erik.sundvall I sometimes say this in [lectures about openEHR.](https://docs.google.com/presentation/d/12ZrXvdIpp7HRG1tSNCat5ISjW_Aswk2p/edit#slide=id.p301): ![image|690x385](upload://z29USUFn31yAykHUdBJTw5qSsKi.png) ...and since you want to use RUST, the third rule probalby appllies to you :slight_smile: My advice regarding storage would be to pick a database that: 1. is good at storing and querying tree-structures, 2. capable of storing an existing standardised canonical openEHR serialisation format, wich currently would be JSON or XML and 3. the database should also already have a built in query language for graphs or tree-structures There are many such DBs, our research in [the ORBDA paper](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0190028) indicates that for example ElasticSearch and Couchbase could be interesting for large scale deployments. If you are targeting something smaller with Rust like running openEHR in a resurce constrained environment like a Raspberry Pi then other options are likely better. When it comes to AQL you then "just" need to make a translator that transforms AQL to the DB's built in query language. Likely using a [RUST-based parser generator](https://www.google.com/search?q=rust+based+parser+generator) and the [published AQL grammar files](https://specifications.openehr.org/releases/QUERY/Release-1.1.0). (A long time ago I was invovled in doing this [in Java with XQuery as target](https://github.com/LiU-IMT/EEE/tree/master/src/main/java/se/liu/imt/mi/eee/AQL_Parser) since we used an XML database). Then there is the problem of eating the openEHR elephant... Although some things have changed and the REST-spec that became openEHR standard is different than in the [REST+openEHR-paper](https://bmcmedinformdecismak.biomedcentral.com/articles/10.1186/1472-6947-13-57) the suggested architecure breakdown into smaller pieces ([Figure 3](https://bmcmedinformdecismak.biomedcentral.com/articles/10.1186/1472-6947-13-57/figures/3)) likely still would be useful and let you complete one useful Rust-based piece at a time: ![image|690x484](upload://epotd333BTNAkykWpRXg4KdoEZJ.png) You could for example make an excellent RUST-based read-only openEHR CDR (useful as backup copy, research DB or for load-splitting/sharing of read operations) It could have full AQL capabilities without you even having to first learn and make complex validation stuff. It could be used also in read+write scenarios by piggy-backing on EHRbase or any other openEHR CDR with validation - or just call and use their input validation code until you some time in the future may have written one (which may be a challenge in RUST). --- ## Post #22 by @ian.mcnicoll > @ian.mcnicoll](https://discourse.openehr.org/u/ian.mcnicoll) Where were you with the same advice when I popped-up here ! Yes - sorry about that!! I failed in my duty as life-guard, trying to prevent newbie divers going straight to the highest 10m board!! ![image|618x499](upload://tBxsPPETWX9zRuiQONXBHPuMDyx.jpeg) --- ## Post #23 by @pintariching Okay that's a lot of replies I did not expect this :sweat_smile:. Let me get through them one at a time. [quote="ian.mcnicoll, post:20, topic:3489"] To store data in the CDR, no you don’t!! You just send the instance data json blob, and tell it the name of the template to validate against (already uploaded and registered). the CDR will do the validation and give you validation errors. [/quote] Okay I think I'll start with web templates then and put off the .opt parsing until I get a better idea about the system. [quote="erik.sundvall, post:21, topic:3489"] My advice regarding storage would be to pick a database that: 1. is good at storing and querying tree-structures, 2. capable of storing an existing standardised canonical openEHR serialisation format, wich currently would be JSON or XML and 3. the database should also already have a built in query language for graphs or tree-structures [/quote] I didn't really put a lot of thought in the database, I'm starting with PostgreSQL as EHRBase is also using it. If I wanna go the "Rust purist" way, perhaps [SurrealDB](https://github.com/surrealdb/surrealdb/tree/c2465338b248cca0d6c558e653d1ba094f1713bb) could be a good fit? It's still in development but I think it has a lot of features that could help with implementing an openEHR compliant CDR. Also I want to thank everyone for your quick replies, they've been very helpful and gave me quite a lot to think about! I think I'll start either like @erik.sundvall recommended with a read-only CDR with full AQL capabilities, or a CDR with only web templates support or maybe both I'll see how much time I can dedicate! --- ## Post #24 by @erik.sundvall If you want to build a CDR I think database query considerations will be pretty central. You likely don't want to reinvent a system for tree traversing queries unless it is your main interest or if it would be a PhD project of yours... I don't know much about SurreadlDB, but it looks like a relational database. Plain relational DBs are not a good fit for (very variable) tree/graph structures (like openEHR) in themselves. Experienced openEHR system vendors like https://www.better.care/ that I believe offer their customers very many options regarding what relational DB to use for storage usually have other tricks like an inverted index (like [Lucene](https://lucene.apache.org/)) for the AQL path traversal/resolution and use the relational DB maily as a blob/JSON/XML storage plus some predifined index fields. I believe the reason EHRbase gets away with using what looks like a relational DB is that they have focused on using PostgresQL that has a lot of nice JSON (tree) traversing functions built in, right @birger.haarbrandt and @christian? Look for things like JSONB and JsonPath in https://ehrbase.readthedocs.io/en/latest/03_development/05_technical_docs/index.html (or the source code) for clues. (But just cloning the EHRbase approach in another language might not be the most fun and productive use of time.) So if using a relational DB you would in the "[Figure 3](https://bmcmedinformdecismak.biomedcentral.com/articles/10.1186/1472-6947-13-57/figures/3)" approach above likely need to add an inverted index adjacent to or as part of the "EHR database" block. Or use tree-relational tricks like Dewey encoding as described in https://pubmed.ncbi.nlm.nih.gov/28423824/ Anyway if you in Rust make what in the figure is labelled as "R/W interfaces" and "Query executors" you can experiment with different storage implementations without having to redesign the rest of your application stack. Please note that there is nothing in [the official openEHR REST specs](https://specifications.openehr.org/releases/ITS-REST/latest) corresponding to the "Versioned Object" box of [Figure 3](https://bmcmedinformdecismak.biomedcentral.com/articles/10.1186/1472-6947-13-57/figures/3) - since when makng the standard (some years after the openEHR-REST-paper publication) in the [SEC](https://openehr.org/programs/specification/board_members) we chose to split that into more specialised resources like ["/ehr/{ehr_id}/composition"](https://specifications.openehr.org/releases/ITS-REST/latest/ehr.html#tag/COMPOSITION) and some of it's siblings. The "Query" box corresponds fairly well to the ["/query" API](https://specifications.openehr.org/releases/ITS-REST/latest/query.html) plus some query storage from the ["/definition/query" API](https://specifications.openehr.org/releases/ITS-REST/latest/definition.html). --- ## Post #25 by @sebastian.iancu Back to your earlier questions: - OAS generated code is now at https://github.com/openEHR/specifications-ITS-REST/tree/feature/codegen_testing/codegen/oas-ehr; this was an experiment around best way of describing OAS model schema so that we get best outcome when we generate code in various languages; in earlier stages I also added Rust, but looks like I did not included in the end that branch anymore. If you are interested in helping in reviewing that code, then I can then keep you updated next time I'm working on it. Or you could also try to generate done using some very dev-mode tooling I made in that branch. - about plans on v2: we will work it year on it, and it is not yet decided how many of the outstanding issues will be in v2 and how many in v1 as release-1.1.0; this subject is still under discussion. In any case, v2 will include a few small breaking changes as we try to fix some inconsistencies in v1, but the goal is anyway to minimize them. More links about plans: [CRs - Change Requests](https://specifications.openehr.org/releases/ITS-REST/Release-1.1.0/changes), [Open Issues](https://specifications.openehr.org/releases/ITS-REST/open_issues) and [this work-overview page](https://openehr.atlassian.net/wiki/spaces/spec/pages/1919811604/Overview+for+REST+API+1.1.0+work+to+be+done). --- ## Post #26 by @ian.mcnicoll [quote="pintariching, post:3, topic:3489"] One thing I wasn’t entirely sure was, that EHRBase uses headers a lot to return minimal data and I was looking through the specifications if there’s anything about that, but I guess there isn’t? [/quote] I think we missed that - yes they are documented but slightly confusingly perhaps, in the Intro https://specifications.openehr.org/releases/ITS-REST/latest/overview.html#tag/Requests_and_responses --- ## Post #27 by @pintariching [quote="erik.sundvall, post:24, topic:3489"] I don’t know much about SurreadlDB, but it looks like a relational database. [/quote] A quick note about SurrealDB and why I'm thinking about it is, that it [supports almost everything you can think of in a database](https://surrealdb.com/features#datamodel). I still need to see if it's a good fit for this use case by trying it out! :smiley: [quote="erik.sundvall, post:24, topic:3489"] Anyway if you in Rust make what in the figure is labelled as “R/W interfaces” and “Query executors” you can experiment with different storage implementations without having to redesign the rest of your application stack. [/quote] So the "R/W interfaces" and "Query executors" are meant like an interface between your database and your REST API, that's how I imagine it? --- ## Post #28 by @pintariching [quote="sebastian.iancu, post:25, topic:3489"] If you are interested in helping in reviewing that code, then I can then keep you updated next time I’m working on it. Or you could also try to generate done using some very dev-mode tooling I made in that branch. [/quote] I'll try and play around with the tooling and check out the repo and see if I can make something. Also yes, sure I can help reviewing it :smile: [quote="sebastian.iancu, post:25, topic:3489"] In any case, v2 will include a few small breaking changes as we try to fix some inconsistencies in v1, but the goal is anyway to minimize them [/quote] Okay if V2 is going to be similar to V1 then I don't think it's going to be too hard to move to it! --- ## Post #29 by @pintariching Yeah no I'm just dumb and missed that :sweat_smile: --- ## Post #30 by @ian.mcnicoll I missed it too when trying to explain it to a colleague!! Knew it wassomwhere , justcouldn;lt find it. @sebastian.iancu - I think we need to somehow connect the generic Header infomation with each of the resources that might need it, and add headers to the examples. Happy to help - does this need a CR? --- ## Post #31 by @sebastian.iancu That's in a way one of the reason we moved to OpenAPI format, so that we can better specify examples, and formally indicate supported headers for each path/resource. These changes will come (to some extent) with the next release, and yes, it would be helpful to have a CR. --- ## Post #32 by @birger.haarbrandt Yes, EHRbase is using a hybrid approach with larger parts represented in tables, complemented by JSONb. However, this is currently being re-evaluated and we will change towards a different model that will make EHRbase more independent of Postgres and comes closer to ANSI/ISO SQL specifications. --- ## Post #33 by @pablo [quote="pintariching, post:7, topic:3489"] Okay so start from back to front, thank you! [/quote] [quote="ian.mcnicoll, post:9, topic:3489"] I suspect @Pablo was indirectly suggesting the opposite!! [/quote] It depends on what you call back and front :) My approach would be middle to back, then front What's middle? First design the architecture needed to retrieve data from an abstract storage, figure out what metadata is needed in your DB to get that data out, then design your DB. Then figure out how to get your data in, and finally design the internal management of versions. All design first. And I would also follow that order for the implementation. Then the last thing is to put the API over your platform (data first - API last approach, with strong emphasis on the architecture design). Note: if you try to implement AQL, that will affect the structure of your database, since you need some kind of document storage (for AQL you need the trees in your DB or have a hybrid DB like relational + document). It's not easy, I have the scars to prove it! hehe --- ## Post #34 by @pintariching I suppose if a hybrid approach is needed, [SurrealDB](https://surrealdb.com/) could be the right choice. I'll have to play around with it. I'll report back here if I manage to get something done, I'm a bit busy right now :sweat_smile: But thank you for your help, I'll make sure to follow your recommendations! --- ## Post #35 by @pablo [quote="pintariching, post:34, topic:3489"] I suppose if a hybrid approach is needed [/quote] I would say "needed" is a strong word. The right choice will be the one that allows you to implement the openEHR specification and comply with your platform requirements. So there is no "needed" approach, just a set of possibilities (architectures and technologies) to choose from. There is no such thing as "one fits all". --- ## Post #36 by @erik.sundvall Surreal DB does seem to have some object/tree storage and query capabilities so it might work for translated AQL-queries if you are lucky. That would be one of the first things I'd investigate when picking a DB if you want AQL support in your CDR. (On the other hand it of course would help to have the knowledge level of people like @Seref that spent part of his PhD on AQL reasoning or @chunlan.ma that invented AQL/EQL) If the object/tree-querying mechansims in the DB are too weak for AQL then you'll need a hybrid solution as discussed earlier (e.g. Dewey encoding or a DB-external inverted index). Another design choice to investigate when it comes to DBs that (like SurrealDB]?) do have support for tree-shaped object/document schema defintions is, if you either want: 1. to validate data using an openEHR (operational) template based validator before the incoming data gets pushed to the DB, or instead 2. to push as much validation as possible into the validation features of the DB itself by first auto-creating a schema for each (operational) template that gets loaded/configured into the CDR. Just to clarify, both 1 & 2 validation methods above would likely configure their validation based on an admin uploading an operational template using for example [ADL 1.4 API](https://specifications.openehr.org/releases/ITS-REST/latest/definition.html#tag/ADL1.4/operation/definition_template_adl1.4_upload) or the [ADL2 API](https://specifications.openehr.org/releases/ITS-REST/latest/definition.html#tag/ADL2/operation/definition_template_adl2_upload) for operational templates. #2 may result in very many schema in the DB over time (which may be fine if the DB was designed for handling that). #1 makes your validation algorithms DB-independent so that it is easier to change DB engine later and easier to use your validator as a freestanding component in other contexts. One thing that is easy to forget for beginners when it comers to planning openEHR validation (or tool design) is that you need to validate based both on the rules in the (operational) template AND the model/rules of the openEHR RM. --- ## Post #37 by @maxwell_flitton Hey guys. I'm Maxwell Flitton. The author of the Packt book "Rust web programming" and the other book "Speed up your Python with Rust". I'm currently writing the O'Reilly book on async Rust. I've recently joined the SurrealDB team as a core dev. I have a deep interest in medical applications. Before going back to study physics I was a nurse in A and E for 7 years. I'm based in London. If anyone on here wants to talk to me about collaboration shoot me an email at: maxwell.flitton@surrealdb.com would love to hear from you about how I can get SurrealDB into the medical field as SurrealDB also supports embedded, in-memory, server, and cloud. I had a lot of options and chose to sign with SurrealDB because once SurrealDB can comply with the medical community, it's going to be a pretty good Swiss army knife to solve a lot of problems. I also plan on integrating transaction logging and encryption. This open-source database will hopefully empower hospital and research departments. --- ## Post #38 by @joostholslag Hi maxwell, welcome to the community. What do you like best about surrealDB? Did you know openehr does not have a relations datamodel? Usually it’s json objects. Because medicine has highly (hourly) changing datamodels with complex variable structures. Postgres with json support usually works acceptably. But performant querying across/within json structures is still a major engineering challenge solve by inleveren a handful of parties. How do you look at this problem? --- ## Post #39 by @ian.mcnicoll Hi Maxwell and welcome. Just to add to what Joost has said, as you may not get the answer here you were hoping for!! openEHR is all based on the idea that we can construct vendor/tech-neutral datastores (CDR) where the interfaces are all based on open standards but critically where new data models can be implemented without changing the internal database schema, and where different dB technologies like SurrealDb (with possibly veery different internal schema, can be swapped in to a CDR without disturbing the application layer , where everything is logically arranged and queried according to the models uploaded by clinical informaticians. So this is a very opinionated world that any new DB has to fit into, and which may negate many of the advantages of flexibility that you mentioned. In an openEHR world no-one is talking directly to the DB. It is the job of the CDR implementer to make that magic happen, and to choose their DB / schema approach. It's not hard to build an openEHR CDR, it is hard to make it perform but we have many examples of this approach working at scale now. Of course many hospitals etc will continue to use traditional wired-to-DB approaches but we think that the complexity of clinical data and the need to be vendor/tech-neutral will quickly make that a thing of the past. --- ## Post #40 by @borut.jures I’m well aware that wired-to-DB approaches are not advised for openEHR but with new databases and DevOps tools it would be interesting to try such an approach just for the fun of it. It would mean using the native features that databases like SurrealDB offer and expecting the full performance of the database engine. Basically to update the native schema description after each openEHR template is deployed and deploy the schema to the database. If everything is automated the clinicians wouldn’t depend on the IT (for them such an approach shouldn’t be any different as it is with the existing CDRs). I know it is a crazy idea but it could be an interesting way to spend a few months on it – maybe as a proof-of-concept for presenting SurrealDB to the openEHR community. But @maxwell_flitton re-read this thread and read twice what @ian.mcnicoll says (the short answer is don't do it :blush:). SurrealDB has some interesting features for the hierarchical data (a summary that might be relevant to openEHR from their site – @erik.sundvall also hinted on these in other databases): * As a multi-model database, SurrealDB enables developers to use multiple techniques to store and model data, without having to choose a method in advance. With the use of tables, SurrealDB has similarities with relational databases, but with the added functionality and flexibility of advanced nested fields and arrays. Inter-document record links allow for simple to understand and highly-performant related queries without the use of JOINs, eliminating the N+1 query problem. * SurrealDB doesn't force you into setting up your data model in any one way. Instead you can choose between simple documents, documents with embedded fields, or related graph connections between records. Use schemafull or schemaless tables giving you the flexibility to store whatever you need. Once stored in SurrealDB, all data is strongly typed. * With full graph database functionality, SurrealDB enables more advanced querying and analysis. Records (or vertices) can be connected to one another with edges, each with its own record properties and metadata. Simple extensions to traditional SQL queries allow for multi-table, multi-depth document retrieval, efficiently in the database. * The primary method of querying SurrealDB is using SurrealQL, a similar but modified version of traditional SQL. SurrealQL enables linked documents to be traversed and queried efficiently, while still using an imperative language which remains understandable by data scientists. * SurrealDB can be run as a single in-memory node, or as part of a distributed cluster - offering highly-available and highly-scalable system characteristics. Designed from the ground up to run in a distributed environment, SurrealDB makes use of special techniques when handling multi-table transactions, and document record IDs - with no use of table or row locks. Source: https://surrealdb.com Interesting examples on: https://github.com/surrealdb/surrealdb --- ## Post #41 by @ian.mcnicoll Thanks Borut, You are of course correct, it is 'legitimate' to have the dbSchema updated with each new template, and I think that is how our Chinese colleagues implemented their CDR. Most current CDRs though do have a fixed schema which is not changed in response to new models being deployed. I wouldn't say 'don't do it', I d just say that the engineering challenge required is a bit different than most people expect - fixed schema, grunt performance and clever indexing is what has worked so far, not adaptability. e.g It might be interesting to emulate the EhrBase approach, based on Postgres and JSONB as a start point. --- ## Post #42 by @erik.sundvall [quote="borut.jures, post:40, topic:3489"] the short answer is don’t do it :blush:). [/quote] But @maxwell_flitton the long answer is more fun if you are a database nerd or a PhD candidate in the field, see slide 124-126 in https://drive.google.com/file/d/167z7YF7PdMGOSHwjiSrGD1Jwud9HIaP6/view?usp=sharing follow links there and re-read the older hints in this discourse thread. (Don't get too scared by @ian.mcnicoll and @joostholslag, they are just MDs that want you to spend your time on something clinically more useful like end user applications rather than implementing yet another openEHR CDR :wink: ) ![image|292x500](upload://r0mmEUaxmWnwT0YywruxaBYR7sx.jpeg) --- ## Post #43 by @borut.jures I agree – implement a CDR for fun. I’m guilty of doing many projects for fun. I was introduced to [TypeDB](https://vaticle.com) and “had” to generate the RM model for it. I hope you (@maxwell_flitton ) start your CDR on SurrealDB and report your progress. I’m sure that @ian.mcnicoll will help you regardless :blush: He cannot stop himself from being helpful to IT people. --- ## Post #44 by @joostholslag [quote="erik.sundvall, post:42, topic:3489"] Don’t get too scared by @ian.mcnicoll and @joostholslag, they are just MDs that want you to spend your time on something clinically more useful like end user applications rather than implementing yet another openEHR CDR [/quote] Yes, please don’t listen to me, and do something you actually like yourself! I’m a little worried we scared of maxwell some posts ago:p --- ## Post #45 by @maxwell_flitton Hey guys thanks for the responses. Was in a real coding groove the other day and ended up finishing at 1am. Was building async actors and unit tests for them and fusing a Rust async runtime into a Python system. joostholslag “What do you like best about surrealDB?” I like that it’s built in Rust, it’s still pretty new, and that the founders are open to suggestions. They asked me if I wanted to work for them and after looking at the code and how clean it was, I was happy to. It’s also a good team, the head of Redis JSON is now working on the SurrealDB team. I think it’s an opportunity for me to drive change in a tech perspective in the NHS. I left the NHS due to not being utilised properly. “But performant querying across/within json structures is still a major engineering challenge solve by inleveren a handful of parties. How do you look at this problem?” I’ll have to have a look into this in more depth to give you a decent answer. ian.mcnicoll It makes sense that the CDR needs to be independent. I guess it’s more building optimizations and specific support for the CDR that can be embedded into SurrealDB? NeoEHR “Basically to update the native schema description after each openEHR template is deployed and deploy the schema to the database. If everything is automated the clinicians wouldn’t depend on the IT (for them such an approach shouldn’t be any different as it is with the existing CDRs).” I think adding features to aid in the CDR integration and offering quick wins would be a good use of time. Thank you and Ian for suggesting that I stay away from building another CDR. erik.sundvall I appreciate the context. As I still have memories from my clinical days I want to do something that’s useful too. I’ll look at the presentation today. Thanks for sharing it. Seeing as you guys know a lot more about the landscape here, what project would you suggest I start on? I can build a side project and fuse it directly into the surrealDB engine. --- **Canonical:** https://discourse.openehr.org/t/openehr-api-implementation-in-rust/3489 **Original content:** https://discourse.openehr.org/t/openehr-api-implementation-in-rust/3489