Veredictum: a machine-readable conformance framework for openEHR

Conformance
1

Hi everyone,

A few days ago I posted here about FerroEHR, my Rust CDR. Since then I got feedback that pointed at a problem I had been circling for a while: my conformance test suite lived in the same repository as the server it graded. When the two disagreed, I was the one deciding which side to adjust, and that is exactly the situation a conformance claim is supposed to rule out.

So I split them. The CDR stays FerroEHR, and the conformance work now lives in its own repository: Veredictum (veredictum.eu). Most of that repository is not Rust code, and this post is mostly about the part that is not.

The conformance framework

The full design lives in ARCHITECTURE.md. It builds on the community CNF work from 2021–2022, keeps that vocabulary on purpose, and cites where each part comes from. The concepts are not mine; what I tried to fix is the operating model that made the original effort stall. The short version:

  • One versioned catalogue is the single normative source, the Abstract Test Suite in ISO terms. It holds 1103 case cores, one file per case, each citing the released specification section it enforces (here is one), and the released text is vendored beside it so every citation resolves against text you can read. Where the specs turned out to be silent or contradicting themselves, that went into an ambiguity register instead of me deciding quietly.
  • Meaning and wire are separate. A case core says what an operation means in Service Model vocabulary and never names a status code; the 247 per-ITS operation bindings do that (the binding for the case above). Supporting a new protocol means writing new binding files and leaving the catalogue alone.
  • The claim model comes from the ISO conformance-testing tradition. Your ICS declares which CNF tiers and capabilities you claim (CORE, STANDARD, OPTIONS, SEC-BASIC; FerroEHR’s statement is a committed example), your IXIT declares how to reach your deployment (example), and the verdict certifies exactly that claim. The certification ladder starts at an ISO/IEC 17050 supplier’s declaration, with witnessed verification and accredited assessment as later rungs.
  • A verdict is a pure function of the statement, the recorded exchanges, the catalogue and the capability matrix. The same record always gives the same bytes, and a claimed capability with no evidence reads not_evidenced instead of quietly disappearing. Measured performance classes and a security baseline sit beside the functional profiles.
  • Everything is schema’d data. Every artifact family has a JSON Schema in schemas/, and no test definition lives in Rust code.
  • Any harness can prove itself. The repository ships a runner-verification pack: replay the recorded transcript and reproduce every adjudicated verdict byte for byte. My own runner has to pass the same pack.

Because the whole catalogue is data, nothing ties it to Rust. Someone could write a runner in Java, TypeScript or Python against the same schemas, replay the verification pack to prove it correct, and get the same verdicts. I would genuinely like that to happen. And to be clear, I am not claiming this as any kind of standard. Something like that should be owned by the community, and ARCHITECTURE.md describes what governance I think it would take.

The Rust instrument

To show the framework runs end to end, I built an instrument on it:

  • The CLI (crates.io, cargo install veredictum): validate the catalogue, run against your server’s REST wire, verdicts from the recordings. perf, stress and aql-probe share the same discipline.
  • The web console (ghcr.io/rubentalstra/veredictum): connect a CDR, paste or compose your claim, watch the run live, read results with the wire transcript, and export a signed record with a public /verify page. gpg --verify accepts the same bundle without Veredictum installed, because trust is good and verification is better.

The EHDS

The timing matters too. The European Health Data Space regulation (Regulation (EU) 2025/327, in force since March 2025, with the EHR-system obligations applying in phases over the coming years) will require EHR systems handling priority data categories to carry harmonised software components, pass a conformity self-assessment, and carry an EU declaration of conformity with CE marking. The regulation foresees European digital testing environments for pre-market testing of the harmonised components, and it states that manufacturers remain fully responsible for conformity even after using them. That last part is what interests me. If the responsibility stays with the manufacturer, the evidence behind a declaration is worth more when someone else can reproduce and check it, and producing that kind of evidence is what this framework is for. Where EHR software also qualifies as a medical device under the MDR (Regulation (EU) 2017/745), conformity assessment for classes IIa and above involves a notified body, so substantiated evidence matters there too.

v0.1.0

I just released v0.1.0. Do not read too much into the word stable: the version number is stable, the product is young. I am the only maintainer, things will break, and the catalogue has gaps. Every correction needs a specification citation to back it, so the catalogue gets better with each report instead of just bigger.

What would help me most:

  • Point it at your CDR. A wrong-looking verdict is the most useful report I can get: it is either your server, my runner, or my catalogue, and finding out which is the entire product.
  • Challenge the citations, and the ambiguity register. Every case names the spec section it enforces, and an expectation can be refuted by a better reading of the released text.
  • Read the architecture. If you were part of the 2021–2022 CNF work, I would especially value your eyes on what I carried forward and what I got wrong.

Everything is Apache-2.0 (vendored openEHR material under its own terms); docs at veredictum.eu/docs. And to say it plainly: openEHR® is the registered trademark of the openEHR Foundation. Veredictum is an independent community project, not an official openEHR Foundation product, and it has no affiliation with the Foundation’s CNF programme. If the Foundation’s conformance work resumes, I would be glad to align with it.

Thanks for reading.

Ruben

PS: to be transparent about how this is built. The idea and the framework design are mine, but the coding and a good part of the writing are AI-assisted. The repository documents this in its AI statement.

2

This is very important work. Most impressive how speedily you are progressing with it, but I can well imagine the paddling required beneath the surface!

I posted some brief comments about quantum computing in reply to your recent post. They didn’t appear and maybe moderators deemed them off topic for this thread. I will send them to you and would be interested in your take. Can you DM me at my UCL address, d.ingram@ucl.ac.uk. Many thanks again for your contributions.

3

Thank you, @DavidIngram. I have just sent you an email, and I look forward to your notes on quantum computing.

4

Hi @rubentalstra good stuff!

Did you check GitHub - ppazos/openehr-conformance-verification: Conformance verification framework or openEHR implementations · GitHub ?

5

Hi @pablo,

Honestly no, I did not know about that repo. I built it from the specifications-CNF repo on GitHub, the Confluence pages (the 2017 openEHR Conformance page and the 2014 roadmap notes it references), the ISO conformity-assessment documents, plus the ehrbase mirror and your CaboLabs article. Your repo never surfaced through any of those.

The chapters of the official Test Schedule (master06 to master09) are yours, and 328 of the 1107 case cores in my catalogue cite those chapters, so roughly a third of what I built stands on your foundation, one step removed. I read the 2019 SEC slides and the 2023 Framework Design PDF this evening: the SM-not-REST split, the Conformance Statement, and the “a test report is not a certificate” point are all in your material before they were in mine. I will fix the citations this weekend and credit you properly in ARCHITECTURE.md.

Two things I want to ask you:

  • Can I reuse your test cases and data sets in Veredictum, cited back to you? Your repo has no license file, so I need your OK (or better, add one, Apache 2 like your 2019 slides said).
  • Does the QUERY test case doc from the “76 syntax variants” slide still exist? master11 has zero official cases and your demographic OPTs plus that doc are where I want to start.
6

Hey @rubentalstra

No worries, I guess that’s a little obscure and didn’t get much publicity. Only a handful of people working on openEHR conformance verification know about it. It was referenced back on the EHRCON23 (my talk was about openEHR conformance and the spec in that repo).

That’s work that started back in 2019 when I was working for Hanover Medical School on HiGHmed on what became EHRbase (I wrote the original conformance tests for EHRbase). So some of the tests and data you found on the EHRbase’s repo might be the same or very similar to what I have on that repo, though expanded and formalized, since I continued working on conformance after my contract finished with Hanover.

I was also in talks with @thomas.beale who did the initial CNF spec, since I thought that was underdeveloped/incomplete. We agreed on some points, though we never got to update the spec.

In the document you found in my repo you’ll see that I consider conformance verification to have a broader scope, and not only be just a bunch of curl/postman calls to the API and checking for response codes. The final goal for this level of formality is really software certification, and not only for CDRs but or any kind of software that implements openEHR, like a CKM, demographic app, clinical app, etc.

The demographic part should be updated since that’s based on my proposal for the demographic API, which I’ve implemented in Atomik, but IIRC recently the SEC published the initial demographic API, so the tests should be based on the official API not in my own implementation.