Conformance Testing: implementation alternatives

Hi all, I did some tests trying an alternative to Robot/Python for conformance test implementation.

I’m used to write tests in Spock Framework and created a couple of test cases from the Conformance Test Case Specification

Spock uses Java/Groovy (which runs in the JVM) and has a DSL to express test cases and data sets.

In the script above you can find two test cases, the second one is called data driven testing and the test data sets are defined as tables in code (the Spock DSL).

The idea of this is to analyze what would be better for the community in terms of maintaining and running the conformance test suites.

Though there is no issue of having multiple implementations of the tests at all, and this is not a Robot vs. Spock discussion, it’s more about what openEHR implementers are used to and what is easier for them.

I’ve been working with Robot for a couple of years now in EHRBASE, and it was great for implementing test cases that were not so complex, because of it’s declarative nature. When the complexity grew, and these test suites were refactored to reuse code, it got a little messy very fast. On the other hand, Spock is imperative, which most devs are most used to. Robot is Python and Spock is Groovy, I’m sure most developers and not familiar with any of those, though Groovy is closer to Java/C# than Python, and runs on the JVM.

One last comment is for Robot we have a lot of test cases already implemented, and for Spock we just have this little test I did in a day, so if people like Spock, there is some migration work that is needed.

We discussed this mainly with @thomas.beale who is working on the CNF spec. It would be useful to get other points of view. Thanks!

1 Like

I missed this when it was first posted. We are currently examining the Robot suite and trying to identify the level of effort it would take to remove the ehrbase-specific parts and replace them with more generic adapters. It’s possible that starting from scratch with a more familiar language (we use Kotlin/Java) might actually be faster.

Have your thoughts on this changed at all over the summer?

Hi @joseph.kane as mentioned in my initial message, in my roadmap I have the implementation of the test cases in Spock https://spockframework.org/ just because of that reason: be more friendly for Java/Groovy (maybe Kotlin too) devs.

I don’t have a timeline for this since I’m doing it in my free time. Help is welcome.