“A picture speaks 1000 words” to quote a proverb. I rephrase this to “A drawing replaces 100 (textual) user stories”. I will talk about how we improved our quality assurance by drawing, initially with pen and paper, where we have developed graphical representations of the system under test (SUT). Our SUT is a national portal of e-health services in Norway which has millions of monthly visits (it had 73M visits in 2020) with many integrations.
Drawings of SUT enabled systematic extraction of abstract tests with minimal cognitive load. Graphical representations help identifying tests which is difficult to extract from (textual) user stories, especially negative test cases. Furthermore, drawings can be used not only for quality assurance purposes but also as a tool for learning (for newcomers) and collaboration (to get in common understanding between various roles in team). The graphical drawings have been presented internally in our organization in various forums. The abstract test cases extracted from drawings are further refined and implemented as automated test cases.
As of today, we have over 700 automated tests executed daily as a part of our CI/CD pipeline. We have carried out drawing (modeling) for testing purposes in two important cases for helsenorge.no.
The first case is when testing access control. We have used tree structure to draw (model) and obtained holistic view of portal’s access control. Nodes of the tree represent attributes that influence access while edges are values of those attributes. The leaf of the tree represents a scope which is a grouping of individual services to which a user has access to. When generating tests from the access control tree our focus has been on negative tests (i.e., verifying absence of access) rather than positive tests (i.e., verifying presence of access).
The second case covers testing GDPR (General Data Protection Regulation) solution in 2018. By drawing states and transitions (finite state machine notation) we have visually illustrated interdependencies between privacy levels and authorizations of the portal. Graphs were usefully especially for systematic identification of forbidden transitions (negative tests).
Those two cases have been presented in the following (peer-reviewed) conferences:
- Davrondzhon Gafurov, et al., “Access Control Tree for Testing and Learning”, IEEE/ACM International Conference on Automated Software Engineering, 2021. (Rank A* – core2021).
- Davrondzhon Gafurov, et al., “Applying Lightweight Model-based Testing for National e-Health Portal in Norway”, IEEE/ACM International Conference on Automated Software Engineering, 2020. (Rank A – core2020).
