Coverage Criteria for Testing DMM Specifications

Abstract

Behavioral modeling languages are most useful if their behavior is specified formally such that it can e.g. be analyzed and executed automatically. Obviously, the quality of such behavior specifications is crucial. The rule-based semantics specification technique Dynamic Meta Modeling (DMM) honors this by using the approach of Test-driven Semantics Specification (TDSS), which makes sure that the specification at hand at least describes the correct behavior for a suite of test models. However, in its current state TDSS does not provide any means to measure the quality of such a test suite.

In this paper, we describe how we have applied the idea of test coverage to TDSS. Similar to common approaches of defining test coverage criteria, we describe a data structure called invocation graph containing possible orders of applications of DMM rules. Then we define different coverage  criteria based on that data structure, taking the rule applications caused by the test suite’s models into account. Our implementation of the described approach gives the language engineer using DMM a means to reason about the quality of the language’s test suite, and also provides hints on how to improve that quality by adding dedicated test models to the test suite.