Dissertation

Title:

It’s about Time —
Model-Based Mutation Testing
for Synchronous and Asynchronous Timed Systems

Download:

Download thesis.

Abstract:

The amount of software in nowadays life is increasing rapidly. This affects various different areas, including the automotive industry. In 2009, the amount of code in a typical premium-class automobile was estimated to be hundred million lines of code. This code regulates everything from basic functionality to the most safety-critical parts. Many of these functionalities must comply to a strict real-time behaviour. Such systems, where the timing behaviour is as important as correct functional behaviour, are called real-time systems. Some examples from the automotive domain are the brakes and the airbag. A delayed reaction of either might end fatally, and must be prevented at any cost.

Testing has proven to be an effective method for detecting bugs and gaining confidence in a system. However, the manual creation of high quality test case is a tedious and error prone task. Consequently, automated test-case generation is an active and important research area. One of its main fields is model- based testing, where test cases are derived from a formal specification of the system. These tests are usually generated according to specified coverage criteria, as for instance transition coverage or state coverage, in case of graphical models. One special instance of model-based testing is model-based mutation testing: it is a fault-based approach that alters the correct specification according to predefined fault models and generates test cases that successfully detect these alterations.

The main goal of this thesis was to extend this model-based mutation testing technique to real-time systems. This proved to be a rather imprecise goal, given the vast amount of different types of real-time systems. Thus, the goal was refined to applying the technique to two instances of oppositional types of models, asynchronous and synchronous models.

As an instance of the first type, timed automata were chosen. They are among the most established and well-known models for timed systems, and attracted a high volume of research activities in the last decades. We developed a bounded model-checking algorithm for model-based mutation testing of timed automata, that was implemented via SMT-solving. The approach is restricted to deterministic models, which is a rather limiting restriction in practice. Thus we also developed a bounded determinization approach for timed automata. While it is well known that timed automata can not be determinized in general, this approach still can be applied to non-determinizable timed automata, due to the bounded setting. Furthermore, we investigated how the mutation-based approach can be used for the localization and repair of bugs in faulty systems-under-test.

We developed requirement interfaces as an instance of synchronous models. They are a contract based formalism we propose as a means for easily building traceability between natural language requirements and corresponding parts of the specification. The contracts are composed via conjunction. In a first step, we developed a test purpose driven test-case generation for untimed requirement interfaces. Then we integrated model-based mutation testing by generating test purposes leading to the introduced faults. Finally, we investigated how to model and test real-time constraints with requirement interfaces.

We implemented the techniques both for the asynchronous and the synchronous systems. The two implementations are called MoMuT::TA and MoMuT::Reqs. We evaluated these tools in several case studies. The most relevant ones were an airbag chip of Infineon and an adjustable speed limiter of Volvo.