How do you actually build a generic test system?
In a nutshell, the task was: “Build a generic test system”.
As simple as the task may seem, it gives rise to many different questions.
We know!
The background
In a nutshell, the task was: “Build a generic test system”.
As simple as the task may seem, it gives rise to many different questions.
What does the client want to achieve with the generic test system? Does everyone understand the terms SUT (System under Test), EUT (Equipment under Test), DUT (Device under Test), MUT (Module under Test) and test object in the same way? Is a module a hardware component that I can install in a system and use to run system tests? Or is it a software element for unit tests? With the unit tests, should I also demonstrate the test coverage with regard to requirements or source code? Is the code coverage per line enough or must every branch also be covered with tests? Should I now build a test system that supports all possible tests (unit, module, system, function, type, routine, production, integration, debug, developer) using “hardware in the loop” or even simulated elements?
What are the environmental conditions? Do we test at room temperature in our laboratory or do we need an accredited testing laboratory? Should the device be tested in a climatic chamber or even in the field in all seasons? Should a HALT (Highly Accelerated Life Test) with different temperature cycles, vibrations or even EMC influence be carried out? How many devices are available for the tests and can we look for the load limit? Or is it a case of stress, load and performance tests that ought to be documented in order to identify weak points? Does the developer want a test system to debug his software on the target controller more efficiently?
Figure: generic test architecture
Solution approach
A structured approach is important in order to find the right answer in this maze of questions. We agreed to formulate the test system requirements in the first instance, using the infrastructure already in place. The test system should only be used in the laboratory and expanded gradually. The main users are developers who want to automate their development tests.
We use the test architecture shown on the left for discussions about other requirements and concepts.
The tests are carried out as black box tests with test points TP1 and TP2. Future upgrades should also enable white box tests with test points TP3 and TP4 . The TP5 environmental conditions do not need to be considered at this stage.
Summary
There are many ways to define, set up and conduct a test. First of all, therefore, the following questions must be asked and answered:
- Which concepts do I want to use as a team/company and how?
- What do I want to achieve with the test?
- What is my test architecture like?
- How do I make sure that the test is transparent? Does it actually have to be transparent, and if so, for whom?