Testing non-functional requirements, a contradiction?

In practice, discussions often arise about what constitutes functional and non-functional requirements for a system.

Could a suitable structuring of requirements by relevant topics make this discussion obsolete?
How can proof be provided that all requirements are met? 

In this blog post, we describe a possible approach. 

Requirements and Their Verification 

Traditionally, requirements have been divided into functional requirements and the rest (non-functional requirements). This classification persists despite newer standards such as [ISO/IEC 25010:2011] "System and software quality models" no longer using the term "non-functional."

Under time pressure, this traditional classification often leads to implementing only the "function" while postponing the "rest" for later.

The effect could be that the system/product does not achieve the desired success in the market. 

Figure 1: Traditional classification of requirements into two main categories

For a feasibility study, fulfilling only the minimal functional requirements may be sufficient (keyword MVP: "Minimum Viable Product").

However, to ensure that a system/product fulfills its purpose sustainably, additional requirements must be formulated, implemented, and verified. 
 
We recommend dividing requirements into multiple topic areas while simultaneously assigning appropriate verification methods.

A proposed classification of topics is shown in Figure 2, while additional information can be found in the following standards: 

• [ISO/IEC/IEEE 29148:2018] «Systems and software engineering —Life cycle processes — Requirements engineering»
• [ISO/IEC 25010:2011] "System and software quality models"
• [Capturing Architectural Requirements, FURPS+ :2005] 
   
  
  

In addition to traditional "testing," other verification methods exist.

The standard [ISO/IEC/IEEE 29148:2018] describes four standard verification methods to provide objective proof that requirements have been met: 

• Inspection 
• Analysis or simulation
• Demonstration
• Testing (measurement)
  
  

The advantage of such classification and direct assignment of verification methods is that requirements are systematically recorded per topic area, reducing the risk of missing elements.

Additionally, tests and proofs become easier to plan and execute since the correct methods are already defined. 

Figure 2: Requirements by topic and assignment of verification methods

Note: Human and Organisational Factors (HOF) include, among other things, requirements for HMI design, such as haptics, etc.
A good overview of HOF topics is provided by [ERA-HOF] and [RailHOF-Design]. 

Implementation Tips

• Group requirements into multiple topics and define the verification method when writing the requirement.
• Avoid using the term "non-functional requirement."
• Consider whether verification is feasible when writing the requirements. 

Conclusion

Are "non-functional requirements" testable?

Yes, but two aspects must be considered:

First, requirements should be structured according to specific topics such as "performance," "interface," "physical properties," etc., rather than simply being categorized as "functional" or "non-functional."

Second, the various verification possibilities for each category should be analyzed, compared, and the most suitable one defined.

With such structuring and a systematic approach, even seemingly non-testable requirements can be verified, ensuring a product with the desired quality. 
 
CSA Engineering has extensive expertise in requirements engineering and test engineering and can select and apply the appropriate methods for a project.

By aligning requirements with verification methods from the outset, a solid foundation for the project is established. 
 
We are happy to support you with your project. Do not hesitate to contact us. 

Get in touch now >

Sources and References 

• [ISO/IEC/IEEE 29148:2018] «Systems and software engineering —Life cycle processes — Requirements engineering» 
• [ISO/IEC 25010:2011] "Systems and software engineering – Systems and software Quality Requirements and Evaluation (SQuaRE) - System and software quality models"
• [ERA-HOF] Human and Organisational Factors (HOF) | European Union Agency for Railways, accessed on 10.03.2025
• [RailHOF-Design] HOF in Design Archives - RailHOF, accessed on 10.03.2025
• [Capturing Architectural Requirements, FURPS+], IBM, Peter Eeles, 2005, accessed from web.archive.com on 13.03.2025
• From Idea to Product: Requirements Engineering Defines the Direction - CSA.ch, Blog Post by Daniel Niggli, 20 Aug. 2024