The DOSS Component Tester – Comprehensive security testing of IoT devices

By June 27, 2024July 2nd, 2024Insights

By Sascha Hackel, Martin Schneider, Ramon Barakat and Luca Jungnickel, Fraunhofer FOKUS

Introduction and motivation

IoT devices surround us every day, whether they are integrated into cars, medical devices, smart home applications, or critical infrastructure. These devices are an integral part of our daily lives, and the need for secure and robust devices is becoming more important. Security testing of these devices is still challenging and remains an open research topic due to the deep integration of hard- and software, resource constraints and detection capabilities for vulnerabilities.

Given the complexity and the dynamic nature of IoT systems, traditional security testing methods often fall short. In response to these challenges, we approach the development of Interactive Application Security Testing (IAST) that combines the strengths of both Static Application Security Testing (SAST) and Dynamic Application Security Testing (DAST) [1]. This method leverages the code analysis capabilities of SAST to detect potential vulnerabilities without executing the code. These findings then inform the subsequent application of DAST, which tests the devices in a dynamic, running state. It is here that the IAST approach offers a compelling solution. By integrating these approaches, IAST enables the analysis and testing of the dynamic behavior of the system, making it particularly adept at handling the multifaceted security challenges presented by IoT devices [2][3]. This comprehensive approach not only enhances the detection of vulnerabilities but also streamlines the remediation process, thus significantly bolstering the security posture of these critical devices.

Component Tester module approach


Building upon the foundation laid by IAST, the Component Tester (CT) emerges as an essential tool in maintaining the security of IoT devices. It implements testing and validation methodologies, procedures, tools, and an integrated system which assures that all components of an IoT operation will be secure and reliable. Before any components are incorporated into the subsequent (system) architecture, they are subjected to thorough security testing and validation. This process involves evaluating devices based on their Device Security Passport (DSP) and scrutinizing third-party software through binary code validation techniques. Additionally, to examine open-source applications, a software security verification platform is designed and developed to specifically predict and detect vulnerabilities effectively. For proprietary developments, integration into a DevSecOps environment will be achieved, featuring predefined quality gates.

Independent of the subject of the validation, third-party code, proprietary code, binary code, the validation process begins with SAST, which scans the IoT device’s codebase to identify potential vulnerabilities without executing the code. These findings serve as input for the subsequent DAST process. Detailed information on this comprehensive SAST approach is given in another blog post [4].

Utilizing the vulnerabilities identified by SAST, the DAST approach is implemented. This involves dynamically testing the IoT devices. DAST not only attempts to verify these vulnerabilities but also generates specific test cases that can trigger them. Following the testing, DAST provides a verdict on whether the identified vulnerabilities are false positives or genuine security concerns.

Extended IAST approach within the CT

IAST can be used in different variants as described in [1X]. In the CT, the IAST is mainly used to verify the findings of vulnerability prediction (see also [5]) by means of dynamic test execution. For this purpose, specific test cases are generated that are intended to execute the predicted vulnerability [6]. The interaction of static and dynamic analysis can take place one after the other or in parallel. As shown in Figure 1, the information is exchanged either after completion of an analysis (Figure 1 (a) or between Figure 1 (b)). The findings from the dynamic analysis can be returned to the static analysis and a new iteration with further analyses can be started.

­­Figure 1 – Difference of sequential and parallel iterative IAST

The test cases generated by DAST that successfully trigger vulnerabilities are invaluable. They confirm the presence of vulnerabilities and are essential in the vulnerability handling process as required by the Cyber Resilience Act. To further improve this process, we employ more advanced fuzzing techniques, such as directed fuzzing. Unlike traditional fuzzing, directed fuzzing targets specific areas in the code suspected of harboring vulnerabilities, thus increasing both the efficiency and effectiveness of the testing process.

For the remaining potential vulnerabilities that are not detected, the probability that they are false positives is indicated. This is done using statistical estimation methods to allow the manufacturer to estimate the residual risk that a suspected vulnerability exists.

Patch validation

To enhance the process of vulnerability identification and subsequent patching, directed fuzzing techniques are employed. They support the patch development by validating if a given security patch completely fixes a vulnerability. Since patches sometimes overfit to a given attack, further variations of the original attack are still feasible even after deploying this patch. By generating a test suite that triggers the vulnerability in different ways during patch development. These test cases can then be using during patch development and before its deployment to ensure that the patch is effective. Directed fuzzing techniques further allow to identify if new vulnerabilities have been introduced through a security patch.

Bare-metal firmware fuzzing

One established method, which proved to be effective in discovering security vulnerabilities, is fuzzing. This dynamically tests random-like inputs and checks if the System Under Test (SUT) behaves unexpectedly, which could be a security vulnerability. An effective way to fuzz an embedded device is to emulate its firmware completely. This speeds up the fuzzing process and no actual hardware is needed for testing – only the firmware binary is required [7]. This also has the advantage that no source code is needed, and closed-source, as well as closed third-party libraries integrated into the device can be tested. Fuzzing firmware binaries still comes with many open research questions – one is the problem of so-called Sanitization [8]. To fuzz a firmware binary more effectively, the binary should be sanitized, which makes errors that are hard to detect observable even if they would have only internal effects. An example is when an input overwrites another variable which it shouldn’t be allowed to. Since this normally does not lead to a crash of the emulated device, current firmware fuzzers cannot detect this behavior. We are currently working on a solution to this problem that contributes to this field and improves current firmware fuzzing becomes more precise in terms of vulnerability detection.

The Component Tester module integrates all these aspects into a unified system, delivering a holistic solution for testing and validating IoT components. This comprehensive approach ensures that only secure and reliable components are integrated into the IoT service architecture.

[1] Barakat, R., Blanckenburg, J.v., Kraus, R., Jezuita, F., Lüdtke, S., Schneider, M.A. (2024). Interactive Application Security Testing with Hybrid Fuzzing and Statistical Estimators. In: Sadovykh, A., Truscan, D., Mallouli, W., Cavalli, A.R., Seceleanu, C., Bagnato, A. (eds) CyberSecurity in a DevOps Environment . Springer, Cham.
[2] Ammar, M., Russello, G., & Crispo, B. (2018). Internet of Things: A survey on the security of IoT frameworks. Journal of Information Security and Applications, 38, 8-27.
[6] Barakat, R., Weiß, P., Blanckenburg, J.v., Kraus, R., Schneider, M.A. ” Enhancing Software Security Analysis: Targeted Test Case Generation through Constraint Solving in Interactive Application Security Testing,” 2024 IEEE 24th International Conference on Software Quality, Reliability and Security Companion (QRS-C), Cambridge, United Kingdom (to be published)
[7] Tobias Scharnowski, Nils Bars, Moritz Schloegel, Eric Gustafson, Marius Muench, Giovanni Vigna, Christopher Kruegel, Thorsten Holz and Ali Abbasi. Fuzzware: Using Precise MMIO Modeling for Effective Firmware Fuzzing. In 31st USENIX Security Symposium, 2022.
[8] Joschua Schilling, Andreas Wendler, Philipp Görz, Nils Bars, Moritz Schloegel, and Thorsten Holz. Binary-level Thread Sanitizer or Why Sanitizing on the Binary Level is Hard. In 33rd USENIX Security Symposium, 2024.

Leave a Reply