The System Integration Test (SIT) is a software test that is also a stage in the software development life cycle that usually comes after the unit test and the acceptance test. During the SIT, various components are tested simultaneously, confirming if they are functioning correctly.
Among every kind of software testing, the SIT is the most cumbersome and labor-intensive, not to mention it has the most chance for potential problems. Not only is there a considerable amount of test content in the SIT, but the delivery schedule of each item can also affect the testing schedule, potentially delaying the delivery times.
SIT Introduction
- The SIT includes the following content:
- SIT Integration Test: Confirms whether the components of the system can be integrated.
- Functional Test: Confirms whether the functions of the system are normal.
- Interface Test: Confirms whether the interfaces between the systems are connected and whether they are interacting correctly.
- Efficiency Test: Confirms whether the system response time, throughput, and other measurements meet normal expectations.
- Security Test: Confirms whether the system has sufficient security protection such as authentication, authorization, and more.
- Reliability Test: Confirms whether the system is reliable and stable enough by testing if it’s prone to crash or if it easily recovers.
- Usability Test: Tests the usability and user experience of the system, such as interface design and the operation mode.
The main purpose of an SIT is to ensure the functions; performance, security, stability, and usability of the system meet the requirements. Integrating multiple resources for testing also ensures the interactions between modules are correct. This can prevent the modules from being damaged during the development and maintenance process.
- Advantages of SIT:
- Finding problems fast: The SIT is usually carried out in the later stages of system development, detecting integration and interface issues between various subsystems. These problems can be found and solved early to avoid affecting the system’s operations.
- Improving system quality: The SIT allows you to test the system functions, performance, and security, which improves the quality and reliability of the entire system.
- Reducing risk: The SIT helps ensure stability and security in your system. Through testing, potential risks and problems can be found, reducing the possibility of system failure.
- Improving efficiency: The SIT helps find integration issues, allowing testers to solve problems more effectively.
The SIT is an integral part of the system development process because it can help ensure the quality and stability of the system.
- The Importance of the SIT to Market:
- Improving system quality: The SIT ensures the stability of the system after integration. If there are serious issues or failures in the system, it may have a serious impact on the company’s reputation and image.
- Improving user satisfaction: The stability and reliability of the system are the basis of any product. If there are serious issues or failures in the system, it will hurt the user experience. With the SIT, the quality of the system can be improved, which then improves user satisfaction.
- Reducing maintenance costs: If there are serious issues or failures in the system, more time and resources would be spent on maintenance and repairs. The SIT can ensure the stability of the system after integration, reducing maintenance costs.
- Improving market competitiveness: Competition in the market is becoming more fierce, so the quality and performance of products are very important for consumers. With the SIT, the quality and performance can be improved, enhancing the market competitiveness of the product.
The SIT can be very important in the market. It improves system quality, user satisfaction, and market competitiveness, and reduces maintenance costs at the same time.
Data Analysis and Case Studies
- Issue Analysis:
According to our data analysis of past experiences, most issues during the manufacturing and development stage can be divided into the categories shown in the table above. The categories BIOS, Platform (CPLD/CPU/HDD_SSD_SD/Input/Memor/ODD/PCIe_SSD/Power), Workstation Software, and Operating System accounted for 72.8% of the total issues.
- Case Studies:
Issue 1 – Background
In the SIT, there is a WHQL test. This checks whether the system’s PnP is functioning properly. During the test, the function is disabled and then enabled. The configuration of the WHQL test is to fully connect all of the cards that support the PCIe interface of the server, for example, inserting all PCIe add-in cards including graphics cards, network cards, Zoom cards, serial cards, and Teradici cards.
During testing, we found that if the capacity of the DIMM is greater than 1 TB, it causes problems that result in system failures. For example, the toolbar on the desktop would disappear (Figure 1), the software or app would disappear after rebooting, error messages would pop up (Figure 2), combined keyboard shortcuts would fail (Figure 3), folders would fail to open, and more. After verifying and discussing with various parties, chip manufacturers determined that the system failures were due to the overload of PCIe.
Issue 1 – Purpose and Benefits
The WHQL test is a driver test within the SIT, mainly used for Windows hardware devices and drivers. It’s designed to ensure compatibility between the Windows operating system, hardware devices, and drivers. Passing this test can improve the stability of the driver and prevent issues that affect the system.
In this case, we successfully helped the customer determine the problems in the PCIe chip and drive on the server, and we also assisted them in passing the WHQL test, making sure that the driver complied with Microsoft’s certification standards.
Issue 2 – Background
In the SIT, when installing the driver of the VGA graphics card, BSOD will occur from time to time if the graphics card is not connected to the monitor (Figure 4). After verifying and discussing with various parties, we found an issue with the VGA driver. After updating the VGA driver to the latest version, the problem was solved.
Issue 2 – Purpose and Benefits
In the SIT, drivers are a very important element, because drivers are directly related to interactions between hardware and software. If there’s an issue with the driver, it could lead to problems such as poor system stability, loss of function, and low performance. In some cases, the system may crash or fail to start. So in the SIT, it’s necessary to conduct a comprehensive test on the drivers to ensure that there aren’t any problems with stability and compatibility.
In this case, the issue we found caused a system crash. So, we conducted a comprehensive test on the driver, which eventually improved the overall stability and reliability of the entire system.
Issue 3 – Background
During the power cycling test, if the S4 test is finished, the average testing time falls within 5-6 minutes. We found that when a specific version of the BIOS is loaded, the server equipped with Win10 takes over 10 minutes to run the S4 test (3/10 SUT fail rate). After verifying and discussing with various parties, the issue was resolved by upgrading the VROC driver.
Issue 3 – Purpose and Benefits
During the SIT, the purpose of the power cycling test is to test the stability of the system after multiple restarts. It can simulate real situations and make sure the system can run stably after long operation times or multiple reboots.
In this case, we recommended for each version update pass the power cycling test. Through repeated startups, potential problems can be found regarding stability and quality.
Allion has the most complete test environment in the industry, including various power equipment and air conditioners dedicated to server testing, not to mention an access control system with 24-hour control with security mechanisms. We can formulate a complete SIT plan according to our client’s needs, providing the test scope, scenarios, cases, and a large selection of test methods and tools. Our experts with professional knowledge and rich experience can provide proposals tailored to our client’s needs, effectively analyzing and solving issues during the test process.
Allion’s SIT service can effectively reduce your investments in manpower and equipment. If you have further needs or inquiries, please contact us at service@allion.com.