Allion Labs
Since the TR-398 standard was released, several updates and corrections have been made, the latest being TR-398 Issue 2 Corrigendum 1. In this article, we will compare six wireless network APs on the market based on TR-398 Issue 2 Corrigendum 1. Among the six wireless network APs we analyzed, three models offer 802.11ax (Wi-Fi 6) support—namely the Telstra DNA0332TLS1, the Eero Mesh Wifi Modem M110315, and the Netcomm Modem NF20MESH-01. For the 802.11ax comparative analysis, only these three models will be compared.
Table 1: Specifications of 6 wireless network APs
6.1.1 – Receiver Sensitivity Test
This test checks each wireless AP antenna’s receiver sensitivity.
Figure 1: 6.1.1 – 802.11n and 802.11ac Test Results
In the 802.11n and 802.11ac tests, four out of the six models passed the test. The Kogan and Vodafone APs did not meet the test requirements, because, with these two models, the Modulation Coding Scheme (MCS) of the wireless device connected to the AP is set to zero, resulting in the APs failing the Ping test.
Figure 2: 6.1.1 – 802.11ax Test Results
On the 802.11ax test side, all three wireless APs supporting 802.11ax were certified to meet the test requirements, with scores 5~10% higher than average. The results suggest that in real-world use cases, decent transmission efficiency can be maintained even if the wireless AP is far away. For our reader’s reference, the following are the requirements specified in TR398.
6.2.1 – Maximum Connection Test
This test verifies if an AP can stay connected and maintain packet transmission under a high connection volume.
Figure 3: 6.2.1 – 802.11n & 802.11ac Test Results
From the 802.11n and 802.11ac test results, we observed that Telstra’s wireless AP had the best overall performance. Both the Eero and the Kogan models failed due to high packet loss rates, which could translate into poor audio and video streaming quality for the end user.
Figure 4: 6.2.1 – 802.11ax Test Results
In the 802.11ax test, only the Netcomm wireless AP passed. At 5GHz, packet loss rates for both Eero and Telstra APs exceeded the standard value, and Telstra’s transmission capacity is also significantly affected during downloads. While the packet loss rate of the Eero AP was higher than specified, it was only by 1.7%, which is negligible from an end-user’s perspective. Following are the TR398 specifications and the scores of the APs that failed the test.
6.2.2 – Maximum Throughput Test
This test checks the transmission capability of each wireless AP.
Figure 5: 6.2.2 Test Results
In this test, only the two wireless APs from Telstra and Netcomm met the standard. Eero had the best transmission performance at 2.4GHz, and Telstra had the best performance at 5GHz. Although not all wireless APs met the test standards, data suggests that all APs’ transmission capabilities were adequate to satisfy the general user’s needs for YouTube and Netflix streaming as well as for mobile games. Below are the associated TR398 specifications.
6.2.3 – Airtime Fairness Test
This test checks the wireless AP’s capabilities to allocate fair airtime.
Figure 6: 6.2.3 – 802.11n/ 802.11ac Test Results
In this test, the blue vertical line shows the transmission performance standard when both devices are at the same close range. The red vertical line represents the transmission performance standard when one device is moved to a longer distance. The green vertical line represents the transmission standard when two devices are within close range, but one of them is set to an older connection mode. In terms of packet loss ratio, it is required to be less than 0.0001.
For the 802.11n and 802.11ac tests, only Netcomm’s wireless AP met the standards. Some wireless APs’ transmission capabilities were affected at different distances, while other wireless APs had packet loss ratios that exceeded the specification.
Figure 7: 6.2.3 – 802.11ax Test Results
With regard to the 802.11ax test, none of the three wireless APs were able to pass the test standard. In terms of transmission performance, Netcomm performed the best at 5GHz. In terms of packet loss ratio, both Netcomm and Telstra models outperformed the Eero AP.
6.2.4 – Dual-band Throughput Test
The test item evaluates the AP’s transmission capabilities when simultaneously transmitting at 2.4GHz and 5GHz.
Figure 8: 6.2.4 – 802.11n/ 802.11ac Test Results
In the 802.11n and 802.11ac tests, none of the wireless APs met the test requirements. Telstra performed the best regardless of whether the distance was set to long-, mid-, or close-range. Eero, Kogan, and Google wireless APs were the worst performers at respectively close-range, mid-range, and long-range distances.
Figure 9: 6.2.4 – 802.11n and 802.11ac Test Results
Likewise in the 802.11ax test, none of the three wireless APs were able to meet the test standards. Telstra was still the best performer. Eero had a weaker performance at close- and mid-ranges, while Netcomm had the worst performance at a long-range distance. Below are the associated TR398 specifications.
6.2.5 – Bidirectional Throughput Test
This test is to check that the packet loss ratio will not exceed the specified value when the wireless AP performs uplink and downlink transmissions simultaneously.
For this test, the packet loss ratio is required to be below 0.0001. Among the models tested, only Telstra’s wireless AP conformed to the standard. Although the packet loss ratios of most wireless APs did not meet the test standard, the results were still considerably low and had a near-neglectable impact on the end-user experience.
This article compared wireless APs’ receiver sensitivity and throughput (bandwidth) performance. In our follow-up articles, we will conduct a comparative analysis of other test items.
The Allion Lab has years of product testing experience. Our expertise includes not only customized testing services in line with real-life use scenarios but also in-depth market research and competition analysis. Our goal is to help manufacturers improve the quality of their products so they can foster better user experiences and create master product positioning plans—thereby standing out from the competition in highly competitive markets.