Allion Labs | Romonz Kao
Mobile communication technologies have been flourishing in recent years with 4G LTE being one of its most compelling innovations. This innovation has also given rise to products that support mobile network applications, such as portable routers and LTE routers. LTE routers enable users to share mobile connectivity over Wi-Fi or Ethernet. When LTE routers transmit mobile data via different interfaces, defects in product design or compatibility issues between chipsets and hardware devices can degrade throughput performance.
Think of the following scenario: when an LTE router has a good throughput conversion, people can enjoy smooth-streaming 4K movies. On the other hand, if the router has a low throughput conversion, it may cause latency or take a longer time to load. The unstable network speed annoys users, seeing as they are stuck on a loading screen. These problems not only frustrate customers but also result in a negative user experience.
To put it simply, 2 major problems will degrade LTE router performance:
- A low throughput conversion ratio
- A throughput performance that does not meet the specifications
This time we chose three popular LTE routers available on the market. For each, we designed conversion tests to verify the change in throughput performance in different LTE categories (Category 4 or Category 6) and transmission interfaces (Wi-Fi or Ethernet). This test review could end up being a reference for future product development.
Device Under Test (DUT)
The specifications of the three DUT are listed below:
| A | B | C | |
| Wi-Fi 2.4GHz | 802.11b/g/n 20/40MHz | 802.11b/g/n 20/40MHz | 802.11b/g/n 20/40MHz |
| Wi-Fi 5GHz | 802.11ac 20/40/80MHz | 802.11ac 20/40/80MHz | Not support |
| LTE Category | Category 4 | Category 4、6 | Category 4 |
Table 1: LTE Router Specifications
Test Items
LTE Category
- LTE Category 4
- LTE Category 6
Transmission Interface
- Wi-Fi Mode
- Wi-Fi 2.4GHz: 802.11b/g/n 20/40MHz
- Wi-Fi 5GHz: 802.11ac 20/40/80MHz
- Ethernet
Throughput Conversion Ratio Test Environment
Test Configuration
We run the test in an electromagnetic shielding environment. The radio communication tester is connected to the LTE router via an RF cable, and the test bed is connected to the LTE router via Wi-Fi.
Figure 1: Test Configuration
Test Equipment and Environment
Radio Communication Tester (R&S CMW500) provides a stable mobile network for the test. The maximum transmission values are set up as below:
- LTE Cat.4 : Downlink / Uplink: 127Mbps / 51Mbps
- LTE Cat.6 : Downlink / Uplink: 254Mbps / 51Mbps
The RSSI value is set to maintain the maximum Wi-Fi speed (-30dBm to -35dBm).
Testing Methods
The radio communication tester will send a signal to the tested LTE router. The LTE router will receive, convert and transmit that signal to the test bed. We define the following two values to evaluate the LTE performance:
- Throughput Conversion Value: The data throughput collected by the test bed (LTE routers receive the signal from the radio communication tester, and later transmit the signal to the test bed)
- Conversion Ratio = Throughput Conversion Value / Transmission Value of Radio Communication Tester
According to our rich testing experience, we suggest that the conversion ratio should be above 90%.
Test Results
For LTE Category 4, we verify whether the conversion performance of the three DUT in different interfaces meets the criterion—the throughput conversion ratio should be above 90%.
Based on the test results (Please refer to Figure 3 and Figure 5), for uploading, all three LTE routers meet the criterion. Plus, their throughput conversion values and ratios do not vary a great deal (The conversion values are around 49 Mbps).
However, for downloading (Please refer to Figure 2 and Figure 4), all three LTE routers do not meet the criterion because their conversion ratios stay below 90%. Moreover, for conversion values, Brand A has a lower one in either 20MHz or 40MHz. For example, in the case of 40MHz, Brand A’s conversion value is 97Mbps while two other values from Brand B and Brand C are around 107Mbps. Thus, compared with two other brands, Brand A performs poorly.
For 802.11 ac, we can only test Brand A and Brand B because Brand C does not support the 5GHz band.
Based on the test results (Please refer to Figure 7, Figure 9, and Figure 11), for uploading, Brand A and Brand B meet the criterion. Their conversion values and ratios do not vary a great deal (both conversion values are around 49 Mbps). However, for downloading (Please refer to Figure 6, Figure 8, and Figure 10), only Brand B meets the criterion at 80MHz, and Brand B’s conversion value is also higher than that of Brand A. In conclusion, Brand B performs better at 802.11ac 5GHz.
Based on the test results (Please refer to Figure 12 and Figure 13) for downloading, the conversion ratios of three LTE routers on Ethernet meet the criterion. However, for uploading, Brand C has a much lower conversion ratio (only 71%).
Test Results Summary of LTE Category 4
Low Throughput Conversion Ratio
- Wi-Fi Uploading: All three LTE routers’ conversion values are above 90%.
- Wi-Fi Downloading: Only Brand B meets the criterion at 802.11ac 80MHz. Brand A and Brand C have significantly lower conversion ratios (60% to 85%).
- LTE to Ethernet Uploading: Brand C’s throughput conversion ratio is only 71%, which is much lower than those of the two other brands.
- LTE to Ethernet Downloading: The conversion ratios of all brands meet the criterion.
Differences in LTE Throughput Performance
- Wi-Fi Uploading: All brands meet the criterion and their values do not vary a lot, except Brand B, whose value is obviously lower on Ethernet.
- Wi-Fi Downloading: Only Brand B meets the criterion at 802.11ac 80MHz.
- Conversion Ratios while Downloading: Although Brand C performs better at 802.11bgn 20MHz, Brand B’s overall performance is the best.
In addition, Brand B supports the LTE Category 6. Does Brand B perform well enough as it does in LTE Category 4? Let’s take a look at the following test results.
In Figure 15, we can see that Brand B’s uploading conversion values meet the criterion. However, for downloading, the conversion ratio in Category 6 is obviously lower (Please refer to Figure 14). Compared with the downloading condition of Category 4, the Category 6 conversion value is lower. Thus, Brand B does not achieve the throughput standard following Category 6 specifications which it claims to support.
In Figure 17, we can see that Brand B’s uploading conversion value is above 90%. However, for downloading in Ethernet Port (Please refer to Figure 16), Brand B performs as poorly as what we saw in the previous Wi-Fi section—Brand B in Category 6 does not meet the criterion as it does in Category 4. Even though the value of Ethernet (128 Mbps) seems to be better than that of Wi-Fi (116 Mbps), the value (128 Mbps) fails to meet the throughput standard in Category 6.
Test Results Summary of LTE Category 6
In short, throughput performance does not meet the specifications. Based on our abundant experience in testing services, we discover that the maximum download transmission value for LTE Category 6 should be 300 Mbps. However, Brand B’s download conversion values in Wi-Fi and Ethernet are only 116Mbps and 128Mbps respectively. Those values are obviously lower than the standard. Thus, in LTE Category 6, Brand B does not meet the throughput conversion standard.
Conclusion
Based on the above test results, we pointed out the following problems:
- Low throughput conversion ratio:
The conversion values of Wi-Fi downloading are generally low. Most Ethernet conversion values meet the criterion, but Brand C’s conversion ratio on Ethernet is significantly lower. - Throughput performance does not meet the specifications:
Among three LTE routers, Brand B performs the best. However, it is not up to par in Category 6.
- Differences in LTE Throughput Performance:
Overall, Brand A has the least ideal throughput conversion performance.
Take the Next Step
In this test review, we have discussed certain issues facing LTE routers, such as low throughput conversion ratio, low transmission speed, and the performance failing to follow product specifications, resulting in frustrated customers and a damaged brand image. To solve these issues that vendors face, Allion can help by customizing the best testing services and solutions to suit your needs. We provide a TR-398 standard test, following the first Wi-Fi in-premises performance testing standard in the world. This test can help ensure your router quality, build customer trust, and establish your brand reputation. If you’re interested in testing your LTE routers, don’t hesitate to contact us directly through email or a contact form.