As a leading global engineering validation and consulting company, Allion has already cultivated and built strong capabilities in the field of wireless device validation and testing. With the increasing penetration of the IEEE 802.11ac wireless standard across a wide range of electronic devices, increasing the overall availability and quality of compatible wireless access points (APs) is attracting more market attention.
Operating both as signal transmitting and relay stations, wireless APs play a central role when it comes to building out wireless local area network (WLAN) infrastructure. Wi-Fi usage scenarios such as the smart home, modern mobile office, in-vehicle infotainment, or wireless urban spaces all require a solid wireless infrastructure that delivers reliable and high-capacity throughput. The irony is that as wireless technology advances the level of service that consumers expect also rises accordingly.
In recent years, an increasing proportion of electronic devices such as smartphones, tablet PCs, digital cameras, and digital home entertainment systems include WLAN connectivity using Wi-Fi technology. Wi-Fi technology includes a range of WLAN standards including IEEE 802.11a, 802.11b, 802.11g, 802.11n, and 802.11ac. The newest WLAN standard, 802.11ac, features increased data throughput and improved performance. The introduction of 802.11ac has also rejuvenated the wireless access point market, since its new specifications are not supported by older equipment. Wireless AP devices are now necessary at home, in the office and in public spaces since most electronic devices require wireless network support. Even if the simplicity and usability add greatly to wireless AP usage, the essential problems exist with poor qualities of signal coverage and unstable performance as with any wireless AP user experience in the marketplace.
A Preliminary Competitive Analysis of Three Wireless APs
One particularly effective wireless test methodology that Allion has developed involves a competitive analysis of wireless AP devices. This methodology involves a series of wireless device validation scopes and test items. In this report, we present sample test results for a competitive analysis of three wireless APs currently available on the market. The specifications for the three sample APs are presented in Table 1.
As shown in Table 2, the seven validation scopes are based on wireless AP usage scenarios and key success factors. Each validation scope includes one or more test items that involve a functional or comparative analysis of one key component of wireless performance. Using this methodology, Allion analyzes the strengths, weaknesses, opportunities, and potential threats to wireless AP devices.
I. Conductive Throughput Test
One of the main issues for end users is wireless throughput. One way to measure wireless throughput is with conductive throughput testing. This test measures wireless throughput under ideal conditions and thereby delivers results that are more reflective of maximum possible throughput. Allion’s conductive throughput tests evaluate both raw throughput and signal stability. The first test method simulates different bandwidth channels to detect any instability in throughput over time. The second test method involves measuring path loss using signal attenuation as an analog for wireless range. Using these test methods (Throughput x Channels x Range), Allion can reliably determine the stability and throughput of any wireless AP.
In this case, conductive throughput testing was used to compare the performance of the three sample wireless APs (see Table 1). Figure 1 shows the results of testing with a 20 MHz channel bandwidth. Under this condition, AP3 has the best performance on channels 36 and 64, but on channel 128 its performance drops dramatically. AP2 also has consistently better performance than AP1 across all three channels for these tests.
Figure 2 shows the test results with the “Auto” channel bandwidth setting enabled. With this setting enabled, the wireless AP will search for the most suitable bandwidth to transfer the signal. Under this condition, AP1 once again has the worst performance on channels 36 and 64. Significantly, we found that the default channel bandwidth setting for AP1 is “Auto”. Since most users would not change this default setting, AP1 would likely have the worst throughput performance in actual use.
Due to the poor throughput performance of AP1, we focused especially on its test results. Figure 3 shows that AP1 throughput performance is best on channel 128 and worst on channel 36. In other words, end users should change the channel setting to 128 to experience the best throughput performance possible. However, if end users rely on default settings, the throughput performance will prove to be relatively unstable due to automatic channel switching. In this case, Allion would recommend that the manufacturer could improve AP1 throughput performance by modifying the “auto bandwidth transfer” setting in the device firmware.
II. Over The Air (OTA) Throughput Test
The Conductive Throughput Test is an excellent test, but it does have some limitations. For example, since the RF signal is sent directly from the test device to the test equipment, the wireless antennas are bypassed and therefore the test results are rather one-dimensional. By comparison, Over The Air (OTA) throughput tests are more accurate, since they measure the actual signal power sent and received from antennas in three dimensions. However, this method also has drawbacks, primarily its susceptibility to signal interference and reflections. Allion’s OTA throughput tests simulate multi-angle Line of Sight (LOS) to observe the signal distribution at different angles including 0°, 45°, 90°, 135°, 180°, 225°, 270° and 315°.
Figures 4 and 5 show typical OTA test results for the sample APs (see Table 1).
Figure 4 shows clearly that AP1 performs worse than AP2 and AP3 on channels 36 and 64. This result matches the Conductive Throughput test results presented in the previous section. As shown in Figure 5, when the channel bandwidth is set to “Auto”, AP1 performance is extremely poor on channels 36 and 64; and much worse in the 90° and 225° angles on channel 128. In this case, Allion would suggest that the antenna design and configuration issues need further work to improve performance, so that AP1 could be more stable in every direction and angle.
III. Association Capacity Test
The purpose of the Association Capacity Test is to ensure that a wireless AP can support its maximum client capacity and maintain throughput when multiple users connect simultaneously. In order to simulate real world conditions, we used a small office for our test setting. An Azimuth Systems test device was used to simulate a heavy wireless usage environment (most of the WLAN devices in the office are used via one wireless AP by hundreds of users at the same time, so we did not even try to over-raise the capacity limit), and to measure the performance of the sample APs.
The Association Capacity test results for the sample APs (see Table 3) show that AP1 and AP2 pass the test, but AP3 fails with only 36 users connected in 5GHz wireless mode. In other words, compared with AP1 and AP2, AP3 has the worst performance at this higher frequency.
IV. Overtime Stress Test
To test the performance of wireless APs under high traffic flow and multi-tasking conditions, the Overtime Stress Test is applied. With this test, Allion test engineers can observe data throughput and changes in excess capacity over a longer time period. This test evaluates the design and quality of wireless AP software and hardware.
As shown in Table 4 and Figure 6, the three sample APs exhibit similar overtime stress test performance. It is worth noting that although AP1 showed the worst performance on the conductive and OTA throughput results (see sections I and II), on this test, AP1 performed comparatively well. We speculate that its poor performance on those previous tests might be due to the AP1 antenna design, placement, or related component matching issues. The design of the AP1 base plate or related component matching issues might also have caused interference and thereby reduced throughput.
V. Extreme Temperature Test
To test the performance of wireless APs under extreme environmental conditions, the Extreme Temperature Test can expose test devices to both extreme cold (-50°C) and extreme heat (150°C) for up to 12 hours. Under these conditions, Allion engineers test device performance (data throughput) and design (structural integrity). Validating the safety and durability of the constituent components and the overall design is another important goal of this test.
The Extreme Temperature test results for the sample APs are shown in Table 5; while the temperatures and times shown in Table 5 are plotted in Figure 7. These test results show that the sample APs continued operating for the entire test period. Once again, AP1 showed the worse transmission performance (45 Mbps) compared with the test results for AP2 (86 Mbps) and AP3 (72 Mbps).
VI. Heat Map Test
The heat map test involves the collection and visualization of wireless AP location, maximum transmission speeds, signal strengths, and area coverage for a set period of time. Using this graphical method, our test engineers can more easily understand and compare the impact of changes in internal (wireless AP settings) and external (RF interference) conditions. For the sample AP heat maps (Figures 8 and 9 below), the color green represent areas of higher signal strength while the color yellow represents areas of lower signal strength.
In Figure 8 we can see that AP3 and AP2 had better signal quality than AP1. In fact, for the highest signal strength (bright green) AP1 only managed to cover roughly 25% of the area covered by its competition. In view of this poor performance, Allion proposed two possible solutions using our previous test experience. First, we proposed adding an absorber on the AP1 to reduce interference. Second, we proposed positioning the AP1 upside down to change the antenna orientation. As shown in Figure 9, implementing these two proposals significantly boosted the signal coverage. This suggests that the manufacturer of the AP1 should add shielding or an absorber to reduce noise interference. They should also redesign and calibrate the antenna position, to improve its transmission efficiency and coverage range.
VII. Streaming Media Performance Test
The Streaming Media Performance test simulates typical multimedia usage through video, audio and photo playback. The test is useful for investigating and resolving potential issues with multimedia quality that end users might experience, such as interrupted playback or signal interference. Table 6 shows that the sample APs can all play standard media streaming formats.
This test also includes thermal heat measurement of the printed circuit board and chipset components using an infrared imaging camera. The infrared images shown in Figure 10 were taken while the sample APs were playing streaming media. These images show that AP1 had the lowest core temperature (36.5 °C), AP2 was slightly hotter (40.6 °C), and that AP3 was much hotter (49.1 °C). In other words, AP1 has the best overheating protection mechanisms. However, taking into account all the test results, it is obvious that AP1 performs poorly in general. We can infer that the AP1 chipset is structurally sound, but the component configuration needs further RF validation and signal integrity testing to discover the root causes of its dysfunction.
Allion Labs – A Trusted Wireless Testing Service Provider
This report provides a simple preliminary competitive analysis of three generic wireless APs currently available on the market. Allion provides more detailed test data and advice to its clients. As the wireless industry continues to grow, we are committed to expanding our wireless testing services, to provide even more comprehensive and detailed validation testing for our clients around the world.
Remark: This article provides test results collected using our proprietary test methodologies and therefore further details will not be disclosed. If you are interested in hearing more about our wireless test services or you have any questions or concerns about this policy, please contact service@allion.com.