Allion Labs / Raymond Chang
Mobile phones are indispensable devices for people nowadays, and the voice assistants developed by various manufacturers make voice recognition becomes an important function. However, in the user experience, the most frustrating and inconvenient parts for the users are listing below:
- The mobile phone has poor reception.
- The correctness of the sound/audio receiving of the smartphone’s microphone.
When these conditions occur, users will no longer trust the quality of the product as well as impact on the brands.
At Allion, we use listening room, environment simulation, and various voices to simulate the situation that users may encounter when riding motorbike. The tool that we use is Google Assistant, which will help us to find out possible problems of smartphone manufacturers in designing microphone sound.
How to Build up the Test Environment?
The environment we simulate is to put smartphone on the stand and fix it on the motorbike (figure 1).
- External volume set up to 70~75dbC.
- Use a professional speaker to simulate the normal people’s speaking volume, which is about 90dbC.
- Use the smartphone stand to keep the phone hanging.
Figure 1
The distance between DUT and the speaker (see figure 2).
Figure 2
Target
We set up 3 scenarios to validate the microphone’s sound:
- The sound/receiving performance of the phone in a quiet situation.
- The sound/receiving performance of the phone in the road.
- In the reasonable distance situation that normal people may use; we combine 1 and 2 to see the sound /receiving performance of the phone.
In the meantime, the use of Bluetooth® headsets is getting more popular; therefore, we also set up a scenario that users may encounter when using Bluetooth® earphone in the road.
Three scenarios in real environment
- Indoor environment.
- On the road (normal, 75dbC), usually comes with few car and engine sound only (figure 3).
Figure 3
- On the road (noisy, 85dbC), usually comes with lots of cars and engine sounds (figure 4).
Figure 4
Test Equipment
We use the current mainstream Bluetooth® headphone on the market, and use Google assistant and dummy heads to simulate real people speech.
Figure 5
Test Results
1. Indoor environment:
When in the indoor environment (about 65dbC), the performance of brand A shows normal, but brand B cannot correctly figure 2 sentences out (figure 6). On the other hand, despite in different distances, the phone of brand B cannot figure the speech in the awakened mode.
In terms of Bluetooth® headset test, all the headsets have good response and reply when the environment is indoor.
Figure 6
Figure 7
2. On the road (only few cars, 75 dbC):
In this scenario, the performance of brand A is slightly stable compared to brand B. Brand B has the bad performance significantly and even in the simulated maximum distance of 60 cm, there is still no reaction.
From the test result, we can know that the recognition rate starts to decrease and the frequency of incorrect response is gradually increasing.
Figure 8
Figure 9
3. Noisy road (about 85dbC):
The last test we simulate the noisy road which has a lot of cars and engine sounds. The test result shows that the recognition of both brands have poor performance however, the performance of the headset 2 is slightly better than headset 1.
Figure 10
Figure 11
From the above 3 scenario tests, we can conclude that although in the same environment, the test results can be vary due to the design of hardware.
In the design of phone’s microphone, brand A has two anti-noise microphones to dilute environmental sound, but brand B has only one. These differences in design are reflected in the test results as well as represent that users may encounter these situations.
In the headset’s test, we can conclude that headset 2 has better performance when the test environment is in a noisy environment. The reason is because headset 2’s design has four microphones for receiving sound, which has better anti-noise function than headset 1.
Come to Allion for the professional voice assistant test!
Voice assistant will be added more functions in smartphone in the future, which will lead the UX design to be more complex. Allion has built up professional listening room and has a lot of professional equipment for validate smartphone, headset, and earphone. Moreover, as TWS earpods getting popular, we also offer RF test to validate Bluetooth® performance in your product.