Allion Labs

Due to the emergence of video streaming services during recent years, Parks Associates indicated in its report that more than half of the households possessing smart TVs may use such TVs as the main devices to play streaming media. A person does not merely “watch” TV, because the TVs produced in modern times are usually equipped with new functions, including social media, games, Internet, video on demand, TV APPs, etc., all of which can be operated on TVs. Thus, TV has become an indispensable part of digital family recreation.

Compared to the past, TV manufacturers now need to take various new functionalities, user habits, and multiple scenarios into consideration when developing a new product. However, along with the prolonged use time of the end-users, complex wired/wireless environments, and complicated and frequent usage habits and approaches, some problems that have never appeared before gradually came to the stage.

The Dilemma of the TV Manufacturer

Based on Allion Labs’ many years of experience in TV testing, about 5~10% of serious problems in a general smart TV usually occur probabilistically after repetitive operations or long-term usage. Therefore, such problems cannot be easily detected in general testing. In fact, the potential risks are higher than those stated in the statistics!

 

Fig.: Data analysis of 10 projects of different scales selected randomly

 

Although the occurrence of malfunctioning is just a probability, if the user’s usage frequency is added with the sales volume on the market, it is expected that the number of actual malfunctioning cases is far greater than the probability! Nowadays, most of the consumers would search relevant information and feedback on the Internet before purchasing products. If these probability issues cannot be effectively reduced before product launch, there might be great pressure on customer complaints and returns of goods. Based on the foregoing reasons, even though the probability of a series problem is just a few digits, it shouldn’t be ignored!

Nevertheless, if long-term or repetitive testing of probability issue is conducted manually, human errors may occur due to insufficient concentration, while the lengthened testing duration is required because of prolonged time and fatigued human labor. A company may be stuck in a dilemma when facing the foregoing difficulties.

In the followings, Allion Labs is going to explain how to effectively assist a customer in finding out the potentially serious problems, to increase product competitiveness by satisfying both conditions – “time cost reduction” and “going beyond the ordinary”.

 “Automation Kit” and “Customized Testing Script” are indispensable: complementary and supplementary testing

 

To solve the pain points for our customers, Allion Labs has integrated the testing concepts of hardware testing, software testing, compatibility, and VFT (virtual field trip) , and has developed the ACSTS (Allion Critical Scenario Test Solution), in order to compensate for any shortcoming that cannot be achieved through manual testing.

First, through the ACSTS Automation kit developed by Allion Labs, we can execute a series of pressure-related tests. In addition to extending the detection time in order to increase the time utilization rate, we can also reduce human errors, increase detection accuracy, and overcome various limitations of manual testing.

Second, based on the testing and market survey experiences that we obtained over the years, we analyze users’ scenarios and key operations and conditions in order to customize the testing script. As we include the Log results, the report will be automatically generated in order to effectively increase the problem detection rate!

With the abovementioned advantages, Allion Labs’ key scenario solution is indeed far better than ordinary manual testing in terms of time utilization rate, detection accuracy rate, or problem detection rate.

 

Item	General manual testing	Allion Labs’ solution
Time utilization rate	Rest is needed after a period of manual operation.	Only one-time programming is needed and can be used repetitively for a long time
Detection accuracy rate	There may be misjudgment risks when the concentration is reduced due to a long time of manual operation.	With automated testing, there will be no difference after a long time.
Problem detection rate	Aimless testing scopes are prone to be distanced from the real users’ scenarios – twice the work, half the effect.	Through the analyses, the test subjects that can be close to users’ scenarios can be set – half the work, twice the effect.

 

What are the serious problems hidden in smart TVs?

• Case background: The customer’s TV had probabilistic series issues from time to time. In the past, it was examined manually by repetitive or long-time detections and verifications; however, in addition to the excessive manpower demand and time cost, it did not contribute to the desired effect, unable to achieve a comprehensive examination of product problems. Thus, when developing new products, this customer can come to Allion Labs for getting assistance in detections, hoping to effectively reduce the probabilistic problems and enhance product quality.

• Understand the customer’s pain points and draft a validation strategy:
• ・Understand the product’s current status
• ・Collect and analyze the problems from users’ feedback
• Based on the results of the analyses, formulate the testing purposes for the object’s target functions and problem types
• Based on the past testing experiences, key operations and conditions that are prone to failure are also included
• ・Based on the foregoing benchmarks, a testing script is designed
• Execute the testing through the ACSTS automation kit
• ・Obtain the key log data
• Make sure that the operations and conditions are consistent in the process of validation

• After the problems are detected:

After performing the testing, the following problems are detected and compiled:

1. After playing several streaming videos, the TV crashes after switching to the digital TV signal (4K). The probability is 2/100.
2. When playing Netflix (Dolby Atmos) and switching the audio output to AVR, an error message appears, and the TV fails to play the video. The probability is 3/100.
3. In HDMI (4K 120Hz), when the TV is restarted due to a power outage, it will not detect the HDMI signal. The probability is 2/100.
4. After switching to a different APP to play the video, the remote control fails to work. The probability is 1/100.
5. When being connected to a Bluetooth speaker and after playing videos from Netflix and YouTube, the wireless network is disconnected. The probability is 3/100.
6. When being connected to next-generation game consoles and switched to game mode, the TV screen blacks out. The probability is 4/100.

Through the foregoing real test results, it is obvious that some problems may be able to reappear in certain scenarios by means of certain testing approaches. Thus, if we just follow the general testing procedures, we cannot really detect them.

What can we do with the bugs that come one after another? Allion Labs’ key scenario testing helps you increase the detection efficiency

 

From the following three aspects, you should not underestimate the benefits

• How much manpower time can be saved?

Allion Labs’ solution can save more than 90% of the time compared to manual operation, which can significantly reduce the overall testing timeframe as well as potential human error.

• How many potential risks can be detected?

 

Take the problem of “After playing several streaming videos, the TV crashes after switching to the digital TV signal (4K). The probability is 2/100” as an example. The action of switching the APP to digital TV for watching the video may be executed at least 3~5 times a day. If we calculated based on the sales volume of 10,000 devices:

2% of probability * 3 times/day *10,000 devices = 600 times/day, which means that the problem may occur 600 times every day in the market. The potential customer complaint risks should not be ignored indeed!

 

• Not only the results but also the problem log?!

In addition to finding out the root cause so that we can fix the problem, Allion Labs will also provide the log report so that manufacturers can archive and conduct subsequent follow-ups based on the log data. Such data can be used as the reference for future product designs and developments, thereby  reducing the chance of occurrence of the same problem as well as repair costs.

The effective reduction of serious probabilistic problems is definitely an indispensable part for manufacturers to improve product competitiveness. In the long run, it is crucial to maintain brand value! If you are interested in smart family key scenario testing solutions, please refer to Smart Family Key Scenario Testing or fill out the inquiry form. We will get in touch with you.