Allion Labs
How do you manage and keep track of all your accounts and passwords in the digital age? Most users decide on their own what passwords to set. However, as the Internet continuously drives traditional services towards the cloud, there are more and more accounts to manage and countless login passwords to memorize. This increases the difficulty of account management and makes sensitive personal data more susceptible to leakage. Data leakage has become increasingly common as the Internet evolves. Website vulnerabilities or viruses may expose account passwords at any time.
A survey taken by Google last year found that 70% of the population in Taiwan had experienced personal data leakage, second only to Vietnam (78%) among the Asian countries surveyed; 50% of people’s passwords were considered overly simple; 86% use the same set of passwords for multiple websites. Due to the increasing need for stronger account security, Two-factor Authentication (2FA) has emerged as an extremely important tool in the digital world.
Two-factor Authentication, as the name suggests, requires two verifications to log into an account, thus reducing the likelihood of cyber-attack and account hacking single-authentication logins may induce. Common 2FA methods include text messages and email links. In today’s article, we will cover an authentication method that requires triple factors but is not yet prevalent in Taiwan–the Two-factor Physical Security Key Authentication. Two-factor Physical Security Key Authentication has been around in America, Europe, and Japan for quite some time now, and is a popular information security product.
The varied physical keys from Yubico
Most users use SMS or Email authentication link as their second verification factor. However, many users may not have constant internet connection or mobile signals and would prefer personalized hardware-based authentication. Additionally, during the pandemic, many western businesses have allowed their employees to work remotely. Logging into enterprise services in various home environments without proper security deployment increased the chances of information leakage. Therefore, physical security services have been preferred by large enterprises as something to provide to their employees.
A physical security key is very easy to use. After plugging it into the PC, the key’s driver will start the installation process automatically. Then, the user goes to the two-factor verification page and enables various settings. Finally, a master key is registered and named. The entire process only takes about three minutes or less.
The 2FA key setup page of Facebook
Later, when the user needs to log in to the service, he or she only has to enter the login password during the first authentication stage. Then a page will pop up, asking the user to insert the physical key. After doing so, the user simply taps on the sensor button of the key and completes the verification.
After entering the first password, a browser pop-up will ask the user for the physical key that he or she previously registered. If the physical key has not been registered, verification will fail.
Because each physical key is unique, once the key is broken or lost, it may be impossible for the user to log in to the service again. As a result, users are advised to prepare multiple keys for registration or set up a backup account rescue method with a password or SMS to prevent having to face this scenario.
It is hard to introduce physical security keys without mentioning the information security regulations they are based on – the Fast Identity Online (FIDO). FIDO refers to a set of network identification standards developed by the FIDO Alliance, a non-profit group with the goal of ensuring server security and terminal device protocols during the login process. This set of identification standards uses a public key encryption framework for multi-factor authentication (MFA) and biometric login to protect cloud data.
Readers can refer to other websites for detailed information about the history of FIDO. Here, we will only introduce the three most important authentication protocols of FIDO.
FIDO UAF (Universal Authentication Framework)
A multi-factor authentication protocol that primarily uses biometrics for password-less logins. Common examples include fingerprint recognition, voice recognition, and more.
FIDO U2F (Universal Second Factor)
This refers to two-factor authentication. The physical security key we introduced in this article is based on this protocol. This authentication method achieves password-less logins by utilizing encrypted physical keys.
FIDO2
The latest FIDO protocol jointly specified by W3C and FIDO integrates both Web Authentication (WebAuthn) and Client-to-Authenticator Protocols (CTAP). This protocol defines multi-factor authentication and its support for various browsers and platforms.
FIDO U2F and UAF certification marks. L1 is the security level supported by the product.
While two-factor authentication physical key is popular in Europe and the United States, customer dissatisfaction is still sometimes reported. We collected feedback for a popular physical security key product sold on Amazon that focuses on fingerprint identification with support for U2F and UAF, and concluded the negative reviews as below:
| Types of customer complaints | No. of cases | Category | Proportion |
| Windows 10 driver cannot be installed or the system cannot recognize the device after driver installation | 43 | Windows OS compatibility issue | 53% |
| No Windows 11 support | 10 | ||
| Operation failure after updating to Windows 10 / Version compatibility issues | 5 | ||
| Inability to work on laptops running Windows 7 | 4 | ||
| Unrecognizable by HP notebook, crashing issues, and driver errors | 2 | ||
| Unable to operate on Dell PCs | 2 | ||
| Unable to be used with Surface | 1 | ||
| Unable to operate on Acer laptops | 1 | ||
| Poor durability | 16 | Poor HW durability | 12% |
| Low fingerprint recognition success rate | 14 | Poor detection function | 11% |
| Difficult installation and insufficient instructions | 8 | Poor user experience | 6% |
| Doesn’t support U2F as advertised, doesn’t work with MS or Google accounts, or has poor web support | 8 | False advertisement | 6% |
| Unstable use after installation, and occasional login failures after restarting | 6 | Poor stability | 12% |
| Crashing during operation, code:10 | 5 | ||
| Unexpected crashes | 2 | ||
| The key is altered with the transfer of PC. For example, after setting up the key using a Google account, Google cannot recognize the key on other work computers. | 1 | ||
| Instability and failure to work when used via a hub | 1 |
We observed that the product seemed to have quite a number of compatibility issues with Windows OS, which is the most popular operating system on the market. We cannot, of course, rule out the possibility that these compatibility issues were caused by the user’s own software or hardware setup, but such a high negative feedback rate indicates that there is indeed room for improvement in compatibility.
To find out the real reasons behind these issues, Allion purchased the seven best-selling U2F physical keys from Amazon and conducted a compatibility test to verify if there are really compatibility problems. The products we selected are listed below:
| Brand | Model | Price | Interface |
| Yubico | Yubico – YubiKey 5C | $70 | Lighting, USBC NFC |
| Thetis | Thetis FIDO U2F Security Key | $29 | USBA, BT |
| GoTrust | GoTrust Idem Key – A. | $22 | USBA, NFC |
| Yubico | Yubico – YubiKey NFC | $25 | USBA, NFC |
| Yubico | YubiKey 5 NFC – USB-A | $45 | USBA NFC |
| Feitian | Feitian ePass NFC FIDO U2F Security key | $25 | USBA, NFC |
| Hypersecu | HyperFIDO Titanium PRO FIDO2 Security Key | $19 | USBA |
All of the physical keys above are advertised to support the U2F protocol and are best sellers on Amazon US.
Based on past experience in compatibility testing and the products’ characteristics, we designed the following test combination.
Laptop
| OS | Version |
| Windows | Windows10 20H2 19042 |
| macOS | mac OS Monterey 12.3.1 |
| Chrome OS | 100.0.4896.133 |
Browser
Chrome 101.0.4951.67
There are many software and hardware combinations that can be used to test compatibility. You may consider the following combinations to perform a complete scan.
OS & Platform
OS & platform are the most important factors in this type of product compatibility testing. We must take into account all generations of Windows, macOS, Chrome OS, Android/iOS systems for mobile phones and tablets, and even Linux OS in order to fully design the test.
Browser
Including the four most used browsers: Microsoft Edge, Google Chrome, Apple Safari, and Firefox Mozilla.
Web service
Web services that are commonly used with U2F must be tested. These include Google account, Microsoft account, Facebook, Twitter, Dropbox, and GitHub, to name just a few.
Connection/Authentication interface
There are currently four types of security key interfaces: USB-A, USB-C, Bluetooth®, and NFC.
The purposes of testing a physical key are relatively simple. With regard to compatibility verification, the design is as follows:
| Test Summary | Check Point |
| Initial Set-up | The system should recognize the key and automatically install it. The device admin should be able to see the device category. |
| NFC/BT logon | |
| Windows Hello Check | Add and remove the physical security key |
| Check whether there is a device conflict that causes the Hello setting to fail | |
| Log out and log in test
S3/S4 Screen saver log out and log in test |
|
| Web APP test | Should be able to add keys to and remove keys from different web services |
| Log in and log out test | |
| Convert Test | Hub connection test to verify key functionality |
Through the above combination and test design, we conducted a simple compatibility test on seven commercially available multi-factor authentication physical keys, and observed the following issues:
| Model | Issue |
| Thetis | 1. First connection to Win10 was successful and could be installed automatically. However, when another set of keys was inserted at the same time and removed, Thetis could not be recognized by the system, not even after re-plugging. |
| 2. After waking up from sleep in Windows, when inserting the key for a second-stage authentication, the key cannot be recognized. The phenomenon disappears after Windows restart. | |
| 3. In the Win10 environment, after setting up two-factor authentication in Google account and manually logging out, only the password is required to re-login. This problem does not occur on Win7. | |
| Idem Key | In the Win10 environment, after setting up two-factor authentication in Google account and manually logging out, only the password is required to re-login. This problem does not occur on Win7. |
| Yubico USBA | After setting two-factor authentication on Facebook, logging out, logging in, and holding down the key, a window pops up asking for a pin code that has not been set before. Login to Facebook thus fails. |
| Yubico 5Ci | 1. When connecting to the test website of demo.yubico.com, even if the key has been registered and two-factor authentication has been activated, the key may still occasionally fail to be recognized. The chance is about one out of five. |
| 2. When connecting to the test site of demo.yubico.com, cannot use NFC to unlock by touching the upper part of the iPhone | |
| FeiTian | When connecting to the Google account on a MacBook to set up the physical key, the Mac displays “Your FT device cannot be recognized, it must be recognized before it can be used.” |
| GoTrust | When connected to a Chromebook for testing, two-factor authentication can work properly, but there is occasional detection failure when using an A-to-C adapter. The problem is resolved after re-plugging in. |
We conclude our test findings as below:
| 1. The keys generally work on Windows/Chromebook/MacBook, but occasionally there will be problems with device detection errors on Windows or Yellow Bang issues, but on the actual network account, the key can function properly. |
| 2. The registered key sometimes cannot be recognized by some web services, which may result from incorrect operations. Therefore, during physical key verification setup, it is better to have a backup verification method to avoid loss of account. |
| 3. The two-factor authentication on a MacBook only sends a 6-digit text message to the user’s mobile phone, with no other authentication methods in place. It may be due to the fact that built-in touch ID already provides a layer of protection. Still, in terms of diversity, it is not as secure as Windows or Chrome, which allows for the setup of physical keys. |
| 4. The Chromebook/MacBook models tested this time were all USB Type-C models, while most of the tested objects used USB-A. We therefore utilized A-to-C adapters. To our surprise, there were not as many detection errors and failures. However, it is still highly recommended that the test combination of adapter/hub/docking station be used, as it is a common application scenario for end-users. |
Information security products are associated with sensitive user information and intellectual properties. As a result, they have extremely high thresholds for design and development, and stringent requirements in testing. Compatibility also must not be overlooked, as users may sometimes need to switch to a different platform or log into a different browser.
Take the Next Step
Allion has years of experience and expertise in testing, as well as more than 20,000 test devices covering most brands and models on the market. Our test environment is top-notch with diverse compatibility test equipment. In addition to compatibility tests, we also service functional and durability tests that are tailored to our customers’ unique product specifications and requirements. Please do not hesitate to contact us for a bespoke compatibility test.
For more information on our services, please contact us at service@allion.com.