MULTI-FACTOR AUTHENTICATION EVALUATION GUIDE

Does your organisation need multi-factor authentication? Yes. But what next? How do you choose the right solution for your specific situation? There is no one-size-fits-all answer!

This guide will outline the different aspects of multi-factor security, which need to be taken into consideration, and help you evaluate them in the context of your requirements.

So what should be considered?

Typically, organizations need to consider their threat model (and the security impact), privacy requirements, as well as availability, feature-set and cost constraints.

Additionally, you may want to consider the direction in which the overall security landscape is moving. Is your solution future-proof, or will you have to change it in a year?

For simplicity’s sake, we have highlighted 3 factors, which – according to Gartner1 – are the leading product selection criteria among the leading IT companies.

Afterwards, we have prepared a technical appendix, in which to explore the remaining requirements in more detail.

Trust

You need to trust your security products, but where does trust come from? The technical strength of a discrete authentication transaction is important, but trust is about much more than that.

Trust is also about control. What data does the solution require, and where will it be stored? On your premises, or in the cloud? Is the code available for audit, or is it a black box? Nowadays data is currency, so evaluate carefully what data your solutions require from you. Where they will put the data, how they will store it, and what they wish to do with it. This is the solution’s data model.

Also consider vendor reliability and support. Will your vendor provide technical support? Consider the terms, SLA, and also use cases. Will your vendor support your use cases directly? Or will you have to assemble a solution from components provided by many different vendors?

To trust a solution, you need to trust its security model, its data model and its support model.

Trust the Security Model

The security model of a solution is not a decisive factor on its own, but it is certainly important. The technological underpinnings of a solution can severely limit its real- world applications, impact and scalability. Crucially, the technology should never rely on human behaviour – e.g. keeping secrets safe manually, or remembering and never divulging them. Expect human operators to be the weakest link in the chain, and never expect anybody to act against self interest.

The technological foundations

A fundamental choice each solution has to make, is what technology to base itself on. Typically the options are:

  • Home-grown cryptography. It may try to account for future constraints (e.g. quantum computers) at the expense of current ones, or in the worst case – it may be reinventing the wheel. Consider if you want your production systems to be used as a research experiment.
  • Various shared-secret schemes. The technology is inherently limited, and it relies on human operators to keep secrets safe
  • Public-key cryptography (PKI). While not perfect, only solutions based on PKI are provably immune to brute-force attacks. They are also the least-reliant on human operators. PKI has a long track record of being used in the real world – including the banking sector, e-commerce and military.

Consider who is in Control (pt. 1)

Being in control means knowing where your solution is hosted, what other services it communicates, and finally – knowing what goes on under the hood. Above all, it means having certainty about these things, and being able to verify everything yourself.

On-premise or Saas

Consider your infrastructure. Does your organisation have on-premise tools, hardware or applications, which need to be tightly integrated with MFA? Are your users stored in Active Directory? Do you want to add MFA to your VPN concentrator, which is accessing your RADIUS servers?

  • Verify if your internal resources can be integrated with the chosen MFA solution.
  • Check if your corporate security policies are compatible with any external requirements.
  • Consider which country hosts your data centers. Are they legally allowed to? (see the next point).

Auditable communications

What other services is your MFA solution communicating with? Do you know for a fact, where it sends data and why?

  • Consider if your solution has clearly documented network communications.
  • Will you be able to notice deviations from the official network communications model?

Consider who is in Control (pt. 2)

What goes on under the hood?

Trust is earned. If necessary – can you find out for certain, what is going on under the hood? What is your MFA solution doing with its cryptographic keys? How it is verifying transactions? Can you be absolutely certain it has no backdoors?

  • Prioritize solutions with open source components.
  • Prioritize audited solutions.

Transaction verification

Can you independently verify transactions? Or does your MFA solution simply respond with “OK” or “NOT OK”?

  • Can you access the technical response, and verify it independently?

Don’t Forget About Regulation

In the European Union, when it comes to regulation, companies need to consider the requirements of Revised Payment Services Directive (PSD2) and General Data Protection Regulation (GDPR). PSD2 stipulates strong authentication for FinTech companies, whereas GDPR limits what and where personal data can be stored. Do you trust your MFA solution to be compliant?

Where will the data be stored?

Does your solution have a SaaS business model? If so, consider what data it expects from you, and in which countries it will host this information. Does it comply with GDPR requirements?

  • Consider where a SaaS solution is hosted.

Do you need to hand off personal data?

Does your solution store user information, including personally identifiable information? Can it work without this information? Can you easily delete this information, if GDPR mandates it?

  • The less data an MFA solution needs, the better.
  • Can you easily bulk-delete user data?

Fintech customers

Are you a FinTech company in the EU? You may need to be compatible with the strong authentication requirements of PSD2.

  • Will your MFA solution be used by your customers?
  • If so – does it qualify as a strong authentication provider under PSD2?

Resiliency and Availability

An essential part of trust, is having the product work when you need it. Can you count on your chosen MFA solution to be highly available?

  • Does the solution have high technical availability?
  • Does the solution remain usable under heavy load?
  • SaaS – what is the promised SLA? What is the historical SLA?
  • On-premise – are there tools and documentation for maintaining a highly available setup?
  • On-premise – is the highly-available configuration supported natively? Does it depend on third-party products? (see section about hidden costs)

Total Cost of Ownership

This is what every business boils down to. Everybody needs to keep costs low, and the trick to success is accurately measuring the true costs of various day-to-day issues.

When it comes to software products, there is only one visible cost – the advertised price (the license fee for on-premise products, or the per-user fee for SaaS products). However, it does not come close to telling the whole story.

Capital expenditure can include user and administrator training costs, as well as hidden up-front costs for hardware, or dependent software licenses.

You also need to consider the operational expenditure – day to day administrative costs, implied cost differences from changes in employee productivity, support costs, feature upgrade and technical maintenance costs.

Evaluating a product’s TCO is about paying attention to the fine print, anticipating your future requirements and accurately estimating changes in employee productivity.

Beware Product Dependencies

It can be tempting for product vendors to hide fees in the fine print, while advertising a lower price. You can avoid falling for it if you know what to look out for. Hidden fees are typically hidden under hardware costs, support costs, and software dependencies. Consider if the product can be used on its own, as-is, or does it require certified auxiliary products for certain features?

Dependencies on other products

The ideal product can cover your requirements on its own. Double check the fine print for dependencies on other software – either first-party or third-party.

  • Does the product need a specific, licensed operating system to run?
  • Consider external requirements – databases, API servers, service providers. Does the evaluated product need any of this? Is any of this required specifically to help the product provide its features?
  • Does the product provide its own high availability features? Or do you need additional software for service discovery, clustering functionality, health monitors etc.?
  • What about additional hardware for load balancing, and assigning floating IP addresses?

Dependency Maintenance

Maintenance fees

  • Would you have use for the additional products on their own? If not, consider them a part of the product being evaluated.
  • Will the dependencies cost you time & money to maintain?
  • Consider the time it takes to train people to use and maintain additional components in your IT infrastructure.

Human Resource Costs

Every product has indirect costs due to training (both day-to-day usage, and maintenance), and support. A product that causes many support requests, is indirectly more expensive. Conversely, if a product causes employees to spend less time requesting support, and improves their overall productivity, it offers better value for money.

Day-to-day use and maintenance

Consider support response times. How long will employees have to wait, in case of an outage?

  • Is it possible and realistic to solve problems in-house, in case support is taking too long? Does the vendor provide tools/documentation for this?
  • SaaS – consider availability history (is this data available? If not – why?)
  • On-premise – how often will you need expected manual attention? Can these tasks be automated?

Time to Deployment can be Costly

Not every solution can be set-up easily. How quickly can you start testing the solution in a real-life scenario? How quickly can you scale the test to 10 people? To the whole organization?

Technical Set-up time

Consider how easily the solution can be “turned on”, including all the minutiae. This includes signing up for an account, configuring client security, configuring firewalls, finding software that adds the required security, or finding software-development kits and writing the required functionality.

Consider if there are pre-made software components for your specific use-case (e.g. VPN, Windows access etc.).

  • Consider if you can start using the product in 1 day? If not – are there compelling reasons?

Short excerpt from real life

Be Careful with SMS

SMS is a historically popular one-time-password delivery channel. However, be aware of its two major shortcomings:

  • It is unnecessarily costly to send SMS OTP for every transaction. Alternative delivery methods can be more efficient and cost less.
  • It is a very insecure delivery channel. SMS messages can be easily redirected. They can be easily intercepted. They can be easily redirected to other phones.

Don’t Believe?

The US National Institute of Standards and Technology’s (NIST) has been recommending against using SMS for multi-factor authentication solutions since July 2016.

Prior to that – in 2012 – Australian Telecommunications Alliance declared SMS insecure and inappropriate for high-value transactions.

https://www.itnews.com.au/news/comms-alliance-says-sms-unsafe-for-bank-transactions-322586

There are numerous horror stories about SMS-based security gone horribly wrong. Just google yourself.

Be careful with SMS. Consider if you have an alternative, and complement SMS with additional layers of authentication.

Opportunity Costs

Product feature sets open new opportunities. Other opportunities are closed, however. For example – by implementing a company-wide security policy based on smartcard, you get all the associated benefits, but may prevent your company from utilizing a lower-cost or more efficient product where applicable. Always consider what your chosen solution implies for the rest of your tech stack – now and in the future.

Consider the future

  • Does the solution align with the latest trends? Will it be relevant in the near future? Will other technological products in the near future be compatible with it?
  • Consider if the chosen product precludes the use of some other technology.
  • Consider how easy it is to migrate away from a chosen product, if the need or an opportunity arises.
  • Consider your business – are you heading in a specific technological direction? Will it be compatible with your chosen MFA solution?

User Experience

According to Gartner – user-experience in many cases is the deciding factor. It is not enough to be trustworthy and cost-effective, software products still need to be enjoyable to use. This makes sense – software products can easily be “secure enough”, and their price can be “good enough”. User satisfaction, however, can always be improved.

User experience is important for every software product, but doubly so for security products. Heightened security typically makes a product more cumbersome to use than products from other categories. Products that provide authentication are typically used often-enough for a bad user-experience to translate into real annoyance.

If users start disliking a product, they may try to avoid it or circumvent it, undermining technology, which “on paper” is highly secure. Alternatively, users may disengage or put off doing certain tasks. This can undermine the product’s cost-effectiveness through lower customer engagement. For example, consider how likely users are to use online banking daily when it is protected with a hardware token (cumbersome experience) versus a smartphone’s fingerprint reader (better experience).

Clearly, user experience matters.

Hardware Experience

If users avoid security products, then security is compromised. A good product should get out of the way, not require too much focus, and generally perform as many tasks as it can, without asking for user input. Otherwise, they may avoid, or circumvent the product, thus compromising security. The accessory type plays a huge role in this.

  • Consider the hardware involved. Is the MFA product based on the smartphone-as- a-token, or on USB keys? Or traditional hardware PIN pads? Does the MFA solution support different types of hardware natively?
  • Consider your threat model – what is the easiest user experience you can afford?
  • If your users need to approve transactions often, a hardware PIN pad may put them off.
  • If your users need to perform infrequent, but sensitive operations, maybe a hardware PIN pad is appropriate.
  • When considering frequency of use, consider what happens to lost devices. Can they be revoked? Will the user notice, if the device is missing?

Software Experience

Software also has a huge impact on the user experience. A smartphone accessory, just like other types of hardware, can offer wildly different user experiences. Compare SMS-based one-time passwords to QR code scanning, to push-based approval. Even push-based transactions can differ – as some can be interacted with, and some require the application to be loaded first.

  • Does the solution support an individual PIN/password? Is it required? Is it optional? Can it be mandatory, and delegated on-demand (i.e. to the smartphone’s OS-level PIN)?
  • Does the solution require any type of manual code input?
  • Does the solution support offline use? If so – can it offer visual (e.g. QR code based) transaction signing?
  • Does the solution offer backups? Can I continue using the product if I lose access to a hardware accessory?
  • Is the solution fast?

Administrator Experience

Administrators are a different type of user, but nevertheless, they are users and their experience matters. If the administrator experience is bad, then the solution might be incorrectly or partially rolled out and not maintained correctly. These factors can compromise security a lot.

  • Is the solution well documented?
  • Are common maintenance tasks automated?
  • Does the solution require the use of specific applications, to be administered? E.g. Internet Explorer?
  • Consider if the solution pays attention to details. E.g. smaller things like autocomplete, responsive user interface etc.

Privacy is Part of the Experience 

Are there privacy-oriented features?

Consider if the MFA solution supports advanced features, which could improve the privacy of its users. For example, does it support a geofence? If so, you could white-list approved locations for access, and eliminate the scenario, where your employees are accessing your applications from an insecure, public WiFi (e.g. the airport).

Alternatively, consider where the user information is stored. Is it on your servers? That makes you liable for keeping this data secure. You already have a lot on your plate, you don’t need to add more to it. Is the information stored with the user? Then consider if you are satisfied with how securely this information is stored, and if it matches your security requirements.

  • Consider if the product has advanced features that contribute to user privacy.
  • Can the product’s design and feature-set reduce your company’s liability?


What Next?

We hope that the mentioned points will help kickstart an internal discussion, and – hopefully – to a rational, and beneficial choice.

Security is important, so you need to consider your options carefully. However, don’t forget that any security is better than no security. If you can not decide on a solution to commit to long-term, try some free/cheap options to get a feel for it. In the short term, it will help you understand the capabilities, and constraints better, as well as provide security while you choose.

For more information, consult industry leading analysts, such as Gartner. Check out all the different vendors, their feature set and price lists, as well as consider additional points, which may be relevant to your situation: do you need an offline mode? What is your threat model? How reliable/highly available the solution needs to be?

Consider your needs, and weigh the pros/cons against them. It is easy to wish for a product that scores 100% in every category, but in reality – every product is a different combination of various tradeoffs.

Keep that in mind, and good luck!