• About Us
    • PRODUCTS

  • Blog
  • Contact
Blog Security What Is mTLS and How Does it Work? 

DATE:

June 14 2023

AUTHOR:

Table of Contents

What Is mTLS and How Does it Work?

mTLS, or mutual Transport Layer Security, is an extension of the TLS protocol that authenticates both the client and server in a communication session. It provides an additional layer of security by verifying the identities of both parties, preventing man-in-the-middle attacks and other types of security threats. mTLS operates by using digital certificates to authenticate both the client and server, which are issued by a trusted Certificate Authority.

What Is Mutual TLS (mTLS)?

By default, the TLS protocol only proves the identity of the server to the client using public key infrastructure X.509 certificates, and the authentication of the client to the server is left to the application layer. TLS also offers client-to-server authentication using client-side X.509 authentication. As it requires provisioning of the certificates to the clients and involves a less user-friendly experience, it’s rarely used in end-user applications.

Mutual TLS authentication (mTLS) is more often used in business-to-business (B2B) applications, where a limited number of programmatic and homogeneous clients are connecting to specific web services or cloud services, the operational burden is limited, and security requirements are usually much higher as compared to consumer environments.

mTLS is also used in microservices-based applications based on runtimes such as Dapr, via systems like SPIFFE. In mTLS, each microservice in a service mesh verifies the other’s certificate and uses the public keys to create encryption keys unique to each conversation.

What Is TLS?

TLS, or Transport Layer Security, is a cryptographic protocol that provides secure communication over networks. It ensures that data transmitted between clients and servers cannot be intercepted or modified by attackers. TLS is commonly used in web browsing, email, instant messaging, and other types of online communication.

Public Key and Private Key

The established symmetric key never leaves the client or server and, accordingly, cannot be intercepted by an attacker. For each new communication session, a new, so-called “temporary” symmetric key is created. Even in the worst case, for example, if the attacker knows the private key of the server, it is impossible to decrypt the previously recorded ones.

Public key encryption is used only in the TLS Handshake procedure during initial connection setup. After the tunnel is set up, symmetric cryptography comes into play, and communication within the current session is encrypted precisely with the established symmetric keys. This is necessary to increase performance, since public key cryptography requires much more computing power.

TLS Certificate

In TLS, chains of trust are based on certificates of authenticity provided by special authorities called CAs. Certification authorities perform checks and, if the issued certificates are compromised, they are revoked.

A chain of trust is formed from the issued certificates. Its root is the so-called Root CA certificate – a certificate signed by a large center, the trust in which is undeniable. In general, the chain of trust looks something like this:

TLS Handshake

The main steps in the procedure for creating a secure communication session:

  1. Since TLS works over TCP – Transmission Control Protocol, a TCP connection is first established between the client and the server.
  2. After TCP is established, the client sends the specification in plain text to the server (namely the protocol version it wants to use, supported encryption methods, etc).
  3. The server approves the version of the protocol used, selects the encryption method from the provided list, attaches its certificate and sends a response to the client (if desired, the server can also request a client certificate).
  4. The protocol version and encryption method are currently considered approved, the client checks the sent certificate and initiates either RSA or Diffie-Hellman key exchange, depending on the set parameters.
  5. The server processes the message sent by the client, checks the MAC, and sends the final (‘Finished’) message to the client in encrypted form.
  6. The client decrypts the received message, checks the MAC, and if everything is fine, the encrypted TLS connection is considered established and the application data exchange begins.

How Does mTLS Work?

In mutual TLS connection, not only the server certificate is verified, but also the client certificate. In the process of establishing a connection, the parties exchange certificates and public keys that are in them. Furthermore, each party can verify the integrity of the certificate – after all, it was transmitted over an open network – by verifying the signature on the public key of the CA that issued these certificates. In the ideal scenario, there is a chain of CAs all the way down to the root CA.

After that, the parties make sure that the opposite party has a private key that matches its certificate. To do this, the parties exchange messages encrypted with public keys from their respective certificates. The correct decoding confirms the identities of each of the parties. After agreeing on the mTLS connection parameters and generating session keys, an encrypted TLS session is established.

Thus, mTLS between services solves two problems at once: the problem of mutual authentication and the problem of protecting transmitted data from eavesdropping and forgery.

Is mTLS a New Protocol?

mTLS, or mutual Transport Layer Security, is not a new protocol. It is a variation of the widely used Transport Layer Security (TLS) protocol. The primary difference between TLS and mTLS is that while TLS only authenticates the server to the client, mTLS authenticates both the server and the client to each other.

The use of mutual TLS has become increasingly important in recent years, especially as the number of connected devices and the volume of data transmitted over the internet has grown. As organizations rely more and more on networked systems and internet-connected devices, the need for secure communication between these systems and devices has become paramount.

mTLS has emerged as a key technology in this regard, providing an additional layer of security that ensures mutual trust and authentication between clients and servers. It has proven effective in preventing various types of attacks, including credential stuffing, on-path attacks, spoofing attacks, and phishing attacks.

Types of Certificates Involved in mTLS

The first type of certificate involved in mTLS is the root certificate, which serves as the foundation of the entire certificate chain. The root certificate is typically issued by a trusted Certificate Authority (CA) and is used to validate other certificates in the chain.

Next are the intermediate certificates, which link the root certificate to the server or client certificates. Intermediate certificates are issued by the CA and are used to create a chain of trust between the root certificate and the server or client certificate.

Finally, there are the server and client certificates, which are used to authenticate the identities of the server and the client. Server certificates are issued to web servers and are used to verify the identity of the server to the client. Client certificates, on the other hand, are issued to clients and are used to verify the identity of the client to the server.

Certificate Authorities in mTLS

Unlike traditional TLS, certificate authorities in mTLS are usually maintained by organizations themselves and the responsibility for issuing certificates rests solely on the company’s shoulders. All servers and clients receive certificates that are linked by a chain of trust to the root certificate. The root certificate is the most critical component of the entire certificate chain, as its compromise could lead to the compromise of all other certificates in the chain. As such, it is essential that the root certificate be protected from any potential threats.

With proper security measures in place, mTLS can provide a high level of security for communication between clients and servers, protecting against a wide range of threats and ensuring mutual trust and authentication.

Why Use mTLS?

mTLS adds an extra layer of security to the organization’s network or applications login process, preventing the known vulnerabilities of the other password-based authentication methods. Additionally, it provides mutual authentication for devices like IoT that do not follow the login process.

mTLS is effective in preventing various types of attacks, from phishing to brute force attacks.

On-path attacks, where an attacker intercepts or modifies communications between a client and server, are almost impossible to carry out with mTLS because on-path attackers cannot authenticate to either party.

Spoofing attacks, where an attacker attempts to impersonate a web server or user, are much more difficult when both sides are required to authenticate with TLS certificates.

When it comes to credential stuffing attacks, where attackers use leaked credentials to try to log in as legitimate users, mTLS also proves extremely effective. Without a legitimately issued TLS certificate, this type of attack cannot succeed.

mTLS effectively prevents brute force attacks, which use rapid trial and error to guess a password, as a password alone is not enough to gain access to an organization’s network.

Phishing attacks, where attackers try to steal user credentials, are also thwarted by mTLS. Even if a user falls for a phishing attack, the attacker would still need a TLS certificate and corresponding private key to use the stolen credentials.

Finally, mTLS is an effective security measure for API requests, ensuring that only legitimate authenticated users can send requests and stopping attackers from exploiting vulnerabilities or subverting the intended functionality of the API.

Does mTLS Have Any Drawbacks?

While mTLS (mutual Transport Layer Security) offers numerous advantages in terms of security and trust in communication between clients and servers, it is not without its drawbacks. In this brief, we will explore some of the potential drawbacks of mTLS.

Identity Verification

One potential drawback of mTLS is the complexity of identity verification. With both the server and client requiring certificates to authenticate their identities, managing and verifying these certificates can be challenging. Furthermore, the use of multiple certificates can create potential points of failure in the authentication process, which can compromise the security of the communication.

Security of Private Certificates

Another potential drawback of mTLS is the security of private certificates. The responsibility for issuing and managing certificates falls entirely on the organization, and any compromise of the private certificate can have serious consequences. Organizations must ensure that proper measures are in place to protect the private certificate, including secure storage and access controls.

Step-Up Authentication in TLS 1.3

The use of step-up authentication in TLS 1.3 can be a drawback for mTLS in some scenarios. Step-up authentication requires additional credentials beyond the initial authentication, such as a smart card or biometric verification. While this provides additional security, it can also add complexity to the authentication process and make it difficult to implement in some environments.

Conclusion

mTLS mutual Transport Layer Security is an important security protocol that provides an additional layer of trust and security in communication between clients and servers. By requiring both the client and server to authenticate their identities with TLS certificates, mTLS helps prevent various kinds of attacks and ensures secure communication. Helenix provides advisory assistance for the safe and secure implementation of security systems in the IT systems of organizations. You can learn more about our competencies in the Custom Development section.

FAQ

mTLS is not used in all sites on the internet because it requires a higher level of technical expertise and infrastructure than standard TLS. It also introduces additional latency and complexity to the communication process, which can be a trade-off for users and organizations.

mTLS helps prevent various types of attacks, including on-path attacks, spoofing attacks, credential stuffing, brute force attacks, phishing attacks, and malicious API requests. By requiring mutual authentication of both the client and server, mTLS provides secure communication and prevents unauthorized access.

TLS and SSL secure communication between a client and server, but only authenticate the server. mTLS authenticates both the client and server, providing an extra layer of security for sensitive transactions and IoT devices.

mTLS is recommended for any communication that requires an extra layer of security, such as sensitive transactions or IoT devices. It is also recommended in API security to prevent malicious requests.

OAuth 2.0 is an authorization framework for delegating access to resources, while mTLS is a security protocol for secure communication between clients and servers. OAuth 2.0 is often used in conjunction with mTLS to provide a more comprehensive security solution.