The importance of cybersecurity cannot be overstated. The SSL 2.0 handshake protocol plays a great role in ensuring data remains secure and does not reach the wrong hands. But it does have some weaknesses, and hence the emergence of SSL 3.0.
While current internet standards implement TLS 1.3 (Transport Layer Security), we’ve made significant strides in securing communication over the internet. The journey began with the development of SSL (Secure Sockets Layer) 1.0, which was never publicly released due to serious flaws. This was followed by SSL 2.0, which introduced a handshake protocol before establishing a connection between the client and server. Every protocol has its own vulnerabilities, and to this day, no single protocol provides a completely secure solution. However, in cybersecurity, we continuously strive to develop ‘better’ solutions, recognising that while these may not be perfect, they are the best options available at present.
The SSL 2.0 handshake protocol aims to send ‘ClientHello’ from the client and ‘ServerHello’ from the server over an insecure channel, typically over a Transmission Control Protocol (TCP) connection. The purpose of these messages is to establish the parameters for the secure session that will follow, including negotiating the encryption methods and exchanging the necessary cryptographic data. This includes a ‘cipher suite’ which is a list of encryption algorithms such as RC4-MD5, DES-CBC-MD5, and RC2-CBC-MD5 sent to the server to choose the most secure algorithm that both machines support, as shown in Figure 1.
The server responds with a ‘ServerHello’ message including the selected cipher suite, another random value (nonce), and the server’s certificate containing its public key. After these initial messages are exchanged, the process of key exchange and session establishment begins, and eventually, the communication is encrypted using the session key derived from the handshake process. This encryption secures subsequent data exchanges.
Key exchange methods
The generation of the PreMasterSecret (PMS) depends on the key exchange method being used during the SSL/TLS handshake. Figure 1 shows two of the typical methods.
RSA key exchange
- Step 1: Client generates the PreMasterSecret: The client randomly generates a 48-byte value known as the PreMasterSecret
- Step 2: Encryption with server’s public key: The client encrypts this PreMasterSecret using the server’s public key, which the client obtained from the server’s digital certificate sent during the ServerHello.
- Step 3: Client sends the encrypted PreMasterSecret: The encrypted PreMasterSecret is sent to the server as part of the ClientKeyExchange message.
- Step 4: Server decrypts the PreMasterSecret: The server decrypts the PreMasterSecret using its private key.
Diffie-Hellman (DH) key exchange
- Step 1: Client and server exchange DH parameters: Both the client and the server exchange Diffie-Hellman parameters (g, p, and public values g^a mod p and g^b mod p).
- Step 2: Client and server independently compute the PreMasterSecret: Both the client and server compute the PreMasterSecret independently using their own private key and the other party’s public key. For example:
- Client computes: PMS = (g^b mod p)^a mod p
- Server computes: PMS = (g^a mod p)^b mod p
Both computations result in the same shared PreMasterSecret, but it’s never transmitted over the network, providing security even if the session is intercepted.
Forward secrecy
Both RSA and Diffie-Hellman (DH) have their strengths and weaknesses in the context of SSL/TLS, and the choice between them depends on the specific requirements and threats. However, the method following Diffie Hellman is more secure in terms of forward secrecy, which means that even if the server’s private key is compromised in the future, past communications cannot be decrypted. RSA key exchange does not provide this because the same private key is used across multiple sessions. If an attacker later obtains the server’s private RSA key, they can decrypt any previously recorded communications that used RSA for the key exchange. This is because PreMasterSecret is encrypted directly with the server’s public key, and the security of the session relies on the secrecy of the server’s private key.
The Diffie-Hellman Ephemeral (DHE) or Elliptic-Curve Diffie-Hellman Ephemeral (ECDHE) key exchange methods provide forward secrecy because they use ephemeral (temporary) key pairs for each session. Even if the server’s private key is compromised, past session keys cannot be derived, as the ephemeral keys are discarded after the session ends. Thus, the attacker may manage to intercept future sessions but not any of the previous or current sessions.
Performance
RSA operations, particularly decryption on the server side, are computationally intensive but still faster than traditional Diffie-Hellman. Given that session keys are generated for each session, handling 1,000 sessions per minute could strain resources with Diffie-Hellman due to the overhead of key generation. In such high-load scenarios, RSA, even with its larger key sizes (e.g., 2048-bit keys), may prove more efficient and practical. The efficiency of RSA in these cases makes it a more suitable choice when balancing performance and security in environments with heavy traffic. Elliptic-Curve Diffie-Hellman (ECDH), however, is significantly faster and requires smaller key sizes for equivalent security, making it more efficient than RSA and traditional DH.
Hashing
In SSL 2.0, MD5 is used both during the handshake and after the connection is established.
Session key derivation: MD5 is used in combination with other hashing algorithms to help derive session keys. For example, MD5 is used along with SHA-1 to generate key material for the session keys based on the pre-master secret and other handshake information. The handshake involves negotiating cryptographic algorithms and generating session keys. MD5 is used to hash various handshake parameters, including the pre-master secret (PMS), client random(C), and server random(S), to derive the session keys from key material of 128-bit hash value.
Message authentication code (MAC) calculation: After the handshake and session key are established, MD5 is used to generate the MACs for ensuring data integrity and authenticity of the messages (M) transmitted over the secure connection. The MAC is computed using MD5 and a secret key (K) derived during the handshake. Once the secure connection is established, each message transmitted over the connection is accompanied by a MAC. This MAC is computed using MD5 and a secret key, ensuring that the message has not been tampered with during transmission.
Lack of protection for handshake and message close phase
The SSL 2.0 handshake process is crucial for establishing a secure session. It involves exchanging keys, agreeing on encryption methods, and authenticating the parties. Proper SSL/TLS implementations ensure that both parties close the connection correctly, confirming that all data has been transmitted securely. However, SSL 2.0 does not adequately protect the handshake or the explicit message close phase. This means that an attacker could intercept and modify messages during these phases without detection. Since the handshake is not fully protected, an attacker could potentially alter the negotiation process (such as choosing a weaker cipher suite to manipulate data easily) or inject malicious data. Similarly, the absence of a secure message close could allow attackers to tamper with or inject additional data into the communication stream. This vulnerability means that attackers could perform man-in-the-middle attacks, intercepting and manipulating the communication between the client and server without the parties being aware.
Length extension attack
Figure 2 depicts how the attacker utilises the potential vulnerability of the MD5 hashing algorithm, which allows the internal state to be extended by appending additional data to the original message. This allows recalculation of MAC even without knowing the secret key used, using libraries allowing one to do so. This processes input in blocks (64-byte blocks).
These vulnerabilities are solely focused on weak integrity checks and have led to the development of the next-generation algorithm, SSL 3.0. This version mitigates the above issues by introducing the ‘Finished’ message concept, where client and server ‘Finished’ messages are passed through pseudo-random functions (PRFs) derived from the session key. Additionally, the use of hash-based message authentication code (HMAC) has been introduced, which incorporates hashing along with secret key usage, significantly enhancing security.
