Http Https And Web Protocols

Category: Transport and Application Layer Protocols
Type: Network Concepts
Generated on: 2025-07-10 08:56:25
For: Network Engineering, Administration & Technical Interviews

This cheatsheet covers HTTP/HTTPS and related web protocols, providing a practical guide for students and professionals in computer networking.

1. Quick Overview

HTTP (Hypertext Transfer Protocol): The foundation of data communication for the World Wide Web. It’s an application-layer protocol that defines how clients (browsers) and servers communicate to exchange hypertext (HTML) documents and other resources.

HTTPS (Hypertext Transfer Protocol Secure): A secure version of HTTP. It encrypts communication between the client and the server using SSL/TLS, protecting data integrity and confidentiality.

Why Important?

Foundation of the Web: Enables web browsing, e-commerce, and many other online services.
Data Exchange: Facilitates the transfer of various data types, including HTML, images, videos, and APIs.
Security: HTTPS ensures secure communication, protecting sensitive information like passwords and credit card details.
API Communication: HTTP methods are central to RESTful API design.

2. Key Concepts

Client-Server Model: HTTP operates on a client-server model. The client (e.g., browser) initiates requests, and the server responds with requested resources.
Stateless Protocol: HTTP is stateless, meaning the server does not retain information about past client requests. Each request is treated independently.
HTTP Methods: Verbs that define the action to be performed on a resource (e.g., GET, POST, PUT, DELETE).
URI (Uniform Resource Identifier): A string of characters that identifies a resource. URLs (Uniform Resource Locators) are a specific type of URI.
HTTP Headers: Metadata fields in HTTP requests and responses that provide additional information.
HTTP Status Codes: Three-digit codes returned by the server to indicate the outcome of a request (e.g., 200 OK, 404 Not Found, 500 Internal Server Error).
TCP/IP: HTTP/HTTPS runs on top of TCP/IP, using TCP port 80 (HTTP) or 443 (HTTPS) by default.
SSL/TLS: Secure Sockets Layer/Transport Layer Security. Cryptographic protocols that provide secure communication over a network. HTTPS uses SSL/TLS to encrypt data.
Ciphersuites: A set of encryption algorithms used by SSL/TLS to secure communication.
Certificates: Digital documents that bind a public key to an identity (e.g., a website).
Caching: Mechanisms to store and reuse previously retrieved resources, improving performance.
Cookies: Small text files stored on the client’s computer by a website to remember information about the user.
REST (Representational State Transfer): An architectural style for designing networked applications, often using HTTP for communication.
JSON (JavaScript Object Notation): A lightweight data-interchange format commonly used in web APIs.
Content Negotiation: The process of selecting the best representation of a resource for the client based on its capabilities (e.g., language, encoding).
Proxy Server: A server that acts as an intermediary between the client and the origin server. It can be used for caching, security, or load balancing.
Reverse Proxy: A proxy server that sits in front of one or more web servers, hiding their internal structure and providing security features.

3. How It Works

HTTP Request-Response Cycle:

+--------+      TCP Connection      +--------+
| Client | ------------------------> | Server |
+--------+      (Port 80)           +--------+
     |                                    |
     |  HTTP Request                   |
     |  (GET /index.html HTTP/1.1)      |
     | ------------------------>         |
     |                                    |
     |                                    | HTTP Processing
     |                                    |
     | <------------------------         |
     |  HTTP Response                  |
     |  (HTTP/1.1 200 OK)              |
     |                                    |
     |                                    |
     +------------------------         |
                                        |
                                        |  Delivers HTML/Content
                                        |
                                        +--------+

HTTPS Request-Response Cycle (Simplified):

+--------+      TCP Connection      +--------+
| Client | ------------------------> | Server |
+--------+      (Port 443)          +--------+
     |                                    |
     |  TLS Handshake                  |
     |  (Negotiate Ciphersuite, etc.)  |
     | <------------------------>         |
     |                                    |
     |  Encrypted HTTP Request         |
     |  (GET /index.html HTTP/1.1)      |
     | ------------------------>         |
     |                                    |
     |                                    | HTTP Processing
     |                                    |
     | <------------------------         |
     |  Encrypted HTTP Response        |
     |  (HTTP/1.1 200 OK)              |
     |                                    |
     |                                    |
     +------------------------         |
                                        |
                                        |  Delivers Encrypted HTML/Content
                                        |
                                        +--------+

HTTPS TLS Handshake (Simplified):

Client Hello: Client sends a “Client Hello” message to the server, proposing TLS versions, cipher suites, and compression methods.
Server Hello: Server responds with a “Server Hello” message, selecting the TLS version, cipher suite, and compression method. The server also sends its digital certificate.
Certificate Verification: The client verifies the server’s certificate against a trusted Certificate Authority (CA).
Key Exchange: Client generates a pre-master secret, encrypts it with the server’s public key (from the certificate), and sends it to the server. Alternatively, Diffie-Hellman key exchange can be used.
Master Secret Generation: Both client and server independently calculate the master secret using the pre-master secret (or Diffie-Hellman parameters) and random values exchanged during the handshake.
Encryption and Authentication: The master secret is used to derive encryption keys and MAC keys for encrypting and authenticating subsequent data.
Finished Messages: Client and server send “Finished” messages, encrypted with the newly established keys, to verify the handshake’s integrity.
Secure Communication: Application data is now encrypted and transmitted between the client and server.

4. Protocol Details

HTTP Request Structure:

<method> <URI> <HTTP version>
<header1>: <value1>
<header2>: <value2>
...
<headerN>: <valueN>

<body>  (Optional)

Example HTTP Request:

GET /index.html HTTP/1.1
Host: www.example.com
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Connection: keep-alive

HTTP Response Structure:

<HTTP version> <status code> <status text>
<header1>: <value1>
<header2>: <value2>
...
<headerN>: <valueN>

<body>  (Optional)

Example HTTP Response:

HTTP/1.1 200 OK
Date: Tue, 29 Jun 2023 12:00:00 GMT
Server: Apache/2.4.41 (Ubuntu)
Content-Type: text/html; charset=UTF-8
Content-Length: 1234

<!DOCTYPE html>
<html>
<head>
<title>Example Website</title>
</head>
<body>
<h1>Hello, World!</h1>
</body>
</html>

Common HTTP Methods:

GET: Retrieve a resource.
POST: Submit data to be processed (e.g., form submission).
PUT: Replace an existing resource or create a new one.
DELETE: Delete a resource.
PATCH: Apply partial modifications to a resource.
HEAD: Retrieve the headers of a resource without the body.
OPTIONS: Retrieve communication options for a resource.
TRACE: Perform a message loop-back test along the path to the target resource.
CONNECT: Establish a tunnel to the server identified by the target resource.

Common HTTP Headers:

Host: Specifies the domain name of the server.
User-Agent: Identifies the client software.
Accept: Lists the MIME types that the client can handle.
Accept-Language: Lists the preferred languages for the response.
Content-Type: Specifies the MIME type of the body.
Content-Length: Specifies the size of the body in bytes.
Cookie: Contains cookies sent by the client to the server.
Set-Cookie: Sets a cookie on the client’s browser.
Cache-Control: Controls caching behavior.
Authorization: Contains credentials for authentication.
Location: Used in redirects to specify the new location of a resource.
Referer (misspelled): Indicates the URL of the previous web page from which a link to the currently requested page was followed.
ETag: Entity Tag, a unique identifier for a specific version of a resource.

Common HTTP Status Codes:

1xx (Informational): Provisional response.
- 100 Continue
- 101 Switching Protocols
2xx (Success): The request was successfully received, understood, and accepted.
- 200 OK
- 201 Created
- 204 No Content
3xx (Redirection): Further action needs to be taken by the client in order to fulfill the request.
- 301 Moved Permanently
- 302 Found (Previously “Moved Temporarily”)
- 304 Not Modified
4xx (Client Error): The request contains bad syntax or cannot be fulfilled.
- 400 Bad Request
- 401 Unauthorized
- 403 Forbidden
- 404 Not Found
- 405 Method Not Allowed
5xx (Server Error): The server failed to fulfill an apparently valid request.
- 500 Internal Server Error
- 502 Bad Gateway
- 503 Service Unavailable
- 504 Gateway Timeout

5. Real-World Examples

Web Browsing: A user types https://www.example.com into their browser. The browser sends an HTTPS GET request to the server. The server responds with HTML, CSS, and JavaScript, which the browser renders to display the webpage.
API Calls: A mobile app sends a POST request to an API endpoint (https://api.example.com/users) to create a new user account. The request body contains JSON data with the user’s information. The server responds with a 201 Created status code and the new user’s ID.
E-commerce: A user adds items to their shopping cart and proceeds to checkout. The browser sends a POST request to the server with the order details. The server processes the order, updates the inventory, and returns a confirmation page to the user. HTTPS is crucial for secure transmission of payment information.
Content Delivery Network (CDN): A CDN stores copies of website content on servers around the world. When a user requests a resource, the CDN serves it from the nearest server, reducing latency and improving performance. HTTP caching mechanisms are heavily used in CDNs.
Load Balancing: A load balancer distributes incoming HTTP requests across multiple web servers, preventing any single server from becoming overloaded. This improves the availability and scalability of the website. Reverse proxies are often used for load balancing.
WebSockets: Provides full-duplex communication channels over a single TCP connection. Used in applications that require real-time data updates, such as chat applications and online games. The HTTP handshake is used to upgrade the connection to WebSocket.

6. Common Issues

Connection Errors: Problems establishing a TCP connection to the server (e.g., firewall issues, DNS resolution failures).
- Troubleshooting: Use ping and traceroute to check network connectivity. Verify firewall rules and DNS settings.
SSL/TLS Certificate Errors: Invalid or expired certificates, untrusted CAs.
- Troubleshooting: Ensure the certificate is valid and issued by a trusted CA. Update the client’s trust store. Check the server’s certificate configuration.
Mixed Content Errors: An HTTPS website loads resources (e.g., images, scripts) over HTTP. Browsers often block mixed content for security reasons.
- Troubleshooting: Update all links to use HTTPS. Configure the server to redirect HTTP requests to HTTPS.
HTTP Status Code Errors: 4xx or 5xx errors indicate problems with the request or the server.
- Troubleshooting: Examine the error message and the request/response headers for clues. Check server logs for errors. Verify the request URL and parameters.
Caching Issues: Stale or incorrect cached content.
- Troubleshooting: Clear the browser cache. Use the Cache-Control header to control caching behavior. Implement cache-busting techniques (e.g., adding a version number to the URL).
Cookie Problems: Cookies are not being set or are being rejected by the browser.
- Troubleshooting: Check the Set-Cookie header in the response. Ensure the cookie’s domain and path are correctly configured. Verify the browser’s cookie settings.
Performance Issues: Slow page load times.
- Troubleshooting: Optimize images and other resources. Use a CDN. Enable HTTP compression (e.g., gzip). Minimize the number of HTTP requests. Use browser developer tools to profile page load times.
HTTP/2 and HTTP/3 Issues: Compatibility problems, performance bottlenecks.
- Troubleshooting: Verify server and client support for the protocol. Check network configuration for MTU issues. Monitor connection statistics for signs of packet loss.

7. Configuration Examples

Apache HTTP Server (HTTPS):

<VirtualHost *:443>
    ServerName www.example.com
    DocumentRoot /var/www/example.com

    SSLEngine on
    SSLCertificateFile /etc/ssl/certs/example.com.crt
    SSLCertificateKeyFile /etc/ssl/private/example.com.key

    <Directory /var/www/example.com>
        Options Indexes FollowSymLinks
        AllowOverride All
        Require all granted
    </Directory>
</VirtualHost>

Nginx (HTTPS):

server {
    listen 443 ssl;
    server_name www.example.com;

    root /var/www/example.com;
    index index.html;

    ssl_certificate /etc/ssl/certs/example.com.crt;
    ssl_certificate_key /etc/ssl/private/example.com.key;

    location / {
        try_files $uri $uri/ =404;
    }
}

.htaccess (Redirect HTTP to HTTPS):

RewriteEngine On
RewriteCond %{HTTPS} off
RewriteRule ^(.*)$ https://%{HTTP_HOST}%{REQUEST_URI} [L,R=301]

iptables (Firewall - Allow HTTP/HTTPS):

# Allow HTTP (port 80)
iptables -A INPUT -p tcp --dport 80 -j ACCEPT

# Allow HTTPS (port 443)
iptables -A INPUT -p tcp --dport 443 -j ACCEPT

# Save iptables rules (on Debian/Ubuntu)
sudo netfilter-persistent save

Using curl for testing:

# Basic HTTP request
curl http://www.example.com

# HTTPS request with verbose output
curl -v https://www.example.com

# POST request with data
curl -X POST -d "param1=value1&param2=value2" http://www.example.com/api

# Get headers only
curl -I https://www.example.com

Using openssl for certificate inspection:

openssl s_client -connect www.example.com:443 -showcerts

8. Interview Questions

Q: What is the difference between HTTP and HTTPS?

A: HTTP is an unencrypted protocol, while HTTPS is encrypted using SSL/TLS. HTTPS provides secure communication by encrypting data exchanged between the client and the server, protecting it from eavesdropping and tampering.

Q: What are HTTP methods? Give examples.

A: HTTP methods (or verbs) define the action to be performed on a resource. Examples include: * GET: Retrieves a resource. * POST: Submits data to be processed. * PUT: Replaces an existing resource or creates a new one. * DELETE: Deletes a resource.

Q: What is an HTTP status code? Give examples.

A: HTTP status codes are three-digit codes returned by the server to indicate the outcome of a request. Examples include: * 200 OK: Request was successful. * 404 Not Found: The requested resource was not found. * 500 Internal Server Error: The server encountered an error.

Q: What is the purpose of HTTP headers? Give examples.

A: HTTP headers provide additional information about the request or response. Examples include: * Host: Specifies the domain name of the server. * User-Agent: Identifies the client software. * Content-Type: Specifies the MIME type of the body. * Content-Length: Specifies the size of the body. * Cache-Control: Controls caching behavior.

Q: Explain the TLS/SSL handshake process.

A: (See detailed explanation in Section 3). Briefly: The client and server negotiate a secure connection by exchanging information such as supported TLS versions, cipher suites, and random values. The server presents its digital certificate, which the client verifies. A shared secret is established, and encryption keys are derived from it. Finally, the client and server exchange “Finished” messages to confirm the handshake’s integrity.

Q: What are cookies, and how are they used?

A: Cookies are small text files stored on the client’s computer by a website. They are used to remember information about the user, such as login credentials, preferences, and shopping cart items. The server sends cookies to the client via the Set-Cookie header, and the client sends them back to the server with subsequent requests via the Cookie header.

Q: What is caching and why is it important?

A: Caching is a mechanism to store and reuse previously retrieved resources. It’s important because it reduces latency, improves performance, and reduces server load. Browsers, CDNs, and proxy servers all use caching.

Q: What is a CDN and how does it work?

A: A Content Delivery Network (CDN) is a distributed network of servers that stores copies of website content on servers around the world. When a user requests a resource, the CDN serves it from the nearest server, reducing latency and improving performance.

Q: What are the differences between HTTP/1.1, HTTP/2, and HTTP/3?

A: * HTTP/1.1: The original version of HTTP, using a text-based protocol and a single TCP connection per request. * HTTP/2: Introduced multiplexing (multiple requests over a single TCP connection), header compression, and server push. * HTTP/3: Uses QUIC protocol (UDP-based) for improved performance and reliability, especially in lossy networks. Addresses head-of-line blocking issues in TCP.

Q: What is CORS and why is it important?

A: CORS (Cross-Origin Resource Sharing) is a security mechanism that allows web pages from one domain to access resources from a different domain. It’s important because it prevents malicious websites from accessing sensitive data from other websites. Browsers enforce CORS policies to protect users.

Q: What is the purpose of a load balancer?

A: A load balancer distributes incoming network traffic across multiple servers. This prevents any single server from becoming overloaded and improves the availability and scalability of the application.

TCP/IP Model: Understanding the underlying transport layer (TCP) is crucial for HTTP/HTTPS.
DNS (Domain Name System): HTTP relies on DNS to resolve domain names to IP addresses.
Network Security: Concepts such as firewalls, intrusion detection systems (IDS), and web application firewalls (WAFs) are important for protecting web servers and applications.
Web Development: Knowledge of HTML, CSS, JavaScript, and server-side programming languages is helpful for understanding how HTTP/HTTPS are used in web applications.
RESTful APIs: HTTP methods are fundamental to RESTful API design.
OAuth: An open standard for access delegation, commonly used with HTTP for authentication and authorization.
WebSockets: A protocol that provides full-duplex communication channels over a single TCP connection.
QUIC: A UDP-based transport protocol developed by Google, used by HTTP/3.

Further Reading:

RFC 9110, 9111, 9112, 9113, 9114 (HTTP/1.1, Caching, HTTP/2, HTTP/3): The official HTTP specifications.
RFC 5246 (TLS 1.2): The official TLS 1.2 specification.
RFC 8446 (TLS 1.3): The official TLS 1.3 specification.
OWASP (Open Web Application Security Project): A valuable resource for web security best practices.
Mozilla Developer Network (MDN): Comprehensive documentation on web technologies.