Intrusion Detection And Prevention Systems

Category: Network Security
Type: Network Concepts
Generated on: 2025-07-10 09:06:42
For: Network Engineering, Administration & Technical Interviews

Intrusion Detection and Prevention Systems (IDPS) Cheatsheet

What is it? An Intrusion Detection and Prevention System (IDPS) is a security tool that monitors network or system activities for malicious behavior or policy violations. IDPSs can detect and/or prevent intrusions by analyzing network traffic, system logs, and other data sources.

Why is it important? IDPSs are essential for:

Early threat detection: Identifying malicious activity before it causes significant damage.
Incident response: Providing valuable information for investigating and responding to security incidents.
Compliance: Meeting regulatory requirements for security monitoring and protection.
Proactive security: Preventing attacks from succeeding and minimizing their impact.
Network Visibility: Provides insights into network traffic patterns and potential vulnerabilities.
Intrusion Detection System (IDS): Monitors network traffic and system activity for suspicious behavior and alerts administrators. It’s primarily a monitoring and reporting tool.
Intrusion Prevention System (IPS): Goes a step further than IDS by actively blocking or preventing detected intrusions. It’s an active security control.
Signature-based Detection: Identifies attacks based on known patterns or signatures of malicious code or network traffic.
Anomaly-based Detection: Identifies attacks by detecting deviations from normal network or system behavior. Requires a baseline of “normal” activity.
Policy-based Detection: Identifies attacks based on predefined security policies.
False Positive: An alert triggered by legitimate activity that is incorrectly identified as malicious.
False Negative: Failure to detect malicious activity.
True Positive: Correctly identifying malicious activity.
True Negative: Correctly identifying legitimate activity.
Host-based IDPS (HIDS): Installed on individual hosts to monitor system activity and logs.
Network-based IDPS (NIDS): Monitors network traffic for malicious activity.
Evasion Techniques: Methods used by attackers to bypass IDPS detection.
Honeypot: A decoy system designed to attract attackers and provide information about their techniques.
Sandbox: An isolated environment used to analyze suspicious files or code.
SIEM (Security Information and Event Management): A system that collects and analyzes security logs from various sources, often integrating with IDPS.

A. NIDS (Network-based IDPS):

+-----------------+   Network Traffic   +-----------------+   +-----------------+
| Network Segment | -------------------> |   NIDS Sensor  | --> |   Analysis Engine |
+-----------------+                      +-----------------+   | +-----------------+
                                          | Capture & Analyze|   |  (Rules, Signatures)|
                                          +-----------------+   | +-----------------+
                                                               | |  Alerts/Prevention|
                                                               | +-----------------+
                                                               |
                                                               v
                                                       +-----------------+
                                                       | Security Console|
                                                       +-----------------+

Traffic Capture: The NIDS sensor captures network traffic. This is often done using a network tap or a SPAN port on a switch.
Analysis: The NIDS engine analyzes the captured traffic using various detection methods (signature-based, anomaly-based, policy-based).
Alerting/Prevention: If malicious activity is detected, the NIDS generates an alert or takes preventative action (e.g., blocking the traffic).
Reporting: The NIDS reports the detected activity to a security console.

B. HIDS (Host-based IDPS):

+-----------------+   System Activity   +-----------------+   +-----------------+
|  Host Machine   | -------------------> |   HIDS Agent   | --> |   Analysis Engine |
+-----------------+                      +-----------------+   | +-----------------+
                                          | Monitor Logs,   |   |  (Rules, Signatures)|
                                          | File Integrity  |   | +-----------------+
                                          +-----------------+   | |  Alerts/Prevention|
                                                               | +-----------------+
                                                               |
                                                               v
                                                       +-----------------+
                                                       | Security Console|
                                                       +-----------------+

Activity Monitoring: The HIDS agent monitors system logs, file integrity, and other system activity.
Analysis: The HIDS engine analyzes the monitored activity using various detection methods.
Alerting/Prevention: If malicious activity is detected, the HIDS generates an alert or takes preventative action (e.g., terminating a process).
Reporting: The HIDS reports the detected activity to a security console.

IDPSs analyze various network protocols. Key protocols to understand include:

TCP (Transmission Control Protocol): Used for reliable, connection-oriented communication. IDPSs analyze TCP headers for flags (SYN, ACK, FIN, RST), port numbers, and sequence numbers to detect port scanning, SYN floods, and other attacks.
UDP (User Datagram Protocol): Used for connectionless communication. IDPSs analyze UDP headers for port numbers and payload data to detect DNS amplification attacks and other UDP-based attacks.
ICMP (Internet Control Message Protocol): Used for network diagnostics and error reporting. IDPSs analyze ICMP messages to detect ping floods, Smurf attacks, and other ICMP-based attacks.
HTTP (Hypertext Transfer Protocol): Used for web communication. IDPSs analyze HTTP headers and content to detect web application attacks (e.g., SQL injection, cross-site scripting).
DNS (Domain Name System): Used for resolving domain names to IP addresses. IDPSs analyze DNS queries and responses to detect DNS tunneling and other DNS-based attacks.
SSL/TLS (Secure Sockets Layer/Transport Layer Security): Used for encrypting network traffic. IDPSs often perform SSL/TLS inspection to decrypt traffic and analyze its content. This requires having access to the private key of the server.

Example: TCP Header Analysis for SYN Flood Detection

A SYN flood attack involves sending a large number of SYN packets to a server without completing the TCP handshake. An IDPS can detect this by monitoring the rate of SYN packets received by the server.

TCP Header:

Source Port: 12345
Destination Port: 80
Sequence Number: 1234567890
Acknowledgment Number: 0
Flags: SYN (Synchronize)
Window Size: 65535
...

An IDPS rule might look for a high rate of SYN packets with no corresponding ACK packets.

Detecting a SQL Injection Attack: An IDPS can analyze HTTP requests for suspicious characters or patterns in URL parameters or form fields that are indicative of SQL injection.

Example HTTP Request:
```
GET /product.php?id=1' OR '1'='1 HTTP/1.1
```
The IDPS would detect the ' OR '1'='1 string as a potential SQL injection attempt.
Preventing a Distributed Denial-of-Service (DDoS) Attack: An IPS can detect a DDoS attack by monitoring network traffic for a sudden surge in traffic from multiple sources. It can then block traffic from the attacking sources to mitigate the impact of the attack.
Detecting Malware Infections: A HIDS can monitor system logs and file integrity to detect malware infections. For example, it can detect the creation of new executable files in suspicious locations or modifications to critical system files.
Detecting Port Scanning: A NIDS can detect port scanning activity by monitoring network traffic for a large number of connection attempts to different ports on a target system.
Monitoring for Phishing Attacks: An IDPS can analyze email traffic and web traffic for phishing links or suspicious attachments. It can then block access to the phishing websites or quarantine the malicious emails.
High False Positive Rate: Generating too many false positives can overwhelm security teams and make it difficult to identify real threats. Tuning and proper configuration are crucial.
- Troubleshooting: Review IDPS rules and signatures, adjust thresholds, and create exceptions for legitimate traffic.
Performance Impact: IDPSs can consume significant system resources, especially when analyzing large volumes of traffic.
- Solutions: Optimize IDPS configuration, use dedicated hardware, and consider using a distributed IDPS architecture.
Evasion Techniques: Attackers can use various techniques to bypass IDPS detection, such as fragmentation, encryption, and obfuscation.
- Mitigation: Keep IDPS signatures up-to-date, use advanced detection methods (e.g., anomaly-based detection), and implement multiple layers of security.
SSL/TLS Inspection Challenges: Decrypting SSL/TLS traffic can be complex and resource-intensive. It also raises privacy concerns.
- Solutions: Use SSL/TLS inspection selectively, obtain user consent, and implement appropriate security controls to protect decrypted data.
Configuration Complexity: IDPSs can be complex to configure and manage, requiring specialized knowledge and expertise.
- Solutions: Provide adequate training for security personnel, use configuration management tools, and leverage vendor support.
Log Management and Analysis: IDPSs generate large volumes of logs, which can be difficult to manage and analyze.
- Solutions: Use a SIEM system to collect and analyze IDPS logs, implement log retention policies, and automate log analysis tasks.

A. Snort (NIDS):

Basic Snort Rule:
```
alert tcp any any -> 192.168.1.0/24 80 (msg:"Possible Web Attack"; content:"<script>"; sid:1000001; rev:1;)
```
This rule alerts if it detects the string <script> in HTTP traffic destined for the 192.168.1.0/24 network on port 80.
Running Snort in IDS Mode:
Terminal window
```
snort -c /etc/snort/snort.conf -i eth0
```
This command runs Snort with the configuration file /etc/snort/snort.conf and monitors the eth0 interface.

Running Snort in Inline IPS Mode (using NFQUEUE and iptables):

# iptables configuration to send traffic to NFQUEUE
iptables -A INPUT -j NFQUEUE --queue-num 0
iptables -A OUTPUT -j NFQUEUE --queue-num 0

# Run Snort in inline mode
snort -c /etc/snort/snort.conf -i eth0 -q 0

This configuration uses iptables to redirect traffic to the NFQUEUE, which Snort then processes. Snort can drop malicious packets directly.

B. OSSEC (HIDS):

OSSEC Configuration (ossec.conf):

<ossec_config>
  <global>
    <email_notification>yes</email_notification>
    <email_to>security@example.com</email_to>
  </global>

  <syscheck>
    <directories check_all="yes">/etc,/usr/bin,/usr/sbin</directories>
    <frequency>7200</frequency> <!-- Check every 2 hours -->
  </syscheck>

  <rootcheck>
    <rootkit_files>/var/ossec/etc/rootcheck/rootkit_files.txt</rootkit_files>
    <rootkit_trojans>/var/ossec/etc/rootcheck/rootkit_trojans.txt</rootkit_trojans>
  </rootcheck>
</ossec_config>

This configuration enables email notifications, monitors key directories for file changes, and performs rootkit detection.

Q: What is the difference between an IDS and an IPS?
- A: An IDS detects intrusions and alerts administrators, while an IPS actively blocks or prevents intrusions. IDS is passive; IPS is active.
Q: What are the different types of IDPS detection methods?
- A: Signature-based, anomaly-based, and policy-based detection.
Q: What are the advantages and disadvantages of signature-based detection?
- A: Advantages: Accurate for known attacks, relatively easy to implement. Disadvantages: Ineffective against new or modified attacks (zero-day exploits), requires constant signature updates.
Q: What are the advantages and disadvantages of anomaly-based detection?
- A: Advantages: Can detect unknown attacks, adaptable to changing network behavior. Disadvantages: High false positive rate, requires a baseline of normal activity.
Q: How does a SYN flood attack work, and how can an IDPS detect it?
- A: A SYN flood attack involves sending a large number of SYN packets to a server without completing the TCP handshake. An IDPS can detect this by monitoring the rate of SYN packets received by the server and looking for a high rate of SYN packets with no corresponding ACK packets.
Q: What are some common evasion techniques used by attackers to bypass IDPS detection?
- A: Fragmentation, encryption, obfuscation, protocol manipulation, and resource exhaustion.
Q: What is the role of a SIEM in relation to an IDPS?
- A: A SIEM (Security Information and Event Management) system collects and analyzes security logs from various sources, including IDPSs. It provides a centralized view of security events and helps security teams identify and respond to threats more effectively. The SIEM correlates data from multiple sources, providing context and reducing false positives.
Q: Explain how you would configure an IPS to block traffic from a known malicious IP address.
- A: This would typically involve creating a rule or policy within the IPS management interface. The rule would specify the malicious IP address as the source, the destination (e.g., internal network), and the action to take (e.g., block or drop the traffic). Example (Snort): drop tcp [malicious IP] any -> any any (msg:"Blocking Malicious IP"; sid:1000002; rev:1;)
Q: What is a honeypot and how can it be used with an IDPS?
- A: A honeypot is a decoy system designed to attract attackers. It can be used with an IDPS to monitor attacker activity and gather information about their techniques. When an attacker interacts with the honeypot, the IDPS can log their actions, analyze their tools, and potentially identify their origin.
Firewalls: Control network access based on predefined rules.
Vulnerability Management: Identifying and mitigating vulnerabilities in systems and applications.
Penetration Testing: Simulating attacks to identify security weaknesses.
Security Auditing: Evaluating security controls and practices.
SIEM (Security Information and Event Management): Centralized log management and security analysis.
Endpoint Detection and Response (EDR): Focuses on detecting and responding to threats on individual endpoints (e.g., computers, servers).
Network Traffic Analysis (NTA): Deep packet inspection and behavioral analysis of network traffic.
Zero Trust Security: A security framework based on the principle of “never trust, always verify.”

Further Reading:

SANS Institute: https://www.sans.org/
OWASP (Open Web Application Security Project): https://owasp.org/
NIST (National Institute of Standards and Technology): https://www.nist.gov/

This cheatsheet provides a comprehensive overview of Intrusion Detection and Prevention Systems. Remember to continuously update your knowledge and skills to stay ahead of evolving security threats.