Performance And Latency Problems

Category: Network Troubleshooting
Type: Network Tools & Commands
Generated on: 2025-07-11 01:39:17
For: Network Engineering, Administration & Technical Interviews

This cheatsheet provides a practical guide to common network tools for diagnosing performance and latency issues. It covers syntax, usage examples, output interpretation, and troubleshooting scenarios for network administrators and engineers.

1. Tool Overview

Tool	Description	Use Case
`ping`	Tests basic network connectivity by sending ICMP echo requests.	Verify host reachability, basic latency measurement, quick network health check.
`traceroute`/`tracert`	Maps the path packets take to a destination, identifying each hop.	Identify routing loops, pinpoint latency hotspots, diagnose path selection issues.
`mtr`	Combines `ping` and `traceroute` for continuous latency and packet loss analysis.	Long-term network monitoring, identify intermittent connectivity problems.
`tcpdump`/`Wireshark`	Captures and analyzes network traffic.	Deep packet inspection, troubleshoot application-layer issues, security analysis.
`netstat`/`ss`	Displays network connections, routing tables, and interface statistics.	Identify open ports, check listening services, diagnose connection issues.
`iperf3`	Measures network bandwidth and performance.	Determine maximum achievable throughput, identify network bottlenecks.
`nmap`	Network exploration and security auditing tool.	Port scanning, service discovery, OS fingerprinting.

2. Basic Syntax

ping
Terminal window
```
ping [options] <destination>
```

traceroute (Linux/macOS) / tracert (Windows)

traceroute [options] <destination>
tracert [options] <destination>

mtr
Terminal window
```
mtr [options] <destination>
```
tcpdump
Terminal window
```
tcpdump [options] <filter expression>
```
netstat (Deprecated, use ss in Linux)
Terminal window
```
netstat [options]
```
ss
Terminal window
```
ss [options]
```

iperf3

iperf3 -s  # Server mode
iperf3 -c <server_ip> [options] # Client mode

nmap
Terminal window
```
nmap [options] <target>
```

3. Practical Examples

ping:

ping google.com

PING google.com (142.250.185.142): 56 data bytes
64 bytes from 142.250.185.142: icmp_seq=0 ttl=118 time=12.5 ms
64 bytes from 142.250.185.142: icmp_seq=1 ttl=118 time=12.3 ms
64 bytes from 142.250.185.142: icmp_seq=2 ttl=118 time=12.4 ms
^C
--- google.com ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 12.314/12.424/12.504/0.080 ms

traceroute (Linux/macOS):

traceroute google.com

traceroute to google.com (142.250.185.142), 30 hops max, 60 byte packets
 1  192.168.1.1 (192.168.1.1)  1.234 ms  1.345 ms  1.456 ms
 2  10.0.0.1 (10.0.0.1)  5.678 ms  6.789 ms  7.890 ms
 3  ...

tracert (Windows):

tracert google.com

Tracing route to google.com [142.250.185.142]
over a maximum of 30 hops:

  1    <1 ms   <1 ms   <1 ms  192.168.1.1
  2     2 ms     1 ms     1 ms  10.0.0.1
  3     ...

mtr:
Terminal window
```
mtr google.com
```
(Displays a continuous updated report of latency and packet loss per hop)
tcpdump:
Terminal window
```
# Capture traffic on port 80
tcpdump port 80
```
(Outputs detailed packet information to the console.)
netstat (Deprecated, use ss):
Terminal window
```
netstat -ant | grep :80
```
ss:
Terminal window
```
ss -ant | grep :80
```
(Displays TCP connections on port 80)
iperf3:

Server:
Terminal window
```
iperf3 -s
```
Client:
Terminal window
```
iperf3 -c 192.168.1.100
```
(Displays bandwidth and performance statistics.)
nmap:
Terminal window
```
nmap google.com
```
(Displays open ports and service information.)

4. Common Options

Tool	Option	Description
`ping`	`-c <count>`	Number of ping requests to send.
	`-i <interval>`	Interval between ping requests (in seconds).
	`-s <size>`	Size of the ping packet (in bytes).
`traceroute`/`tracert`	`-m <max_hops>`	Maximum number of hops to trace.
`mtr`	`-r`	Report mode (shows final statistics).
	`-n`	Disable DNS resolution.
`tcpdump`	`-i <interface>`	Specify the network interface to capture traffic on.
	`-n`	Do not resolve hostnames.
	`-w <file>`	Write captured packets to a file.
	`-s <snaplen>`	Capture only the first bytes of each packet.
`netstat`/`ss`	`-a`	Show all connections (listening and non-listening).
	`-n`	Show numerical addresses instead of resolving hostnames.
	`-t`	Show TCP connections.
	`-u`	Show UDP connections.
	`-p`	Show the PID and name of the program to which each socket belongs.
`iperf3`	`-P <parallel>`	Number of parallel client threads to run.
	`-t <seconds>`	Duration of the test (in seconds).
	`-b <bandwidth>`	Target bandwidth (e.g., 10M for 10 Mbps).
`nmap`	`-p <ports>`	Specify port(s) to scan (e.g., `22,80,443`, `1-1000`).
	`-sS`	TCP SYN scan (stealth scan).
	`-sV`	Service version detection.
	`-O`	OS detection.

5. Advanced Usage

ping:

# Flood ping (use with caution - can impact network performance)
ping -f <destination>

traceroute:

# Use TCP SYN packets for traceroute (bypasses some firewalls)
traceroute -T -p 80 <destination>

tcpdump:

# Capture traffic to/from a specific IP address on a specific port
tcpdump -i eth0 host 192.168.1.100 and port 80

# Capture only HTTP GET requests
tcpdump -i eth0 -A 'tcp[((tcp[12:1] & 0xf0) >> 2):4] = 0x47455420' # GET

ss:

# Show all established TCP connections sorted by state
ss -tan | sort

iperf3:

# Run a UDP test with a specific bandwidth target
iperf3 -c <server_ip> -u -b 10M -t 10

nmap:

# Scan a network range for live hosts
nmap -sn 192.168.1.0/24

# Perform a version detection scan on all TCP ports
nmap -sV -p 1-65535 <target>

6. Troubleshooting Scenarios

Problem	Tool(s)	Solution
Host unreachable	`ping`, `traceroute`	Check physical connectivity (cables, interfaces), verify IP address configuration, check routing tables, investigate firewall rules.
High latency to a specific destination	`ping`, `traceroute`, `mtr`	Identify the hop(s) with high latency, investigate network congestion, check link utilization, consider routing optimization.
Packet loss	`ping`, `mtr`	Check for network congestion, faulty hardware (cables, routers), excessive buffer overflows, duplex mismatch.
Slow application performance	`tcpdump`, `iperf3`, `netstat`/`ss`	Identify the bottleneck (network bandwidth, server resources, application code), analyze network traffic for retransmissions, measure bandwidth between client and server, check server CPU/memory utilization.
Port blocked by firewall	`nmap`, `telnet`	Verify firewall rules on the client, server, and intermediate devices. Use `nmap` to identify filtered ports.
DNS resolution issues	`ping`, `nslookup`, `dig`	Verify DNS server configuration, check DNS server reachability, investigate DNS server performance.
Intermittent connectivity	`mtr`, `ping` with logging	Use `mtr` to monitor latency and packet loss over time, log ping results to a file for analysis. Investigate possible causes of intermittent network congestion or hardware failures.
Network bandwidth limitations	`iperf3`	Measure available bandwidth between client and server, identify bandwidth bottlenecks, consider upgrading network infrastructure or optimizing traffic shaping policies.
High CPU usage on network devices	`top`, `netstat`/`ss` (Linux)	High CPU usage can indicate a network attack, routing loop, or misconfigured device. Investigate processes consuming CPU, check routing tables for inconsistencies, and implement appropriate security measures. Using `ss -p` (linux) can help correlate high socket usage with specific processes.

7. Output Interpretation

ping:
- time=XX ms: Round-trip time (RTT) in milliseconds. Lower is better.
- ttl=XX: Time-to-live. Decreases by one at each hop. Indicates the number of hops remaining.
- packet loss: Percentage of packets lost. 0% is ideal.
traceroute/tracert:
- Lists each hop along the path to the destination.
- Shows the IP address and hostname (if resolved) of each hop.
- Displays the RTT for each hop. High RTT indicates potential latency issues at that hop. * indicates a timeout.
mtr:
- Loss%: Percentage of packet loss at each hop.
- Snt: Number of packets sent.
- Last: RTT of the last packet.
- Avg: Average RTT.
- Best: Minimum RTT.
- Wrst: Maximum RTT.
- StDev: Standard deviation of RTT. Higher StDev indicates more variable latency.
tcpdump:
- Requires knowledge of network protocols to interpret.
- Examine packet headers for source/destination IP addresses and ports.
- Analyze packet content for application-layer data.
- Look for retransmissions (duplicate sequence numbers) indicating packet loss.
netstat/ss:
- State: State of the TCP connection (e.g., ESTABLISHED, LISTEN, TIME_WAIT).
- Local Address: IP address and port the local host is using.
- Foreign Address: IP address and port of the remote host.
iperf3:
- Transfer: Amount of data transferred during the test.
- Bandwidth: Achieved bandwidth (in bits/sec or bytes/sec).
- Jitter: Variation in packet delay.
nmap:
- Lists open, closed, and filtered ports.
- Provides service version information (if -sV is used).
- May provide OS fingerprint information (if -O is used).

8. Security Considerations

ping: Relatively safe, but flooding can be used for denial-of-service attacks. Disable ICMP if security is paramount.
traceroute/tracert: Can reveal network topology, which can be used for reconnaissance. Some firewalls block traceroute requests.
mtr: Same security considerations as traceroute.
tcpdump: Captures sensitive data (passwords, credit card numbers) if not used carefully. Limit capture to specific traffic and use encryption where possible. Store capture files securely. Consider using a dedicated capture machine.
netstat/ss: Reveals open ports and network connections, which can be used for reconnaissance.
iperf3: Can be used for denial-of-service attacks if not configured properly. Restrict access to the iperf3 server.
nmap: Can be considered aggressive and may trigger intrusion detection systems. Use with caution and obtain permission before scanning networks you don’t own. Avoid advanced scanning options like OS detection on sensitive networks without explicit permission.

9. Platform Differences

traceroute: Command name is traceroute on Linux/macOS and tracert on Windows. Options and output format may vary slightly.
netstat: Deprecated on Linux in favor of ss.
tcpdump: Available on Linux and macOS. Requires WinPcap/Npcap on Windows (Wireshark includes this).
Firewall Configuration: Firewall configuration methods differ greatly between Linux (iptables/nftables), macOS (pf), and Windows Firewall.
Path to executables: May differ slightly based on distribution. e.g., /usr/sbin/traceroute or /bin/traceroute. Check your $PATH.

This cheatsheet provides a foundation for troubleshooting network performance and latency issues. Always consult the tool’s manual page (man <tool>) for complete documentation and advanced options. Remember to use these tools responsibly and ethically, and always obtain permission before scanning or capturing traffic on networks you don’t own.