Configuration Management Basics

Category: DevOps and System Tools
Type: Linux Commands
Generated on: 2025-07-10 03:21:15
For: System Administration, Development & Technical Interviews

Configuration Management Basics Cheatsheet (Linux Commands - DevOps and System Tools)

This cheatsheet covers essential Linux commands used in configuration management, focusing on their practical application in DevOps and system administration.

1. rsync - Remote Synchronization

Command Overview: rsync is a powerful tool for synchronizing files and directories between two locations, either locally or over a network. It’s highly efficient because it only transfers the differences between files, minimizing bandwidth usage. Used extensively for backups, deployments, and mirroring.
When to Use:
- Backing up data to a remote server.
- Deploying code to multiple servers.
- Mirroring a website or file server.
- Synchronizing files between development and production environments.
Basic Syntax:
Terminal window
```
rsync [options] source destination
```

Practical Examples:

# Synchronize a local directory to a remote server (preserving permissions, ownership, times)
rsync -avz /path/to/local/directory user@remote_server:/path/to/remote/directory

# Synchronize a remote directory to a local directory
rsync -avz user@remote_server:/path/to/remote/directory /path/to/local/directory

# Dry run: See what would be transferred without actually transferring anything
rsync -avn /path/to/local/directory user@remote_server:/path/to/remote/directory

# Delete extraneous files from destination (mirroring)
rsync -avz --delete /path/to/local/directory user@remote_server:/path/to/remote/directory

Sample Output (Dry Run):

sending incremental file list
./
file1.txt
file2.txt

sent 176 bytes  received 53 bytes  458.00 bytes/sec
total size is 0  speedup is 0.00 (DRY RUN)

Common Options:
- -a, --archive: Archive mode; preserves permissions, ownership, timestamps, symlinks, etc. Equivalent to -rlptgoD. Crucial for maintaining file integrity.
- -v, --verbose: Increase verbosity. Use -vv for even more details.
- -z, --compress: Compress file data during the transfer. Useful for slow network connections.
- -r, --recursive: Recurse into directories.
- -l, --links: Copy symlinks as symlinks.
- -p, --perms: Preserve permissions.
- -t, --times: Preserve modification times.
- -g, --group: Preserve group ownership.
- -o, --owner: Preserve owner (requires root privileges on the destination).
- -D: Preserve device files and special files.
- --delete: Delete extraneous files from the destination directory that are not present in the source. USE WITH CAUTION!
- -n, --dry-run: Perform a trial run without making any changes. Always use this before running with --delete.
- --exclude='pattern': Exclude files matching the specified pattern. e.g., --exclude='*.log'
- --include='pattern': Include files matching the specified pattern. Useful in conjunction with --exclude.
- --progress: Show progress during transfer.
- --bwlimit=KBPS: Limit I/O bandwidth; useful for not saturating a network.
- -e ssh: Specify the SSH command to use for remote transfers. Useful for non-standard SSH ports or options.

Advanced Usage:

# Exclude specific files and directories
rsync -avz --exclude '.git/' --exclude 'node_modules/' /path/to/source user@remote_server:/path/to/destination

# Rsync using a specific SSH port
rsync -avz -e "ssh -p 2222" /path/to/source user@remote_server:/path/to/destination

# Backup with incremental backups using --link-dest
# First full backup
rsync -avz /path/to/source /path/to/backup/full_backup
# Subsequent incremental backups (hard links to unchanged files)
rsync -avz --link-dest=/path/to/backup/full_backup /path/to/source /path/to/backup/incremental_backup

# Rsync over SSH with a custom identity file
rsync -avz -e "ssh -i /path/to/private_key" /path/to/source user@remote_server:/path/to/destination

Tips & Tricks:
- Use trailing slashes carefully. rsync -avz /path/to/source/ destination copies the contents of source into destination, while rsync -avz /path/to/source destination copies the source directory itself into destination.
- Consider using --delete only after thorough testing with --dry-run.
- For very large datasets, consider using screen or tmux to keep the rsync process running even if your SSH connection drops.
- Use ionice to reduce the I/O priority of rsync to avoid impacting other processes. e.g., ionice -c 3 rsync ...
Troubleshooting:
- “Permission denied (publickey)”: Ensure your SSH key is properly configured on the remote server and that you’re using the correct username.
- “rsync: command not found”: Install rsync on both the source and destination servers.
- Slow transfer speeds: Check network connectivity, compress data (-z), or limit bandwidth (--bwlimit).
- Files not being updated: Check timestamps and permissions. Ensure you’re using the correct options (especially -a).
Related Commands:
- scp: Secure copy (simpler but less efficient than rsync).
- tar: Archiving utility (often used with rsync for backups).
- cp: Copy files locally.

2. sed - Stream Editor

Command Overview: sed is a powerful stream editor used for performing text transformations on files or input streams. It’s often used for find and replace, filtering, and other text manipulation tasks. Extremely valuable for automating configuration file changes.
When to Use:
- Replacing text in configuration files.
- Filtering log files.
- Automating repetitive text editing tasks.
- Performing batch modifications on multiple files.
Basic Syntax:
Terminal window
```
sed [options] 'command' input_file
```

Practical Examples:

# Replace "old_text" with "new_text" in file.txt and print to stdout
sed 's/old_text/new_text/' file.txt

# Replace all occurrences of "old_text" with "new_text" (global replacement)
sed 's/old_text/new_text/g' file.txt

# Replace in place (modifies the file directly) - requires the -i option
sed -i 's/old_text/new_text/g' file.txt

# Delete lines containing "pattern"
sed '/pattern/d' file.txt

# Insert a line before a line containing "pattern"
sed '/pattern/i New line of text' file.txt

# Append a line after a line containing "pattern"
sed '/pattern/a New line of text' file.txt

Sample Output (replace and print to stdout):

This is a new_text example.
Another line with old_text.

Common Options:
- -i: Edit files in place (modifies the original file). Use with extreme caution! It’s recommended to create a backup first. Often used as -i.bak to create a backup with the extension .bak.
- -n: Suppress automatic printing of pattern space. Useful when used with the p command (print).
- -e: Specify multiple commands.
- -f script_file: Read commands from a script file.
- -r: Use extended regular expressions (ERE).

Advanced Usage:

# Replace only on lines starting with "prefix"
sed '/^prefix/s/old_text/new_text/g' file.txt

# Use a different delimiter (useful when the pattern contains forward slashes)
sed 's#old_text#new_text#g' file.txt

# Perform multiple replacements
sed -e 's/old_text1/new_text1/g' -e 's/old_text2/new_text2/g' file.txt

# Read sed commands from a file
echo "s/foo/bar/g" > sed_script.txt
sed -f sed_script.txt file.txt

# Using capture groups (backreferences) - swap two words
echo "Hello World" | sed 's/\(Hello\) \(World\)/\2 \1/'  # Output: World Hello

Tips & Tricks:
- Always back up your files before using sed -i.
- Use sed -n 'p' to print only the lines that match a pattern.
- For complex replacements, consider using awk or perl instead of sed.
- When using variables in sed commands, use double quotes to allow variable expansion. e.g., sed "s/$VAR/replacement/" file.txt
- Use \ to escape special characters like /, $, ^, *, etc.
Troubleshooting:
- **“sed: -e expression #1, char X: unterminated s' command"**: Check for missing delimiters or unescaped special characters in your s` command.
- Changes not being saved: Ensure you’re using the -i option for in-place editing.
- Unexpected behavior: Double-check your regular expressions and escape characters.
- Permission denied: Make sure you have write permissions on the file you are trying to modify with -i.
Related Commands:
- awk: Pattern scanning and processing language.
- grep: Find lines matching a pattern.
- cut: Extract sections from each line of a file.
- tr: Translate or delete characters.

3. awk - Pattern Scanning and Processing Language

Command Overview: awk is a powerful text processing tool that scans input files line by line, searching for patterns and performing actions on matching lines. It’s essentially a programming language designed for text manipulation. Excellent for extracting data, generating reports, and transforming data.
When to Use:
- Extracting specific columns from a file.
- Generating reports from log files.
- Performing calculations on data in files.
- Transforming data from one format to another.

Basic Syntax:

awk [options] 'pattern { action }' input_file

Practical Examples:

# Print the first column of each line
awk '{print $1}' file.txt

# Print the first and third columns, separated by a comma
awk '{print $1 "," $3}' file.txt

# Print lines where the second column is greater than 10
awk '$2 > 10 {print}' file.txt

# Print the number of lines in the file
awk 'END {print NR}' file.txt

# Print lines that contain the word "error"
awk '/error/ {print}' file.txt

Sample Output (print the first column):

Line1
Line2
Line3

Common Options:
- -F fs: Specify the field separator. Default is whitespace.
- -v var=value: Assign a value to a variable.
- -f script_file: Read awk commands from a script file.

Advanced Usage:

# Calculate the sum of the second column
awk '{sum += $2} END {print sum}' file.txt

# Print lines where the first column matches a regular expression
awk '$1 ~ /^A/ {print}' file.txt  # Lines starting with 'A'

# Using variables
awk -v threshold=10 '$2 > threshold {print $1, $2}' file.txt

# Reading awk script from file
echo '{print $1}' > awk_script.awk
awk -f awk_script.awk file.txt

# Example:  Sum the sizes of files listed by `ls -l`, print total in MB
ls -l | awk '{sum += $5} END { print sum / (1024*1024) " MB" }'

Tips & Tricks:
- $0 represents the entire line.
- $1, $2, etc., represent the first, second, etc., fields.
- NF represents the number of fields in the current line.
- NR represents the number of the current record (line).
- Use the BEGIN block to perform actions before processing any input lines.
- Use the END block to perform actions after processing all input lines.
- You can define your own functions within awk scripts.
- For more complex logic, write your awk scripts in separate files and use the -f option.
Troubleshooting:
- “awk: syntax error near line X”: Check for syntax errors in your awk code, such as missing braces or semicolons.
- Incorrect output: Double-check your field separators and regular expressions.
- Variables not being assigned correctly: Ensure you’re using the -v option or assigning variables within the awk script correctly.
- No output: Make sure the pattern you’re searching for actually exists in the input file.
Related Commands:
- sed: Stream editor.
- grep: Find lines matching a pattern.
- cut: Extract sections from each line of a file.
- sort: Sort lines of text files.

4. find - Find Files and Directories

Command Overview: find is a powerful command-line tool for locating files and directories based on various criteria, such as name, size, modification time, and permissions. Essential for searching for configuration files, log files, and other important data.
When to Use:
- Locating configuration files.
- Finding log files older than a certain date.
- Deleting temporary files.
- Changing permissions on multiple files.
Basic Syntax:
Terminal window
```
find [path] [expression]
```

Practical Examples:

# Find all files named "config.txt" in the current directory and its subdirectories
find . -name config.txt

# Find all files with the ".log" extension
find . -name "*.log"

# Find all directories
find . -type d

# Find all files that are larger than 10MB
find . -size +10M

# Find all files modified in the last 7 days
find . -mtime -7

Sample Output (find files named config.txt):

./directory1/config.txt
./config.txt

Common Options:
- .: The current directory (default path if none is specified).
- -name pattern: Search for files with names matching the specified pattern (supports wildcards).
- -type type: Search for files of the specified type (e.g., f for file, d for directory, l for symbolic link).
- -size n[cwbkMG]: Search for files of the specified size (e.g., +10M for greater than 10MB, -10k for less than 10KB).
- -mtime n: Search for files modified n days ago. +n for older than n days, -n for newer than n days.
- -atime n: Search for files accessed n days ago.
- -ctime n: Search for files whose status was changed n days ago.
- -user username: Search for files owned by the specified user.
- -group groupname: Search for files owned by the specified group.
- -perm mode: Search for files with the specified permissions.
- -exec command {} \;: Execute a command on each found file. {} is replaced with the filename. \; terminates the command. Consider using + instead of \; for better performance.
- -ok command {} \;: Like -exec, but prompts for confirmation before executing the command.

Advanced Usage:

# Find all files larger than 10MB and delete them (USE WITH CAUTION!)
find . -size +10M -delete

# Find all files with the ".log" extension and change their permissions to 644
find . -name "*.log" -exec chmod 644 {} \;

# Find all files owned by user "john" and change their ownership to "jane"
find . -user john -exec chown jane {} \;

# Find files modified in the last day and compress them using gzip
find . -mtime -1 -exec gzip {} \;

# Find files and then use xargs for parallel processing (more efficient than -exec for many files)
find . -name "*.txt" -print0 | xargs -0 grep "pattern"

Tips & Tricks:
- Use -print0 with xargs -0 to handle filenames containing spaces or special characters.
- Use -ok instead of -exec when you’re unsure about the command you’re about to execute.
- For complex searches, consider using grep to filter the results of find.
- Use locate for faster searches if you have a regularly updated mlocate database. However, locate might not show recently created files until the database is updated.
- The -path predicate can be used to exclude entire directory trees.
- Combine -o (OR) and -a (AND) to create complex search criteria.
Troubleshooting:
- **“find: missing argument to -exec'"**: Ensure you have the ;at the end of the-exec` command.
- Permission denied: You may not have permissions to access certain directories or files. Try running find with sudo.
- Unexpected results: Double-check your search criteria and make sure you’re using the correct options.
- Slow performance: Avoid using find on very large file systems if possible. Consider using locate or optimizing your search criteria.
Related Commands:
- locate: Find files by name (faster but requires an updated database).
- grep: Find lines matching a pattern within files.
- xargs: Build and execute command lines from standard input.
- chmod: Change file permissions.
- chown: Change file ownership.
- rm: Remove files or directories (use with extreme caution with find -exec rm ...).

5. systemctl - Control the Systemd System and Service Manager

Command Overview: systemctl is the primary command for managing the systemd system and service manager. It allows you to control the state of services, manage system boot processes, and inspect the system’s overall health. Fundamental for managing services in modern Linux distributions.
When to Use:
- Starting, stopping, restarting, and reloading services.
- Enabling or disabling services to start at boot.
- Checking the status of services.
- Managing system targets (runlevels).
- Inspecting the systemd journal (logs).
Basic Syntax:
Terminal window
```
systemctl [command] [unit]
```

Practical Examples:

# Start a service
sudo systemctl start nginx

# Stop a service
sudo systemctl stop nginx

# Restart a service
sudo systemctl restart nginx

# Reload a service (e.g., after changing its configuration)
sudo systemctl reload nginx

# Check the status of a service
systemctl status nginx

# Enable a service to start at boot
sudo systemctl enable nginx

# Disable a service from starting at boot
sudo systemctl disable nginx

Sample Output (systemctl status nginx):

● nginx.service - A high performance web server and a reverse proxy server
   Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled)
   Active: active (running) since Tue 2023-10-24 10:00:00 UTC; 1h ago
     Docs: man:nginx(8)
  Process: 12345 ExecStart=/usr/sbin/nginx -g daemon off; (code=exited, status=0/SUCCESS)
 Main PID: 67890 (nginx)
    Tasks: 2 (limit: 1000)
   Memory: 5.2M
   CGroup: /system.slice/nginx.service
           └─67890 nginx: master process /usr/sbin/nginx -g daemon off;

Oct 24 10:00:00 hostname systemd[1]: Started A high performance web server and a reverse proxy server.

Common Options:
- start: Start a unit.
- stop: Stop a unit.
- restart: Restart a unit.
- reload: Reload a unit’s configuration.
- status: Show the status of a unit.
- enable: Enable a unit to start at boot.
- disable: Disable a unit from starting at boot.
- is-enabled: Check if a unit is enabled.
- mask: Mask a unit (prevent it from being started manually or automatically).
- unmask: Unmask a unit.
- list-units: List all active units.
- list-unit-files: List all installed unit files.
- show unit: Show detailed information about a unit.
- journalctl -u unit: Show the journal logs for a specific unit.
- daemon-reload: Reload the systemd daemon configuration (after modifying unit files).

Advanced Usage:

# Check the status of all services
systemctl list-units --type=service

# Show the dependencies of a service
systemctl list-dependencies nginx

# Mask a service to prevent it from being started
sudo systemctl mask nginx

# Unmask a service
sudo systemctl unmask nginx

# Show the journal logs for a specific service, following the logs in real-time
journalctl -u nginx -f

# Show the journal logs for a specific service, for the last hour
journalctl -u nginx --since "1 hour ago"

# Create a custom systemd service unit file (e.g., /etc/systemd/system/my-app.service)
# [Unit]
# Description=My Application
# After=network.target

# [Service]
# ExecStart=/path/to/my/application
# Restart=on-failure
# User=myuser
# Group=mygroup

# [Install]
# WantedBy=multi-user.target

# Reload systemd daemon, then enable and start the service
sudo systemctl daemon-reload
sudo systemctl enable my-app.service
sudo systemctl start my-app.service

Tips & Tricks:
- Use systemctl daemon-reload after making changes to unit files.
- Use journalctl to view logs and troubleshoot service issues.
- Pay attention to the After= and Requires= directives in unit files to manage service dependencies.
- Use WantedBy= in the [Install] section to specify the target that should start the service.
- Use Restart=on-failure in the [Service] section to automatically restart a service if it crashes.
- Check the systemd documentation for more advanced configuration options.
Troubleshooting:
- “Failed to start service: Unit not found”: Ensure the unit file exists and is correctly named.
- Service fails to start: Check the journal logs (journalctl -u service_name) for error messages. Verify the ExecStart path in the unit file. Ensure the user specified in User= has the necessary permissions.
- Service starts but doesn’t function correctly: Check the service’s configuration file for errors.
- “Failed to reload daemon: Access denied”: You need to run systemctl daemon-reload with sudo.
Related Commands:
- journalctl: View systemd journal logs.
- systemd-analyze: Analyze system boot performance.
- ps: Process status.
- top: Display top CPU-using processes.
- init: System initialization (older systems, systemd replaced init).

This cheatsheet provides a strong foundation for using these essential Linux commands for configuration management. Remember to always test commands in a non-production environment before applying them to production systems and to back up your data regularly. Good luck!