Table of contents
- Issue 1: Unable to Start a Service
- Issue 2: High CPU Usage
- Issue 3: Network Connectivity Issues Between Servers
- Issue 4: Unable to Mount a Filesystem
- Issue 5: Filesystem corrupted
- Issue 6: Canβt cd to the directory even if the user has sudo privileges
- Issue 7: Running Out of Memory
- Issue 8: Add/Extend the Swap Space
- Issue 9: Unable to Run Certain Commands
- Issue 10: System Unexpectedly reboot and process restart?
Issue 1: Unable to Start a Service
π οΈ Approach / Solution:
βββ Check if the service is installed
βββ Verify the service configuration file
βββ Check the service status using systemctl or other command
βββ Inspect the service logs for any errors
βββ Ensure there are no port conflicts
βββ Review firewall rules and SELinux settings
βββ Restart the service and check for error messages
βββ Inspect system resource usage with tools like top or htop
βββ ...
Issue 2: High CPU Usage
π οΈ Approach / Solution:
βββ Identify the process causing high CPU usage using top or htop
βββ Check if the issue is intermittent or continuous
βββ Review logs for any error messages or known issues
βββ Inspect running processes and their resource consumption
βββ Investigate potential malware or unauthorized processes
βββ Consider optimizing or scaling the application
βββ Monitor system metrics over time to identify patterns
βββ Apply performance tuning based on the specific application
βββ ...
Issue 3: Network Connectivity Issues Between Servers
π οΈ Approach / Solution:
βββ Check if other servers on the same network are accessible
βββ Verify firewall rules on both source and destination servers
βββ Inspect routing tables to ensure correct routes are set
βββ Use tools like traceroute or mtr to trace the network path
βββ Check for any network hardware failures or misconfigurations
βββ Review system logs for network-related errors
βββ Test connectivity using tools like telnet or nc
βββ Investigate potential DNS or hostname resolution problems
βββ Consider network segmentation or VLAN configurations
βββ ...
Issue 4: Unable to Mount a Filesystem
π οΈ Approach / Solution:
bash
βββ Check if the filesystem is specified in /etc/fstab
βββ Verify the device path and UUID in /etc/fstab
βββ Ensure the filesystem type is correct
βββ Check for errors in /var/log/messages or dmesg
βββ Confirm that the device is accessible and not failing
βββ Use the mount command manually to check for errors
βββ Investigate if the filesystem needs repair (fsck)
βββ Inspect disk space on the target mount point
βββ Check for any SELinux or AppArmor restrictions
βββ ...
Issue 5: Filesystem corrupted
π οΈ Approach / Solution:
βββ One of the error that causes the system unable to BOOT UP
βββ Check /var/log/messages, dmesg, and other log files
βββ If we have bad sector logs, we have to run fsck
β βββ True:
β β βββ Reboot the system into rescue mode by booting it from CDROM by applying ISO
β β βββ Proceed with option 1, which mounts the original root filesystem under /mnt/sysimage
β β βββ Edit fstab entries or create a new file with the help of blkid and reboot
βββ ...
Issue 6: Canβt cd to the directory even if the user has sudo privileges
π οΈ Approach / Solution:
βββ Reasons and Resolution
β βββ Directory does not exist
β βββ Pathname conflict: relative vs absolute path
β βββ Parent directory permission/ownership
β βββ Doesn't have executable permission on the target directory
β βββ Hidden directory
βββ ...
Issue 7: Running Out of Memory
π οΈ Approach / Solution:
βββ Types
β βββ Cache (L1, L2, L3)
β βββ RAM
β β βββ Usage
β β β βββ #free -h
β β β β βββ Total (Total assigned memory)
β β β β βββ Used (Total actual used memory)
β β β β βββ Free (Actual free memory)
β β β β βββ Shared (Shared Memory)
β β β β βββ Buff/Cache (Pages cache memory)
β β β β βββ Available (Memory can be freed)
β β β βββ /proc/meminfo
β β β β βββ file active
β β β β βββ file inactive
β β β β βββ anon active
β β β β βββ anon inactive
β βββ Swap (Virtual Memory)
βββ Resolution
β βββ Identify the processes that are using high memory using top, htop, ps, etc.
β βββ Check the OOM in logs and also check if there is a memory commitment in sysctl.conf
β βββ Kill or restart the process/service
β βββ Prioritize the process using nice
β βββ Add/Extend the swap space
β βββ Add more physical more RAM
βββ ...
Issue 8: Add/Extend the Swap Space
π οΈ Approach / Solution:
βββ Due to running out of memory, we would need to add more swap space
β βββ Create a file with #dd, as it will reserve the blocks of disk for the swap file
β βββ Set permission 600 and give root ownership
β βββ #mkswap
β βββ Now Turned swap on #swapon
β βββ fstab entry for persistence
βββ ...
Issue 9: Unable to Run Certain Commands
π οΈ Approach / Solution:
βββ Troubleshooting and Resolution
β βββ command
β β βββ Could be the system-related command which non-root user does not have the access
β β βββ Could be the user-defined script/command
β βββ Troubleshooting
β β βββ permission/ownership of the command/script
β β βββ sudo permission
β β βββ absolute/relative path of command/script
β β βββ not defined in user $PATH variable
β β βββ command is not installed
β β βββ command library is missing or deleted
βββ ...
Issue 10: System Unexpectedly reboot and process restart?
π οΈ Approach / Solution:
βββ Troubleshooting and Resolution
β βββ System reboot/crash reasons
β β βββ CPU stress
β β βββ RAM stress
β β βββ Kernel fault
β β βββ Hardware fault
β βββ Process restart
β β βββ System reboot
β β βββ Restart itself
β β βββ Watchdog application
β β β βββ To prevent high stress on system resources
β β β βββ If the application is causing stress, so it will restart or terminate
β βββ Troubleshooting
β β βββ After logged in, check the status by using commands like uptime, top, dmesg, journalctl, iostat -xz 1
β β βββ syslog.log, boot.log, dmesg, messages.log, etc
β β βββ custom log path of application
β β βββ if not completely accessible, so take the virtual console like from ILO, IDRAC, etc
β β βββ open a case and reach out a vendor
βββ ...
Β