Network Issues
The key to network troubleshooting is layered diagnosis, progressively narrowing the scope. This article follows the order from the physical layer to the application layer, introducing a systematic network troubleshooting approach.
Troubleshooting Approach: Bottom to Top
Section titled “Troubleshooting Approach: Bottom to Top”Network troubleshooting should follow the OSI model from the bottom up:
- Physical / Link layer — Cables, NIC, link status
- Network layer — IP address, routing, gateway
- DNS — Name resolution
- Firewall — firewalld / iptables rules
- Transport / Application layer — Ports, service listening status
Principle: Do not skip lower layers and jump straight to higher layers. If the IP address is misconfigured, troubleshooting DNS is pointless.
Step 1: Check the Physical Link
Section titled “Step 1: Check the Physical Link”$ ip link showLook for state UP vs. state DOWN. If the interface is in DOWN state:
$ sudo ip link set eth0 up$ lspci | grep -i ethernet$ lsmod | grep -i e1000 # Example: Intel NIC$ sudo ethtool eth0Look for Link detected: yes in the output. If it shows no, there is a physical connection problem (cable, switch port, virtual network configuration, etc.).
$ ip -s link show eth0Watch the errors and dropped counters. A high count may indicate hardware issues or a duplex mismatch.
Step 2: Check IP Configuration
Section titled “Step 2: Check IP Configuration”$ ip addr show$ ip route showConfirm the following:
- The interface has the correct IP address
- The subnet mask is correct
- A default route exists (
default via x.x.x.x)
$ ping -c 3 $(ip route show default | awk '{print $3}')If pinging the gateway fails, the problem is in the local network configuration or physical connection.
Using nmcli to Check and Configure Networking
Section titled “Using nmcli to Check and Configure Networking”$ nmcli connection show$ nmcli connection show "Wired connection 1"$ nmcli device status$ sudo nmcli connection down "Wired connection 1" && sudo nmcli connection up "Wired connection 1"Common IP Configuration Issues
Section titled “Common IP Configuration Issues”DHCP failed to obtain an address:
$ sudo nmcli connection modify "Wired connection 1" ipv4.method auto$ sudo nmcli connection up "Wired connection 1"$ journalctl -u NetworkManager --since "5 minutes ago" | grep -i dhcpSetting a static IP:
$ sudo nmcli connection modify "Wired connection 1" \ ipv4.method manual \ ipv4.addresses "192.168.1.100/24" \ ipv4.gateway "192.168.1.1" \ ipv4.dns "8.8.8.8 8.8.4.4"$ sudo nmcli connection up "Wired connection 1"Step 3: Check DNS Resolution
Section titled “Step 3: Check DNS Resolution”If the IP layer is working (you can ping IP addresses) but you cannot access domain names:
$ cat /etc/resolv.conf$ dig google.com$ dig @8.8.8.8 google.com$ dig +short google.com$ dig -x 8.8.8.8Common DNS Issues
Section titled “Common DNS Issues”resolv.conf being overwritten:
NetworkManager manages /etc/resolv.conf. If manual changes keep being overwritten:
$ sudo nmcli connection modify "Wired connection 1" ipv4.dns "8.8.8.8 114.114.114.114"$ sudo nmcli connection up "Wired connection 1"Slow DNS resolution:
$ time dig google.comIf latency is high, the DNS server may be unreachable or responding slowly. Try a different DNS server:
$ dig @114.114.114.114 google.com$ dig @223.5.5.5 google.comLocal hosts file issues:
$ cat /etc/hosts$ grep hosts /etc/nsswitch.confThe normal configuration is typically: hosts: files dns myhostname
Step 4: Check the Firewall
Section titled “Step 4: Check the Firewall”The firewall is one of the most common causes of network issues on EL systems. If a service is running but cannot be accessed externally, nine times out of ten it is the firewall.
$ sudo firewall-cmd --state$ sudo firewall-cmd --list-all$ sudo firewall-cmd --list-services$ sudo firewall-cmd --list-portsTemporarily Allow a Port for Testing
Section titled “Temporarily Allow a Port for Testing”$ sudo firewall-cmd --add-port=8080/tcp$ sudo firewall-cmd --add-service=httpPermanent Rules
Section titled “Permanent Rules”$ sudo firewall-cmd --add-port=8080/tcp --permanent$ sudo firewall-cmd --reload$ sudo firewall-cmd --add-service=http --permanent$ sudo firewall-cmd --add-service=https --permanent$ sudo firewall-cmd --reloadDiagnosing Traffic Blocked by firewalld
Section titled “Diagnosing Traffic Blocked by firewalld”$ sudo firewall-cmd --set-log-denied=all$ sudo journalctl -f | grep REJECTAfter testing, disable logging:
$ sudo firewall-cmd --set-log-denied=offQuickly Ruling Out the Firewall
Section titled “Quickly Ruling Out the Firewall”If you need to quickly determine whether the firewall is the issue:
$ sudo firewall-cmd --zone=trusted --change-interface=eth0Warning: This completely opens the interface. Use only for temporary testing and restore immediately afterward.
$ sudo firewall-cmd --zone=public --change-interface=eth0Step 5: Check Services and Ports
Section titled “Step 5: Check Services and Ports”$ sudo ss -tlnp$ sudo ss -ulnp$ sudo ss -tlnp | grep :80If the service is not listening on the expected port, check the service status:
$ systemctl status nginx$ journalctl -u nginx -n 30 --no-pagerTesting HTTP Services with curl
Section titled “Testing HTTP Services with curl”$ curl -v http://localhost/$ curl -I http://localhost/$ curl -v http://localhost:8080/Using traceroute to Trace Routes
Section titled “Using traceroute to Trace Routes”$ sudo dnf install -y traceroute$ traceroute 8.8.8.8$ sudo traceroute -T -p 80 example.comUsing tcpdump for Packet Capture
Section titled “Using tcpdump for Packet Capture”$ sudo tcpdump -i eth0 port 80 -nn -c 20$ sudo tcpdump -i eth0 port 443 -w /tmp/capture.pcap -c 100Common Network Failure Scenarios
Section titled “Common Network Failure Scenarios”Scenario 1: Server is Pingable but HTTP is Not Working
Section titled “Scenario 1: Server is Pingable but HTTP is Not Working”Troubleshooting order:
- Confirm the web service is running:
systemctl status nginx - Confirm the service is listening on the correct port:
ss -tlnp | grep :80 - Confirm local access works:
curl http://localhost/ - Check the firewall:
firewall-cmd --list-all - Check SELinux:
ausearch -m avc -ts recent
Scenario 2: DNS Resolves but Connection Times Out
Section titled “Scenario 2: DNS Resolves but Connection Times Out”$ timeout 5 bash -c 'echo > /dev/tcp/TARGET_IP/TARGET_PORT' && echo "Port reachable" || echo "Port unreachable"Possible causes:
- Remote firewall is blocking traffic
- Routing issue (use traceroute to diagnose)
- ISP or intermediate network device blocking
Scenario 3: NetworkManager Connection Keeps Disconnecting and Reconnecting
Section titled “Scenario 3: NetworkManager Connection Keeps Disconnecting and Reconnecting”$ journalctl -u NetworkManager --since "30 minutes ago" | grep -E '(connect|disconnect|fail)'$ dmesg | grep -i 'link'Possible causes:
- Loose cable connection
- NIC driver issue
- DHCP lease conflict
Scenario 4: Network Configuration Lost After Migration
Section titled “Scenario 4: Network Configuration Lost After Migration”After migrating from CentOS, network interface names or configuration files may change:
$ ls /etc/NetworkManager/system-connections/$ ls /etc/sysconfig/network-scripts/ifcfg-* 2>/dev/null$ sudo nmcli connection add type ethernet con-name "Primary" ifname eth0$ sudo nmcli connection modify "Primary" ipv4.method auto$ sudo nmcli connection up "Primary"Network Troubleshooting Quick Reference
Section titled “Network Troubleshooting Quick Reference”| Target | Command |
|---|---|
| View interface status | ip link show |
| View IP addresses | ip addr show |
| View routing table | ip route show |
| Test connectivity | ping -c 3 target |
| DNS resolution | dig domain |
| View listening ports | ss -tlnp |
| Trace route | traceroute target |
| Firewall rules | firewall-cmd --list-all |
| View connections | nmcli connection show |
| Packet capture | tcpdump -i interface port port-number |
| HTTP test | curl -v URL |
| NetworkManager logs | journalctl -u NetworkManager |