When systemd‑resolved stops answering local hostnames: a quick hunt for /etc/hosts overrides

On a busy home‑lab or a small office server, the first thing you notice when ping myapp.local suddenly fails is that systemd‑resolved has stopped resolving the name. The culprit is almost always a mis‑configured or stale entry in /etc/hosts. This post walks through a systematic hunt, shows how to keep the file tidy, and highlights security implications of host‑name resolution.

Why /etc/hosts still matters

systemd‑resolved reads /etc/hosts before querying DNS. It treats the file as a local override list, so any line that matches a query will short‑circuit DNS. That design is handy for internal services, but it also means that a stray duplicate or a typo can silently break name resolution for the whole system.

The file is parsed by systemd-resolve each time it starts or when systemd‑resolved receives a SIGHUP. A change that doesn’t trigger a reload will stay invisible until the next restart, which explains why the issue can appear “out of the blue”.

Quick checklist

Verify the service status
```
systemctl status systemd-resolved
```
Look for Active: active (running) and any error messages in the journal.
Confirm the resolver is in use
```
resolvectl status
```
The DNS Servers line should list your local DNS or 127.0.0.53. If it shows none, the stub resolver is disabled.
Test a known local hostname
```
ping -c 1 myapp.local
```
If it fails with Name or service not known, the problem is local.
Check /etc/hosts for the hostname
```
grep -i myapp.local /etc/hosts
```
A missing line or a commented‑out entry is a red flag.
Look for duplicate IPs
```
grep -i myapp /etc/hosts | awk '{print $1}' | sort | uniq -d
```
Duplicate IPs can cause ambiguity; the first match wins, but the second remains a silent source of confusion.

Common pitfalls and how to fix them

Symptom	Likely cause	Fix
`ping myapp.local` resolves to the wrong IP	Duplicate or conflicting entry	Remove the older line or consolidate into a single, authoritative record.
`systemd-resolved` never reloads after editing `/etc/hosts`	Service not reloaded	`systemctl restart systemd-resolved` or `systemctl reload systemd-resolved`.
Hostname resolves via DNS instead of `/etc/hosts`	Entry commented out or wrong order	Ensure the line is not prefixed with `#` and appears before any wildcard or generic entries.
Security audit flags a “public” IP in `/etc/hosts`	Unintended exposure	Verify that only internal, non‑public IPs are listed.

Example: cleaning up a cluttered `/etc/hosts`

# Backup first
sudo cp /etc/hosts /etc/hosts.bak

# Show all entries for the domain
grep -i myapp /etc/hosts

# Suppose we see:
# 192.168.1.10  myapp.local
# 10.0.0.5     myapp.local

# Decide which IP is correct
# Remove the wrong one
sudo sed -i '/10\.0\.0\.5/d' /etc/hosts

# Verify
grep -i myapp /etc/hosts
# 192.168.1.10  myapp.local

# Reload resolver
sudo systemctl reload systemd-resolved

Security considerations

Least‑privilege for /etc/hosts
The file should be owned by root and readable by all, but writable only by root. A mis‑configured ACL can let an unprivileged user add entries that redirect traffic to malicious IPs.
```
sudo chown root:root /etc/hosts
sudo chmod 644 /etc/hosts
```
Avoid exposing internal names to the public DNS
If you accidentally publish an internal hostname in a public zone, attackers can map your internal network. Keep /etc/hosts entries strictly for local use and never sync them to external zones.
Use systemd‑resolved’s DNSSEC validation
Enable DNSSEC in /etc/systemd/resolved.conf to guard against spoofed DNS responses that might bypass /etc/hosts if you rely on DNS for other names.
```
[Resolve]
DNSSEC=yes
```
Audit changes
Track who edits /etc/hosts with auditd or a simple git repository. A change log helps spot accidental or malicious edits.
```
sudo apt install auditd
sudo auditctl -w /etc/hosts -p wa -k hosts_change
```

When `/etc/hosts` is not the culprit

If /etc/hosts looks clean but the problem persists, consider:

DNS cache corruption – systemd-resolve --flush-caches clears the in‑memory cache.
Network interface changes – a new NIC with a different IP may cause systemd‑resolved to rebuild its view. Restart the service after such changes.
Misconfigured resolv.conf – if /etc/resolv.conf points to a non‑existent DNS server, systemd‑resolved may fall back to /etc/hosts. Ensure /run/systemd/resolve/resolv.conf is a symlink to /etc/resolv.conf and points to a valid server.

A minimal reproducible test

Create a temporary /etc/hosts entry and verify resolution:

echo "127.0.0.1   test.local" | sudo tee -a /etc/hosts
ping -c 1 test.local
# → 127.0.0.1

# Remove it
sudo sed -i '/test.local/d' /etc/hosts

If this works, the issue lies elsewhere; if it fails, the problem is with the resolver configuration or the file’s permissions.

Takeaway

systemd‑resolved is a powerful tool, but its reliance on /etc/hosts means that a small typo can bring down local name resolution. By keeping the file lean, securing its permissions, and using the built‑in reload commands, you can avoid downtime and maintain a predictable network stack.

For deeper dives into systemd internals, the official docs are a solid reference: https://systemd.io/ and the kernel’s networking guide at https://www.kernel.org/doc/html/latest/networking/.