rsync scheduled jobs that stall on authentication: a journalctl‑first approach
When an rsync job launched by a systemd timer stops at the authentication step, the first place to look is the journal. The logs contain every attempt to open a connection, every key exchange, and every error message that the ssh daemon emits. With the right filters you can turn a vague “stalled” symptom into a concrete failure reason.
1. Know the unit that runs your rsync
Most modern distributions ship a template unit [email protected] that is started by a timer such as [email protected]. The instance name encodes the target host, e.g. [email protected]. Check the status:
systemctl status [email protected]
The output shows the unit’s name, its last start time, and any immediate exit codes. If the unit never reaches the “running” state, the journal will contain the reason.
2. Pull the rsync logs with journalctl
Once you know the unit, fetch its messages:
journalctl -u [email protected] -p err -S
-ulimits the output to the unit.-p errshows only error‑level entries, which is usually enough for authentication problems.-Ssorts by timestamp, newest first.
If the unit is a simple wrapper that calls rsync directly, the journal will contain the command line and any output that rsync writes to stdout or stderr. Look for lines that start with rsync: or rsync: auth: .
3. Correlate with the sshd logs
rsync uses SSH for remote authentication. The SSH daemon logs authentication events under the sshd unit. To see what happened on the remote side, run:
journalctl -u sshd -p err -S --since "2026-10-05 00:00" --until "2026-10-05 02:00"
Adjust the time window to match the timer’s schedule. Search for:
sshd[12345]: Authentication failure
sshd[12345]: Connection closed by remote host
If the client side was waiting for a password prompt, the server will log “Connection closed by remote host” or “Connection closed by 192.0.2.5 port 22”.
4. Use tags and priority filters
journalctl can filter by tag (-t) or priority (-p). If your rsync unit sets a tag, e.g. -t rsync, you can isolate it:
journalctl -t rsync -p err -S
If you’re not sure whether the unit uses a tag, search the unit file:
grep -i 'Tag=' /etc/systemd/system/[email protected]
5. Inspect the key exchange
A common cause of stalls is a missing or unreadable private key. The sshd log will contain:
sshd[12345]: Authentication refused: bad ownership or modes for private key
Check the key’s permissions on the client:
ls -l ~/.ssh/id_rsa
The file must be -rw------- (600). If the key is in a directory that is not owned by the user, ssh will refuse to use it.
6. Verify the SSH agent
If the rsync job relies on an SSH agent, the agent must be running in the same user session that starts the timer. Systemd can start an agent with a Service unit:
[Unit]
Description=SSH Agent for rsync
After=network.target
[Service]
Type=forking
ExecStart=/usr/bin/ssh-agent -s
Environment=SSH_AUTH_SOCK=%t/ssh-agent.socket
Then the rsync unit can reference the socket:
Environment=SSH_AUTH_SOCK=/run/user/1000/ssh-agent.socket
If the agent is not running, the journal will show:
rsync[12345]: auth: cannot open private key
7. Check host key verification
When connecting to a new host, ssh will ask to confirm the host key. In a non‑interactive job, this will block indefinitely. The sshd log will show:
sshd[12345]: Connection closed by 192.0.2.5 port 22
On the client side, you’ll see:
rsync[12345]: auth: cannot open private key
or
rsync[12345]: auth: connection closed
The usual fix is to pre‑populate the known‑hosts file or disable strict host key checking in the rsync command line with -e "ssh -o StrictHostKeyChecking=no"—but that’s a trade‑off you should weigh carefully.
If you’ve followed these steps and the logs still don’t explain the stall, double‑check that the timer actually fires by looking at the timer’s status:
systemctl list-timers | grep rsync
Sometimes the timer is disabled, or the unit file has a typo that prevents it from starting. Once the timer is firing, the journal will give you the missing piece of the puzzle.
TAGS: rsync, systemd, ssh, troubleshooting, journalctl
See also
- Fixing GNOME’s broken audio output after an ALSA upgrade
- When systemd‑resolved ignores /etc/hosts entries for local subdomains
- How to Fix Permission‑Denied Errors When Mounting Host Paths in Rootless Podman Containers
- When Ubuntu’s `apt dist-upgrade` Removes Your Custom Kernel Module – A Practical Pinning Fix
- When initramfs drops into emergency mode after a kernel upgrade: how to recover the root partition on Ubuntu 24.04