How to use journalctl to pinpoint why a scheduled rsync job stalls during authentication

rsync scheduled jobs that stall on authentication: a journalctl‑first approach

When an rsync job launched by a systemd timer stops at the authentication step, the first place to look is the journal. The logs contain every attempt to open a connection, every key exchange, and every error message that the ssh daemon emits. With the right filters you can turn a vague “stalled” symptom into a concrete failure reason.

1. Know the unit that runs your rsync

Most modern distributions ship a template unit [email protected] that is started by a timer such as [email protected]. The instance name encodes the target host, e.g. [email protected]. Check the status:

systemctl status [email protected]

The output shows the unit’s name, its last start time, and any immediate exit codes. If the unit never reaches the “running” state, the journal will contain the reason.

2. Pull the rsync logs with journalctl

Once you know the unit, fetch its messages:

journalctl -u [email protected] -p err -S
  • -u limits the output to the unit.
  • -p err shows only error‑level entries, which is usually enough for authentication problems.
  • -S sorts by timestamp, newest first.

If the unit is a simple wrapper that calls rsync directly, the journal will contain the command line and any output that rsync writes to stdout or stderr. Look for lines that start with rsync: or rsync: auth: .

3. Correlate with the sshd logs

rsync uses SSH for remote authentication. The SSH daemon logs authentication events under the sshd unit. To see what happened on the remote side, run:

journalctl -u sshd -p err -S --since "2026-10-05 00:00" --until "2026-10-05 02:00"

Adjust the time window to match the timer’s schedule. Search for:

sshd[12345]: Authentication failure
sshd[12345]: Connection closed by remote host

If the client side was waiting for a password prompt, the server will log “Connection closed by remote host” or “Connection closed by 192.0.2.5 port 22”.

4. Use tags and priority filters

journalctl can filter by tag (-t) or priority (-p). If your rsync unit sets a tag, e.g. -t rsync, you can isolate it:

journalctl -t rsync -p err -S

If you’re not sure whether the unit uses a tag, search the unit file:

grep -i 'Tag=' /etc/systemd/system/[email protected]

5. Inspect the key exchange

A common cause of stalls is a missing or unreadable private key. The sshd log will contain:

sshd[12345]: Authentication refused: bad ownership or modes for private key

Check the key’s permissions on the client:

ls -l ~/.ssh/id_rsa

The file must be -rw------- (600). If the key is in a directory that is not owned by the user, ssh will refuse to use it.

6. Verify the SSH agent

If the rsync job relies on an SSH agent, the agent must be running in the same user session that starts the timer. Systemd can start an agent with a Service unit:

[Unit]
Description=SSH Agent for rsync
After=network.target

[Service]
Type=forking
ExecStart=/usr/bin/ssh-agent -s
Environment=SSH_AUTH_SOCK=%t/ssh-agent.socket

Then the rsync unit can reference the socket:

Environment=SSH_AUTH_SOCK=/run/user/1000/ssh-agent.socket

If the agent is not running, the journal will show:

rsync[12345]: auth: cannot open private key

7. Check host key verification

When connecting to a new host, ssh will ask to confirm the host key. In a non‑interactive job, this will block indefinitely. The sshd log will show:

sshd[12345]: Connection closed by 192.0.2.5 port 22

On the client side, you’ll see:

rsync[12345]: auth: cannot open private key

or

rsync[12345]: auth: connection closed

The usual fix is to pre‑populate the known‑hosts file or disable strict host key checking in the rsync command line with -e "ssh -o StrictHostKeyChecking=no"—but that’s a trade‑off you should weigh carefully.


If you’ve followed these steps and the logs still don’t explain the stall, double‑check that the timer actually fires by looking at the timer’s status:

systemctl list-timers | grep rsync

Sometimes the timer is disabled, or the unit file has a typo that prevents it from starting. Once the timer is firing, the journal will give you the missing piece of the puzzle.

TAGS: rsync, systemd, ssh, troubleshooting, journalctl


See also