Running sshd with Privilege Seperation drops connection on password change

[Nasim, Kam]

Hi folks,

I came across this issue on both stock CentOS(v6.4) and Ubuntu(14.04 LTS) and was wondering if any of you have seen it.
As far as I can tell this seems like a day-1 bug to me.

PROBLEM:
If I expire a linux user's password (passwd -e <user>) and then log in via ssh, it will prompt you for a password change.
On changing the password successfully, sshd will drop the connection and client has to reconnect.


ANALYSIS:

Looking at sshd debug logs, it would appear that the child process that runs passwd SIGCHLD's to the parent which appears to be treating that signal as a SIGTERM:

Dec 3 18:36:17 knasim-ubuntu1 passwd[3152]: pam_unix(passwd:chauthtok): password changed for wrs <<<<<<<
Dec 3 18:36:17 knasim-ubuntu1 sshd[3151]: debug1: Received SIGCHLD. <<<<<<<<<<<<<<<<<<
Dec 3 18:36:17 knasim-ubuntu1 sshd[3151]: debug1: session_by_pid: pid 3152
Dec 3 18:36:17 knasim-ubuntu1 sshd[3151]: debug1: session_exit_message: session 0 channel 0 pid 3152
Dec 3 18:36:17 knasim-ubuntu1 sshd[3151]: debug2: channel 0: request exit-status confirm 0 <<<<<<<<<<<


By default, in sshd "Privilege Separation" is enabled. What does this mean?

It means that sshd spawns an unprivileged process that does basic authentication and that guy spawns a child process running as root to run passwd (in this case).

After password change, the child SIGCHLD's the parent which instead of handling it properly treats it like a SIGTERM and closes the socket.

The following process tree should give a better idea of the process nestings:

With Privilege Separation:
***@knasim-ubuntu1:~$ pstree
init─┬─acpid
├─atd
├─cron
├─dbus-daemon
├─dhclient
├─7*[getty]
├─rsyslogd───3*[{rsyslogd}]
├─sshd───bash───ssh
├─sshd───bash───pstree
├─sshd───sshd───sshd───passwd
├─systemd-logind
├─systemd-udevd
├─upstart-file-br
├─upstart-socket-
└─upstart-udev-br

Without Privilege Separation:
init─┬─acpid
├─atd
├─cron
├─dbus-daemon
├─dhclient
├─7*[getty]
├─rsyslogd───3*[{rsyslogd}]
├─sshd─┬─sshd───bash───ssh
│ ├─sshd───bash───pstree
│ └─sshd───sshd
├─systemd-logind
├─systemd-udevd
├─upstart-file-br
├─upstart-socket-
└─upstart-udev-br

If I disable Privilege Seperation ("UsePrivilegeSeperation no") in sshd config then the problem goes away but that opens up a security loophole where the process is running at root privilege even prior to authentication.


What do you guys think? Have others come across this? Is there a patch available for this?

Thanks,
Kam

[Damien Miller]

Hi,

We'll need to see a full debug log from the server - it's not possible to
see what is going one from the abridged ones you sent.

-d

Post by Nasim, Kam
Hi folks,
I came across this issue on both stock CentOS(v6.4) and Ubuntu(14.04 LTS) and was wondering if any of you have seen it.
As far as I can tell this seems like a day-1 bug to me.
If I expire a linux user's password (passwd -e <user>) and then log in via ssh, it will prompt you for a password change.
On changing the password successfully, sshd will drop the connection and client has to reconnect.
Dec 3 18:36:17 knasim-ubuntu1 passwd[3152]: pam_unix(passwd:chauthtok): password changed for wrs <<<<<<<
Dec 3 18:36:17 knasim-ubuntu1 sshd[3151]: debug1: Received SIGCHLD. <<<<<<<<<<<<<<<<<<
Dec 3 18:36:17 knasim-ubuntu1 sshd[3151]: debug1: session_by_pid: pid 3152
Dec 3 18:36:17 knasim-ubuntu1 sshd[3151]: debug1: session_exit_message: session 0 channel 0 pid 3152
Dec 3 18:36:17 knasim-ubuntu1 sshd[3151]: debug2: channel 0: request exit-status confirm 0 <<<<<<<<<<<
By default, in sshd "Privilege Separation" is enabled. What does this mean?
It means that sshd spawns an unprivileged process that does basic authentication and that guy spawns a child process running as root to run passwd (in this case).
After password change, the child SIGCHLD's the parent which instead of handling it properly treats it like a SIGTERM and closes the socket.
init─┬─acpid
├─atd
├─cron
├─dbus-daemon
├─dhclient
├─7*[getty]
├─rsyslogd───3*[{rsyslogd}]
├─sshd───bash───ssh
├─sshd───bash───pstree
├─sshd───sshd───sshd───passwd
├─systemd-logind
├─systemd-udevd
├─upstart-file-br
├─upstart-socket-
└─upstart-udev-br
init─┬─acpid
├─atd
├─cron
├─dbus-daemon
├─dhclient
├─7*[getty]
├─rsyslogd───3*[{rsyslogd}]
├─sshd─┬─sshd───bash───ssh
│ ├─sshd───bash───pstree
│ └─sshd───sshd
├─systemd-logind
├─systemd-udevd
├─upstart-file-br
├─upstart-socket-
└─upstart-udev-br
If I disable Privilege Seperation ("UsePrivilegeSeperation no") in sshd config then the problem goes away but that opens up a security loophole where the process is running at root privilege even prior to authentication.
What do you guys think? Have others come across this? Is there a patch available for this?
Thanks,
Kam
_______________________________________________
openssh-unix-dev mailing list
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev

[Darren Tucker]

There's 3 ways that sshd will force you to change an expired password
when you have PAM enabled (which you do):
1) if you're using keyboard-interactive authentication, sshd will call
pam_chauthtok() as part of the authentication. This should work with
or without privsep.

If you authenticate some other way (eg password), sshd will force you
to change your password once you start your session.
2) if you have privilege separation enabled, by the time your session
starts sshd has dropped the privileges needed to successfully call
pam_chauthtok(). Instead, it'll exec /bin/passwd.
3) if you have privsep disabled, sshd is still running as root and
will call pam_chauthtok().

On Sat, Dec 5, 2015 at 3:23 AM, Nasim, Kam <***@windriver.com> wrote:
[...]

Post by Nasim, Kam
If I expire a linux user's password (passwd -e <user>) and then log in via ssh, it will prompt you for a password change.
On changing the password successfully, sshd will drop the connection and client has to reconnect.

That's working as intended, and you should have seen text to that
effect. In session.c (simplified):

static void
do_pwchange(Session *s)
{
fprintf(stderr, "WARNING: Your password has expired.\n");
if (s->ttyfd != -1) {
fprintf(stderr,
"You must change your password now and login again!\n");
execl(_PATH_PASSWD_PROG, "passwd", (char *)NULL);

Post by Nasim, Kam
If I disable Privilege Seperation ("UsePrivilegeSeperation no") in sshd config then the problem goes away but that opens up a security loophole where the process is running at root privilege even prior to authentication.

RIght, see #3 above.