GNU/Linux: why doesn't chroot() really protect?

[DiTBho]

chroot() is a function usually implemented as "command", whose binary is usually "/bin/chroot", and it's usually part of sys-apps/coreutils, intended to limit access to a filesystem by changing its root, so, after chroot($xxx) a process will see only those which are limited at the top-level by the $xxx parameter passed to the chroot().

Code: [Select]

    char_t root_name[]="/stage/stage4-2008-04";
    ...
    /* do chroot() */
    chdir(root_name);
    ans=chroot(root_name);
    if( ans isNotEqualTo 0)
    {
        panic(module, fid, "probably wrong", root_name);
    }
(user space application)

chroot() needs a kernel syscall, which invokes "set_fs_root()" for a process. In no way chroot()  limits the syscalls a process can make! It's not a sandbox! It's not a virtualizer! It simply "sets" the root in the info structures passed to a process (usually /bin/bash).

Using chroot mainly to use very old rootfs (e.g. 2008 Stage4 on modern 2024 kernels) without having to resort to a virtual machine or anything else!

Code: [Select]

bind { /dev /proc /sys } /stage/stage4-2008-04
chroot /stage/stage4-2008-04 /bin/bash
source /etc/profile
I also use chroot to compile a new rootfs, and to test it.

And right here I was told that ... chroot() is not the best choice for "testing" a rootfs because "an exploit" (evil code) could very well reach even /etc/passwd.

This is because chroot() does not in any way limit access to inodes outside the new root.

What other vulnerabilities does it have? Has anyone ever seen a real exploit?

[madires]

From 'man 2 chroot':

This  call changes an ingredient in the pathname resolution process and does nothing else.  In particular, it
is not intended to be used for any kind of security purpose, neither to fully sandbox a process  nor  to  re-
strict  filesystem system calls.  In the past, chroot() has been used by daemons to restrict themselves prior
to passing paths supplied by untrusted users to system calls such as open(2).  However, if a folder is  moved
out  of  the  chroot directory, an attacker can exploit that to get out of the chroot directory as well.  The
easiest way to do that is to chdir(2) to the to-be-moved directory, wait for it to be moved out, then open  a
path like ../../../etc/passwd.

A  slightly trickier variation also works under some circumstances if chdir(2) is not permitted.  If a daemon
allows a "chroot directory" to be specified, that usually means that if you want to prevent remote users from
accessing files outside the chroot directory, you must ensure that folders are never moved out of it.

[DiTBho]

Stages1-4 ? Folders are never moved out of the chrooted!

Is there any other vulnerability?

[amwales]

Well, if you are able to run mknod while in a chroot you have access to devices you may not normally have access to.

[bsdphk]

Chroot(2) was never intended to be a security mechanism, and it isn't, there are many ways around it.

If you want the nitty gritty details, read the first part of Robert's and my Jail paper: https://papers.freebsd.org/2000/phk-jails/

[DiTBho]

Well, if you are able to run mknod while in a chroot you have access to devices you may not normally have access to.

some application should have *root privileges only*
sys-apps/coreutils { /bin/mknod, ... }
sys-apps/util-linux { /bin/mount, /bin/unmount, /sbin/mkfs.*, ... }
sys-fs/dosfstools { /usr/sbin/mkfs.*fat, ... }
sys-fs/xfsprogs { * }
...

and in the chroot /dev shouldn't be populated with all dev-name, only the minimal ones


-

this is my default profile for every stage1-4.
I know, it's not enough 

[Nominal Animal]

The Wikipedia article on OS-level virtualization contains a table comparing some common OS-level virtualization implementations, but doesn't mention their security implications.

chroot() is only a filesystem isolation mechanism, not a security one.  (Consider a lab, but with an unlockable door/airlock.)  It is very useful for e.g building native second-stage compilers and base C libraries (see LinuxFromScratch and its general instructions), because builds and installations within the chrooted environment will not be affected by the host system changes outside the chroot.  It is particularly nice when the build uses cmake or autotools, which test to see things the build host supports.  Generally, Linux build tools try to behave very nicely, and usually don't do anything that could break them out of a chroot.

Leaking descriptors (referring to directories) to the chrooted process, or passing one via an Unix domain socket (via SCM_RIGHTS ancillary message), is a common way to escape a chroot.  The moving directories inside-outside trick is another, but can be avoided if the chroot is a separate partition (because you cannot move/rename/hard-link stuff across partitions: they have to be copied instead).  I personally think POSIX O_CLOEXEC flag should be the default for all open() variants, especially C library opendir() implementations due to the risk of leaking descriptors to child processes executing helper programs; it's super-nasty for multithreaded programs where multiple threads open and close files, and at least one of the threads forks child processes and executes helper programs.  One can always duplicate the desired descriptors in the child process, and remove the O_CLOEXEC flag from the duplicate using fcntl(descriptor,F_SETFD,0) thus allowing the descriptor to survive the exec() boundary, safely avoiding fork()+exec() races against other threads in the same process.  Setting the flag after opening is always at risk of such races and leaking the descriptor.

I've always preferred the FreeBSD jail mechanism.  Thank you for implementing them, bsdphk!
For similar functionality in Linux, you need to use cgroups.

If you need to isolate an unprivileged process, you can bolster chroot() with a seccomp filter that disables most syscalls, but it approaches security the exact wrong way: starting from an unsecure situation and trying to plug up the known holes.  The correct approach is to start by locking everything down, and only allow known safe operations.  But be very, very suspicious about what you consider safe.

Nowadays, I like to use virtual machines for stuff I don't really trust.  There is a bit of overhead (mostly memory use), but it is lightweight enough and nearly effortless.  It also makes post-crapout cleanup much easier – I checkpoint the vm before testing the stuff, and revert back to the checkpoint afterwards.

If I were to suspect something evil like a virus or a worm, then I'd use physically separate dedicated hardware, with the storage used completely scrubbed without mounting, afterwards.

[DiTBho]

One can always duplicate the desired descriptors in the child process, and remove the O_CLOEXEC flag from the duplicate using fcntl(descriptor,F_SETFD,0) thus allowing the descriptor to survive the exec() boundary, safely avoiding fork()+exec() races against other threads in the same process.  Setting the flag after opening is always at risk of such races and leaking the descriptor.

damn, this is terrible 

[DiTBho]

builds and installations within the chrooted environment will not be affected by the host system changes outside the chroot.  It is particularly nice when the build uses cmake or autotools, which test to see things the build host supports.  Generally, Linux build tools try to behave very nicely, and usually don't do anything that could break them out of a chroot.

yup, this is how Catalyst works for Gentoo, and I thought about this on a stage4 to be regenerated starting from binaries for which I don't have the sources. Made by a vagabond dude I can't reach out by email, and the only MIPS4/R16k (experimental) stage I've found ...

Call me "paranoid", but ... you know, the recent backdoor found in the xz-tool(1), so ... if it were a stage3 downloaded from the Gentoo repo... there would be fewer problems, even official repositories can be contaminated(1), but I don't really trust myself to put something like that on disk, bind it to /dev!!! (mount -o bind /dev/ /stage/dev), then invoke Catalyst which in turn does chroot, and then uses the binaries that it finds to fill out a new stage1-3


edit:
(1) Who? What? 
For those who don't know about this story, I'll summarize it briefly (read more here)

The maintainer of the xz compression tool suffered a social engineering attack in which he was made to believe that the tool had a lot of flaws to fix and that the help of a second maintainer was needed. So he "gave" trust to someone who was pretending to be "a good guy", who inserted malicious code directly into the tool's sources

I don't know how many thousands of machines have been infected, but this kind of stuff ... inside a chrooted tipical Gentoo building scenario can make damages!

[DiTBho]

If I were to suspect something evil like a virus or a worm, then I'd use physically separate dedicated hardware, with the storage used completely scrubbed without mounting, afterwards.

this sounds the perfect job for my ramdisk-tower (16Gbyte emulated by sdram, connected to the host via fiber-optic).
you can literally commit yourself to inserting the evilest code as well as possible, but if I pull the plug ...
... wait 5 minuts to have all the ram cell capacitors discharged and their bytes will be snowflakes that don't fall anywhere.

[Haenk]

I'm pretty sure I used that for running ftp environments for our customers - that was way before any "easy" all-in-one ftp-server software was available. My script essentially created a basic mini environment for each customer. Since being "ftp only", there was no risk of escaping (not considering the likely exploits, which weren't a thing back then).

[DiTBho]

I'm pretty sure I used that for running ftp environments for our customers

a few ftp could be configured to chroot() each client upon connection.

Code: [Select]

net-ftp/vsftpd
      Homepage:      https://security.appspot.com/vsftpd.html
      Description:   Very Secure FTP Daemon
      License:       GPL-2
This one allows to chroot to user's home directory.
If you want to chroot all users to one fixed directory, just add the following to your /etc/vsftpd/vsftpd.conf:

Code: [Select]

local_root=/home/ftp


[Whales]

N.B. If you're after isolation then linux namespaces might be what you're after. 

[Nominal Animal]

I'm pretty sure I used that for running ftp environments for our customers - that was way before any "easy" all-in-one ftp-server software was available. My script essentially created a basic mini environment for each customer. Since being "ftp only", there was no risk of escaping (not considering the likely exploits, which weren't a thing back then).

Correct.  Note that it was the ftpd service process that chroot()ed itself, for easier control of pathnames –– which it used exclusively for server-side access ––, it was essentially isolating itself.

If the service fork()s a child process, fully drops privileges (by switching to the remote user identity), makes sure all extraneous file descriptors are closed (leaving only the client socket and a pipe for error logging), then chroot()s itself, it is pretty well isolated.  It is no cgroups isolation, but if the child process only uses a small subset of syscalls –– avoiding anything exec, mount, unmount, fork (and clone), setuid/setgid, etc. –– limiting itself to open/create, close, recv/read/readv/pread, send/write/writev/pwrite, chmod, unlink, link/rename, chdir, mkdir/rmdir, clock_gettime/gettimeofday/time, exit/exit_group –– it typically suffices.  Using seccomp here would mean that even if the child process had exploitable buffer overruns or similar bugs, they couldn't be used to grant extra access, without help from someone with access to the root system.

How secure is that sort of thing?  Sufficient for isolation between publicly accessible web servers, for example.  It is a sufficient barrier between clients.  It is vulnerable to shenanigans outside any chroots, so it would be paramount that all clients be limited thus.  It is also paramount that the chroots are on a different mount, and never on the same mount as root /, /usr, or /var, because that would risk hard-link exploits; and is not mounted under root/, /boot, /etc, /tmp, or /var/tmp, because these either are privileged directories with configuration information only, or are temporary directories with special modes (sticky group) with many users creating and deleting directories often.

(It's been a few years since I last worked with this kind of isolation, so I may have forgotten some details, too.  When developing this kind of services, you quickly develop a rather paranoid mode of thinking, which one basically has to drop when interacting with other humans in a mutually beneficial manner.  If you don't develop that paranoia, your services will most likely be exploitable.)