I've just been playing with the VFS aliasing feature (aka bind mounts) on Linux 2.4. It's deeply shiny and cool. I like it already.
Evil Mike just accosted me at work, and explained that he had a problem: someone had sent him some custom-built versions of gcc which were compiled to assume they were installed in /usr/local, and he needed to install them in that precise location on an x86 Linux box in order to use them. I have an x86 Linux box on my desk, which is why he came to me; but of course I was not enthusiastic about the idea of randomly installing stuff of unknown provenance in /usr/local for any old random (i.e. me) to stumble over by mistake.
Bind mounts to the rescue:
adduser evilmike
mkdir /home/evilmike/{jail,home,usrlocal}
mount --bind / /home/evilmike
mount --bind /home/evilmike/home /home/evilmike/jail/home/evilmike
mount --bind /home/evilmike/usrlocal /home/evilmike/jail/usr/local
mount --bind /proc /home/evilmike/jail/proc
# similarly for /dev/pts and other sub-filesystems
... and the very spitting image of my real VFS springs into life underneath /home/evilmike/jail, except that where Evil Mike's home directory should be there's actually his directory "home", and where /usr/local should be there's actually his directory "usrlocal". A few judicious chowns and chmods later, and a setuid program he can run to chroot into that area, and he's suddenly using a system which looks almost exactly like mine but happens to have /usr/local owned by him. And apart from the minimal effort required to set this up, it hasn't bothered me in the slightest.
This is fantastically shiny. I can't think of any easy way to achieve this sort of thing without bind mounts. It's just deeply cute.
(Now all we need is a way for a non-root user to set themselves up with such an ersatz system, without even the minimal sysadmin intervention I needed to perform here ... :-)