My webapp allows users to execute some arbitrary code in a sandbox. To prevent forkbombs, the application calls setrlimit and limits RLIMIT_NPROC to 50 before executing user code. This worked great in Ubuntu 12.04 up till Ubuntu 13.04. However, after upgrading to Ubuntu 13.10 (which ships with Apache 2.4 and Linux 3.11), we hit the limit of 50 www-data processes, even when Apache2 is idle!
The problem is most easily reproduced by running bash as user www-data with ulimit. First switch into user www-data and start bash:
jeroen@Ubuntu:/$ sudo su www-data
$ bash
www-data@Ubuntu:/$
Now gradually lower RLIMIT_NPROC until we hit problems:
#RLIMIT_NPROC=100: works fine   
www-data@Ubuntu:/$ ulimit -u 100
www-data@Ubuntu:/$ ls
bin    dev   initrd.img      lib64   mnt   root  srv  usr      vmlinuz.old
boot   etc   initrd.img.old  lost+found  opt   run   sys  var
cdrom  home  lib         media   proc  sbin  tmp  vmlinuz
#RLIMIT_NPROC=50: limit reached
www-data@Ubuntu:/$ ulimit -u 50
www-data@Ubuntu:/$ ls
bash: fork: retry: No child processes
bash: fork: retry: No child processes
bash: fork: Resource temporarily unavailable
Hence after setting RLIMIT_NPROC to 50, the process can no longer fork. This implies that there are already 50 or more processes running as user www-data. However, this does not seem to be the case, the server is just a blank idle Apache 2.4. According to ps, there are currently only 2 procs owned by www-data:
jeroen@Ubuntu:~$ ps aux | grep www-data
www-data 11473  0.0  0.5 631296 46164 ?        Sl   14:28   0:01 /usr/sbin/apache2 -k start
www-data 11474  0.0  0.5 565656 45632 ?        Sl   14:28   0:01 /usr/sbin/apache2 -k start
jeroen   12136  0.0  0.0  13644   956 pts/4    S+   14:51   0:00 grep --color=auto www-data
So why is www-data is hitting the RLIMIT_NPROC limit of 50 in Apache 2.4, even when idle?