I was told that my server refused to accept client network connections at a specific port could be due to the lack of file descriptors. I looked up what this is all about and read about it here: http://www.netadmintools.com/art295.html
So I tested my system and I got this:
cat /proc/sys/fs/file-nr
1088 0 331287
What does this mean?
The second column actually stays at 0 even after I shutdown my server, it even stays at 0 even right after a boot!
You want to look at /proc/sys/fs/file-max instead
From recent linux/Documentation/sysctl/fs.txt:
file-max & file-nr:
The kernel allocates file handles dynamically, but as yet it doesn't free them again.
The value in file-max denotes the maximum number of file- handles that the Linux kernel will allocate. When you get lots of error messages about running out of file handles, you might want to increase this limit.
Historically, the three values in file-nr denoted the number of allocated file handles, the number of allocated but unused file handles, and the maximum number of file handles. Linux 2.6 always reports 0 as the number of free file handles -- this is not an error, it just means that the number of allocated file handles exactly matches the number of used file handles.
Attempts to allocate more file descriptors than file-max are reported with printk, look for "VFS: file-max limit reached".
EDIT: the underlying error is probably not the system running out of global filedescriptors, but just your process. It seems likely that the problem is the maximum size limit of select.
It does not look like you are hitting the system file desriptor limit. See this answer.
Perhaps your server process uses select and is thus limited to 1024 descriptors? If you switch to another mechanism, e.g. poll you will not be limited to 1024 descriptors anymore.
select() works with fd_sets
This is from the POSIX documentation of select.h:
The following shall be defined as a macro:
FD_SETSIZE
Maximum number of file descriptors in an fd_set structure.
Try to find or output FD_SETSIZE on your system.
If you find FD_SETSIZE is too low for you, I would rather try to move away from select than trying to increase FD_SETSIZE which is typically harder.
What error are you getting from accept() under these circumstances? Check errno and report it accordingly.
According to the man page, accept() will give EMFILE or ENFILE if the per-process or overall file descriptor limit has been reached, it would be helpful to know which (or if there was something else).
There is a per-process file descriptor limit which is often set to 1024 - but can easily be increased.