A quick web search with key words Solaris stdio open file descriptors results in numerous references to stdio's limitation of 255 open file descriptors on Solaris.
#1085341: 32-bit stdio routines should support file descriptors >255, a 14 year old RFE explains the problem and the bug report links to handful of other bugs which are some how related to stdio's open file descriptors limitation.
Now the good news: the wait is finally over. Sun Microsystems finally made a fix/workaround available to the community in the form of a new library called
The workaround does not require any source code changes or re-compilation of the objects. You just need to increase the file descriptor limit using
However applications fiddling with
The following example shows the failure and the steps to workaround the issue. Note that with the extendedfile library pre-loaded, the process can open upto 65532 files excluding stdin, out and err.
* Test case (simple C program tries to open 65536 files):
* Re-producing the failure:
* Showcasing the Solaris workaround:
For further information about the
extendedFILE
enable_extended_FILE_stdio
fopen
fdopen
__________
Technorati tags:
Sun | Solaris | OpenSolaris
#1085341: 32-bit stdio routines should support file descriptors >255, a 14 year old RFE explains the problem and the bug report links to handful of other bugs which are some how related to stdio's open file descriptors limitation.
Now the good news: the wait is finally over. Sun Microsystems finally made a fix/workaround available to the community in the form of a new library called
extendedFILE
. If you are running Solaris Express (SX) 06/06 or later builds, your system already has the workaround. You just need to enable it to get around the 255 open file descriptors problem with stdio's API. This workaround will be part of the Update 3 release of Solaris 10, which is due in October 2006. There are some plans to backport this workaround to Solaris 9 and 10. However there is no clear timeline for completion of this backport, at the moment.The workaround does not require any source code changes or re-compilation of the objects. You just need to increase the file descriptor limit using
limit
or ulimit
commands; and pre-load /usr/lib/extendedFILE.so.1
before running the application.However applications fiddling with
_file
field of FILE
structure may not work. This is because when extendedFILE
library is pre-loaded, descriptors > 255 will be stored in an auxiliary location and a fake descriptor will be stored in the FILE
structure's _file
field. In fact, accessing _file
field was long discouraged; and to discourage non-standard practices even further _file
has been renamed to _magic
starting with SX 06/06. So, applications which access _file
directly rather than with fileno()
function, may encounter compilation errors starting with S10 U3. This step is necessary to ensure that the source code is in a clean state, so the resulting object code is not vulnerable to data corruption during run-time.The following example shows the failure and the steps to workaround the issue. Note that with the extendedfile library pre-loaded, the process can open upto 65532 files excluding stdin, out and err.
* Test case (simple C program tries to open 65536 files):
% cat files.c
#include <stdio.h>
#define NoOfFILES 65536
int main()
{
char filename[10];
FILE *fds[NoOfFILES];
int i;
for (i = 0; i < NoOfFILES; ++i)
{
sprintf (filename, "%d.log", i);
fds[i] = fopen(filename, "w");
if (fds[i] == NULL)
{
printf("\n** Number of open files = %d. fopen() failed with error: ", i);
perror("");
exit(1);
}
else
{
fprintf (fds[i], "some string");
}
}
return (0);
}
% cc -o files files.c
* Re-producing the failure:
% limit
cputime unlimited
filesize unlimited
datasize unlimited
stacksize 8192 kbytes
coredumpsize unlimited
descriptors 256
memorysize unlimited
% uname -a
SunOS sunfire4 5.11 snv_42 sun4u sparc SUNW,Sun-Fire-280R
%./files
** Number of open files = 253. fopen() failed with error: Too many open files
* Showcasing the Solaris workaround:
% limit descriptors 5000
% limit
cputime unlimited
filesize unlimited
datasize unlimited
stacksize 8192 kbytes
coredumpsize unlimited
descriptors 5000
memorysize unlimited
% setenv LD_PRELOAD_32 /usr/lib/extendedFILE.so.1
% ./files
** Number of open files = 4996. fopen() failed with error: Too many open files
% limit descriptors 65536
% limit
cputime unlimited
filesize unlimited
datasize unlimited
stacksize 8192 kbytes
coredumpsize unlimited
descriptors 65536
memorysize unlimited
%./files
** Number of open files = 65532. fopen() failed with error: Too many open files
(Note that descriptor 0, 1 and 2 will be used by stdin, stdout and stderr)
% ls -l 65531.log
-rw-rw-r-- 1 gmandali ccuser 11 Aug 9 12:35 65531.log
For further information about the
extendedFILE
library and for the extensions to fopen, fdopen, popen
, .. please have a look at the new and updated man pages:extendedFILE
enable_extended_FILE_stdio
fopen
fdopen
__________
Technorati tags:
Sun | Solaris | OpenSolaris