ulimit of programs run by mpiexec?

[David Golden]

On 2006-04-26 16:02:13 -0700, Garrick Staples wrote:

> The ulimit problem should be solved within PBS by setting default or max
> memory limits at the server or queue.
>

Assuming linux:

Just in case: Is that true anymore? I think >2.0.0p8 on linux might (only
just glanced at the code) have stopped implicitly adjusting the
stack limit given a mem limit:

from resmom/linux/mom_mach.c

/* UMU vmem patch sets RLIMIT_AS rather than RLIMIT_DATA and RLIMIT_STACK */

(probably rightly, IMHO, the stack ulimit is an independent issue
to how much total memory a process might need...)


FWIW, as an interim bodge, I passed the burden on to the user by 
asking them to link in a call to a tiny c function post-mpi-init
to bump the soft ulimit programmatically [*].

(one could also eliminate the soft ulimit I guess, globally in linux
in /etc/security/limits.conf, or IIRC what
matters is the ulimit of the pbs_mom, so if you e.g. were to set it
in a typical /etc/init.d/torque_mom service starter script, its TM children
will probably inherit, but having some sort of soft stack limit on by default
tends to catch bugs in code, so I was loath to do that).


[*]

#include <sys/resource.h>
#include <sys/time.h>
#include <unistd.h>
#include <stdio.h>

/* trailing underscore for intel fortran linking convention */
void stacklim_() {
         struct rlimit l;
         if (getrlimit(RLIMIT_STACK, &l) != 0) {
                perror("stacklim:");
         }

         l.rlim_cur = RLIM_INFINITY;

         if (setrlimit(RLIMIT_STACK, &l) != 0) {
                perror("stacklim:");
         }
        return;
}