Mpiexec runs 1 process only

[Pete Wyckoff]

dsternkopf at hpce.nec.com wrote on Fri, 27 Aug 2004 18:28 +0200:
> I'am just testing the mpiexec program due to the handicap of PBS.
> On my machine is PBSPro 5.4.1 and MPICH 1.2.5.2 installed.
> 
> I could compile the mpiexec code without problems.
> Now I want to run a two CPU job, but mpiexec starts always
> one process only. Here the example:
> 
> asama.ess.nec.de:~/mpiexec-0.76 2910> qsub -lnodes=1:ppn=2 -I -N"test_mpiexec"
> qsub: waiting for job 227.asama to start
> qsub: job 227.asama ready
> 
> 18:24:43
> asama.ess.nec.de:~/mpiexec-0.76 2909> ./mpiexec -nostdout -nostdin -comm shmem ./hello
> hello from 0/1 hostname asama.ess.nec.de pid 16724 with 0 args:
> 
> I saw in the source code that tasks which have the same node and same config but
> possibly different cpu# will be squeezed together.
> 
> Why does mpiexec do that?
> Sorry, but I'am not so familiar with MPI.
> 
> What could be the reason for the above problem?

The shmem device in mpich expects that mpiexec will spawn one process,
then within the mpich library it forks itself into as many as requested
on the SMP, during MPI_Init().  mpiexec tells the mpich library using
the environment MPICH_NP which should be 2 in your case.

You're pretty sure you complied mpich using --with-device=ch_shmem and
not, say, ch_p4 or something else?  Other devices expect that mpiexec
will spawn all the tasks, one at a time.

If you send me the output of "qstat -f $PBS_JOBID" within the batch job,
and run "mpiexec -v -v -v ..." to show lots of its debugging messages
too, maybe we'll be able to see a problem.

		-- Pete