order of assign MPI_rank to nodes

[Federico Paladin]

On Thu, 2004-12-02 at 16:02, Pete Wyckoff wrote:
> f.paladin at cineca.it wrote on Thu, 02 Dec 2004 11:33 +0100:
> >   I have an MPI application that generate data locally to nodes in which
> > it is dispatched. More, another part of this application will
> > collect/use this nodes.
> > 
> > The two part have to dispatched in the same order, that is, the same
> > MPI_rank have to go in the same nodes (as with mpirun -pgfile).
> > 
> > Is that garanteed by mpiexec?
> > If not, there is a way to force mpiexec to do it?
> > 
> > We have to use mpiexec under pbs to launch the 2 part, in the same pbs
> > session or, worst case, in 2 separated pbs session (in this case, we
> > have obviusly to use pbs keywords to force to go in the same nodes)
> 

Thanks for answering

> If I understand you correctly, you have two parallel programs that must
> run concurrently on the same set of nodes, 

No, I didn't explain sufficent clearly: the situation is more simple,
the 2 part have not to go in parallel. First part1, then part2. 
Part1 and Part2 are the same program (let's say P) with different
parameter.
They have to go on the same nodes.

The 2 part can go in the same pbs section or not:

Case A)
  "qsub file" where file contains 
  ...
  mpiexec P <arg1>
  ...
  mpiexec P <arg2>
  ...

Case B) 
  "qsub file1" and, when the computation finish, "qsub file2"
   where file1 contains
   ...
   mpiexec P <arg1>
   ...
   and file2 contains
   ...
   mpiexec P <arg2>
   ...
  
> in the same order.  The first
> thing to worry about is that PBS allows only a single mpiexec in a batch
> job, so you cannot do, say, this:
> 
>     mpiexec -np 2 job1 &
>     mpiexec -np 2 job2
> 
> If you do manage to convince PBS to start two jobs on the same set of
> nodes, MPI will honor the node list order as given by PBS in starting
> the jobs.  The order of the nodes given in the output of "qstat -n",
> which is the same order as given in $PBS_NODEFILE, will be used to assign
> tasks to processors, starting from 0.

OK, so for Case A) it is sure that in the 2 mpiexec same task goes in
the same node (I think that $PBS_NODEFILE will not change in the same
session, do you confirm? )

For Case B) in the "qsub file2" I have to manage with some pbs keyword
to go exactly in the same node allocated by pbs in "qsub file1".
OK, in this case the question is: is it sure that in the second
submission $PBS_NODEFILES is ordered like in the first submission?
Or have I to use the option -conf as I just saw in your answer to the
precedent mail thread?

Thanks,
  Federico


> 
> 		-- Pete
>