order of assign MPI_rank to nodes

[Pete Wyckoff]

f.paladin at cineca.it wrote on Thu, 02 Dec 2004 17:06 +0100:
> OK, so for Case A) it is sure that in the 2 mpiexec same task goes in
> the same node (I think that $PBS_NODEFILE will not change in the same
> session, do you confirm? )

Yes.  Guaranteed not to change during the course of a single PBS job.

> For Case B) in the "qsub file2" I have to manage with some pbs keyword
> to go exactly in the same node allocated by pbs in "qsub file1".
> OK, in this case the question is: is it sure that in the second
> submission $PBS_NODEFILES is ordered like in the first submission?
> Or have I to use the option -conf as I just saw in your answer to the
> precedent mail thread?

True again.  This sounds like a more risky approach as now you have to
hope all the nodes stay up between your jobs, and convince the scheduler
to allocate the same set of nodes again, in exactly the same order.  I'm
not sure if anything in PBS guarantees the same node _order_ for a given
node _set_.  Mpiexec just uses what it is given by PBS.

I suppose you could manage a different order by remembering the original
set of nodes then building a -conf file in your second batch job to
explicitly set the order:

    % cat conf
    node03 : myjob
    node01 : myjob
    node02 : myjob

It won't be very fun to script that up, though.

		-- Pete