mpiexec conf file versus pg file

[Pete Wyckoff]

cdmaest at sandia.gov said:
> All 4 tasks started.
> read_gm_startup_ports: waiting for info
> read_gm_startup_ports: id 2 port 2 board 0 gm_node_id 203 pid  2878
> read_gm_startup_ports: id 3 port 2 board 0 gm_node_id 202 pid 20417
> hello
> goodbye
[..]
> And it hangs ... Is it not possible to do a multiple program job launch with mpiexec using two
> mpiprograms?

It would not be easy.

What's happening above is that mpiexec thinks you have a single MPI job
with four tasks.  It is waiting to get information from each of them so
that it can construct a global map of the GM connections and send it
back out to all the tasks which can then proceed past MPI_Init.  But
"echo" isn't acting like an MPICH/GM code, obviously.  :)

And you can't even get away with something like:
    mpiexec -np 2 code1 &
    mpiexec -np 2 code2
because the TM interface in PBS doesn't allow more than one startup
connection per job for some reason.

It is true that mpiexec could be modified to be able to express what
you're trying to do with some config file extensions, but can I
recommend that you just get two PBS jobs, each with half the nodes (or
processors), and run a single mpiexec in each?  You could also break up
MPI_COMM_WORLD into a pair of communicators, one for each of your
sub-jobs, and let them proceed essentially independently but within the
context of a single MPI parallel process.

		-- Pete