mpiexec conf file versus pg file

[Pete Wyckoff]

cdmaest at sandia.gov said:
> The problem is that mpirun and pg files don't deal well with arguments
> passed into an mpi executable.  
> (I submitted a feature request to mpi-maint at mcs.anl.gov).  So when you do:
> 	mpirun -pg mypgfile echo	# doesn't matter for exe arg
> it fails to read in the arguments for the executables.

We've complained about that too.

> So I'd like to request trying to do this with mpiexec instead.
> 
> qsub -I -l nodes=2:vis+2:storage
> mkpbsmpiexecfile --mpiexecfile=mympiexecfile storage:/usr/local/ioproc/ioproc vis:"/usr/local/ccaffeine/dccafe/cxx/dc/user_iface/demo-client --file /home/user1/processors --port 4539"
> cat mympiexecfile
> --- mympiexec file
> storage1 : /usr/local/ioproc/ioproc
> vis1 : /usr/local/ccaffeine/dccafe/cxx/dc/user_iface/demo-client --file /home/user1/processors --port 4539
> --- mympiecec file
> 
> mpiexec -config=mympiexecfile
> 
> Is this possible?

I think this just works but maybe I'm missing a requirement of yours.
Something similar to your example above:

    $ qsub -I -l nodes=4:ppn=1
    $ cat $PBS_NODEFILE
    amd017
    amd018
    amd019
    amd020
    $ cp hello hello2
    $ cat conf
    amd017 : hello vis01
    amd018 : hello vis02
    amd019 : hello2 storage01
    amd020 : hello2 storage02
    $ mpiexec -config conf
    hello from 0/4 name amd017 with 1 args: vis01
    hello from 1/4 name amd018 with 1 args: vis02
    hello from 2/4 name amd019 with 1 args: storage01
    hello from 3/4 name amd020 with 1 args: storage02

Pretend that the file conf is built with your script, then does this
not do just what you want?  They can all run different programs, with
different arguments, not just "hello" as given here.

Perhaps you need to have mpi id=0 not always go with amd017, i.e. you
need the flexibility to make the IDs stick to particular nodes?  (But
you could build your communicators to figure this out correctly.)

Let me know what I'm missing here.

		-- Pete