How to execute the right binary on each node for each architecture in a heterogeneous system?

[Pete Wyckoff]

wytsang at clustertech.com wrote on Mon, 31 Jan 2005 16:52 +0800:
> How do I execute the right binary on each node for each architecture
> in a heterogeneous system?
> 
> I try to do it with option "-config", but it is not handy to specify the
> node names explicitly everytime.
> 
> Does mpiexec provides any option like LAM-mpiexe option "-arch"?

You are courageous to run heterogenous codes in the first place.

As you point out, -config does work, but it's a bit messy to have to
build the config file given the hostnames and architecture types.  The
other option with the current mpiexec is to write your own wrapper
script and exec that on the command line, something like this (untested);

    #!/bin/bash
    #
    # Front-end architecture-detection script for heterogeneous parallel
    # codes.
    #
    arch=$(arch)
    arch=$(echo $arch | sed 's/i[3-6]86/x86/')
    arch=$(echo $arch | sed 's/sun4/sparc/')
    # or look at uname -s and uname -m.
    code=$0
    exec $code.$arch "$@"


To change mpiexec to add add a LAM-mpiexec style "-arch" option, i.e.,
to permit a construct like:

    mpiexec -arch solaris hello.solaris -arch linux hello.linux

we would need some way for mpiexec to get the architecture information
out of PBS.  I don't see anything immediately in PBS that would provide
that, but each mom certainly knows its own arch.

I'm not going to work on this (although you or anyone else is welcome),
but I will comment on one major drawback of the way LAM-mpiexec works
and what we might get if we did a PBS-mpiexec implementation.  Troy Baer
pointed out that the pbs_mom architecture string for our machines is
"linux", not "linux-ppc" or "linux-x86" or "linux-ia64" or any of the
other hardware architectures on which that OS runs.  Something to keep
in mind if you have a similar target environment.

		-- Pete