mpiexec GMPI_SLAVE env.t problem

[Pete Wyckoff]

brh at unimelb.edu.au said on Wed, 06 Aug 2003 12:02 +1000:
> I've tracked this down to the mpich-gm gmpi_conf.c source and gmpi_getenv 
> routine, and
> have added printf's to see what's happening...

Great debugging info, thanks!

> Under mpirun, the GMPI_.... env't vbles get returned OK, eg.
> BH: gmpi_conf.c : gethostbyname returned node004
> BH: gmpi_getenv var : GMPI_MAGIC , result 7374385
> BH: gmpi_getenv var : GMPI_MASTER , result node040
> BH: gmpi_getenv var : GMPI_PORT , result 8000
> BH: gmpi_getenv var : GMPI_SLAVE , result 172.20.3.4
> BH: gmpi_getenv var : GMPI_ID , result 7
> BH: gmpi_getenv var : GMPI_NP , result 8
> BH: gmpi_getenv var : GMPI_BOARD , result -1
> BH: gmpi_getenv var : GMPI_NUMA_NODE , result (null)
> BH: gmpi_getenv var : GMPI_EAGER , result (null)
> BH: gmpi_getenv var : GMPI_SHMEM , result 1
> BH: gmpi_getenv var : GMPI_RECV , result (null)
[..]
> but under mpiexec, it fails on GMPI_SLAVE,
> BH: gmpi_conf.c : gethostbyname returned node040
> BH: gmpi_getenv var : GMPI_MAGIC , result 210
> BH: gmpi_getenv var : GMPI_MASTER , result node040
> BH: gmpi_getenv var : GMPI_PORT , result 36678
> BH: gmpi_getenv var : GMPI_SLAVE , result (null)
[..]

> I also note that under mpirun GMPI_PORT=8000, whereas as seen above,
> under mpiexec its' getting GMPI_PORT , result 36678.

The magic number is any old made-up thing.  Mpiexec uses the PBS job id.
Port is the listening TCP port on the spawner; this is fine to be
different too.  GMPI_SLAVE is the hostname of each respective compute
node in the parallel job.  Mpiexec gets this from looking at the
equivalent of the output of

    qstat -f jobid | grep exec_host

Can you run mpiexec with one or two "-v" to induce it to print out the
hostnames it gets for the job?  Those should not be null, but maybe some
oddness will make it clear why those get propagated as null to the
spawned job.  Perhaps your /etc/hosts or DNS is strange in some way.

		-- Pete