[myrinet] Re: Re: myrinet and one-sided messaging

[Jarek Nieplocha]

Oops, I meant SHMEM/FM not SHMEM/GM in my question about
the HPVM implementation.

Jarek


> 
> Hi Mario,
> 
> We were considering FM as an alternative protocol to GM
> in our work on ARMCI for its certain advantages over GM
> e.g., excellent support for noncontiguous data transfers and better
> for one-sided messaging. However, a majority of Myrinet sites
> we knew in fact are interested in running production rather
> than research network drivers & communication s/w. Given the fact
> that FM is not compatible with the standard GM installations and
> at this time it does not support operating systems other than
> Windows we really had no choice but to target GM.
> 
> I also have a comment/question about the latency numbers that you
> mentioned for get. If I remember from some previous paper on HPVM
> (HPDC'98?), the low latency in SHMEM/GM was achieved by dedicating  <<<<<<<<<<
> a CPU on an SMP node to service one-sided communication requests.
> For example, on a dual CPU SMP node, only one CPU would run the
> application code and the other an extra FM thread. This thread was
> actively polling the network interface thus allowing FM to avoid
> the cost of interrupt processing that in the current ARMCI/GM
> implementation adds tens of microseconds to the get latency.
> Can you confirm that the latency number you quoted in fact
> corresponds to the configuration with a dedicated CPU actively
> polling? Also what was the get b/w -- you mentioned put b/w and
> get latency?
> 
> Regards,
> 
> Jarek
> 
> Mario Lauria wrote:
> >
> > Bob is right - SHMEM is one of the libraries ported on top
> > of Fast Messages (FM) as part of the HPVM project. Highligths
> > of the SHMEM-FM implementation are 23 usec min latency for a get
> > operation, and 93MB/s peak bandwidth for put. The pointer to
> > Andrew Chien's group site is http://www-csag.ucsd.edu
> >
> > The current version of HPVM runs under Windows NT and requires
> > the old Myrinet boards (M2M). I am a former member of the CSAG group
> > and if I find enough interest in an M3M and/or Linux version of HPVM I
> > could ask Andrew to let me update and redistribute the code.
> >
> >         Mario.
> >
> > > Date: Wed, 10 Jan 2001 11:11:14 -0800 (PST)
> > > From: Bob Felderman <feldy at myri.com>
> > > Subject: Re: [myrinet] myrinet and one-sided messaging
> >
> > > gm-2.0 will support
> > > gm_put (formerly gm_directed_send)
> > > gm_get (not implemented before gm-2.0)
> >
> > > with those two functions, it should be simple to get shmem
> > > ported to GM.
> >
> > > I think HPVM from Andrew Chien's group at UCSD does
> > > have a shared memory option.
> >
> > --
> > -=-=-=-=-=- Mario Lauria, Assistant Professor -=-=-=-=-=-
> > Department of Computer     Email: lauria at cis.ohio-state.edu
> >  and Information Science   Phone: (614) 292-7027
> > The Ohio State University  Fax:   (614) 292-2911
> > Columbus, OH 43210-1277    Radio: N9ZCT
> > -=-=-=-=- http://www.cis.ohio-state.edu/~lauria -=-=-=-=-