[ofa-general] RFC: SRC API

[Gleb Natapov]

On Mon, Jul 30, 2007 at 02:21:30PM +0300, Michael S. Tsirkin wrote:
> > Quoting Gleb Natapov <glebn at voltaire.com>:
> > Subject: Re: [ofa-general] RFC: SRC API
> > 
> > On Mon, Jul 30, 2007 at 12:16:39PM +0300, Michael S. Tsirkin wrote:
> > > More code examples:
> > > 
> > > Create an SRC QP, part of SRC domain:
> > > 
> > > 	attr.qp_type = IBV_QPT_SRC;
> > > 	attr.src_domain = d;
> > > 	qp = ibv_create_qp(pd, &attr);
> > > 
> > > Given remote SRQ number, send data to this SRQ over an SRC QP:
> > > 
> > > 	wr.src_remote_srq_num = src_remote_srq_num;
> > > 	ib_post_send(qp, &wr);
> > > 
> > > Note: SRQ number needs to be exchanged as part of CM private data
> > >       or some other protocol.
> > > 
> > You are too brief. I can come up with one linears based on the API by
> > myself. I am trying to understand how sharing of SRC between processes
> > will work and your example doesn't show this.
> 
> It seems what you are missing is what SRC is, not how to use the API.
So tell us. Because it seems I am not the only one judging by
presentation I've got from Ishai. In this presentation he propose to
create separate receive QPs and send QPs. Is this how it meant to be
working if SRC domain is shared between processes? Because frankly, I don't
see how it can be used in any other way.

> I'll have a working example when I get closer to implementation.
> For now you'll have to look up Dror's preso if you want to
> understand what SRC is.
I looked at Dror's presentation not once. If we are talking about the
same presentation there is no much details there except additional field
in the header with destination SRQ number so HW will be able to demux a packet in
the right SRQ.

> 
> > Can I connected the same
> > SRC to different QPs? If yes, can I send packet to any SRQ connected to
> > the SRC through any QP connected to the same SRC?
> 
> Yes to both.
And can I attach SRQ to SRC domain without creating QP? I suppose yes.

> 
> > If yes how is this
> > different from having regular QPs?
> 
> With regular QP you can only send to a single SRQ.
> But again, look at Dror's preso.
> 
Yes but I can use the same QP for sending and receiving (this is a Queue
Pair after all). Now I'll have to create QP for send and QP for receive.
Overall number of QPs may still be smaller though...

--
			Gleb.