[ofa-general] Application layer support for RMPP using OFED stack.

[Hal Rosenstock]

Hi Anton,

On Wed, 2007-12-05 at 12:15 -0600, Anton Bodner wrote:
>  
> 
> Hello –
> 
>  
> 
> I’m from QLogic Corp, and I’m trying to port some older applications
> to OFED, using the OFED stack and the OFED umad api (specifically –
> opening the /dev/infiniband/umadX, and using read / write).  I am
> using OFED 1.2.5.
> 
>  
> 
> My old application interrogates the SA, and also implements the RMPP
> in the application layer.  In porting this application to OFED, I
> realize that the OFED stack has the capability to do the RMPP on my
> behalf, but that has an adverse effect on my application since my
> infiniband interface in my app transfers 256 byte packets ONLY.  
> 
>  
> 
> Hence – I want my app to do the RMPP, not OFED.

I understand why you want to do this but that was not a design goal. The
design goal was that the kernel always does the RMPP. There were long
discussions of this on the list way back when the approach for this was
being discussed. I think that was over 3 years ago and I've forgotten
all the details and haven't gone back to try to resurrect that
discussion. The bottom line was we wanted one RMPP implementation that
would be more likely to interoperate with other RMPPs. The downside is
the "feature" you are asking about: the ability for userspace to
implement its own RMPP.
 
> So – I disabled RMPP support in OFED stack by registering with RMPP
> version of 0.  This seems effective in stopping OFED stack from doing
> the RMPP for me.   

Yes, but that doesn't mean that userspace can do RMPP but just that your
client is not using RMPP (although in this case it is). You want this to
mean kernel don't do RMPP but that wasn't the design point.

> However – when I try to send the RMPP ack, it fails at the write ().

Yes, that is one of the issues. I think a similar problem also exists on
the receive side.

> I investigated the OFED stack code, and its failing in
> ib_create_send_mad (drivers/infiniband/core/mad.c).  Investigation
> shows that if one registered with RMPP version 0, then the ‘send’
> never allows rmpp to be active…  (code snippet below)

Because that is correct with the current meaning of rmpp_version.

>             if ((!mad_agent->rmpp_version &&
> 
>                  (rmpp_active || message_size > sizeof(struct
> ib_mad))) ||
> 
>                 (!rmpp_active && message_size > sizeof(struct
> ib_mad)))
> 
>     {
> 
>                         return ERR_PTR(-EINVAL);
> 
>     }
> 
>  
> 
> My questions are these:
> 
> 1. Has anyone tried an application layer supported RMPP ?

I suspect not.

>  If so – how did you get past this logic (possible OFED bug??)
> 
>  
> 
> 2. Is the OFED implementation intent to disable RMPP support
> COMPLETELY when registering with a RMPP version of  0?

No.

>  If so, then how does one implement a user level RMPP using the OFED
> stack?  Perhaps no one is doing this at all since the stack already
> does it for you?

That's how it's intended to be used.

> Like I said – the only reason I’m doing this is to port an old/
> existing app to OFED, and there is a considerable level of difficulty
> to rip our app level RMPP support.  

Understood.

> But perhaps this is the alternative I must do…

Not sure if that is the case but that certainly is an alternative and is
likely the most expeditious.

-- Hal

> Thanks
> 
>  
> 
> Anton Bodner
> 
>  
> 
>  Anton Bodner Jr.
> 
> 
> QLogic Corporation
> 
> (610)233-4856
> 
> anton.bodner at qlogic.com
> 
> http://www.qlogic.com
> 
>  
> 
> 
> _______________________________________________
> general mailing list
> general at lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
> 
> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general