[ofa-general] SDP Fixes

[Constantine Gavrilov]

While playing with SDP code in OFED 1.3.1 (latest stable), I have 
encountered a number of bugs in the zero-copy send code:

* sdp_bz_setup() code does not handle the case of kernel data segment 
correctly (kernel sockets)
* sdp_bz_setup() does not pass ENOMEM, EFAULT or other errors to 
sendmsg(). In fact, a negative possible return from get_user_pages() is 
nor handled.
* the deallocation of bz descriptor in sendmsg() is not handled properly 
-- it is allocated many times, but freed once.
* sdp_bzcopy_get() code does not raise reference count for all  pages in 
the bz descriptor (only the "partial" pages will get the count raised).
   However, the send completion code will call put_page() on all 
entries, leading to a crash for page-aligned transfers.

Attached, please find a patch that solves these problems. With this 
patch, I can use SDP and send page-aligned kernel buffers even for 
zero-copy case.

Still, I do not see any performance benefit when using the zero-copy 
method. I have tried various thresholds (32K, 64K, 128K), and zero-copy 
was always slower.

It seems that the penalty of memcpy() is negligible compared to the 
penalty of reconfiguring the card to use different addresses.

Also, looking at the sndmsg() code, I can say that  allocation and 
deallocation of bz descriptor for each iov element is not optimal. 
Instead, an existing bz descriptor can be re-used if it fits.


-- 
----------------------------------------
Constantine Gavrilov
Kernel Developer
Platform Group
XIV, an IBM global brand 
1 Azrieli Center, Tel-Aviv
Phone: +972-3-6074672
Fax:   +972-3-6959749
----------------------------------------


-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: sdp_patch.diff.txt
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20081116/dfedc9ae/attachment.txt>