<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD><TITLE></TITLE>
<META http-equiv=Content-Type content="text/html; charset=us-ascii">
<META content="MSHTML 6.00.6000.16587" name=GENERATOR></HEAD>
<BODY><!-- Converted from text/plain format -->
<P><FONT face=Arial size=2>Just to summarize:</FONT></P>
<P><FONT size=2><FONT face=Arial>The structure ib_ca_attr_t alread contains the
field ipoib_csum (according to your request).<BR>The structure ib_wc_t will look
as:<BR><BR>typedef struct
_ib_wc<BR>{<BR> TO_LONG_PTR(struct
_ib_wc* , p_next)
;<BR>
uint64_t
wr_id;<BR>
ib_wc_type_t
wc_type;<BR><BR>
uint32_t
length;<BR> ib_wc_status_t
status;<BR> </FONT><FONT
face=Arial><FONT
color=#ff0000>uint64_t
vendor_specific;<BR></FONT><BR> union
_wc_recv<BR>
{<BR>
struct
_wc_conn<BR>
{<BR>
ib_recv_opt_t
recv_opt;<BR>
ib_net32_t
immediate_data;<BR><BR>
}
conn;<BR><BR>
struct
_wc_ud<BR>
{<BR>
ib_recv_opt_t
recv_opt;<BR>
ib_net32_t
immediate_data;<BR>
ib_net32_t
remote_qp;<BR>
uint16_t
pkey_index;<BR>
ib_net16_t
remote_lid;<BR>
uint8_t
remote_sl;<BR>
uint8_t
path_bits;<BR>
</FONT><FONT face=Arial><FONT
color=#ff0000>
uint8_t
csum_ok;<BR></FONT><BR>
}
ud;<BR><BR>
struct
_wc_raw_ipv6<BR>
{<BR>
ib_net16_t
remote_lid;<BR>
uint8_t
remote_sl;<BR>
uint8_t
path_bits;<BR><BR>
}
raw_ipv6;<BR><BR>
struct
_wc_raw_ether<BR>
{<BR>
ib_net16_t
remote_lid;<BR>
uint8_t
remote_sl;<BR>
uint8_t
path_bits;<BR>
ib_net16_t
ether_type;<BR><BR>
}
raw_ether;<BR><BR>
}
recv;<BR><BR>} ib_wc_t;</FONT></P></FONT>
<P><FONT face=Arial size=2>As Jan has said, Hardware send checksum
generation has to be controllable on a per packet basis.<BR>For ipoib connected
mode check sum is not being done by the hardware.<BR>We will soon add support
for the three types of recieve checksum to the code.</FONT></P>
<P><FONT size=2><FONT face=Arial>Thanks<BR>Tzachi</FONT><BR></FONT><FONT
size=2><BR>> -----Original Message-----<BR>> From: Fab Tillier [<A
href="mailto:ftillier@windows.microsoft.com">mailto:ftillier@windows.microsoft.com</A>]<BR>>
Sent: Thursday, July 24, 2008 7:12 PM<BR>> To: Tzachi Dar; Sean Hefty;
ofw@lists.openfabrics.org<BR>> Subject: RE: [ofw] changes to ib_wc committed
to svn<BR>><BR>> Hi Tzachi,<BR>><BR>> > -----Original
Message-----<BR>> > From: Tzachi Dar [<A
href="mailto:tzachid@mellanox.co.il">mailto:tzachid@mellanox.co.il</A>]<BR>>
> Sent: Thursday, July 24, 2008 12:58 AM<BR>> ><BR>> > Hi
Fab,<BR>> ><BR>> >> You've also lost the ability to eliminate
receive checksum<BR>> >> validation. IB has 32-bit CRC which
would cause any corrupted<BR>> >> packets to get dropped. If you assume
that packets are<BR>> being put on<BR>> >> the wire correctly, then
if they are received at all they haven't<BR>> >> been
corrupted.<BR>> ><BR>> > This approach has a few problems.<BR>>
> First, Ethernet also has CRC in it and still IP check sum<BR>> is used
on<BR>> > Ethernet. This means that the users of IP believe that there
still<BR>> > should be an end to end check.<BR>> > Second, your
approach works only inside a small cluster that is all<BR>> > windows. If
the cluster has Linux machines in it they will<BR>> through the<BR>> >
packets.<BR>><BR>> No, the checksum is still calculated and set properly
by the<BR>> sender. I agree that if you enable send checksum offload
and<BR>> don't perform the calculation then the packets can get<BR>>
discarded at the receiver. However, if you let the OS do<BR>> send
checksums, you can pass packets to the stack at the<BR>> receiver as if
checksum had been done. This should interop<BR>> just fine with Linux
since the checksums in the packet should<BR>> be correct.<BR>><BR>>
> If there is a gateway to Ethernet, that gateway should be<BR>>
configured<BR>> > to do the checksum once packets are moved to the
Ethernet. I don't<BR>> > think there is such a gateway.<BR>><BR>>
This only applies if you don't calculate checksums when<BR>> putting a packet
on the wire. Receive checksum calculation<BR>> is
independent.<BR>><BR>> > In any case, I believe that we should have 3
possibilities:<BR>> > 1) Never do checksum in hw. Windows will always do
checksum.<BR>> > 2) Never do checksum by windows. The hw will do checksum
if<BR>> it can or<BR>> > send packets with bad checksum if the card
doesn't support that.<BR>><BR>> This one I think can be refined - you can
enable receive<BR>> checksum 'offload' without packet drop. Enabling
send<BR>> checksum 'offload' without hardware support will put
packets<BR>> on the wire with incorrect checksums though, and I think
that<BR>> should be discouraged.<BR>><BR>> > 3) [default] Do
checksum in HW if the hw supports that and do it by<BR>> > software if HW
doesn't support that.<BR>> ><BR>> > This will give anyone the chance
to choose the right option for him.<BR>><BR>> Agreed, this should be
configurable.<BR>><BR>> >> What happens if these options are set and
the underlying<BR>> HCA doesn't<BR>> >> support checksum
offload? Does the WR just get sent or<BR>> does it fail?<BR>>
><BR>> > In the existing code, this flags are not checked by
other<BR>> drivers so<BR>> > things should be fine. This might have to
be changed once<BR>> the 3 modes<BR>> > suggested above will be
available.<BR>><BR>> It seems to me that checksum offload should be a
QP<BR>> attribute, not a work request option. That way clients
can<BR>> do a check to see if the CA attributes indicate the HW<BR>>
supports checksum offload, and then create their QP accordingly.<BR>><BR>>
Ignoring send flags is a bit odd - right now none of the<BR>> flags get
ignored do they? If someone requests a send with<BR>> immediate, it's
expected that the send will happen with<BR>> immediate. Sending the
data without immediate data would<BR>> affect the wire
protocol.<BR>><BR>> >> All in all, I think using the vendor specific
field for checksum<BR>> >> notifications seems lacking proper design,
and more importantly,<BR>> >> community input. A few issues from a
quick glance:<BR>> >> - it should be clear that it matches the<BR>>
>> NDIS_TCP_IP_CHECKSUM_PACKET_INFO definition.<BR>> ><BR>> >
Indeed it matches this field exactly. This was done to<BR>> avoid doing
un<BR>> > needed work on the data path.<BR>><BR>> A comment to that
extent would be good.<BR>><BR>> >> - either checksum offload is
vendor specific or it isn't. <BR>> If it is,<BR>> >>
then:<BR>> >> 1. the
IB_SEND_OPT_TX_XXX_CSUM values should be in the<BR>>
>> IB_SEND_OPT_VEND_MASK
range 2. The ib_wc_t structure stays<BR>>
>> unchanged 3. IPoIB
checks which HCA it runs on and has<BR>>
>> HCA-specific code to
handle checksum offload. 4.<BR>> ib_ca_attr_t<BR>>
>> is unchanged<BR>>
>> If it isn't then the vendor_specific field should get<BR>>
overloaded with<BR>> >> non-vendor specific information and csum_ok
should be put into the<BR>> >> recv.ud part of the ib_wc_t
structure.<BR>> >> - MTHCA needs to be updated so that it properly
clears csum_ok.<BR>> > What is your suggestion, do you want to put
it as vendor<BR>> specific or<BR>> > not? I suggest to have it non
vendor specific.<BR>><BR>> I think it makes sense as non-vendor
specific. So think I'd<BR>> like to see:<BR>> - checksum offload
capabilities are reported in CA attributes.<BR>> - checksum offload as QP
creation attribute (do you support<BR>> this for both UD and RC QPs?
That is, can IPoIB-CM take<BR>> advantage of this too?). If offload is
requested but HW<BR>> doesn't support, QP creation should fail. Clients
should be<BR>> expected to check the capabilities of the HW.<BR>> -
checksum results are reported in ib_wc_t::recv.ud.csum_ok<BR>> -
ib_wc_t::vendor_specific remains as a uint64_t.<BR>><BR>> -Fab<BR>>
</FONT></P></BODY></HTML>