[openib-general] Updated IPoIB IETF WG presentation

[David M. Brean]

Hello,

Some comments/questions about these slides:

* slide 1 - nit: perhaps the title should be "Some Experience with Linux
IPoIB Implementations" since the information is coming from Linux
developers.

* slide 4 - nit: move the first bullet after bullet containing "single 
implementation"

* slide 6 - nit: first bullet should be highlighted as the "problem" and 
the second bullet as the "solution".

* slide 7 and 8 - In section 5.0 of the I-D, there is text stating that 
the "broadcast group may be created by the first IPoIB node to be 
initialized or it can be created administratively before the IPoIB 
subnet is setup".  The mechanism used to administratively create the 
group is intentionally beyond the scope of the I-D.  For example, an 
implementation could enable the fabric (or "network" as you say) 
administrator to control membership in a partition and therefore make 
sure that the first node added to that partition creates the broadcast 
group correctly.  In any case, mentioning the administrative option is 
kinda a "helpful" hint.  All the IPoIB nodes are free to create the 
broadcast group, just like they can create any multicast group, as long 
as the IPoIB node has enough information to specify the necessary 
parameters as required by the SA interface.  The I-D suggests how to 
find the necessary parameters for the multicast groups and leaves open 
how IPoIB nodes obtain that information if they need to create that group.

   Are these slides suggesting that the I-D be changed to specify the 
IPoIB parameters via defaults for the case where the IPoIB node must 
create the broadcast group?

   [Note, Q_Key is provided by broadcast group, so it isn't necessary to 
distribute to all IPoIB nodes.]

* slide 9 and 10 - "Running" may be the description of a state that is 
be OS is beyond the scope of the I-D (does Windows network interface 
support a "running" state?).  However, the I-D does say that an IPoIB 
link is "formed" only when the broadcast group exists.  The I-D doesn't 
say anything about operation in a "degraded" mode, for example, when a 
IPoIB node can't join a multicast group.  Behavior in degraded mode 
seems like an implementation issue.  It's not clear what you would want 
to change in the I-D, perhaps you can suggest what you want changed in 
the presentation.

* slide 12 - I recall that during the email discussion:
1) a boot-time scenario where the IPoIB nodes had to access the SA to 
obtain pathrecord information to fill the pathrecord cache and send 
unicast ARP messages
2) a SM failover/restart scenario

   For #1, the speed at which the IPoIB nodes can begin normal operation 
depends on the fabric and SA implementation.  I guess the question is 
whether this is an architecture or implementation problem.  Is it 
impossible to implement a working system based on the current 
architecture?  I think the proposed alternative would require changes to 
the encapsulation scheme plus specifying some defaults such as the SL so 
that SA queries are eliminated.  Some of that might require input from 
the IBTA.

   For #2, how long is too long for a subnet to operate without 
successful SA queries?  10 seconds?  20 seconds?  Or is this change 
suggesting that the subnet should continue operating, perhaps 
establishing new IP connections (note, this proposal doesn't attempt to 
fix the situation at the IB transport level) even in the case where no 
SA exists.  Please clarify in the slides.

* slide 13 - An IB CA should perform as well as a "dumb" ethernet NIC 
with respect to bandwidth and CPU utilization.  If not, someone should 
look at the overheads in the IB access layer and the CA implementation, 
right?  The statement "not equivalent to ethernet" is highlighting the 
lack offload mechanisms in the CA such as checksum, correct?  If so, 
perhaps that point should be made explicit.

Note, I'm not attempting to respond to the issues raised on the slides 
since that will happen at the meeting, but merely seeking clarification 
of the issues being raised.

-David

Hal Rosenstock wrote:

> Here's an updated presentation based on the comments from yesterday:
>     - Separate slide and more detail on openib
>     - Eliminate checksum slide
> 
> It is also available as
> https://openib.org/svn/trunk/contrib/voltaire/ietf_ipoib/ipoib_exp.pdf.
> 
> -- Hal
> 
>