[ofa-general] IPoIB arp relation to the SM

[Hal Rosenstock]

Todd,

On Thu, 2008-06-05 at 15:05 -0600, Todd Bowman wrote:
> All,
> 
> 
> I am trying to have a better understanding of how the SM and IPoIB
> interact with respect to arp.  If there is a document that describes
> this please point me in the right direction.
> 
> In the past, (OFED 1.0 and pre OFED 1.0) I observed the the following:
> 
> 1) Using hosts A,B
> 
> A> ping B  (succeeded)  
> A> opensm stop
> A> arp -d  B
> A> ping B   (Failed)
> 
> 
> 2) Using opensm-3.1
> 
> A> ping B  (succeeded)
> A> opensm stop
> A> arp -d B
> A> ping B  (succeeded)
> A> ping C  (failed)
> 
> 
> 
> In scenario 2, what allows the arp query for B to be resolved after
> the SM is stopped and arp entry cleared?
> Is this due to connected mode, in that an RD connection is already
> established?
> Or is there an underlying cache that feeds the ARP cache?

There are two things here (assuming nodes A and B have already joined
the IPoIB broadcast group prior to these scenarios).

The end nodes first need to get PathRecords from the SA in order to be
able to unicast prior to being able to ARP. If it is using broadcast,
that information can also be obtained via MCMemberRecord from the SA but
not sure that is used. ARP requests are broadcast and I think that ARP
replies can be unicast or broadcast depending on the type.

The point here is that the ARP/ping can work if the IPoIB broadcast
group is complete enough for the nodes in question and neither end node
needs any PathRecord information for the destination.

I don't think there has been any SM changes which affect this but I do
think there have been changes/optimizations in the end nodes (based on
kernel or OFED version) as to when the PathRecords information is
saved/needed but don't recall all the details. Maybe others on the list
can elaborate on this.

Hope this helps.

-- Hal

> Thanks,
> Todd
> _______________________________________________
> general mailing list
> general at lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
> 
> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general