[openib-general] IPoIB interface for unauthorized partition

[Eitan Zahavi]

Hi Moni,

Sorry it took me a while to get back to you (was out on vacation ...)

Moni Levy wrote:
> On 4/10/06, Eitan Zahavi <eitan at mellanox.co.il> wrote:
> 
>>Hi Hal,
>>
>>
>>>-----Original Message-----
>>>From: Hal Rosenstock [mailto:halr at voltaire.com]
>>>Sent: Monday, April 10, 2006 2:00 PM
>>>To: Eitan Zahavi
>>>Cc: Roland Dreier; openib-general at openib.org
>>>Subject: Re: [openib-general] IPoIB interface for unauthorized
>>
>>partition
>>
>>>Hi Eitan,
>>>
>>>On Mon, 2006-04-10 at 02:35, Eitan Zahavi wrote:
>>>
>>>>Hi Roland,
>>>>
>>>>Roland Dreier wrote:
>>>>
>>>>>    Eitan> I thought the intent of the IB spec when defining P_Key
>>>>>    Eitan> index usage (and not P_Key value) was that the P_Key
>>
>>values
>>
>>>>>    Eitan> would never need to be known above the driver level.
>>
>>To
>>
>>>>>    Eitan> avoid exposing the P_Key values we could use P_Key
>>
>>index
>>
>>>>>    Eitan> for creating the IPoIB interfaces.
>>>>>
>>>>>    Eitan> Does it make sense to work on a patch that would setup
>>>>>    Eitan> IPoIB interfaces by the P_Key index (and not by P_Key
>>>>>    Eitan> value)?
>>>>>
>>>>>I don't see how this is feasible.  The index that a particular
>>
>>P_Key
>>
>>>>>lands at is completely undetermined -- if two nodes wanted to talk
>>
>>on
>>
>>>>>partition 0x8001 say, how does one know which interface to use
>>
>>without
>>
>>>>>knowing the index of that P_Key?
>>>>
>>>>OK, I get it. Actually the way IPoIB defines the broadcast group
>>
>>MGID exposes
>>
>>>P_Key anyway.
>>>
>>>>>    Eitan> Also I think the expected behavior for IPoIB should be
>>
>>that
>>
>>>>>    Eitan> IPoIB "child" interfaces should be "automatically"
>>>>>    Eitan> initialized by the code that brings up the interface
>>>>>    Eitan> (ifconfig scripts). All valid IPoIB partitions (valid =
>>>>>    Eitan> have corresponding broadcast groups) should be
>>>>>    Eitan> initialized. By doing so we provide a centralized
>>
>>control
>>
>>>>>    Eitan> of the partitions and their IPoIB interfaces through
>>
>>the
>>
>>>>>    Eitan> SM.
>>>>>
>>>>>Not sure if this is so.  I may want a partition strictly for
>>
>>storage
>>
>>>>>traffic something like that, so it doesn't make sense to create an
>>>>>IPoIB interface for that partition.
>>>>
>>>>OpenSM provides this capability in the "partition policy":
>>>>Each partition is marked explicitly if to be used for IPoIB or not.
>>>>So through one file one could actually control the IPoIB interfaces
>>>>that will exist in the subnet.
>>>
>>>The end node does not know the SM policy for that partition though.
>>>
>>>
>>>>My intent is to write some extension to ifup for IPoIB such that all
>>
>>sub
>>
>>>>interfaces will be automatically started (based on pre-availability
>>
>>of IPoIB
>>
>>>>broadcast MGID).
> 
> 
> I'm not sure how ifup is related to that. From what I understand you'd
> like ipoib driver to behave as follows:
> 
> 1. Get an event ( or figure it out) when a new PKEY is added to the
> relevant port partition table.
I prefer not to rely on new events. Instead I would like to rely on existing IB Notices:
If we register to multicast group create/delete events (traps 66/67) IPoIB can know about each new partition created.

> 2. Try to join that new MC group with the MGID it created according to
> the PKEY and the spec.  (or maybe query for the MC group existance but
> that's not atomic)
Simply join the group. We rely on these groups to be pre-created by the SM enforcing policy dictating with partitions should
be used for IPoIB and which not.

> 3. In case it fails nothing is done (no relevant MC group was
> pre-created in the SM).
Exactly

> 4. In case it succeeds a new interface is created.
> 
> Is that what you meant ?
> 
> - Moni
> 
> 
>>>If that were to be done, it would be cleanest if the child IPoIB
>>>interface was created only if that IPoIB broadcast group for that
>>>partition exists.
>>
>>[EZ] This is exactly what I had in mind.
>>
>>>-- Hal
>>>
>>>
>>>>> - R.
>>>>>
>>>>
>>_______________________________________________
>>openib-general mailing list
>>openib-general at openib.org
>>http://openib.org/mailman/listinfo/openib-general
>>
>>To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
>>