[ofa-general] Re: [ewg] [PATCH 3/9] ib_core: RDMAoE support only QP1

[Hal Rosenstock]

On Wed, Jun 17, 2009 at 7:57 AM, Eli Cohen<eli at dev.mellanox.co.il> wrote:
> On Wed, Jun 17, 2009 at 07:14:56AM -0400, Hal Rosenstock wrote:
>> On Wed, Jun 17, 2009 at 7:10 AM, Eli Cohen<eli at dev.mellanox.co.il> wrote:
>> > On Mon, Jun 15, 2009 at 02:26:42PM -0400, Hal Rosenstock wrote:
>> >> Should ib_post_send_mad return some error on QP0 sends on RDMAoE ports ?
>> > You can't send anything over QP0 because it is not created and so
>> > there are no data structs corresponding to it.
>>
>> Yes, I understand that's the intention but I didn't see where a MAD
>> posted to QP0 returns an error. Does that occur ? Or is it just
>> silently dropped ?
> But you don't have a "struct ib_qp *" for QP0 that you could use to
> post MADs to QP0...

Understood.

Doesn't an error need to be returned for certain cases of invoking
ib_post_send_mad on this new port type (at least qp0 and maybe some
qp1 things) ? Look at user_mad.c:umad_write calling
ib_post_send_mad().

>
>>
>> >> What QP1 sends are allowed ?
>> > Basically, all QP1 sends are allowed without any changes - QP1
>> > functions as normal.
>> > However, RDMAoE will initially support only the CM. In the future, we
>> > can support additional QP1 services.
>>
>> So what happens with other QP1 sends now ? Do they go into hyperspace
>> and then timeout ?
> SA joins and SA path queries are terminated at the driver. Otherwise,
> post sends on QP1 should be sent on the wire.

So these would just timeout if there's nothing there to consume them ?
Seems better to error them out at the sender.

>>
>> There are a set of tools (infiniband-diags and ibutils) which do send
>> MADs from userspace. I'm concerned that if someone tries these, the
>> right thing will happen.
>>
> What exactly do you mean?

opensm and any IB diag (relying on QP1 and/or QP0) should error out at
the sender.

>From your patches, I'm also not sure how user space sees this port
type (umad needs to know about these port types). Maybe this needs
ioctl exposure (and another change to umad API if it's done this way
(rather than some other way which is what I think Sean was getting at
in his email on node type)); sigh :-(

-- Hal