[openib-general] RFC: process_mad extension

[Michael S. Tsirkin]

Hello!
ib_verbs.h has:

        int                        (*process_mad)(struct ib_device *device,
                                                  int process_mad_flags,
                                                  u8 port_num,
                                                  u16 source_lid,
                                                  struct ib_mad *in_mad,
                                                  struct ib_mad *out_mad);

This function serves both GSI and SMA MADs. It is used internally by
the ib core, and I think its normally not used by ULPs.

process_mad_flags just says whether to enable the MKey check.
The Mkey check is enabled for incoming mad packets that are being
returned to the HCA. When enabled, the HCA is expected to generate
mkey violation trap when appropriate.

I see some issues with this interface:

1. I think for GSI MADs, the HCA (even Tavor) needs more information to
	generate traps in case of errors.
	Consider for example Trap 259 - Bkey violation.
	Per IB spec 1.2, the trap has the following payload.

	Bad B_Key, <B_Key> from
	<LIDADDR>/<GIDADDR>/<QP> attempted
	<METHOD> with <ATTRIBUTEID> and <ATTRIBUTEMODIFIER>

	However, the GIDADDR/QP is never passed to process_mad and so is not
	available to the HCA.


2. For BMA MADs, it seems an additional flag would be needed to enable/
   disable the bkey check for the MAD, same as we do for the mkey?

3. The Arbel memfree (aka native) mode hardware needs the Grh, and some
   bits from the completion of the incoming MAD, to process the MAD.
   Again, this information is not passed to process_mad.


I think it makes sence to address these issues sooner rather
than later :)

I propose extending the interface in the following way:

        int                        (*process_mad)(struct ib_device *device,
                                                  int process_mad_flags,
                                                  u8 port_num,
						  struct ib_wc* in_wc,
						  struct ib_grh* in_grh,
                                                  struct ib_mad *in_mad,
                                                  struct ib_mad *out_mad);

I think its in some sence nicer than passing the slid directly since
when sm builds packets (with mkey check disabled) slid may not be
valid at all.
The new parameters can be NULL in this case, then no trap can be generated.

If you guys agree, I'll prepare a patch fixing the core and mthca.

MST