[ofa-general][PATCH] mlx4_core: Multi Protocol support

[Roland Dreier]

Your email has

 > Content-Type: text/plain; charset=ISO-8859-1; format=flowed

and the format=flowed means that the patch gets corrupted and won't
apply.  So when you resend, please fix.

I don't think we can really apply this as one patch -- it does too many
things at once and needs to be split up... I think pretty much each of
these items is independent and could be a separate patch:

 >         1.  Mlx4 device now holds the actual protocol for each port.
 >             The port types are determined through module parameters of through sysfs
 >             interface. The requested types are verified with firmware capabilities
 >             in order to determine the actual port protocol.
 >         2.  The driver now manages Mac and Vlan tables used by customers of the low
 >             level driver. Corresponding commands were added.
 >         3.  Completion eq's are created per cpu. Created cq's are attached to an eq by
 >             "Round Robin" algorithm, unless a specific eq was requested.
 >         4.  Creation of a collapsed cq support was added.
 >         5.  Additional reserved qp ranges were added. There is a range for the customers
 >             of the low level driver (IB, Ethernet, FCoE).
 >         6.  Qp allocation process changed.
 >             First a qp range should be reserved, then qps can be allocated from that
 >             range. This is to support the ability to allocate consecutive qps.
 >             Appropriate changes were made in the allocation mechanism.
 >         7.  Common actions to all HW resource management (Doorbell allocation,
 >             Buffer allocation, Mtt write) were moved to the low level driver.

Also, on the other hand, the current two patches are too split up: if
I apply this patch then mlx4_ib won't compile until the second patch
goes in too.  Which means someone trying to bisect an mlx4 bug gets into
trouble.  So please make sure that everything still compiles and works
after each patch is applied.

By the way, the multiple EQ stuff is a pretty major change in
behavior... are we really ready for this?  Round robin seems like it
could easily lead to worst-case behavior for some plausible workloads.

Finally, checkpatch.pl shows a few minor whitespace problems... please
fix when you resend.

 - R.