[openib-general] [PATCH] mthca_eq improvements

[Michael S. Tsirkin]

Hello, Roland, all!
Here are some mthca eq improvements, rolled into one patch.
These are all touching the same code and so are related,
but if its too big to be readable, let me know and I'll try to split
it up.

With these changes I am now getting at least 10% more bandwiths
on a non-MSI system than before.

Issues addressed are:

1. general interrupt handling speedup
   I noticed that several operations where performed multiple times,
   for example get_eqe was called multiple times, as another example,
   consumer index was truncated to eq size upon each eqe found.

   These cleanups get me some 5% more bandwith in ip over ib.

   This last change (truncating consumer index only once) is also
   needed for future memfree support, since memfree has full 32 bit
   indices.

2. non-MSI interrupt handling simplification
   I also noticed that interrupt handler re-read the ecr after
   handling all eqs, in case more eqes were written. However, consider
   that the interrupt is *not* cleared second if this happens.

   As a result, after we get out of the handler, we will get an extra
   interrupt with all bits in ecr cleared. And since this is a shared
   interrut, we'll report it as not handled, further confusing the
   system.

   And re-checking ecr is not actually needed, since if some bits are
   set, hardware will assert the interrupt and we will get back to the
   handler eventually.

   Changing this gets me another 5% bandwiths, but only on
   non-MSI system, naturally.

3. Added memfree support in the eq initialisation command.

Please comment.
MST