[openib-general] [ANNOUNCE] Initial trunk checkin of ISERinitiator

[Guy German]

Hi Grant,

On Tue, 2005-08-23 at 11:53 -0700, Grant Grundler wrote:
> On Tue, Aug 23, 2005 at 11:58:54AM +0300, Guy wrote:
> > How much is linux tolerant to code making *long* use of the interrupt
> > context. Are there any limitations (set by the kernel hackers)
> > regarding this issue ? (I know that in rtos this is totally not
> > acceptable, for understandable reasons)
> > 
> > How long does count as *long* ?
> 
> Long enough to service the HW and defer the rest of the work
> to another context where interrupts are enabled.
> I'm not aware of any other "hard" rule on this.
> 
> > Is it measured in jiffies/usecs/locs other ?
> 
> It should be measured in usecs - preferably sub-jiffies.
> I think key is the duration must be deterministic.
> 
> e.g. IDE PIO modes block interrupts for long, deterministic periods
> of time. That's another reason PIO modes suck - but some HW
> only works that way.

What about the SRP implementation, which polls the cq in the cq's upcall
context (interrupt in mthca). I guess that it improves performance, and
if it is good enough for the SRP (and linux) I think it will be good for
the iSER *initiator* as well.

The thing is - even if I find it reasonable to do it this way (after
calculating the estimated amount of time spent in the ISR), I am
not sure it is a good scalable solution, because there can be several
initiators and hcas per machine.

The fact that the primitiveness of linux is not a major issue, makes it
harder to decide on the right way to go here... 

> > Our iser implementation is currently context switching the completion
> > (interrupt context) to a kernel thread for the "handling" and the rest
> > of the polling. But, if linux tolerants code in isr contexts we might
> > consider changing that (at least for the initiator code).
> 
> Can you quantify the problem?
> Can you quantify how much longer you expect the driver to
> sit on the ISR?
> 
> The danger in doing too much work in the interrupt context is
> "live lock".
> 
> > in the request_irq call (mthca_eq.c) - why not set the SA_SAMPLE_RANDOM
> > to contribute to the linux entropy pool ?
> 
> Who is actually using that flag in the interrupt support routines?
> i386, x86-64, ia64, ppc, parisc do not seem to.
> The comments in arch/sparc64/kernel/irq.c: didn't sound too encouraging.

You are probably right. I did not look at the current linux
implementation of it, just the use of the flag in other drivers...

Thanks, 
Guy