Do we care about pre-emption? Was Re: [openib-general] [ANNOUNCE] Initial trunk checkin of ISERinitiator

[Paul Baxter]

From: "Grant Grundler" <iod00d at hp.com>
To: "Guy German" <guyg at voltaire.com>

>> The fact that the <preemptive-ness> of linux is not a major issue, makes 
>> it
>> harder to decide on the right way to go here...
>
> preemptive is a major issue for some uses. But I'm skeptical it is for
> the initial clusters I expect people will use RDMA for. So I'm not
> going to worry about it for now. There are more important issues.

I'm extremely pleased with what's going on in OpenIB at the moment but I 
just wanted to register an alternate view from Grant's, though I may be in a 
minority.

Infiniband is good at moving large quantities of data quickly. I need it to 
lower communications overhead so that my processors can be working and 
responding in real-time. Apps like streaming video processing and/or audio 
processing value this as well.

I don't have batch jobs taking seconds or more, I have parallel data streams 
being time multiplexed for processing across many nodes. I have distributed 
computation that needs to happen in milliseconds and not be locked out for 
milliseconds while a ponderous ISR prevents more important work from 
happening.

Your (current?) design minimising the work in interrupt context seems a good 
starting point. Why do you want to do more of that work in an ISR? Is the 
extra complexity putting it outside an ISR significant? Is the performance 
much worse?

I thought in general on Linux, even without PREEMPT_RT and alike, Linux 
kernel and driver developers were spending more effort in reducing possible 
preemption bottlenecks.

That said, I'm very excited by the work and thank you for the efforts. I'd 
like to log my data streams at 800MB/s please :)

Paul