[openib-general] Re: [PATCH] TX/RX_RING_SIZE as loadable parameters

[Talpey, Thomas]

At 01:20 PM 3/7/2006, Michael S. Tsirkin wrote:
>Quoting r. Roland Dreier <rdreier at cisco.com>:
>> Subject: Re: [PATCH] TX/RX_RING_SIZE as loadable parameters
>> 
>>     Michael> How about we take the ring buffer size from
>>     Michael> dev->tx_queue_len?
>> 
>> But dev->tx_queue_len is a different setting.  It's quite reasonable
>> to have the tx_queue_len be set independently of the underlying
>> pseudo-hardware work queues.
>
>It kind of makes sense to have them related though, does it not?

Again - not necessarily! The tx_queue_len is a software backlog used
if the software overdrives the hardware on transmit. It avoids losing
packets, any of which can be seen in the netstat per-interface "drops"
statistic. It only needs to be "big enough".

In fact, a large device ring means you probably only need a small tx queue.
But tuning this stuff can be a black art. Generally there is no reason to
want a silly-large tx ring. Besides, IPoIB only ever has one message in
flight, right?

In an earlier message you mentioned scaling the rx ring to the tx ring.
I think you should think more about that. The rx ring (hardware) has to
be big enough to keep packets while they await hardware interrupt processing.
So it's dependent on the arrival rate and the interrupt latency (including
any interrupt coalescing), not the transmitter depth.

All these numbers should be settable per-interface, and should attempt
to adhere to the principle of least surprise - do what other drivers in
net/ do.

Tom.