[ofa-general] Very high CPU load during RDMA communication

[Dotan Barak]

Diego Guella wrote:
>
> ----- Original Message -----
>> A newbie question about  InfiniBand: while running an RDMA bandwidth
>> test I noticed that the RDMA communication occupied 99% CPU time,
>> which is much more than I expected for InfiniBand. Is this normal or
>> should I check my setup ?
>>
>> How the test was run:
>> (host 1) ib_rdma_bw -s 104857600
>> (host 2) time ib_rdma_bw -s 104857600 192.168.64.7
>
> I think it's normal, I see the same behavior here.
> If you take a look at ib_rdma_bw code, you can see that the program 
> waits in a while() loop, polling the CQ for the WC to appear.
> I think it's coded that way to get the WC as soon as possible and 
> report a more precise time (that is used after to compute BW).
>
> If you don't want that behavior in your application you could use 
> event notification for the CQ.
If one wants to reduce the CPU usage (on the expense of the latency) he 
can work with CQ events.


Dotan