[ofa-general] retry exceeded problem with rdma_read

[Dotan Barak]

Rajouri Jammu wrote:
> Occasionally, I'm getting a retry exceeded error on the qp (error 12) 
> when doing rdma_reads.
>
> Under what conditions would thins kind of problem happen?
>
> I have the retry_count = 5 and 'am using rdma_cm for all the 
> connection setup.
>
> OFED version is 1.2.5
Does it happen between different HCAs?

If this happens during working with the QPs (not in the first message) 
than check the following thing:

If the QP attributes values of  max_rd_atomic and max_dest_rd_atomic 
this may happen.

The values should be (for sides A and B):
A.max_rd_atomic         <= B.max_dest_rd_atomic
A.max_dest_rd_atomic >= B.max_rd_atomic

(which means that RDMA Reads/atomic as initiator shouldn't be larger 
than the supported value as the destination)

You can check it by query the used QP and verify those values.



If it happens at the beginning of the connection, there may be other 
problem and i need more info  ....

Dotan