[openib-general] QP with large starting sequence adds latency to RDMA READ???

[Arlin Davis]

Sean Hefty wrote:

> Hal Rosenstock wrote:
>
>> Might it be some timeout being hit before this starts working ?
>>
>> Not sure I understand it but it looks like the CM starting_psn is set 
>> to ep_ptr->qp_handle->qp_num in the uDAPL CM.
>
>

Seeding the starting_psn to the QPN is new to this latest drop of uDAPL. 
Every new QPN created in subseqeunt runs is 0x10000 greater then the 
previous and this seems to cause a hit of about 40us. This is why I went 
back and just seeded by hand to see what happens and sure enough for 
every 0x1000 increment on the seed there is an additional ~40us hit.

I am working on a verbs based test to make sure it is not some uDAPL 
issue that I introduced . I will also send out a patch to my dtest.c 
that measures the RDMA reads

> He is over-riding this value before modifying the QP.
>
>> Don't know if that has anything to do with this but shouldn't this be
>> the same PSN set for the QP ?
>
>
> Not sure what the problem is.  Initially, he was seeing a steady 
> increase in RDMA read latency of 40 us per run.  He narrowed it down 
> to setting the PSN. Hard-coding the PSN to 1 eliminated the latency 
> issue.  Setting it to higher values cause the RDMA read latency to 
> increase.


>
> - Sean
>