[openib-general] modifying queue pair attributes - librdmacm

[Dotan Barak]

Hi.

Hoang-Nam Nguyen wrote:

>Hi Sean!
>  
>
>>>struct ibv_qp_attr attr = {
>>>               .path_mtu               = IBV_MTU_2048
>>>       };
>>>       if (ibv_modify_qp(cb->qp, &attr, IBV_QP_PATH_MTU) ){
>>>               fprintf(stderr, "Failed to modify QP\n");
>>>               fprintf(stderr, "%s \n\n", strerror(errno) );
>>>               return 1;
>>>       }
>>>
>>>
>>>And, it fails to do the job (The error string comes up with "Invalid
>>>argument".)! I can't find any function in librdmacm which allows us to
>>>modify the queue pair attributes. What am I doing wrong ?
>>>      
>>>
>>The path MTU should be set when transitioning the QP from INIT to RTR as
>>    
>>
>part of
>  
>
>>the remote node address vector.  The rdmacm sets this based on the path
>>    
>>
>record
>  
>
>>that it obtains from route resolution.  Although the kernel allows users
>>    
>>
>to
>  
>
>>modify the QP, this functionality has not been exposed through the
>>    
>>
>userspace
>  
>
>>APIs.
>>    
>>
>Not sure if I understand you properly, but ibv_modify_qp() is accessible
>from user space, which then comes to the corresponding implementation
>of ib_modify_qp(), e.g. ehca_modify_qp() or mthca_modify_qp().
>Looking at qp_state_table in verbs.c tells me that IBV_QP_PATH_MTU resp.
>IB_QP_PATH_MTU is changeable only for rc/uc and transition INIT -> RTR.
>Above code does not indicate a qp state, so it's a transient transition
>and therefor not allowed. I do assume that the underlying modify_qp()
>uses ib_modify_qp_is_ok() from Roland to validate its args. Correct?
>Regards
>Nam
>  
>
Path MTU can be set in a connected QP context ONLY in the transition 
INIT->RTR.
1) This attribute cannot be changed in other transition
2) During this transition (INIT->RTR) you must set some more attributes 
(according to the QP transport type).
If you set only the Path MTU you will get a failure ...

You should look at Table 91 in the IB spec 1.2 for more details.

I hope this answer helped you ..
Dotan