[openib-general] modifying queue pair attributes - librdmacm

[Hoang-Nam Nguyen]

Hi Sean!
> > struct ibv_qp_attr attr = {
> >                .path_mtu               = IBV_MTU_2048
> >        };
> >        if (ibv_modify_qp(cb->qp, &attr, IBV_QP_PATH_MTU) ){
> >                fprintf(stderr, "Failed to modify QP\n");
> >                fprintf(stderr, "%s \n\n", strerror(errno) );
> >                return 1;
> >        }
> >
> >
> >And, it fails to do the job (The error string comes up with "Invalid
> >argument".)! I can't find any function in librdmacm which allows us to
> >modify the queue pair attributes. What am I doing wrong ?
>
> The path MTU should be set when transitioning the QP from INIT to RTR as
part of
> the remote node address vector.  The rdmacm sets this based on the path
record
> that it obtains from route resolution.  Although the kernel allows users
to
> modify the QP, this functionality has not been exposed through the
userspace
> APIs.
Not sure if I understand you properly, but ibv_modify_qp() is accessible
from user space, which then comes to the corresponding implementation
of ib_modify_qp(), e.g. ehca_modify_qp() or mthca_modify_qp().
Looking at qp_state_table in verbs.c tells me that IBV_QP_PATH_MTU resp.
IB_QP_PATH_MTU is changeable only for rc/uc and transition INIT -> RTR.
Above code does not indicate a qp state, so it's a transient transition
and therefor not allowed. I do assume that the underlying modify_qp()
uses ib_modify_qp_is_ok() from Roland to validate its args. Correct?
Regards
Nam