[ewg] Questions about CC in OFED

[Michael Zhang]

On 6/28/12, Ira Weiny <weiny2 at llnl.gov> wrote:
> On Wed, 27 Jun 2012 11:41:03 +0800
> Michael Zhang <ziwen1203 at gmail.com> wrote:
>
>> Hi everybody,
>>
>>   I focus on the source code of  IB congestion control in OFED, such as
>> CCT
>> related data structure and FECN/BECN process. But the only code related
>> to
>> the congestion control mechanism is found in the module of
>> "infinipath-psm". As I know, psm works for MPI applications and actually
>> I
>> haven't installed psm in our system. I have done experiment through IPoIB
>> and the congestion control still works that makes me confused.
>
> I am not familiar with how psm may set CC parameters.  Perhaps someone from
> QLogic/Intel can speak up here?
>
>>   Could someone tell me where is the CC-related source code located
>> besides
>> "infinipath-psm"? Or some functions are realized in hardware such as the
>> movement of CCT Index when receives BECN message?
>>   Thank you for your help!
>
> The latest version of infiniband-diags (1.6.0)[*] has 2 tools (ibccquery and
> ibccconfig) which will allow one to query and set the CC parameters.  As you
> will note in the man page as well as the help output these tools are not to
> be used lightly.  You can cause instability in your fabric if you set things
> wrong.  However, when used properly they work well.
>
> In addition, Al Chu from LLNL has posted a patch to OpenSM which allows for
> the setting of CC parameters in OpenSM.  Feedback from this patch has been
> minimal but Alex Netes (maintainer of OpenSM) says he will review it soon.
> (http://www.spinics.net/lists/linux-rdma/msg11615.html)
>
> Finally, if you are interested in the ongoing development of this and other
> features of Open Fabrics code you may want to sign up for the linux-rdma
> mailing list (linux-rdma at vger.kernel.org or
> http://vger.kernel.org/vger-lists.html).
>

As I tested, the tool of "ibccquery" and "ibccconfig" in
infiniband-diags (1.6.0) works well to "get" and "set" CC parameters.
It is helpful for us. I really appreciate it.

The tools resolve the configuration of cc related parameters. On the
other side, we also care about the implementation of congestion
control mechanism itself, such as CCTI is increased by CCTI_Increase
when receiving BECN and feedback BECN if receiving data packet with
FECN tag. I find no data structure of CC (like CCTI) in QP Context
since CC is based on QP and no realization of ECN mechanism  as well
in the source code of OFED. Can you give me some cluses of this in
OFED source code?
Thank you for ur help!

Best,
Ziwen


> Ira
>
> [*] git://beany.openfabrics.org/~iraweiny/infiniband-diags.git
>
>>
>> Best,
>> Ziwen
>>
>> --
>> Ziwen(Michael) Zhang Ph.D candidate
>> College of Computer
>> National University of Defense Technology (NUDT) , P.R.China
>> Email: ziwen at nudt.edu.cn ziwen1203 at gmail.com
>
>
> --
> Ira Weiny
> Member of Technical Staff
> Lawrence Livermore National Lab
> 925-423-8008
> weiny2 at llnl.gov
>


-- 
Ziwen(Michael) Zhang Ph.D candidate
College of Computer
National University of Defense Technology (NUDT) , P.R.China
Email: ziwen at nudt.edu.cn ziwen1203 at gmail.com