[ofa-general] IPOIB CM (NOSRQ) patch -memory footprint

[Pradeep Satyanarayana]

Roland,

Is it too late to get this into 2.6.22? If so, I will try for 2.6.23
-please let me know.

Pradeep

Pradeep Satyanarayana wrote:
> 
> If this proposal is acceptable, would you want me to generate a patch
> against Roland's for-2.6.22 git tree, or would for-2.6.23 tree be
> better?
> 
> Pradeep
> 
> Pradeep Satyanarayana wrote:
>> Here are my thoughts about limiting the memory footprint for IPOIB CM
>> (NOSRQ) patch:
>>
>> By default, cap the NOSRQ memory usage to 1GB. The default recvq_size
>> is set to 128. Therefore for 64KB packets this would imply a maximum of
>> 128 endpoints.
>>
>> -Make the maximum number of endpoints a module parameter with a default
>> value of 128.
>>
>> -The NOSRQ limit of 1GB is also made a module parameter. However, 1GB is
>> the default limit and could be changed as needed (by the administrator)
>> depending on the system configuration, application needs and so on. The
>> server would return a "REJ" message upon receiving a "REQ", whenever one
>> of these limits (i.e. max number of endpoints or the max NOSRQ memory
>> usage) is reached. Currently, we only check for the maximum number of
>> endpoints -hard coded to 1024.
>>
>> -The IPOIB CM (NOSRQ) patch is practically transparent to HCAs that
>> support SRQ like the Topspin HCA and, such HCAs should not be
>> impacted at all.
>>
>> -Currently we allocate a default of 64KB for the ring buffer elements,
>> and this buffer size is not linked to the mtu. In the future, we could
>> allocate buffers based on the mtu and link that into the computation of
>> the memory cap. This way customers who might want to use a smaller mtu
>> could use a larger number of endpoints, or a larger recvq_size without
>> exceeding the memory cap.
>>
>>
>> Would this approach address the issues of scalability and enable IPOIB
>> CM to be turned as the default?
>>
>>
>> Pradeep
>>
>> _______________________________________________
>> general mailing list
>> general at lists.openfabrics.org
>> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
>>
>> To unsubscribe, please visit 
>> http://openib.org/mailman/listinfo/openib-general
>>
> 
> 
> _______________________________________________
> general mailing list
> general at lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
> 
> To unsubscribe, please visit 
> http://openib.org/mailman/listinfo/openib-general
>