[ofa-general] IPOIB CM (NOSRQ) patch -memory footprint

[Pradeep Satyanarayana]

If this proposal is acceptable, would you want me to generate a patch
against Roland's for-2.6.22 git tree, or would for-2.6.23 tree be
better?

Pradeep

Pradeep Satyanarayana wrote:
> Here are my thoughts about limiting the memory footprint for IPOIB CM
> (NOSRQ) patch:
> 
> By default, cap the NOSRQ memory usage to 1GB. The default recvq_size
> is set to 128. Therefore for 64KB packets this would imply a maximum of
> 128 endpoints.
> 
> -Make the maximum number of endpoints a module parameter with a default
> value of 128.
> 
> -The NOSRQ limit of 1GB is also made a module parameter. However, 1GB is
> the default limit and could be changed as needed (by the administrator)
> depending on the system configuration, application needs and so on. The
> server would return a "REJ" message upon receiving a "REQ", whenever one
> of these limits (i.e. max number of endpoints or the max NOSRQ memory
> usage) is reached. Currently, we only check for the maximum number of
> endpoints -hard coded to 1024.
> 
> -The IPOIB CM (NOSRQ) patch is practically transparent to HCAs that
> support SRQ like the Topspin HCA and, such HCAs should not be
> impacted at all.
> 
> -Currently we allocate a default of 64KB for the ring buffer elements,
> and this buffer size is not linked to the mtu. In the future, we could
> allocate buffers based on the mtu and link that into the computation of
> the memory cap. This way customers who might want to use a smaller mtu
> could use a larger number of endpoints, or a larger recvq_size without
> exceeding the memory cap.
> 
> 
> Would this approach address the issues of scalability and enable IPOIB
> CM to be turned as the default?
> 
> 
> Pradeep
> 
> _______________________________________________
> general mailing list
> general at lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
> 
> To unsubscribe, please visit 
> http://openib.org/mailman/listinfo/openib-general
>