<html><body>
<p>Some customers have seen strange behaviors with the tcp_mem settings in /sbin/ib_ipoib_sysctl.The problems are similar to what has been reported at:<br>
<br>
<a href="http://lists.linbit.com/pipermail/drbd-user/2009-September/012711.html">http://lists.linbit.com/pipermail/drbd-user/2009-September/012711.html</a><br>
<br>
Essentially a setting of :<br>
/sbin/sysctl -q -w net.ipv4.tcp_mem="16777216 16777216 16777216"<br>
<br>
chews up so much memory that it starves other applications (including file systems) and deadlocks the system. I am not sure which OFED release this crept in, but I <br>
do understand that it is a performance tweak to help IPoIB CM. What may not have been considered is that this vector specifies pages (not bytes) as shown:<br>
<br>
tcp_mem - vector of 3 INTEGERs: min, pressure, max<br>
min: below this number of pages TCP is not bothered about its<br>
memory appetite.<br>
<br>
pressure: when amount of memory allocated by TCP exceeds this number<br>
of pages, TCP moderates its memory consumption and enters memory<br>
pressure mode, which is exited when memory consumption falls<br>
under "min".<br>
<br>
max: number of pages allowed for queueing by all TCP sockets.<br>
<br>
Defaults are calculated at boot time from amount of available<br>
memory.<br>
<br>
In effect (with this setting)TCP will not moderate it's memory consumption below 16M*4K (if page size=4k) i.e. 64GB! This may be more than the RAM available on smaller<br>
systems, and what what about the case when page size is say 64K (1024GB before TCP starts to moderate)?<br>
<br>
Can we consider removing the tcp_mem settings from the /sbin/ib_ipoib_sysctl file for OFED-1.5.3? As mentioned above, defaults are calculated at boot time<br>
based on the memory available and should be good enough for most uses.<br>
<br>
Thanks<br>
Pradeep<br>
pradeep@us.ibm.com</body></html>