<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 TRANSITIONAL//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; CHARSET=UTF-8">
<META NAME="GENERATOR" CONTENT="GtkHTML/3.28.3">
</HEAD>
<BODY>
The tcp memory can only safely be set with some knowledge of the amount of memory available to the kernel. <BR>
<BR>
I agree with Padeep that the default behaviour of letting this be set at boot time is the safest approach.<BR>
<BR>
Where necessary, and in specific cases, this can then be overridden by a start up script. <BR>
<BR>
On Thu, 2010-12-02 at 12:06 -0800, Pradeep Satyanarayana wrote:<BR>
<BLOCKQUOTE TYPE=CITE>
Some customers have seen strange behaviors with the tcp_mem settings in /sbin/ib_ipoib_sysctl.The problems are similar to what has been reported at:<BR>
<BR>
<A HREF="http://lists.linbit.com/pipermail/drbd-user/2009-September/012711.html">http://lists.linbit.com/pipermail/drbd-user/2009-September/012711.html</A><BR>
<BR>
Essentially a setting of :<BR>
/sbin/sysctl -q -w net.ipv4.tcp_mem="16777216 16777216 16777216"<BR>
<BR>
chews up so much memory that it starves other applications (including file systems) and deadlocks the system. I am not sure which OFED release this crept in, but I <BR>
do understand that it is a performance tweak to help IPoIB CM. What may not have been considered is that this vector specifies pages (not bytes) as shown:<BR>
<BR>
tcp_mem - vector of 3 INTEGERs: min, pressure, max<BR>
min: below this number of pages TCP is not bothered about its<BR>
memory appetite.<BR>
<BR>
pressure: when amount of memory allocated by TCP exceeds this number<BR>
of pages, TCP moderates its memory consumption and enters memory<BR>
pressure mode, which is exited when memory consumption falls<BR>
under "min".<BR>
<BR>
max: number of pages allowed for queueing by all TCP sockets.<BR>
<BR>
Defaults are calculated at boot time from amount of available<BR>
memory.<BR>
<BR>
In effect (with this setting)TCP will not moderate it's memory consumption below 16M*4K (if page size=4k) i.e. 64GB! This may be more than the RAM available on smaller<BR>
systems, and what what about the case when page size is say 64K (1024GB before TCP starts to moderate)?<BR>
<BR>
Can we consider removing the tcp_mem settings from the /sbin/ib_ipoib_sysctl file for OFED-1.5.3? As mentioned above, defaults are calculated at boot time<BR>
based on the memory available and should be good enough for most uses.<BR>
<BR>
Thanks<BR>
Pradeep<BR>
pradeep@us.ibm.com<BR>
<BR>
<PRE>
_______________________________________________
ewg mailing list
<A HREF="mailto:ewg@lists.openfabrics.org">ewg@lists.openfabrics.org</A>
<A HREF="http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg">http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg</A>
</PRE>
</BLOCKQUOTE>
<BR>
</BODY>
</HTML>