[openib-general] why sdp connections cost so much memory
zhu shi song
zhushisongzhu at yahoo.com
Tue Aug 15 02:13:24 PDT 2006
(1) ibv_devinfo
HCA: MHES18-XTC
FW: 1.1.0
OFED: OFED-1.1-rc1
(2) Test Bed
On Client:
ib0: 193.12.10.24
test command:
LD_PRELOAD=/usr/local/ofed/lib64/libsdp.so
SIMPLE_LIBSDP=1 ab -c m -n m -X 193.12.10.14:3129
http://www.sse.com.cn/sseportal/ps/zhs/home.shtml
The web page is about 68K.
On Server:
ib0: 193.12.10.14
squid.sdp -d 10 -f squid2.conf (I have changed
squid-cache to support listening on SDP port 3129)
The test result is :
Concurrent Conns(=m) Free Memory Requests
completed
0 926980 0
100 712508 100
200 497372 200
300 282636 256
400 52868 256
500 kernel crashed because of
"out of memory"
>From above, every about 100 concurrent SDP connections
will cost 210M memory. It's too vast for large scale
applications. TCP costs very lower memory than SDP.
The max concurrent connections completed successfully
is 256. it is some bad limit. Who knows how and when
will solve the problem?
I'll test the performance of sdp connection and
compare it with TCP further.
tks
zhu
--- openib-general-request at openib.org wrote:
> Send openib-general mailing list submissions to
> openib-general at openib.org
>
> To subscribe or unsubscribe via the World Wide Web,
> visit
> http://openib.org/mailman/listinfo/openib-general
> or, via email, send a message with subject or body
> 'help' to
> openib-general-request at openib.org
>
> You can reach the person managing the list at
> openib-general-owner at openib.org
>
> When replying, please edit your Subject line so it
> is more specific
> than "Re: Contents of openib-general digest..."
>
>
> Today's Topics:
>
> 1. Re: Conflicting typedefs for "ib_gid_t" (Sean
> Hefty)
> 2. Re: [PATCH] mthca: make IB_SEND_FENCE work
> (Michael S. Tsirkin)
> 3. Re: Conflicting typedefs for "ib_gid_t" (Hal
> Rosenstock)
> 4. sanity check on datapath (Michael S. Tsirkin)
> 5. Re: [PATCH] mthca: make IB_SEND_FENCE work
> (Sean Hefty)
> 6. Re: Conflicting typedefs for "ib_gid_t"
> (Lakshmanan, Madhu)
> 7. Re: IB mcast question (Steve Wise)
> 8. Re: Conflicting typedefs for "ib_gid_t"
> (Michael S. Tsirkin)
>
>
>
----------------------------------------------------------------------
>
> Message: 1
> Date: Mon, 14 Aug 2006 13:38:42 -0700
> From: "Sean Hefty" <sean.hefty at intel.com>
> Subject: Re: [openib-general] Conflicting typedefs
> for "ib_gid_t"
> To: "'Michael S. Tsirkin'" <mst at mellanox.co.il>,
> "Hal Rosenstock"
> <halr at voltaire.com>
> Cc: openib-general at openib.org
> Message-ID:
> <000101c6bfe1$a1cdbe40$8698070a at amr.corp.intel.com>
> Content-Type: text/plain; charset=us-ascii
>
> >Yes, or ib_mad_, or ibmad_. And same for other IB_
> and ib_ names there.
> >We really need to do something about names like
> ib_attr_t.
>
> I like to move away from each library re-defining
> common IB data types.
> Something like ibv_gid should be picked up from
> libibverbs.
>
> IMO, the core of the problem is that opensm include
> files carry too many legacy
> typedefs.
>
> - Sean
>
>
>
> ------------------------------
>
> Message: 2
> Date: Mon, 14 Aug 2006 23:41:54 +0300
> From: "Michael S. Tsirkin" <mst at mellanox.co.il>
> Subject: Re: [openib-general] [PATCH] mthca: make
> IB_SEND_FENCE work
> To: "Sean Hefty" <mshefty at ichips.intel.com>
> Cc: Roland Dreier <rdreier at cisco.com>,
> openib-general at openib.org
> Message-ID: <20060814204154.GK16821 at mellanox.co.il>
> Content-Type: text/plain; charset=us-ascii
>
> Quoting r. Sean Hefty <mshefty at ichips.intel.com>:
> > This could eliminate the warning, and remove an if
> statement from executing on
> > each iteration.
>
> We still need to test size0 to set size0 = size.
> So we just reuse the extra branch, and I agree with
> Roland
> this way code is clearer.
>
> --
> MST
>
>
>
> ------------------------------
>
> Message: 3
> Date: Mon, 14 Aug 2006 23:39:00 +0300
> From: "Hal Rosenstock" <halr at voltaire.com>
> Subject: Re: [openib-general] Conflicting typedefs
> for "ib_gid_t"
> To: mlakshmanan at silverstorm.com, "Michael S.
> Tsirkin"
> <mst at mellanox.co.il>
> Cc: openib-general at openib.org
> Message-ID:
>
>
<5CE025EE7D88BA4599A2C8FEFCF226F589AD1B at taurus.voltaire.com>
> Content-Type: text/plain; charset=iso-8859-1
>
> To answer your questions:
>
> I'm not totally sure about your application but it
> seems to me to fall in the category being discussed.
>
> Not all of the definitions in ib_types.h are
> elsewhere.
>
> I am working on a patch to get you past this issue.
>
> -- Hal
>
> ________________________________
>
> From: mlakshmanan at silverstorm.com
> [mailto:mlakshmanan at silverstorm.com]
> Sent: Mon 8/14/2006 4:35 PM
> To: Hal Rosenstock; Michael S. Tsirkin
> Cc: mlakshmanan at silverstorm.com;
> openib-general at openib.org
> Subject: Re: Conflicting typedefs for "ib_gid_t"
>
>
>
> >>> I don't think the way forward is using iba/ in
> all applications.
> >>> I see it mostly as a legacy header for opensm
> and related apps
> >>> that want their own layer for portability
> between stacks.
> > >
> > > I agree with your view on iba/ib_types.h
>
> Does that imply that the definitions in
> iba/ib_types.h are not expected to be required or
> used
> by user-space applications other than those
> categories mentioned above? If iba/ib_types.h is
> only a legacy header, are the definitions also
> present in another header file?
>
> Madhu
>
>
>
>
>
>
> ------------------------------
>
> Message: 4
> Date: Mon, 14 Aug 2006 23:45:21 +0300
> From: "Michael S. Tsirkin" <mst at mellanox.co.il>
> Subject: [openib-general] sanity check on datapath
> To: "Roland Dreier" <rolandd at cisco.com>,
> openib-general at openib.org
> Message-ID: <20060814204521.GL16821 at mellanox.co.il>
> Content-Type: text/plain; charset=us-ascii
>
> Roland, do we really need code like
>
> if (wr->opcode >= sizeof mthca_opcode / sizeof
> mthca_opcode[0])
> {
> ret = -1;
> *bad_wr = wr;
> goto out;
> }
>
> in mthca on data path? Should this be put within
> ifdef DEBUG or something?
>
> --
> MST
>
>
>
> ------------------------------
>
> Message: 5
> Date: Mon, 14 Aug 2006 13:47:01 -0700
> From: "Sean Hefty" <mshefty at ichips.intel.com>
> Subject: Re: [openib-general] [PATCH] mthca: make
> IB_SEND_FENCE work
> To: "Michael S. Tsirkin" <mst at mellanox.co.il>
> Cc: Roland Dreier <rdreier at cisco.com>,
> openib-general at openib.org
> Message-ID: <44E0E145.60906 at ichips.intel.com>
>
=== message truncated ===
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
More information about the general
mailing list