<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:x="urn:schemas-microsoft-com:office:excel" xmlns:p="urn:schemas-microsoft-com:office:powerpoint" xmlns:a="urn:schemas-microsoft-com:office:access" xmlns:dt="uuid:C2F41010-65B3-11d1-A29F-00AA00C14882" xmlns:s="uuid:BDC6E3F0-6DA3-11d1-A2A3-00AA00C14882" xmlns:rs="urn:schemas-microsoft-com:rowset" xmlns:z="#RowsetSchema" xmlns:b="urn:schemas-microsoft-com:office:publisher" xmlns:ss="urn:schemas-microsoft-com:office:spreadsheet" xmlns:c="urn:schemas-microsoft-com:office:component:spreadsheet" xmlns:odc="urn:schemas-microsoft-com:office:odc" xmlns:oa="urn:schemas-microsoft-com:office:activation" xmlns:html="http://www.w3.org/TR/REC-html40" xmlns:q="http://schemas.xmlsoap.org/soap/envelope/" xmlns:rtc="http://microsoft.com/officenet/conferencing" xmlns:D="DAV:" xmlns:Repl="http://schemas.microsoft.com/repl/" xmlns:mt="http://schemas.microsoft.com/sharepoint/soap/meetings/" xmlns:x2="http://schemas.microsoft.com/office/excel/2003/xml" xmlns:ppda="http://www.passport.com/NameSpace.xsd" xmlns:ois="http://schemas.microsoft.com/sharepoint/soap/ois/" xmlns:dir="http://schemas.microsoft.com/sharepoint/soap/directory/" xmlns:ds="http://www.w3.org/2000/09/xmldsig#" xmlns:dsp="http://schemas.microsoft.com/sharepoint/dsp" xmlns:udc="http://schemas.microsoft.com/data/udc" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:sub="http://schemas.microsoft.com/sharepoint/soap/2002/1/alerts/" xmlns:ec="http://www.w3.org/2001/04/xmlenc#" xmlns:sp="http://schemas.microsoft.com/sharepoint/" xmlns:sps="http://schemas.microsoft.com/sharepoint/soap/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:udcs="http://schemas.microsoft.com/data/udc/soap" xmlns:udcxf="http://schemas.microsoft.com/data/udc/xmlfile" xmlns:udcp2p="http://schemas.microsoft.com/data/udc/parttopart" xmlns:wf="http://schemas.microsoft.com/sharepoint/soap/workflow/" xmlns:dsss="http://schemas.microsoft.com/office/2006/digsig-setup" xmlns:dssi="http://schemas.microsoft.com/office/2006/digsig" xmlns:mdssi="http://schemas.openxmlformats.org/package/2006/digital-signature" xmlns:mver="http://schemas.openxmlformats.org/markup-compatibility/2006" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns:mrels="http://schemas.openxmlformats.org/package/2006/relationships" xmlns:spwp="http://microsoft.com/sharepoint/webpartpages" xmlns:ex12t="http://schemas.microsoft.com/exchange/services/2006/types" xmlns:ex12m="http://schemas.microsoft.com/exchange/services/2006/messages" xmlns:pptsl="http://schemas.microsoft.com/sharepoint/soap/SlideLibrary/" xmlns:spsl="http://microsoft.com/webservices/SharePointPortalServer/PublishedLinksService" xmlns:Z="urn:schemas-microsoft-com:" xmlns:st="" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=Content-Type content="text/html; charset=utf-8">
<meta name=Generator content="Microsoft Word 12 (filtered medium)">
<style>
<!--
/* Font Definitions */
@font-face
{font-family:"MS Gothic";
panose-1:2 11 6 9 7 2 5 8 2 4;}
@font-face
{font-family:MingLiU;
panose-1:2 2 5 9 0 0 0 0 0 0;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:"\@MingLiU";
panose-1:2 2 5 9 0 0 0 0 0 0;}
@font-face
{font-family:"\@MS Gothic";
panose-1:2 11 6 9 7 2 5 8 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
-->
</style>
<!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=EN-US link=blue vlink=purple>
<div class=WordSection1>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Hi,<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>I don’t think that we have ever tried to run neither wsd nor sdp
on .net framework code, so I don’t know if it will work or not.<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>In any case, wsd has performance counters that you can use to
verify if the traffic is going through or not (under ib winsock direct).<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>So please open this performance counters, and see if your
program is using it or not?<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Thanks<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Tzachi<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<div style='border:none;border-right:solid blue 1.5pt;padding:0in 0in 0in 4.0pt'>
<div>
<div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in'>
<p class=MsoNormal><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span
style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>
lock.zhang@gmail.com [mailto:lock.zhang@gmail.com] <b>On Behalf Of </b>suo
zhang<br>
<b>Sent:</b> Wednesday, January 26, 2011 8:25 AM<br>
<b>To:</b> Tzachi Dar<br>
<b>Cc:</b> ofw@lists.openfabrics.org<br>
<b>Subject:</b> Re: [ofw] WSD Detail Sample Code<o:p></o:p></span></p>
</div>
</div>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span
style='color:#1F497D'>Hi </span><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span
style='color:#1F497D'> </span><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span
style='color:#1F497D'>If the point 2 is true, we DO did the test with the
WinSocket based code (we use .Net Framework 4.0 and use IO Completion port to
send/receive the data in async mode), what the problem mentioned is that:</span><o:p></o:p></p>
<p class=MsoListParagraph style='margin-left:.25in;text-align:justify;
text-indent:-.25in'><span style='color:#1F497D'>1.</span><span
style='font-size:7.0pt;color:#1F497D'> </span><span
style='color:#1F497D'>I run my testing code try to send large data set and the
performance throught can’t reach 300MB/s;</span><o:p></o:p></p>
<p class=MsoListParagraph style='margin-left:.25in;text-align:justify;
text-indent:-.25in'><span style='color:#1F497D'>2.</span><span
style='font-size:7.0pt;color:#1F497D'> </span><span
style='color:#1F497D'>I run the SDPConnect.exe tool</span><span
style='font-family:"MS Gothic";color:#1F497D'>（</span><span style='color:#1F497D'>It
can be found in server installed in MLNX_VPI/IB/Tools folder</span><span
style='font-family:"MS Gothic";color:#1F497D'>）</span><span style='color:#1F497D'>
use Overlapped send and receive in both side of the testing machine, it reached
900+MB/s throughput performance result.</span><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span
style='color:#1F497D'> </span><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span
style='color:#1F497D'>So it turns out that we must seek more information from
documentation or SDK for how to write such WinSocket based App, otherwise we just
use InfiniBand as a normal IP based LAN.</span><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span
style='font-size:10.5pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<div>
<p class=MsoNormal>On Tue, Jan 25, 2011 at 5:26 PM, Tzachi Dar <<a
href="mailto:tzachid@mellanox.co.il">tzachid@mellanox.co.il</a>> wrote:<o:p></o:p></p>
<div>
<div>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span
style='font-size:11.0pt;color:#1F497D'>Hi,</span><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span
style='font-size:11.0pt;color:#1F497D'> </span><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span
style='font-size:11.0pt;color:#1F497D'>It seems from your question that there
is some confusion:</span><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span
style='font-size:11.0pt;color:#1F497D'> </span><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span
style='font-size:11.0pt;color:#1F497D'>There are two ways to work with IB:</span><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span
style='font-size:11.0pt;color:#1F497D'> </span><o:p></o:p></p>
<p><span style='font-size:11.0pt;color:#1F497D'>1)</span><span
style='font-size:7.0pt;color:#1F497D'> </span><span
style='font-size:11.0pt;color:#1F497D'>Use native ib program. This means that
you have to write your code differently, in order for it to work. To see
examples of such programs you can download our code from svn://<a
href="http://openib.tc.cornell.edu/gen1/trunk" target="_blank">openib.tc.cornell.edu/gen1/trunk</a>.
there are many examples there (trunk\tools\perftests\user\send_bw, or
trunk\tests\perftest\send_bw). In general in windows there are many apis that
one can use. I’m currently not explaining this thoroughly, because I believe
that you want to go to option #2.</span><o:p></o:p></p>
<p><span style='font-size:11.0pt;color:#1F497D'>2)</span><span
style='font-size:7.0pt;color:#1F497D'> </span><span
style='font-size:11.0pt;color:#1F497D'>Use one of WSD or SDP. With this
approach you don’t have to change anything in your source code. In fact, you
don’t actually need to have the source code at all. For example the sdpconnect
program (source attached) is a native windows socket program. You can find more
information about how to run your program with one of this in the Mellanox
documentation. Please also read the limitations section, as both have their
limitations. </span><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;
margin-left:.25in'><span style='font-size:11.0pt;color:#1F497D'> </span><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span
style='font-size:11.0pt;color:#1F497D'>By the way, when you write
“SDPConnect.exe showed great performance”, how did you run it (wsd?
SDP?)?</span><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span
style='font-size:11.0pt;color:#1F497D'> </span><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span
style='font-size:11.0pt;color:#1F497D'>Thanks</span><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span
style='font-size:11.0pt;color:#1F497D'>Tzachi</span><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span
style='font-size:11.0pt;color:#1F497D'> </span><o:p></o:p></p>
<div style='border:none;border-right:solid blue 1.5pt;padding:0in 0in 0in 4.0pt'>
<div>
<div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in'>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><b><span
style='font-size:10.0pt'>From:</span></b><span style='font-size:10.0pt'> <a
href="mailto:ofw-bounces@lists.openfabrics.org" target="_blank">ofw-bounces@lists.openfabrics.org</a>
[mailto:<a href="mailto:ofw-bounces@lists.openfabrics.org" target="_blank">ofw-bounces@lists.openfabrics.org</a>]
<b>On Behalf Of </b>suo zhang<br>
<b>Sent:</b> Tuesday, January 25, 2011 10:13 AM<br>
<b>To:</b> <a href="mailto:ofw@lists.openfabrics.org" target="_blank">ofw@lists.openfabrics.org</a><br>
<b>Subject:</b> Re: [ofw] WSD Detail Sample Code</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span
style='font-size:10.0pt'>Hi<br>
<br>
Following is our question.<br>
<br>
Thanks<br>
<br>
Suo<br>
<br>
----------------------------------------------------------------------------<br>
--------------------------------------------<br>
<br>
<br>
<br>
We already wrote a IOCP and WinSocket based communication program. The<br>
testing upon Infiniband indicated that this program still go over the<br>
IPOverIP and has the overhead of Windows IP stack.<br>
<br>
<br>
<br>
We saw this PPT and want to know how our WinSocket based program can fully<br>
utilize the capabilities of Inifniband RDMA and how to write code over WSD<br>
to avoid the overhead of Windows IP stack.<br>
<br>
<br>
<br>
We just need a sample code like the source code of tool<br>
SDPConnect.exe(within the Mallanox driver tools folder) etc. It showed great<br>
performance we can’t achieved by our WinSocket code.<br>
<br>
<br>
<br>
We’ve got some progress improvement:<br>
<br>
1. By searching the source code in OpenFabric, it turns
out that these<br>
open source include some tools’ source code, like this c file attached;<br>
<br>
2. There was some code founded in such tool’s source code
like:<br>
<br>
while (ccnt < user_param->iters || rcnt < user_param->iters ) {<br>
<br>
while (scnt < user_param->iters
&& (scnt - ccnt) <<br>
user_param->tx_depth / 2) {<br>
<br>
ib_send_wr_t *bad_wr;<br>
<br>
tposted[scnt]
= get_cycles();<br>
<br>
ib_status
= ib_post_send(qp, &ctx->wr, &bad_wr);<br>
<br>
if
(ib_status != IB_SUCCESS) {<br>
<br>
fprintf(stderr, "Couldn't post send: scnt=%d<br>
ib_status %d\n",<br>
<br>
scnt,ib_status);<br>
<br>
return 1;<br>
<br>
}<br>
<br>
++scnt;<br>
<br>
PERF_DEBUG("scnt = %d \n",scnt);<br>
<br>
}<br>
<br>
<br>
<br>
But no document just have to guess from the source code for what “qp”,<br>
“ctx” means and how to use them. These “ib_” initiated APIs were<br>
absolutely NOT WinSocket APIs. What we asked for is SDK or documents for how<br>
to using such API and which Header files we should include and which lib<br>
have to be linked. A sample project will be OK if no such document exists.<br>
<br>
<br>
<br>
<br>
</span>Thanks <o:p></o:p></p>
<div>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p>
</div>
<div>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Suo<o:p></o:p></p>
<div>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>2011/1/25
Smith, Stan <<a href="mailto:stan.smith@intel.com" target="_blank">stan.smith@intel.com</a>><o:p></o:p></p>
<div>
<div>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span
style='font-size:11.0pt;color:#1F497D'>Please submit modified question to <a
href="mailto:ofw@lists.openfabrics.org" target="_blank">ofw@lists.openfabrics.org</a></span><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span
style='font-size:11.0pt;color:#1F497D'> </span><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span
style='font-size:11.0pt;color:#1F497D'>Furthermore, please identify which
sample code you are referring to in order that someone can respond to your
question.</span><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span
style='font-size:11.0pt;color:#1F497D'> </span><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span
style='font-size:11.0pt;color:#1F497D'>Which Windows OS and processor
(x86,x64,ia64).</span><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span
style='font-size:11.0pt;color:#1F497D'> </span><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span
style='font-size:11.0pt;color:#1F497D'>Generally speaking WSD (WinSock Direct)
is not used directly but utilized by opening a socket(AF_INET) which beneath
the socket layer that has a WSD provider enabled for the specific address
family.</span><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span
style='font-size:11.0pt;color:#1F497D'>For winOFED 2.3 release, run ‘installsp
–l’ to see WSD providers.</span><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span
style='font-size:11.0pt;color:#1F497D'>WSD in no longer enabled by default for
svr2008* – see ‘installsp –I’</span><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span
style='font-size:11.0pt;color:#1F497D'> </span><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span
style='font-size:11.0pt;color:#1F497D'>Stan.</span><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span
style='font-size:11.0pt;color:#1F497D'> </span><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span
style='font-size:11.0pt;color:#1F497D'> </span><o:p></o:p></p>
<div>
<p class=MsoNormal style='mso-margin-top-alt:auto;margin-bottom:12.0pt'>From: <b>Suo
Zhang</b> <<a href="mailto:zhangs@bjcarnation.com.cn" target="_blank">zhangs@bjcarnation.com.cn</a>><br>
Date: 2011/1/20<br>
Subject: WSD Detail Sample Code<br>
To: <a href="mailto:membership@openfabrics.org" target="_blank">membership@openfabrics.org</a><o:p></o:p></p>
<div>
<div>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Hi
<o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>When
I use WSD in my project, I found the sample code is very difficult to use.
Because the sample code is very simple.<o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>I
want to know where can I get a detail sample code for WSD. Because my
application must use zero copy to get good performance.<o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Thanks<o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Best
regards<o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span
lang=ZH-CN style='font-family:MingLiU'>张锁</span> Suo Zhang<o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Sales
Engineer<o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span
lang=ZH-CN style='font-family:"MS Gothic"'>北京石竹科技有限公司</span><span lang=ZH-CN> </span><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Beijing
Carnation Technologies Limited<o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Mobile
+86-186-1175-6167<o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Tel<span
lang=ZH-CN style='font-family:"MS Gothic"'>：</span>+8610-68587971-8029<o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Fax<span
lang=ZH-CN style='font-family:"MS Gothic"'>：</span>+8610-68587975<o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><a
href="http://www.vme.cn/" target="_blank">Http://www.vme.cn</a><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p>
</div>
</div>
</div>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p>
</div>
</div>
</div>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<p class=MsoNormal><o:p> </o:p></p>
</div>
</div>
</body>
</html>