[ewg] [PATCH ofed-1.2 docs] Updates for iWARP and Chelsio
Steve Wise
swise at opengridcomputing.com
Wed Apr 25 10:39:28 PDT 2007
Tziporet,
Below is a patch to your docs git tree. It adds a cxgb3 release notes
file, and updates the OFED release and install docs.
---
Updates for chelsio.
- added cxgb3 release notes file
- updated ofed release notes and installation guide.
Signed-off-by: Steve Wise <swise at opengridcomputing.com>
---
OFED_Installation_Guide.txt | 8 +--
OFED_release_notes.txt | 13 +++-
cxgb3_release_notes.txt | 127 +++++++++++++++++++++++++++++++++++++++++++
3 files changed, 140 insertions(+), 8 deletions(-)
diff --git a/OFED_Installation_Guide.txt b/OFED_Installation_Guide.txt
index ccc01d5..8c096b9 100644
--- a/OFED_Installation_Guide.txt
+++ b/OFED_Installation_Guide.txt
@@ -25,9 +25,9 @@ Table of contents
==============================================================================
This is the OpenFabrics Enterprise Distribution (OFED) version 1.2
-software package supporting InfiniBand fabrics. It is composed of
-several software modules intended for use on a computer cluster
-constructed as an InfiniBand subnet.
+software package supporting InfiniBand and iWARP fabrics. It is composed
+of several software modules intended for use on a computer cluster
+constructed as an InfiniBand subnet or iWARP network.
This document describes how to install the various modules and test them in
a Linux environment.
@@ -76,7 +76,7 @@ The OFED Distribution package generates
3. HW and SW Requirements
==============================================================================
-1) Server platform with InfiniBand HCA or iWARP NIC (see OFED Distribution
+1) Server platform with InfiniBand HCA or iWARP RNIC (see OFED Distribution
Release Notes for details)
2) Linux OS (see OFED Distribution Release Notes for details)
diff --git a/OFED_release_notes.txt b/OFED_release_notes.txt
index e302ea5..e4f780d 100644
--- a/OFED_release_notes.txt
+++ b/OFED_release_notes.txt
@@ -11,7 +11,7 @@ Table of Contents
1. Overview, which includes:
- OFED Distribution Rev 1.2 Contents
- Supported Platforms and Operating Systems
- - Supported HCA Adapter Cards and Firmware Versions
+ - Supported HCA and RNIC Adapter Cards and Firmware Versions
- Tested Switch Platforms
- Third party Test Packages
- OFED sources
@@ -36,7 +36,7 @@ all of its nodes to this new version.
---------------------
The OFED package contains the following components:
o OpenFabrics core and ULPs:
- - HCA drivers (mthca, ipath, ehca)
+ - IB HCA and RNIC drivers (mthca, ipath, ehca, cxgb3)
- core
- Upper Layer Protocols: IPoIB, SDP, SRP Initiator, iSER Host and uDAPL
o OpenFabrics utilities:
@@ -57,6 +57,7 @@ Notes:
3. All other OFED components are of production quality.
4. See release notes for each package in the docs directory.
5. Any Topspin copyright belongs to Cisco Systems, Inc.
+6. cxgb3 driver is in technology preview state.
1.2 Supported Platforms and Operating Systems
---------------------------------------------
@@ -73,9 +74,10 @@ Notes:
- SLES10: 2.6.16.21-0.8-smp
- kernel.org: 2.6.17.x and 2.6.18.x
-1.3 HCAs Supported
+1.3 HCAs and RNICs Supported
------------------
-This release supports HCAs by Mellanox Technologies, Qlogic and IBM.
+This release supports IB HCAs by Mellanox Technologies, Qlogic and IBM as
+well as iWARP RNICs by Chelsio Communications.
o Mellanox Technologies HCAs:
- InfiniHost (fw-23108 Rev 3.5.000)
@@ -96,6 +98,9 @@ This release supports HCAs by Mellanox T
- GX Dual-port 4x IB HCA
- GX Dual-port 12x IB HCA
+ o Chelsio RNICs:
+ - S310/S320 10GbE Storage Accelerators
+ - R310E 10GbE iWARP Adapters
1.4 Switches Supported
----------------------
diff --git a/cxgb3_release_notes.txt b/cxgb3_release_notes.txt
new file mode 100644
index 0000000..85bc774
--- /dev/null
+++ b/cxgb3_release_notes.txt
@@ -0,0 +1,127 @@
+
+ CHELSIO T3 RNIC RELEASE NOTES
+
+Author: Steve Wise
+Last Updated: April, 2007
+
+The iw_cxgb3 and cxgb3 modules provide iWARP and NIC support for the
+Chelsio S310, S320, and R310 adapters. Make sure you choose the 'cxgb3'
+options when generating your ofed-1.2 rpms.
+
+This release is a technology preview.
+
+============================================
+Loadable Module options:
+============================================
+
+The following options can be used when loading the iw_cxgb3 module to
+tune the iWARP driver:
+
+cong_flavor - set the congestion congtrol algorithm. Default is 1.
+ 0 == Reno
+ 1 == Tahoe
+ 2 == NewReno
+ 3 == HighSpeed
+
+snd_win - set the TCP send window in bytes. Default is 32KB.
+
+rcv_win - set the TCP receive window in bytes. Default is 256KB.
+
+crc_enabled - set whether MPA CRC should be negotiated. Default is 1.
+
+markers_enabled - set whether to request receiving MPA markers. Default is
+ 0; do not request to receive markers.
+
+ NOTE: The Chelsio RNIC fully supports markers, but
+ the current OFA RDMA-CM doesn't provide an API for
+ requesting either markers or crc to be negotiated. Thus
+ this functionality is provided via module parameters.
+
+mpa_rev - set the MPA revision to be used. Default is 1, which is
+ spec compliant. Set to 0 to connect with the Ammasso 1100
+ rnic.
+
+ep_timeout_secs - set the number of seconds for timing out MPA start up
+ negotiation and normal close. Default is 10.
+
+The following options can be used when loading the cxgb3 module to
+tune the NIC driver:
+
+msi - whether to use MSI or MSI-X. Default is 2.
+ 0 = only pin
+ 1 = only MSI or pin
+ 2 = use MSI/X, MSI, or pin, based on system
+
+============================================
+Updating Firmware:
+============================================
+
+Contact chelsio to obtain the latest firmware and cxgbtool source.
+
+To build cxgbtool:
+
+# cd <path-to-cxgbtool>
+# make && make install
+
+Then load the cxgb3 driver:
+
+# modprobe cxgb3
+
+Now note the ethernet interface name for the T3 device. This can be
+done by typing 'ifconfig -a' and noting the interface name for the
+interface with a HW address that begins with "00:07:43". Then load the
+new firmware:
+
+# cxgbtool ethxx loadfw <firmware_file>
+# reboot
+
+============================================
+Testing connectivity with ping and rping:
+============================================
+
+Configure the ethernet interfaces for your cxgb3 device. After you
+modprobe iw_cxgb3 you will see one or two ethernet interfaces for the
+T3 device. Configure them with an appropriate ip address, netmask, etc.
+You can use the Linux ping command to test basic connectivity via the
+T3 interface.
+
+To test RDMA, use the rping command that is included in the librdmacm-utils
+rpm:
+
+On the server machine:
+
+# rping -s -a 0.0.0.0 -p 9999
+
+On the client machine:
+
+# rping -c -VvC10 -a server_ip_addr -p 9999
+
+You should see ping data like this on the client:
+
+ping data: rdma-ping-0: ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqr
+ping data: rdma-ping-1: BCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrs
+ping data: rdma-ping-2: CDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrst
+ping data: rdma-ping-3: DEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstu
+ping data: rdma-ping-4: EFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuv
+ping data: rdma-ping-5: FGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvw
+ping data: rdma-ping-6: GHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwx
+ping data: rdma-ping-7: HIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxy
+ping data: rdma-ping-8: IJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz
+ping data: rdma-ping-9: JKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyzA
+client DISCONNECT EVENT...
+#
+
+============================================
+Addition Notes and Issues
+============================================
+
+1) To run uDAPL over the chelsio device, you must export this environment
+variable:
+
+ export DAPL_MAX_INLINE=64
+
+2) If you have a multi-homed host and the physical ethernet networks are
+bridged, then you need to configure arp to only send replies on the
+interface with the target ip address:
+
+ sysctl -w net.ipv4.conf.all.arp_ignore=2
More information about the ewg
mailing list