[ofa-general] ***SPAM*** Re: iSer tuning guide?

[Chris Worley]

It looks like it's the target side at fault.  The target in stalled is
the one pointed to by the OFED wiki:

http://www.voltaire.com/ftp/support-products/source/stgt/scsi-target-utils-0.1-20080828.x86_64.rpm

... it's mind-boggling to believe that any OFED component that's 8
months old would be compatible with the latest release... but that
directory isn't browsable, so it's hard to tell if there is something
newer.

Most of the time, if I restart the tgtd and opensm/openibd on the
target system, then restart just iscsi on the initiators, I can get
iser working again after it hangs... but sometimes the tgtd gets into
a state where it segfaults at launch, and I have to reboot the target
machine before things work again.

It looks like it works great with one initiator and one target... any
more than that and the connections are hung and the target needs
restarted.

I can run "fio" with one thread against two targets, and it's okay...
more than one thread, and it hangs.

Multiple initiators (separate systems) work as long as the access is
serial... accessing two different or the same target from two
different initiators simultaneously, and it hangs.

I have seen it crash the initiator also, but no console to see what
the crash was.

So, I guess I'll just start trying recent OFED versions until I find a
stable iser. If anybody can give me a hint as to what might be both
stable and perform well, it would be appreciated.

Thanks,

Chris
On Fri, Apr 10, 2009 at 2:35 PM, Chris Worley <worleys at gmail.com> wrote:
> What's version of OFED has both a stable and high performance iSer?
>
> I setup the latest OFED (1.4.1rc3), and the speed kicked in, but using
> fio on the initiator using two targets hung quickly on the initiator:
>
>  connection3:0: ping timeout of 5 secs expired, last rx 4557795905,
> last ping 4557800905, now 4557805905
>  connection3:0: detected conn error (1011)
> iser: iscsi_iser_ep_disconnect:ib conn ffff8104dc482ec0 state 2
> iser: iser_cq_tasklet_fn:comp w. error op 0 status 5
> iser: iser_cq_tasklet_fn:comp w. error op 0 status 5
> iser: iser_cq_tasklet_fn:comp w. error op 0 status 5
> iser: iser_cq_tasklet_fn:comp w. error op 0 status 5
> iser: iser_cq_tasklet_fn:comp w. error op 0 status 5
>  connection4:0: ping timeout of 5 secs expired, last rx 4557796701,
> last ping 4557801701, now 4557806701
>  connection4:0: detected conn error (1011)
>  connection1:0: ping timeout of 5 secs expired, last rx 4557799350,
> last ping 4557804350, now 4557809350
>  connection1:0: detected conn error (1011)
>  connection2:0: ping timeout of 5 secs expired, last rx 4557800108,
> last ping 4557805108, now 4557810108
>  connection2:0: detected conn error (1011)
>  connection6:0: ping timeout of 5 secs expired, last rx 4557800224,
> last ping 4557805224, now 4557810224
>  connection6:0: detected conn error (1011)
>  connection5:0: ping timeout of 5 secs expired, last rx 4557800231,
> last ping 4557805231, now 4557810234
>  connection5:0: detected conn error (1011)
> iser: iser_cma_handler:event 10 conn ffff8104dc482ec0 id ffff8105b1502800
> iser: iser_free_ib_conn_res:freeing conn ffff8104dc482ec0 cma_id
> ffff8105b1502800 fmr pool ffff8105f9b555c0 qp ffff81027b346000
> iser: iser_device_try_release:device ffff81081bb81640 refcount 5
> iser: iscsi_iser_ep_disconnect:ib conn ffff8104dc4824c0 state 2
> iser: iser_cq_tasklet_fn:comp w. error op 0 status 5
> iser: iser_cq_tasklet_fn:comp w. error op 0 status 5
> iser: iser_cq_tasklet_fn:comp w. error op 0 status 5
> iser: iser_cq_tasklet_fn:comp w. error op 0 status 5
> iser: iser_cq_tasklet_fn:comp w. error op 0 status 5
>  session3: session recovery timed out after 120 secs
> iser: iser_cma_handler:event 10 conn ffff8104dc4824c0 id ffff8107df2ce400
> iser: iser_free_ib_conn_res:freeing conn ffff8104dc4824c0 cma_id
> ffff8107df2ce400 fmr pool ffff8107f60c90c0 qp ffff8107720d2400
> iser: iser_device_try_release:device ffff81081bb81640 refcount 4
> iser: iscsi_iser_ep_disconnect:ib conn ffff81081fa521c0 state 2
> iser: iser_cq_tasklet_fn:comp w. error op 0 status 5
> iser: iser_cq_tasklet_fn:comp w. error op 0 status 5
> iser: iser_cq_tasklet_fn:comp w. error op 0 status 5
> iser: iser_cq_tasklet_fn:comp w. error op 0 status 5
> iser: iser_cq_tasklet_fn:comp w. error op 0 status 5
>  session4: session recovery timed out after 120 secs
>
>
> On Fri, Apr 10, 2009 at 8:53 AM, Chris Worley <worleys at gmail.com> wrote:
>> I'm running RHEL5.2 w/ the stock OFED RPMs.  I have a target disk that
>> run at 800MB/s locally and QDR IB.  I need to get some performance out
>> of this, as I'm seeing <300MB/s in my benchmarks.
>>
>> On the initiator side, I run a test using 1MB block sizes.
>>
>> On the target side I'm seeing everything in 4K packets.
>>
>> Any idea who's setting this and why?
>>
>> Is there a tuning guide available?
>>
>> Thanks,
>>
>> Chris
>>
> _______________________________________________
> general mailing list
> general at lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
>
> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
>