[openib-general] ibv_reg_mr failure with pvfs on ehca?

[Hoang-Nam Nguyen]

Hi Troy!
> The netpipe code is available with mercurial by:
> hg clone http://source.scl.ameslab.gov/hg/netpipe3-pvfs-dev
> Once you have pvfs2-1.5.1 installed, you should be able to do 'make
> pvfs' in the netpipe3-pvfs-dev directory and build NPpvfs.
> The command line arguments I used to reproduce this were:
> ./NPpvfs -d $PVFS_FILE_PATH -l 32768 -u 268435456 -n 100 -o
> $NETPIPE_OUTPUT_FILE
Thanks for this. I've been struggling  with setting up the systems
to recreate this problem. Please be patient.
Can you please send me the ouput of modinfo ib_ehca (or hcad_mod
in older version)? Also the firmware code level as plained in
previous email. How many memory have you assigned to the partition?
With those data I'd be able to have nearly the same envs like yours.
> This is the dmesg log:
> PU0001 000e0091:ehca_hcall_7arg_7ret HCAD_ERROR  opcode=160
> ret=fffffffffffffff7 arg1=1000000003000004 arg2=5 arg3=4000f830000
> arg4=10000 arg5=e0000000000000 arg6=eb6b6920 arg7=0 out1=0 out2=0
> out3=0 out4=0 out5=0 out6=0 out7=0
> PU0001 00090454:ehca_reg_mr HCAD_ERROR  hipz_alloc_mr failed,
> h_ret=fffffffffffffff7 hca_hndl=1000000003000004
> PU0001 00090478:ehca_reg_mr <<< ret=ffffffea shca=c0000000e796b000
> e_mr=c0000000ce865e80 iova_start=000004000f830000 size=10000 acl=7
> e_pd=c0000000eb6b6920 pginfo=c0000000dfcb3a70 num_pages=10 num_4k=10
> PU0001 00090176:ehca_reg_user_mr <<< rc=ffffffffffffffea
> pd=c0000000eb6b6920 region=c0000000ce861dd0 mr_access_flags=7
> udata=c0000000dfcb3ba0
I got this already from you and Kyle. I meant the full log with
debug traces enabled: modprobe ib_ehca debug_level=1 or for older
versions modprobe hcad_mod debug_level=9999999999999999999999. If
possible, try to get it. Anyway I'll do that with my test env.
Thanks!
Nam