[openib-general] ibv_reg_mr failure with pvfs on ehca?

[Hoang-Nam Nguyen]

Hi Troy!
> I am running PVFS2 on OpenIB, with IBM's ehca.
> When we start writing/reading large files, either with the NetPIPE
> PVFS module we have or a modified GAMESS executable that uses
> libpvfs2 directly, the 'ibv_reg_mr' function fails, and we get an error.
> This is also correlated with kernel log messages like this:
> Oct 16 11:14:45 p5l8 kernel: PU0003 000e0091:ehca_hcall_7arg_7ret
> HCAD_ERROR  opco
> de=160 ret=fffffffffffffff7 arg1=1000000003000004 arg2=5
> arg3=14f0ebc8 arg4=10000
> arg5=e0000000000000 arg6=e3e9f200 arg7=0 out1=0 out2=0 out3=0 out4=0
> out5=0 out6=0
> out7=0
Return code f7 from firmware/hvcall means H_NO_MEM. I'm wondering
if you could provide me with some pre-history of this problem.
Is this a permanent problem? If yes, could you give me more infos
on your testcase resp. scenario eg large file size, NetPIPE options?
Which version of ehca are you using? And which kernel version?
Thanks!
Hoang-Nam Nguyen