[Users] increasing limit of Registerable memory with OFED-3.5-1

["Roberto R. Expósito"]

Hi,

I think you can try with the log_mtts_per_seg parameter.

Regards,

Roberto R. Expósito

El 24/07/13 09:31, Anton Starikov escribió:
> Hello,
>
>
> I'm using OFED-3.5-1 with SL-6.4 (I had to do some minor patching to get it working on 2.6.32-358.14.1.el6.x86_64 kernel due to double export of __pskb_copy).
>
> OpenMPI give known warning about limit of registrable memory (below). But in current modules there is no "log_num_mtt" parameter to tune.
>
> Just in case, My hard and soft limits for maximal locked memory are unlimited.
>
> What should be procedure with latest OFED then?
>
> Thank you,
>
> Anton Starikov
>
> ------------------------------------------
>
> WARNING: It appears that your OpenFabrics subsystem is configured to only
> allow registering part of your physical memory.  This can cause MPI jobs to
> run with erratic performance, hang, and/or crash.
>
> This may be caused by your OpenFabrics vendor limiting the amount of
> physical memory that can be registered.  You should investigate the
> relevant Linux kernel module parameters that control how much physical
> memory can be registered, and increase them to allow registering all
> physical memory on your machine.
>
> See this Open MPI FAQ item for more information on these Linux kernel module
> parameters:
>
>      http://www.open-mpi.org/faq/?category=openfabrics#ib-locked-pages
>
>    Local host:              node1
>    Registerable memory:     32768 MiB
>    Total memory:            262098 MiB
>
> Your MPI job will continue, but may be behave poorly and/or hang.
> _______________________________________________
> Users mailing list
> Users at lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/users