[ofiwg] inserting duplicate addresses into an AV

[Hefty, Sean]

MPI is hitting into an issue that is the result of inserting the same address into an AV more than once.  There is no defined behavior for what a provider should do in this case.  At least one provider allows the duplicate insertion, and at least one fails the call... and neither work with MPI when this occurs.  :/

There are a couple of problems trying to define this.  In the case of the provider that fails the call, the failure is detected when attempting to insert the same address into a hash table.  However, not all providers are easily able to detect duplicates.  Forcing them to do so _may_ require the provider to perform a linear search over the AV looking for a duplicate for every address that is inserted.  At scale, this is a significant overhead.

Even if the decision is made to force detecting duplicates (maybe even making this an AV option), there's the question of how a provider should respond.  Should it insert the address twice -- creating a new fi_addr for it, discard the duplicate -- and return the existing fi_addr, or generate an error.  And does it matter if AV_TABLE or MAP is used?

We need to know what applications need here, and how difficult it will be for providers to detect duplicates.  It is apparently non-trivial for the apps to avoid duplicate insertions.

- Sean