Hmm, it's back. I guess I was not patient enough. Not sure when it all got back. I will have to time it next time, I assume it won't take 6 hours each time :-) I'm working on making this 256-node cluster work over infiniband only, same as our myrinet clusters which are myrinet-only. ron