[Wien] Slow lapw1

[Laurence Marks]

Responding to my own post. I asked the vendor to install Intel MPI,
and after (with a little sweat) compiling the problem vanished and the
speed is rather impressively faster than it was with openmpi:

16 1:       TIME HAMILT (CPU)  =     7.5, HNS =     8.0, DIAG =    61.3
32 2:       TIME HAMILT (CPU)  =     5.1, HNS =     4.4, DIAG =    40.8
48 3:       TIME HAMILT (CPU)  =     4.1, HNS =     3.2, DIAG =    31.8
64 4:       TIME HAMILT (CPU)  =     3.4, HNS =     2.6, DIAG =    25.1

(First number is the cores, second the number of nodes.)

This is 10-20% faster with just 16 cores (on a single node) than I got
using the vendors compilation of openmpi 1.4.5.

A few comments (for the record), maybe not all needed or best in all
cases but adequate:

a) They installed 4.0.3.008 and composer_xe_2011_sp1.11.339

b) I added to my .bashrc
source /opt/intel/composer_xe_2011_sp1.11.339/bin/compilervars_global.sh intel64
 source /opt/intel/impi/4.0.3/intel64/bin/mpivars.sh
export PATH=/opt/intel/impi/4.0.3.008/intel64/bin:$PATH

c) I used the Intel Link advisor page, which says to use for the blacs
libmkl_blacs_intelmpi_lp64.a

d) I used setenv WIEN_MPIRUN "mpirun --rsh=ssh -machinefile _HOSTS_
-envall -np _NP_ _EXEC_"


On Wed, Aug 1, 2012 at 7:04 PM, Laurence Marks <L-marks at northwestern.edu> wrote:
> While testing a vendor's cluster, I came across what might be a
> serious issue for future WIen2k jobs. With the latest architecture it
> appears that 64 cores can in fact be slower than 32, using the
> standard mpi test. The slowdown occurs in both  PDSYGST and PDSYEVX. I
> do not think that this is an issue of the vendor's mpi, but one never
> knows.
>
> I have posted to the Intel list; it will be interesting to see what if
> any responses there are
> http://software.intel.com/en-us/forums/showthread.php?t=106950
>
> I would also encourage others who have new hardware to do similar
> tests, and if they have similar issues to also post to the thread.
>
> --
> Professor Laurence Marks
> Department of Materials Science and Engineering
> Northwestern University
> www.numis.northwestern.edu 1-847-491-3996
> "Research is to see what everybody else has seen, and to think what
> nobody else has thought"
> Albert Szent-Gyorgi



-- 
Professor Laurence Marks
Department of Materials Science and Engineering
Northwestern University
www.numis.northwestern.edu 1-847-491-3996
"Research is to see what everybody else has seen, and to think what
nobody else has thought"
Albert Szent-Gyorgi