[Wien] Slow lapw1

[Laurence Marks]

An important N.B. I had to add "-fc=ifort" to the compilation options
as otherwise the Intel mpif90 was defaulting to gfortran. There is
probably a way during installation to avoid this.

On Fri, Aug 3, 2012 at 9:33 AM, Laurence Marks <L-marks at northwestern.edu> wrote:
> Responding to my own post. I asked the vendor to install Intel MPI,
> and after (with a little sweat) compiling the problem vanished and the
> speed is rather impressively faster than it was with openmpi:
>
> 16 1:       TIME HAMILT (CPU)  =     7.5, HNS =     8.0, DIAG =    61.3
> 32 2:       TIME HAMILT (CPU)  =     5.1, HNS =     4.4, DIAG =    40.8
> 48 3:       TIME HAMILT (CPU)  =     4.1, HNS =     3.2, DIAG =    31.8
> 64 4:       TIME HAMILT (CPU)  =     3.4, HNS =     2.6, DIAG =    25.1
>
> (First number is the cores, second the number of nodes.)
>
> This is 10-20% faster with just 16 cores (on a single node) than I got
> using the vendors compilation of openmpi 1.4.5.
>
> A few comments (for the record), maybe not all needed or best in all
> cases but adequate:
>
> a) They installed 4.0.3.008 and composer_xe_2011_sp1.11.339
>
> b) I added to my .bashrc
> source /opt/intel/composer_xe_2011_sp1.11.339/bin/compilervars_global.sh intel64
>  source /opt/intel/impi/4.0.3/intel64/bin/mpivars.sh
> export PATH=/opt/intel/impi/4.0.3.008/intel64/bin:$PATH
>
> c) I used the Intel Link advisor page, which says to use for the blacs
> libmkl_blacs_intelmpi_lp64.a
>
> d) I used setenv WIEN_MPIRUN "mpirun --rsh=ssh -machinefile _HOSTS_
> -envall -np _NP_ _EXEC_"
>
>
> On Wed, Aug 1, 2012 at 7:04 PM, Laurence Marks <L-marks at northwestern.edu> wrote:
>> While testing a vendor's cluster, I came across what might be a
>> serious issue for future WIen2k jobs. With the latest architecture it
>> appears that 64 cores can in fact be slower than 32, using the
>> standard mpi test. The slowdown occurs in both  PDSYGST and PDSYEVX. I
>> do not think that this is an issue of the vendor's mpi, but one never
>> knows.
>>
>> I have posted to the Intel list; it will be interesting to see what if
>> any responses there are
>> http://software.intel.com/en-us/forums/showthread.php?t=106950
>>
>> I would also encourage others who have new hardware to do similar
>> tests, and if they have similar issues to also post to the thread.
>>
>> --
>> Professor Laurence Marks
>> Department of Materials Science and Engineering
>> Northwestern University
>> www.numis.northwestern.edu 1-847-491-3996
>> "Research is to see what everybody else has seen, and to think what
>> nobody else has thought"
>> Albert Szent-Gyorgi
>
>
>
> --
> Professor Laurence Marks
> Department of Materials Science and Engineering
> Northwestern University
> www.numis.northwestern.edu 1-847-491-3996
> "Research is to see what everybody else has seen, and to think what
> nobody else has thought"
> Albert Szent-Gyorgi



-- 
Professor Laurence Marks
Department of Materials Science and Engineering
Northwestern University
www.numis.northwestern.edu 1-847-491-3996
"Research is to see what everybody else has seen, and to think what
nobody else has thought"
Albert Szent-Gyorgi