[Wien] OS bug/inconveniece with mkl thread/mpi conflicts

[Laurence Marks]

This is an informational report of an issue that can arise with Wien2k
leading to poor performance. I came across it when experimenting with
the "procs=XX" option in msub/qsub with XX=64 as an alternative to the
more standard "nodes=X:ppn=Y".

In an scf cycle lapwdm and (with the next release) mixer always use
lapack routines (and maybe some others) which by default with mkl are
multithreaded and use multiple cores. For a mpi run with lapw0:
specified lapw0, lapw1 and lapw2 all use mpi. A issue can arise with
msub/qsub (and perhaps other) if the number of cores mkl uses is
larger than what has been allocated and there are other mpi tasks
running on the same node. Then the performance of lapwdm can be bad;
in one case rather than taking 30 seconds it took 14 minutes.

I am not sure if this is something "fixable" in WIen2k, and there are
ways in qsub/msub to avoid it using nodes=X:ppn=Y. However, procs=XX
is "nice" as Wien2k in mpi does not really need factors of 2
cores/node (I've tested this) and can fit in the gaps.

N.B., running without mpi and using nodes=X:ppn=1 could also run into
the same problem.

-- 
Laurence Marks
Department of Materials Science and Engineering
MSE Rm 2036 Cook Hall
2220 N Campus Drive
Northwestern University
Evanston, IL 60208, USA
Tel: (847) 491-3996 Fax: (847) 491-7820
email: L-marks at northwestern dot edu
Web: www.numis.northwestern.edu
Chair, Commission on Electron Crystallography of IUCR
www.numis.northwestern.edu/
Electron crystallography is the branch of science that uses electron
scattering and imaging to study the structure of matter.