[Wien] wien2k, gotoblas and multi threads

[Peter Blaha]

Looking on these numbers tells me, that you probably should invest into 
ifort + mkl. It does not make sense to buy expensive new hardware, but 
with bad software it runs slower than on a 6 year old PC.
Compare your timing with the benchmark page to see what is possible.

k-point parallelization: Please read the UG !!! This is fairly simple.

1:localhost:4    utilizes the mpi-parallel version;

you need to put N-lines

1:localhost
1:localhost
...

to specify running N lapw1 processes in parallel.

Todd Pfaff schrieb:
> Peter, thanks for the response.
> 
> I'm getting small speedup from multithreading in libgoto.  Here are
> timings from the wien2k serial benchmark:
> 
> OMP_NUM_THREADS=1: 195.463u 0.307s 3:15.80 99.9%        0+0k 0+33264io 0pf+0w
> OMP_NUM_THREADS=2: 199.565u 0.569s 2:57.40 112.8%       0+0k 0+33264io 0pf+0w
> OMP_NUM_THREADS=3: 204.145u 0.635s 2:51.02 119.7%       0+0k 0+33264io 0pf+0w
> OMP_NUM_THREADS=4: 211.666u 0.736s 2:49.02 125.6%       0+0k 0+33264io 0pf+0w
> OMP_NUM_THREADS=5: 222.604u 1.032s 2:48.41 132.7%       0+0k 0+33264io 0pf+0w
> OMP_NUM_THREADS=6: 231.258u 0.927s 2:47.54 138.5%       0+0k 0+33264io 0pf+0w
> OMP_NUM_THREADS=7: 243.170u 0.996s 2:46.55 146.5%       0+0k 0+33264io 0pf+0w
> OMP_NUM_THREADS=8: 252.584u 0.916s 2:46.57 152.1%       0+0k 0+33264io 0pf+0w
> 
> 
> I would like explore the k-point parallelization.  But when I run
> 'x lapw1 -p' it aborts with an error message about being unable to run
> lapw1c_mpi.  This appears to me like it's trying to run the fine grained
> MPI parallel version.  I'm not building wien2k with mpi so I don't have a
> lapw1c_mpi.  I must be misunderstanding something.  What am I doing wrong
> that's causing it to try to run this lapw1c_mpi executable?
> 
> Which of these are appropriate .machines files to do k-point 
> parallelization across N cpu cores on a single machine?
> 
> This?
> 
>    1:localhost:N
> 
> Or this?
> 
>    N:localhost
> 
> And do I need any of these lines?
> 
>    extrafine
>    granularity:1
>    residue:localhost
> 
> Or do I need something else either in .machines or in some other
> file or on the command line?
> 
> --
> Todd Pfaff <pfaff at mcmaster.ca>
> Research & High-Performance Computing Support
> McMaster University, Hamilton, Ontario, Canada
> http://www.rhpcs.mcmaster.ca/~pfaff
> 
> On Mon, 11 Aug 2008, Peter Blaha wrote:
> 
>> The program lapw1 spends a large fraction in BLAS-routines, thus it can
>> benefit from multithreading of GOTOLIBS (or MKL).
>> Setting the variables you mentioned to 2 (or 4) you should see a
>> speedup. The improvement may depend on many factors but it will be at
>> most about 50%.
>>
>> Another possibility to utilize the multiple cores is to do k-point
>> parallelism.
>> Generate a .machines file with 2,4 or 8  times your machine name
>> and test the performance with     x lapw1 -p.
>> On some architectures (with slow memory bus) it can be that only 4
>> parallel jobs give best performance (because the slow memory bus cannot
>> feed all 8 cpus properly), on others you can use 8 parallel jobs.
>> Sometimes a mixture (4 k-point parallel + OMP_NUM_THREADS=2) is best.
>>
>> Todd Pfaff schrieb:
>>> We're using:
>>>
>>>    wien2k-08.2-20080407
>>>
>>> built with:
>>>
>>>    GNU Fortran (GCC) 4.2.3 (4.2.3-6mnb1)
>>>    GotoBLAS-1.26
>>>
>>> and running on an 8 core (2 x quad core) Xeon machine.
>>>
>>> Can wien2k take advantage of multithreading inherent to GotoBLAS
>>> when either GOTO_NUM_THREADS or OMP_NUM_THREADS is set?
>>>
>>> If so, can someone provide, or direct me to a document about details of
>>> the best way to build and run wien2k for such an environment?
>>>
>>> Thank you,
>>> --
>>> Todd Pfaff <pfaff at mcmaster.ca>
>>> Research & High-Performance Computing Support
>>> McMaster University, Hamilton, Ontario, Canada
>>> http://www.rhpcs.mcmaster.ca/~pfaff
>>> _______________________________________________
>>> Wien mailing list
>>> Wien at zeus.theochem.tuwien.ac.at
>>> http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien
>>
> _______________________________________________
> Wien mailing list
> Wien at zeus.theochem.tuwien.ac.at
> http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien

-- 
-----------------------------------------
Peter Blaha
Inst. Materials Chemistry, TU Vienna
Getreidemarkt 9, A-1060 Vienna, Austria
Tel: +43-1-5880115671
Fax: +43-1-5880115698
email: pblaha at theochem.tuwien.ac.at
-----------------------------------------