[Wien] Pathscale+OpenMPI support

[Peter Blaha]

I see !

setenv WIEN_MPIRUN "mpirun _EXEC_"

You run an mpi job just using   "mpirun executable"; without specifying 
the number of nodes (taking the default defined in the sge-job).

Thus for ONE k-point, the .machines file is actually ignored, i.e. 
wien2k starts ONE mpi job (and this is ALWAYS running on ALL available 
nodes.

Open a terminal window and change into the "case-directory". Execute
x kgen
It asks for some input and use a larger number of k-points (eg. 20).
Observe how many k-points are created by that (cat *.klist). Rerun kgen 
until you get eg. 4 k-points (or a similar number).

The run    x lapw1 -p    (or run_lapw -p -i 1)

Now wien2k will start as many mpi-jobs as lines in .machines (and 
probably you will have more than one mpi-job running on every core,
since your mpirun will always take all hosts).


Scott Beardsley schrieb:
>>> A 32cpu job would always look like this:
>>>
>>> lapw0: compute-0-0:8 compute-0-1:8 compute-0-2:8 compute-0-3:8
>>> 1:compute-0-0:8
>>> 1:compute-0-1:8
>>> 1:compute-0-2:8
>>> 1:compute-0-3:8
>>> granularity:1
>>> extrafine:1
>> Please note: If a user runs a job with only 2 k-points (or with 5 !!), 
>> lots of resources get waisted, because only half of the machines are 
>> actually used (but all 32 nodes are reserved).
> 
> Do you happen to have a somewhat short (time wise - say <15mins for 
> 32cpus) "real world" input that I can test out to demonstrate the 
> differences? I did some runs using strictly mpi parallelization and 
> didn't notice much of a difference (effectively zero) between the above 
> machines file and the below machines file. Results here[1].
> 
> lapw0: compute-0-0:8 compute-0-1:8 compute-0-2:8 compute-0-3:8
> 1:compute-0-0:8 compute-0-1:8 compute-0-2:8 compute-0-3:8
> granularity:1
> extrafine:1
> 
>>> Also, I'm curious if w2web is batch queue aware? If not, I'll have to 
>>> discourage people from using it.
>> In w2web one can configure the "execution type". Predefined is just 
>> "interactive" and "background", but one can configure its own modes,
>> eg sge8, sge16, ... under "configuration" (directly in w2web).
> 
> Thanks for the info!
> Scott
> -----------
> [1] http://wiki.cse.ucdavis.edu/support:hpc:software:wien2k#benchmarks
> _______________________________________________
> Wien mailing list
> Wien at zeus.theochem.tuwien.ac.at
> http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien