<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
<meta name="Generator" content="Microsoft Exchange Server">
<!-- converted from text --><style><!-- .EmailQuote { margin-left: 1pt; padding-left: 4pt; border-left: #800000 2px solid; } --></style>
</head>
<body>
<meta content="text/html; charset=UTF-8">
<style type="text/css" style="">
<!--
p
{margin-top:0;
margin-bottom:0}
-->
</style>
<div dir="ltr">
<div id="x_divtagdefaultwrapper" dir="ltr" style="font-size:12pt; color:#000000; font-family:Calibri,Helvetica,sans-serif">
<p>Dear Professor Blaha</p>
<p><br>
</p>
<p>Thanks a lot for your responses. I have performed some additional testing, which has been delayed because I cannot run lapw0/1/2 from the command-line due to memory issues. Hence, I have had to go through the queue for each test. On top of that, I have been
unable to get information about our installation. However, I finally achieved ~99% CPU efficiency with the following setup:</p>
<p><br>
</p>
<p>CPUs: 2 nodes with 24 cores each (x073 and x082)</p>
<p><br>
</p>
<p>.machines:<br>
</p>
<div><span style="font-family:"Courier New",monospace">dstart:x073:24 x082:24</span><br>
<span style="font-family:"Courier New",monospace">lapw0:x073:24 x082:24</span><br>
<span style="font-family:"Courier New",monospace">1:x073:3</span><br>
<span style="font-family:"Courier New",monospace">1:x082:3</span><br>
<span style="font-family:"Courier New",monospace">1:x073:3</span><br>
<span style="font-family:"Courier New",monospace">1:x082:3</span><br>
<span style="font-family:"Courier New",monospace">1:x073:3</span><br>
<span style="font-family:"Courier New",monospace">1:x082:3</span><br>
<span style="font-family:"Courier New",monospace">1:x073:3</span><br>
<span style="font-family:"Courier New",monospace">1:x082:3</span> # 16 lines total; 8 for each node<br>
<span style="font-family:"Courier New",monospace">1:x073:3</span><br>
<span style="font-family:"Courier New",monospace">1:x082:3</span><br>
<span style="font-family:"Courier New",monospace">1:x073:3</span><br>
<span style="font-family:"Courier New",monospace">1:x082:3</span><br>
<span style="font-family:"Courier New",monospace">1:x073:3</span><br>
<span style="font-family:"Courier New",monospace">1:x082:3</span><br>
<span style="font-family:"Courier New",monospace">1:x073:3</span><br>
<span style="font-family:"Courier New",monospace">1:x082:3</span></div>
<p></p>
<p><br>
</p>
<p>After creating the .machines-file I call 'mpirun run_lapw -p'. The above .machines file is basically a combination of the two examples found on page 86 of the User's Guide (without using OMP, of course). From checking the case.klist_1-16 files, I have verified
that each individual job works on a different subset of the k-points. Can anyone confirm whether this setup is correct; i.e. is it a proper way to parallellize the lapw1/lapw2 cycles? Assuming the compilations of lapw0/1/2_mpi proceeded without errors, which
seems to be the case.</p>
<p><br>
</p>
<p>Best regards</p>
<p>Christian <br>
</p>
</div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="x_divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>Fra:</b> Wien <wien-bounces@zeus.theochem.tuwien.ac.at> på vegne af Peter Blaha <pblaha@theochem.tuwien.ac.at><br>
<b>Sendt:</b> 13. oktober 2020 07:43:16<br>
<b>Til:</b> wien@zeus.theochem.tuwien.ac.at<br>
<b>Emne:</b> Re: [Wien] .machines for several nodes</font>
<div> </div>
</div>
</div>
<font size="2"><span style="font-size:10pt;">
<div class="PlainText">To run a single program for testing, do:<br>
<br>
x lapw0 -p<br>
<br>
(after creation of .machines.)<br>
<br>
Then check all error files, but in particular also the slurm-output <br>
(whatever it is called on your machines. It probably gives some messages <br>
like library xxxx not found or so, which is needed for additional debugging.<br>
<br>
AND:<br>
<br>
We still don't know how many cores your nodes have<br>
<br>
We still don't know your compiler options (WIEN2k_OPTIONS, <br>
parallel_options) and if the compilation of eg. lapw0_mpi did work at <br>
all (compile.msg in SRC_lapw0).<br>
<br>
Am 12.10.2020 um 22:17 schrieb Christian Søndergaard Pedersen:<br>
> Dear everybody<br>
> <br>
> <br>
> I am following up on this thread to report on two separate errors in my <br>
> attempts to properly parallellize a calculation. For the first, a <br>
> calculation utilized 0.00% of available CPU resources. My .machines file <br>
> looks like this:<br>
> <br>
> <br>
> #<br>
> dstart:g004:8 g010:8 g011:8 g040:8<br>
> lapw0:g004:8 g010:8 g011:8 g040:8<br>
> 1:g004:16<br>
> 1:g010:16<br>
> 1:g011:16<br>
> 1:g040:16<br>
> <br>
> With my submit script calling the following commands:<br>
> <br>
> <br>
> srun hostname -s > slurm.hosts<br>
> <br>
> run_lapw -p<br>
> <br>
> x qtl -p -telnes<br>
> <br>
> <br>
> Of course, the job didn't reach x qtl. The resultant case.dayfile is <br>
> short, so I am dumping all of it here:<br>
> <br>
> <br>
> Calculating test-machines in /path/to/directory<br>
> on node.host.name.dtu.dk with PID XXXXX<br>
> using WIEN2k_19.1 (Release 25/6/2019) in <br>
> /path/to/installation/directory/WIEN2k/19.1-intel-2019a<br>
> <br>
> <br>
> start (Mon Oct 12 19:04:06 CEST 2020) with lapw0 (40/99 to go)<br>
> <br>
> cycle 1 (Mon Oct 12 19:04:06 CEST 2020) (40/99 to go)<br>
> <br>
>> lapw0 -p (19:04:06) starting parallel lapw0 at Mon Oct 12 19:04:06 CEST 2020<br>
> -------- .machine0 : 32 processors<br>
> [1] 16095<br>
> <br>
> <br>
> The .machine0 file displays the lines<br>
> <br>
> g004 [repeated for 8 lines]<br>
> g010 [repeated for 8 lines]<br>
> g011 [repeated for 8 lines]<br>
> g040 [repeated for 8 lines]<br>
> <br>
> which tells me that the .machines file works as intended, and that the <br>
> cause of the problem is located somewhere else. Which brings me to the <br>
> second error, which occured when I tried calling mpirun explicitly like so:<br>
> <br>
> srun hostname -s > slurm.hosts<br>
> mpirun run_lapw -p<br>
> mpirun qtl -p -telnes<br>
> <br>
> from within the job script. This crashed the job right away. The <br>
> lapw0.error file prints out "Error in Parallel lapw0" and "check ERROR <br>
> FILES!" a number of times. The case.clmsum file is present and looks <br>
> correct, and the .machines file looks like the one from before (with <br>
> different node numbers). However, the .machine0 file now looks like:<br>
> <br>
> g094<br>
> g094<br>
> g094<br>
> g081<br>
> g081<br>
> g08g094<br>
> g094<br>
> g094<br>
> g094<br>
> g094<br>
> [...]<br>
> <br>
> I.e. there's an error on line 6, where a node is not properly named and <br>
> a line break is missing. The dayfile repeatedly prints out "> stop <br>
> error" a total of sixteen times. I don't know if the above .machine0 <br>
> file is the culprit, but it seems the obvious conclusion. Any help in <br>
> this matter will be much appreciated.<br>
> <br>
> Best regards<br>
> Christian<br>
> <br>
> _______________________________________________<br>
> Wien mailing list<br>
> Wien@zeus.theochem.tuwien.ac.at<br>
> <a href="http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien">http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien</a><br>
> SEARCH the MAILING-LIST at: <a href="http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html">
http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html</a><br>
> <br>
<br>
-- <br>
--------------------------------------------------------------------------<br>
Peter BLAHA, Inst.f. Materials Chemistry, TU Vienna, A-1060 Vienna<br>
Phone: +43-1-58801-165300 FAX: +43-1-58801-165982<br>
Email: blaha@theochem.tuwien.ac.at WIEN2k: <a href="http://www.wien2k.at">http://www.wien2k.at</a><br>
WWW: <br>
<a href="http://www.imc.tuwien.ac.at/tc_blaha-------------------------------------------------------------------------">http://www.imc.tuwien.ac.at/tc_blaha-------------------------------------------------------------------------</a>
<br>
<br>
_______________________________________________<br>
Wien mailing list<br>
Wien@zeus.theochem.tuwien.ac.at<br>
<a href="http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien">http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien</a><br>
SEARCH the MAILING-LIST at: <a href="http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html">
http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html</a><br>
</div>
</span></font>
</body>
</html>