<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Aptos;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Aptos",sans-serif;
mso-ligatures:standardcontextual;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#467886;
text-decoration:underline;}
span.EmailStyle18
{mso-style-type:personal-compose;
font-family:"Aptos",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:11.0pt;
mso-ligatures:none;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="#467886" vlink="#96607D" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal">Hi,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I am trying to use the <i>balance:</i> keyword in .machines to allocate k-points on-the-fly rather than all at once because even though all nodes on the cluster I use are identical, some k-points are computed much faster than others.
<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">When running a band structure calculation (x lapw1 -band -up –p), after the initial k-points are allocated, there is a bug related to assigning remaining k-points. This was run on 4 nodes. Once the first job ends, it tries to access a .machine
(.machine5) file that doesn’t exist—there are only 4 nodes, so only four .machine* files are created. The job still completes correctly, so I’m not sure if this is just a logging issue or if the subsequent k-points are being assigned to the wrong nodes.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">case.klist_band had had 102 k points in it<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Here is the relevant STDOUT:<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.5in">running LAPW1 in parallel mode (using .machines)<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.5in">4 number_of_parallel_jobs<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.5in">[1] 45240<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.5in">[2] 45278<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.5in">[3] 45313<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.5in">[4] 45344<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.5in">LAPW1 END<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.5in">[3] Done ( cd $PWD; $t $ttt; rm -f .lock_$lockfile[$p] ) >> .time1_$loop<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.5in">sort: open failed: .machine5: No such file or directory<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.5in">[5] 111735<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.5in">LAPW1 END<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.5in">[2] Done ( cd $PWD; $t $ttt; rm -f .lock_$lockfile[$p] ) >> .time1_$loop<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.5in">sort: open failed: .machine6: No such file or directory<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.5in">[6] 111966<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.5in">LAPW1 END<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.5in">[5] Done ( cd $PWD; $t $ttt; rm -f .lock_$lockfile[$p] ) >> .time1_$loop<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.5in">LAPW1 END<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.5in">LAPW1 END<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.5in">LAPW1 END<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.5in">[6] + Done ( cd $PWD; $t $ttt; rm -f .lock_$lockfile[$p] ) >> .time1_$loop<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.5in">[4] + Done ( cd $PWD; $t $ttt; rm -f .lock_$lockfile[$p] ) >> .time1_$loop<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.5in">[1] + Done ( cd $PWD; $t $ttt; rm -f .lock_$lockfile[$p] ) >> .time1_$loop<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">.machines file is as follows:<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"> #<o:p></o:p></p>
<p class="MsoNormal"> omp_global:5 #(this is 5 because there are two 10 core processors, and I run 4 mpi jobs per node)<o:p></o:p></p>
<p class="MsoNormal"> lapw0:host1:4 host2:4 host3:4 host4:4<o:p></o:p></p>
<p class="MsoNormal"> balance:<o:p></o:p></p>
<p class="MsoNormal" style="text-indent:.5in">1:host1:4<o:p></o:p></p>
<p class="MsoNormal" style="text-indent:.5in">1:host2:4<o:p></o:p></p>
<p class="MsoNormal" style="text-indent:.5in">1:host3:4<o:p></o:p></p>
<p class="MsoNormal" style="text-indent:.5in">1:host4:4<o:p></o:p></p>
<p class="MsoNormal" style="text-indent:.5in">granularity:1<o:p></o:p></p>
<p class="MsoNormal"> extrafine:1<o:p></o:p></p>
<div>
<p class="MsoNormal"><span style="font-family:"Calibri",sans-serif"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Calibri",sans-serif">I’ve removed the balance: keyword from my .machines file for now.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Calibri",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Calibri",sans-serif">Daniel <o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-ligatures:none"><o:p> </o:p></span></p>
<p class="MsoNormal">Daniel Straus<o:p></o:p></p>
<p class="MsoNormal">Assistant Professor<o:p></o:p></p>
<p class="MsoNormal">Department of Chemistry<o:p></o:p></p>
<p class="MsoNormal">Tulane University<o:p></o:p></p>
<p class="MsoNormal">5088 Percival Stern Hall<o:p></o:p></p>
<p class="MsoNormal">6400 Freret Street<o:p></o:p></p>
<p class="MsoNormal">New Orleans, LA 70118<o:p></o:p></p>
<p class="MsoNormal">(504) 862-3585<o:p></o:p></p>
<p class="MsoNormal"><a href="http://straus.tulane.edu/"><span style="color:#0563C1">http://straus.tulane.edu/</span></a><o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</body>
</html>