<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0in;
font-size:10.0pt;
font-family:"Courier New";}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:Consolas;}
span.EmailStyle22
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;
mso-ligatures:none;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:895555814;
mso-list-template-ids:-1502864164;}
@list l1
{mso-list-id:1834296153;
mso-list-template-ids:2060215966;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal">Hi Peter,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Thanks for the response. In case this is helpful to anyone in the future:
<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">To get w2web working on the cluster and not get banned for accidentally running things on the login node, I modified the w2web source so it does not create a new configuration for every node by fixing the host name in the configuration
part of the script. I then run w2web on a compute node using an interactive job with port forwarding enabled. This way, everything will either run on the compute node running w2web or it will get submitted as a batch job to the queue if I use a submission
script. <o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal"><span style="mso-ligatures:standardcontextual"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="mso-ligatures:standardcontextual">Daniel Straus<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-ligatures:standardcontextual">Assistant Professor<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-ligatures:standardcontextual">Department of Chemistry<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-ligatures:standardcontextual">Tulane University<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-ligatures:standardcontextual">5088 Percival Stern Hall<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-ligatures:standardcontextual">6400 Freret Street<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-ligatures:standardcontextual">New Orleans, LA 70118<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-ligatures:standardcontextual">(504) 862-3585<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-ligatures:standardcontextual"><a href="http://straus.tulane.edu/"><span style="color:#0563C1">http://straus.tulane.edu/</span></a><o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-ligatures:standardcontextual"> </span><span style="mso-ligatures:standardcontextual"><o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b>From:</b> Peter Blaha <peter.blaha@tuwien.ac.at> <br>
<b>Sent:</b> Tuesday, April 16, 2024 1:58 AM<br>
<b>To:</b> wien@zeus.theochem.tuwien.ac.at<br>
<b>Subject:</b> Re: [Wien] SLURM cluster issues<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p>Hi,<o:p></o:p></p>
<p class="MsoNormal"><br>
<o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">I am trying to set up WIEN2k ver 23.2 to run on a SLURM cluster. I have gotten it to work with SCALAPACK, runnning with a slurm batch submission script through w2web by following
the examples.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">I have two issues.<o:p></o:p></p>
<p class="MsoListParagraph" style="margin-left:.25in;text-indent:-.25in;mso-list:l0 level1 lfo1">
<![if !supportLists]><span style="mso-list:Ignore">1.<span style="font:7.0pt "Times New Roman"">
</span></span><![endif]>Is it possible to make the “x dstart” button in the initialize web interface submit its job with a script like is possible for running SCF? Right now, it runs on the w2web node, and I can’t use a preexisting .machines file because all
jobs are submitted through SLURM so the .machines needs to be generated on the fly.<o:p></o:p></p>
</div>
</blockquote>
<p>If you really want to run this in parallel on a compute node (makes only sense for really big cases with 100 or more atoms/unit cell), you have to do it similar as with the scf calculation and execute it via a script.<o:p></o:p></p>
<p>A user could activate the -nodstart option in the initialization and run dstart as "single program" with the submission method.<o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoListParagraph" style="margin-left:.25in;text-indent:-.25in;mso-list:l1 level1 lfo2">
<![if !supportLists]><span style="mso-list:Ignore">1.<span style="font:7.0pt "Times New Roman"">
</span></span><![endif]><o:p> </o:p></p>
<p class="MsoListParagraph" style="margin-left:.25in;text-indent:-.25in;mso-list:l1 level1 lfo2">
<![if !supportLists]><span style="mso-list:Ignore">2.<span style="font:7.0pt "Times New Roman"">
</span></span><![endif]>I cannot seem to get ELPA to work. It compiles fine and passes the ELPA make check tests. However, when I try to run the TiC example in the usersguide, I always get an error. I have a feeling that ELPA is not using the correct kernel—is
there a way to specify that though WIEN2k, or should I set a default ELPA kernel through ELPA ./configure?
<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Here is a link to a zip file with what I hope are the relevant files:
<a href="https://tulane.box.com/s/ozohfwe0xyoipb8jzxeh3ec15imq1eam">https://tulane.box.com/s/ozohfwe0xyoipb8jzxeh3ec15imq1eam</a>. My email got blocked when I attached them directly.
<br>
<br>
<o:p></o:p></p>
</div>
</blockquote>
<p>I guess the only problem is that the TiC case is too small to run in mpi parallel.<o:p></o:p></p>
<p>In WIEN2k one has to adapt the parallelization to the case one is studying.<o:p></o:p></p>
<p>The openMP parallelization (on one node) works always, but is efficient only up to 4 nodes. It should always be used.<o:p></o:p></p>
<p>The k-point parallelization is the second option, it is helpful if one has many k-points in the input and the lapw1 step takes more than eg. 20 seconds.<o:p></o:p></p>
<p>The last parllelization is via mpi using SCALAPACK and/or ELPA. The latter is much more efficient, but works only if the problem size (number of atoms ---> size of the eigenvalue problem) exceeds a few thousands. Even with such intermediate sizes (NMAT between
3000 -20000) the number of cores should not be too large, otherwise communication wins and it takes longer than on fewer cores.<o:p></o:p></p>
<p>For really large problems (NMAT up to 100000x100000) many cores (eg. 512) can be used efficiently.<o:p></o:p></p>
<p><o:p> </o:p></p>
<p>PS: I always compile ELPA without machine specific options and let it decide on runtime what is available on the specific hardware).<o:p></o:p></p>
<p><o:p> </o:p></p>
<p>PPS: I'd also suggest to run the w2web interface ONLY on a local workstation and setting up a case there. Only after the initialization I'd migrate (migrate_lapw) this to a supercomputer and run the scf there.<o:p></o:p></p>
<p><o:p> </o:p></p>
<p>Best regards<o:p></o:p></p>
<pre>-- <o:p></o:p></pre>
<pre>-----------------------------------------------------------------------<o:p></o:p></pre>
<pre>Peter Blaha, Inst. f. Materials Chemistry, TU Vienna, A-1060 Vienna<o:p></o:p></pre>
<pre>Phone: +43-158801165300<o:p></o:p></pre>
<pre>Email: <a href="mailto:peter.blaha@tuwien.ac.at">peter.blaha@tuwien.ac.at</a> <o:p></o:p></pre>
<pre>WWW: <a href="http://www.imc.tuwien.ac.at">http://www.imc.tuwien.ac.at</a> WIEN2k: <a href="http://www.wien2k.at">http://www.wien2k.at</a><o:p></o:p></pre>
<pre>-------------------------------------------------------------------------<o:p></o:p></pre>
</div>
</body>
</html>