<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p><font color="#0000ff">Three additional comments:</font></p>
<p><font color="#0000ff">1) If you are running the slurm.job script
as Non-Interactive [1,2], you might need a "source
/etc/profile.d/ummodules.csh" line like that at [3].<br>
</font></p>
<font color="#0000ff">[1]
<a class="moz-txt-link-freetext" href="https://slurm.schedmd.com/faq.html#sbatch_srun">https://slurm.schedmd.com/faq.html#sbatch_srun</a></font><br>
<font color="#0000ff">[2]
<a class="moz-txt-link-freetext" href="https://wiki.umiacs.umd.edu/umiacs/index.php/SLURM/JobSubmission">https://wiki.umiacs.umd.edu/umiacs/index.php/SLURM/JobSubmission</a></font><br>
<font color="#0000ff">[3]
<a class="moz-txt-link-freetext" href="https://wiki.umiacs.umd.edu/umiacs/index.php/Modules">https://wiki.umiacs.umd.edu/umiacs/index.php/Modules</a></font><br>
<p><font color="#0000ff"> 2) To eliminate any possible conflicts
between Intel compilers (ifort, icc) and GNU compilers
(gfortran, gcc) such as those mentioned in posts [4,5]. I
suggest compiling of WIEN2k using the same C compiler and
Fortran compiler that Open MPI was compiled with.</font></p>
<p><font color="#0000ff">The commands [6] below might help you check
that the Linux environment is using the intended Open MPI and
mpi parallel compilers [7]:<br>
</font></p>
<p>username@computername:~/Desktop$ mpirun -V<br>
mpirun (Open MPI) 4.1.1<br>
<br>
Report bugs to <a class="moz-txt-link-freetext" href="http://www.open-mpi.org/community/help/">http://www.open-mpi.org/community/help/</a><br>
username@computername:~/Desktop$ mpicc --showme:version<br>
mpicc: Open MPI 4.1.1 (Language: C)<br>
username@computername:~/Desktop$ mpifort --showme:version<br>
mpifort: Open MPI 4.1.1 (Language: Fortran)<br>
username@computername:~/Desktop$ mpicc --version<br>
gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0<br>
Copyright (C) 2019 Free Software Foundation, Inc.<br>
This is free software; see the source for copying conditions.
There is NO<br>
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR
PURPOSE.<br>
<br>
username@computername:~/Desktop$ mpifort --version<br>
GNU Fortran (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0<br>
Copyright (C) 2019 Free Software Foundation, Inc.<br>
This is free software; see the source for copying conditions.
There is NO<br>
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR
PURPOSE.<br>
</p>
<font color="#0000ff">[4]
<a class="moz-txt-link-freetext" href="https://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/msg08315.html">https://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/msg08315.html</a></font><br>
<font color="#0000ff">[5]
<a class="moz-txt-link-freetext" href="https://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/msg17052.html">https://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/msg17052.html</a></font><br>
<font color="#0000ff">[6]
<a class="moz-txt-link-freetext" href="https://stackoverflow.com/questions/10056898/how-do-you-check-the-version-of-openmpi">https://stackoverflow.com/questions/10056898/how-do-you-check-the-version-of-openmpi</a></font><br>
<font color="#0000ff">[7]
<a class="moz-txt-link-freetext" href="https://www.open-mpi.org/faq/?category=mpi-apps#general-build">https://www.open-mpi.org/faq/?category=mpi-apps#general-build</a><br>
</font>
<p><font color="#0000ff">3)</font> <font color="#0000ff">Those
cluster administrators are usually more savvy than I am with
installation and optimization of software (using compiler
documentation, e.g. [8,9]) on a high performance computing (hpc)
supercomputer [10,11]. They would know your situation better.
For example, they could login to their administrator account on
the cluster to install WIEN2k only in your user account
directory (/home/users/mollabashi), and they would know how to
set the appropriate access permissions [12]. Alternatively, if
your not using a personal laptop but a computer at the
organization to remotely connect to the cluster then they might
use remote desktop access </font><font color="#0000ff"><font
color="#0000ff">[13] </font>to help you with the installation
within only your account. Or they might use another method.<br>
</font></p>
<font color="#0000ff">[8]
<a class="moz-txt-link-freetext" href="https://software.intel.com/content/www/us/en/develop/articles/download-documentation-intel-compiler-current-and-previous.html">https://software.intel.com/content/www/us/en/develop/articles/download-documentation-intel-compiler-current-and-previous.html</a></font><br>
<font color="#0000ff">[9] <a class="moz-txt-link-freetext" href="https://gcc.gnu.org/onlinedocs/">https://gcc.gnu.org/onlinedocs/</a></font><br>
<font color="#0000ff">[10]
<a class="moz-txt-link-freetext" href="https://www.usgs.gov/core-science-systems/sas/arc/about/what-high-performance-computing">https://www.usgs.gov/core-science-systems/sas/arc/about/what-high-performance-computing</a></font><br>
<font color="#0000ff">[11]
<a class="moz-txt-link-freetext" href="https://en.wikipedia.org/wiki/Supercomputer">https://en.wikipedia.org/wiki/Supercomputer</a></font><br>
<font color="#0000ff">[12]
<a class="moz-txt-link-freetext" href="https://www.oreilly.com/library/view/running-linux-third/156592469X/ch04s14.html">https://www.oreilly.com/library/view/running-linux-third/156592469X/ch04s14.html</a><br>
</font>
<div class="moz-cite-prefix"><font color="#0000ff">[13]
<a class="moz-txt-link-freetext" href="https://en.wikipedia.org/wiki/Desktop_sharing">https://en.wikipedia.org/wiki/Desktop_sharing</a></font><br>
</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">On 5/4/2021 3:40 PM, Laurence Marks
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CANkSMZCQbyFsSLud1i+=_ULypVtTSFh-67yw=YJ+szOnNrGgZA@mail.gmail.com">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<div dir="ltr">
<div dir="ltr">
<div class="gmail_default"
style="font-family:verdana,sans-serif;color:#000000">
<div dir="ltr" style="color:rgb(34,34,34)">
<div class="gmail_default"
style="font-family:verdana,sans-serif;color:rgb(0,0,0)">For
certain, "<span
style="font-family:Arial,Helvetica,sans-serif;font-size:small;color:rgb(34,34,34)">/opt/exp_soft/local/generic/</span><span
style="font-family:Arial,Helvetica,sans-serif;font-size:small;color:rgb(34,34,34)">openmpi/4.1.0_gcc620/bin/</span><span
style="font-family:Arial,Helvetica,sans-serif;font-size:small;color:rgb(34,34,34)">mpiexec
/home/users/mollabashi/codes/</span><span
style="font-family:Arial,Helvetica,sans-serif;font-size:small;color:rgb(34,34,34)">v21.1/run_lapw
-p" is completely wrong. You do not, repear do not use
mpirun or mpiexec to start run_lapw. It has to be
started by simply "run_lapw -p ..." by itself.</span></div>
<div class="gmail_default"
style="font-family:verdana,sans-serif;color:rgb(0,0,0)"><span
style="font-family:Arial,Helvetica,sans-serif;font-size:small;color:rgb(34,34,34)"><br>
</span></div>
<div class="gmail_default"
style="font-family:verdana,sans-serif;color:rgb(0,0,0)"><span
style="font-family:Arial,Helvetica,sans-serif;font-size:small;color:rgb(34,34,34)">I
suggest that you create a very simple job which has
the commands:</span></div>
<div class="gmail_default"
style="font-family:verdana,sans-serif;color:rgb(0,0,0)"><span
style="font-family:Arial,Helvetica,sans-serif;font-size:small;color:rgb(34,34,34)"><br>
</span></div>
<div class="gmail_default"
style="font-family:verdana,sans-serif;color:rgb(0,0,0)"><span
style="font-family:Arial,Helvetica,sans-serif;font-size:small;color:rgb(34,34,34)">which
mpirun</span></div>
<div class="gmail_default"
style="font-family:verdana,sans-serif;color:rgb(0,0,0)"><span
style="font-family:Arial,Helvetica,sans-serif;font-size:small;color:rgb(34,34,34)">which
lapw1_mpi</span></div>
<div class="gmail_default"
style="font-family:verdana,sans-serif">echo $WIENROOT</div>
<div class="gmail_default"
style="font-family:verdana,sans-serif">ldd
$WIENROOT/lapw1_mpi</div>
<div class="gmail_default"
style="font-family:verdana,sans-serif">ldd
$WIENROOT/lapw1</div>
<div class="gmail_default"
style="font-family:verdana,sans-serif">echo env</div>
<div class="gmail_default"
style="font-family:verdana,sans-serif">echo $PATH</div>
<div class="gmail_default"
style="font-family:verdana,sans-serif"><br>
</div>
<div class="gmail_default"
style="font-family:verdana,sans-serif">Run this
interactively as well as in a batch job and compare. You
will find that there are something which are not present
when you are launching your slurm job that are present
interactively. You need to repair these with relevant
PATH/LD_LIBRARY_PATH etc</div>
<div class="gmail_default"
style="font-family:verdana,sans-serif"><br>
</div>
<div class="gmail_default"
style="font-family:verdana,sans-serif">Your problems are
not Wien2k problems, they are due to incorrect
modules/script/environment or similar. Have you asked
your sysadmin for help? I am certain that someone local
who is experienced with standard linux can tell you very
quickly what to do. </div>
<div class="gmail_default"
style="font-family:verdana,sans-serif"><br>
</div>
<div class="gmail_default"
style="font-family:verdana,sans-serif">N.B., there is an
error in your path setting.</div>
</div>
<div style="color:rgb(34,34,34)">
<div class="gmail-adm" style="margin:5px 0px">
<div id="gmail-q_126" class="gmail-ajR gmail-h4"
style="font-size:11px;line-height:6px;background-color:rgb(232,234,237);border:none;clear:both;outline:none;width:24px;color:rgb(80,0,80);border-radius:5.5px"><br>
</div>
</div>
</div>
</div>
</div>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Tue, May 4, 2021 at 3:38
PM leila mollabashi <<a
href="mailto:le.mollabashi@gmail.com"
moz-do-not-send="true">le.mollabashi@gmail.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div dir="ltr">Dear all WIEN2k users,<br>
Thank you for your guides.
<div>>take care on the correct location ...<br>
It is the /usr/share/Modules/init<br>
After adding the “source /usr/share/Modules/init/tcsh”
line in to the script the same error appeared:<br>
mpirun: command not found</div>
<div><br>
In fact, with and without “source
/usr/share/Modules/init/tcsh” it is written in
slurm.out file that “
module load complete
”.</div>
<div><br>
</div>
<div>I noticed that “export” is also the bash command so
I used these commands to path the openmpi and fftw:<br>
setenv LD_LIBRARY_PATH
{$LD_LIBRARY_PATH}:/opt/exp_soft/local/generic/openmpi/4.1.0_gcc620/lib:/opt/exp_soft/local/generic/fftw/3.3.8_gcc620/lib<br>
set path = ($path
/opt/exp_soft/local/generic/openmpi/4.1.0_gcc620/bin:/opt/exp_soft/local/generic/fftw/3.3.8_gcc620/bin)</div>
<div>But result is the same:<br>
bash: mpirun: command not found<br>
</div>
<div><br>
By using this line in the script:<br>
/opt/exp_soft/local/generic/openmpi/4.1.0_gcc620/bin/mpiexec
/home/users/mollabashi/codes/v21.1/run_lapw -p<br>
The calculation stopped with the following error:<br>
mpirun does not support recursive calls</div>
<div><br>
> I wonder that you have only single modules…<br>
There are different versions of ifort and mkl:
ifort/15.0.0, ifort/15.0.3, ifort/17.0.1,
ifort/19.1.3.304(default) mkl/11.2, mkl/<a
href="https://urldefense.com/v3/__http://11.2.3.187__;!!Dq0X2DkFhyF93HkjWTBQKhk!BD88XA2ujgG8Gel0NZaSOKaSrtPN7kq75O9hkG-dnIZQJRbsnQE-ArEqbFqA6XVu7qQFcg$"
target="_blank" moz-do-not-send="true">11.2.3.187</a> mkl/2017.1.132,
mkl/2019.2.187, mkl/2020.0.4(default). I used the
defaults<br>
> you may also wish to make a single module file to
be loaded…<br>
That is a good idea.<br>
> On our cluster we have different W2k modules ….<br>
As you know WIEN2k is not a free code and the users of
the cluster that I am using are not registered WIEN2k
users. Thus, according to my moral commitment to the
WIEN2k developers, I cannot ask the administrator to
install it on the cluster. I should install it on my
user account.</div>
<div><br>
</div>
<div>Sincerely yours,<br>
Leila<br>
>PS.: maybe one should mention this tcsh "problem"
in the slurm.job example on the FAQ page by adding (or
similar)…<br>
That is a good idea. Thank you for your suggestion.<br>
</div>
</div>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr"><br>
</div>
</div>
</div>
</blockquote>
</div>
-- <br>
<div dir="ltr" class="gmail_signature">
<div dir="ltr">Professor Laurence Marks<br>
Department of Materials Science and Engineering<br>
Northwestern University<br>
<a href="http://www.numis.northwestern.edu/" target="_blank"
moz-do-not-send="true">www.numis.northwestern.edu</a>
<div>"Research is to see what everybody else has seen, and
to think what nobody else has thought" Albert <span
style="font-family:Arial,Helvetica,sans-serif;font-size:12.8px">Szent-</span><span
style="font-family:Arial,Helvetica,sans-serif;font-size:small;color:rgb(34,34,34)">Györgyi</span></div>
</div>
</div>
</div>
</blockquote>
</body>
</html>