[Wien] MPI error

Tue May 4 22:37:50 CEST 2021

Dear all WIEN2k users,
Thank you for your guides.
>take care on the correct location ...
It is the /usr/share/Modules/init
After adding the “source /usr/share/Modules/init/tcsh” line in to the
script the same error appeared:
mpirun: command not found

In fact, with and without “source /usr/share/Modules/init/tcsh” it is
written in slurm.out file that “ module load complete ”.

I noticed that “export” is also the bash command so I used these commands
to path the openmpi and fftw:
setenv LD_LIBRARY_PATH
{$LD_LIBRARY_PATH}:/opt/exp_soft/local/generic/openmpi/4.1.0_gcc620/lib:/opt/exp_soft/local/generic/fftw/3.3.8_gcc620/lib
set path = ($path
/opt/exp_soft/local/generic/openmpi/4.1.0_gcc620/bin:/opt/exp_soft/local/generic/fftw/3.3.8_gcc620/bin)
But result is the same:
bash: mpirun: command not found

By using this line in the script:
/opt/exp_soft/local/generic/openmpi/4.1.0_gcc620/bin/mpiexec
/home/users/mollabashi/codes/v21.1/run_lapw -p
The calculation stopped with the following error:
mpirun does not support recursive calls

> I wonder that you have only single modules…
There are different versions of ifort and mkl: ifort/15.0.0, ifort/15.0.3,
ifort/17.0.1, ifort/19.1.3.304(default) mkl/11.2, mkl/11.2.3.187
mkl/2017.1.132,
mkl/2019.2.187, mkl/2020.0.4(default). I used the defaults
> you may also wish to make a single module file to be loaded…
That is a good idea.
> On our cluster we have different W2k modules ….
As you know WIEN2k is not a free code and the users of the cluster that I
am using are not registered WIEN2k users. Thus, according to my moral
commitment to the WIEN2k developers, I cannot ask the administrator to
install it on the cluster. I should install it on my user account.

Sincerely yours,
Leila
>PS.: maybe one should mention this tcsh "problem" in the slurm.job example
on the FAQ page by adding (or similar)…
That is a good idea. Thank you for your suggestion.

On Mon, May 3, 2021 at 4:18 PM Fecher, Gerhard <fecher at uni-mainz.de> wrote:

> Dear Leila
> In your first mail you mentioned that you use
>      slurm.job
> with the added lines
>    module load openmpi/4.1.0_gcc620
>    module load ifort
>    module load mkl
>
> Sorry that my Sunday evening answer was too short, here a little more
> detail:
> I guess your login shell is bash, and you run the command
>      sbatch slurm.job
> that is written for tcsh, but tcsh does not know where the module command
> is, therfore  the job file should tell it where it is,
> e.g.: the beginning of slurm.job should look like
>
> #!/bin/tcsh
> #
> # Load the respective software module you intend to use, here for tcsh
> shell
> # NOTE: you may need to edit the source line !
> source /usr/share/lmod/lmod/init/tcsh
> module load openmpi/4.1.0_gcc620
> module load ifort
> module load mkl
>
> take care on the correct location it may be in: /usr/share/modules/init/csh
> if you do not find its correct location then ask your administrator
>
> I wonder that you have only single modules for ifort and mkl and not
> different version,
> I guess that are defaults, but which ? ask your administrator;
> you may also wish to make a single module file to be loaded, and
> you may also whish to send the output to the data nirvana by using >&
> /dev/null
> in that case you may have only the lines (as an example)
> source /usr/share/lmod/lmod/init/tcsh
> module load Wien2k/wien2k_21_intel19 >& /dev/null
> echo -n "Running Wien2k" $WienVersion
>
>
> PS.: maybe one should mention this tcsh "problem"  in the slurm.job
> example on the FAQ page by adding (or similar)
>   #  NOTE: you may need to edit the following line !
>   #  source /usr/share/lmod/lmod/init/tcsh
> as modules are frequently used on clusters and allow easily to change
> between different versions.
> On our cluster we have different W2k modules that have been compiled with
> different libraries, compilers, and/or settings.
>
> PSS.: I am not aware of typos ;-)
>
> Ciao
> Gerhard
>
> DEEP THOUGHT in D. Adams; Hitchhikers Guide to the Galaxy:
> "I think the problem, to be quite honest with you,
> is that you have never actually known what the question is."
>
> ====================================
> Dr. Gerhard H. Fecher
> Institut of Physics
> Johannes Gutenberg - University
> 55099 Mainz
> ________________________________________
> Von: Wien [wien-bounces at zeus.theochem.tuwien.ac.at] im Auftrag von leila
> mollabashi [le.mollabashi at gmail.com]
> Gesendet: Montag, 3. Mai 2021 00:35
> An: A Mailing list for WIEN2k users
> Betreff: Re: [Wien] MPI error
>
> Thank you.
>
> On Mon, May 3, 2021, 3:04 AM Laurence Marks <laurence.marks at gmail.com
> <mailto:laurence.marks at gmail.com>> wrote:
> You have to solve the "mpirun not found". That is due to your
> path/nfs/module -- we do not know.
>
> ---
> Prof Laurence Marks
> "Research is to see what everyone else has seen, and to think what nobody
> else has thought", Albert Szent-Györgyi
> www.numis.northwestern.edu<http://www.numis.northwestern.edu>
>
> On Sun, May 2, 2021, 17:12 leila mollabashi <le.mollabashi at gmail.com
> <mailto:le.mollabashi at gmail.com>> wrote:
> >You have an error in the LD_LIBRARY_PATH def you sent -- it needs to be
> "...:$LD_LIB..."
> Thank you. I have corrected it but I still have error in x lapw1 “mpirun:
> command not found”
> >Why not load the modules in the script to run a job? I have loaded but
> this error happened “bash: mpirun: command not found”.
>
> On Mon, May 3, 2021 at 2:23 AM Laurence Marks <laurence.marks at gmail.com
> <mailto:laurence.marks at gmail.com>> wrote:
> You have an error in the LD_LIBRARY_PATH def you sent -- it needs to be
> "...:$LD_LIB...".
>
> Why not load the modules in the script to run a job?
>
> ---
> Prof Laurence Marks
> "Research is to see what everyone else has seen, and to think what nobody
> else has thought", Albert Szent-Györgyi
> www.numis.northwestern.edu<http://www.numis.northwestern.edu>
>
> On Sun, May 2, 2021, 16:35 leila mollabashi <le.mollabashi at gmail.com
> <mailto:le.mollabashi at gmail.com>> wrote:
> Dear all WIEN2k users,
> Thank you for your reply.
> >The error is exactly what it says -- mpirun not found. This has something
> to do with the modules, almost certainly the openmpi one. You need to find
> where mpirun is on your system, and ensure that it is in your PATH. This is
> an issue with your OS, not Wien2k. However...
> which mpirun:
> /opt/exp_soft/local/generic/openmpi/4.1.0_gcc620/bin/mpirun
> I have installed WIEN2k by loading ifort, mkl, openmpi/4.1.0_gcc620,
> fftw/3.3.8_gcc620 modules. when I added the path in my .bashrc file as
> followes:
> export
> LD_LIBRARY_PATH=/opt/exp_soft/local/generic/openmpi/4.1.0_gcc620/lib:/opt/exp_soft/local/generic/fftw/3.3.8_gcc620/lib:LD_LIBRARY_PATH
> export
> PATH=/opt/exp_soft/local/generic/openmpi/4.1.0_gcc620/bin:/opt/exp_soft/local/generic/fftw/3.3.8_gcc620/bin:$PATH
> wien2k does not run:
> error while loading shared libraries: libiomp5.so: cannot open shared
> object file: No such file or directory
> 0.000u 0.000s 0:00.00 0.0%      0+0k 0+0io 0pf+0w
> but without the path and by loading the modules it runs.
> > First do "x lapw0 -p", send the .machines file and the last few lines of
> your *.output0*. Then we can confirm if that worked right, did not or what.
> .machines:
> lapw0:e0183:4
> 1:e0183:4
> 1:e0183:4
> Almost end of *output0000:
> TOTAL VALUE = -10433.492442     (H)
> :DEN  : DENSITY INTEGRAL  =        -20866.98488444   (Ry)
> Almost end of *output0001
> TOTAL VALUE = -10433.492442     (H)
> >Assuming that you used gcc
> Yes.
> >For certain you cannot run lapw2 without first running lapw1.
> Yes. You are right. When x lapw1 –p has not executed I have changed the
> .machines file and run in kpoint parallel mode then changed the .machines
> file again and run lapw2 –p.
> >How? Do you mean that there are no error messages?
> Yes and I also checked compile.msg in SRC_lapw1
>
> Sincerely yours,
>
> Leila
>
> On Mon, May 3, 2021 at 12:42 AM Fecher, Gerhard <fecher at uni-mainz.de
> <mailto:fecher at uni-mainz.de>> wrote:
> I guess that module does not work with tcsh
>
> Ciao
> Gerhard
>
> DEEP THOUGHT in D. Adams; Hitchhikers Guide to the Galaxy:
> "I think the problem, to be quite honest with you,
> is that you have never actually known what the question is."
>
> ====================================
> Dr. Gerhard H. Fecher
> Institut of Physics
> Johannes Gutenberg - University
> 55099 Mainz
> ________________________________________
> Von: Wien [wien-bounces at zeus.theochem.tuwien.ac.at<mailto:
> wien-bounces at zeus.theochem.tuwien.ac.at>] im Auftrag von Laurence Marks [
> laurence.marks at gmail.com<mailto:laurence.marks at gmail.com>]
> Gesendet: Sonntag, 2. Mai 2021 21:32
> An: A Mailing list for WIEN2k users
> Betreff: Re: [Wien] MPI error
>
> Inlined response and questions
>
> On Sun, May 2, 2021 at 2:19 PM leila mollabashi <le.mollabashi at gmail.com
> <mailto:le.mollabashi at gmail.com><mailto:le.mollabashi at gmail.com<mailto:
> le.mollabashi at gmail.com>>> wrote:
> Dear Prof. Peter Blaha and WIEN2k users,
> Now I have loaded the openmpi/4.1.0 and compiled Wine2k. The admin told me
> that I can use your script in >http://www.wien2k.at/reg_user/faq/slurm.job
> <
> https://urldefense.com/v3/__http://www.wien2k.at/reg_user/faq/slurm.job__;!!Dq0X2DkFhyF93HkjWTBQKhk!A4zeMc6H184Nsbinv0lWLQyzxpdvRUetaqlHDTUV8sC-k8WlE7z_qcoC_7AzO5s6X8cPOw$
> ><
> https://urldefense.com/v3/__http://www.wien2k.at/reg_user/faq/slurm.job__;!!Dq0X2DkFhyF93HkjWTBQKhk!G_67ZheBzKx4rn9SJ-7AOPNV2M9DFC6mHQ4b1S_sPZITO1RwQsLYLGNWwENJJwPKlowiXQ$>
> . I added this lines to it too:
> module load openmpi/4.1.0_gcc620
> module load ifort
> module load mkl
> but this error happened “bash: mpirun: command not found”.
> The error is exactly what it says -- mpirun not found. This has something
> to do with the modules, almost certainly the openmpi one. You need to find
> where mpirun is on your system, and ensure that it is in your PATH. This is
> an issue with your OS, not Wien2k. However...
>
> In an interactive mode “x lapw0 –p” and “x lapw2 –p” are executed MPI but
> “x lapw1 –p” is stoped with following error:
> w2k_dispatch_signal(): received: Segmentation fault
> Is this mpi mode? None of lapw0/1/2 can work in true parallel without
> mpirun, so there is something major wrong here. I doubt that anything
> really executed properly. For certain you cannot run lapw2 without first
> running lapw1. What is your .machines file? what is the content of the
> error files? (cat *.error).
>
> First do "x lapw0 -p", send the .machines file and the last few lines of
> your *.output0*. Then we can confirm if that worked right, did not or what.
>
> --------------------------------------------------------------------------
> I noticed that the FFTW3 and OpenMPI installed on the cluster are both
> compiled by gfortan. But I have compiled WIEN2k by intel ifort. I am not
> sure whether the problem originates from this inconsistency between gfortan
> and ifort.
> Almost everything in FFTW3 and OpenMPI is in fact c. Assuming that you
> used gcc there should be no problem. In general there should be no problem.
>
> I have checked that lapw1 has compiled correctly.
> How? Do you mean that there are no error messages?
>
>
> Sincerely yours,
>
> Leila
>
>
>
> _______________________________________________
> Wien mailing list
> Wien at zeus.theochem.tuwien.ac.at<mailto:Wien at zeus.theochem.tuwien.ac.at>
> http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien<
> https://urldefense.com/v3/__http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien__;!!Dq0X2DkFhyF93HkjWTBQKhk!A4zeMc6H184Nsbinv0lWLQyzxpdvRUetaqlHDTUV8sC-k8WlE7z_qcoC_7AzO5uDU05KOQ$
> >
> SEARCH the MAILING-LIST at:
> http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html<
> https://urldefense.com/v3/__http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html__;!!Dq0X2DkFhyF93HkjWTBQKhk!A4zeMc6H184Nsbinv0lWLQyzxpdvRUetaqlHDTUV8sC-k8WlE7z_qcoC_7AzO5swXEtm0g$
> >
> _______________________________________________
> Wien mailing list
> Wien at zeus.theochem.tuwien.ac.at<mailto:Wien at zeus.theochem.tuwien.ac.at>
>
> https://urldefense.com/v3/__http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien__;!!Dq0X2DkFhyF93HkjWTBQKhk!A4zeMc6H184Nsbinv0lWLQyzxpdvRUetaqlHDTUV8sC-k8WlE7z_qcoC_7AzO5uDU05KOQ$
> SEARCH the MAILING-LIST at:
> https://urldefense.com/v3/__http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html__;!!Dq0X2DkFhyF93HkjWTBQKhk!A4zeMc6H184Nsbinv0lWLQyzxpdvRUetaqlHDTUV8sC-k8WlE7z_qcoC_7AzO5swXEtm0g$
> _______________________________________________
> Wien mailing list
> Wien at zeus.theochem.tuwien.ac.at<mailto:Wien at zeus.theochem.tuwien.ac.at>
> http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien<
> https://urldefense.com/v3/__http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien__;!!Dq0X2DkFhyF93HkjWTBQKhk!CZnbK8WwpYpOc2mUYWb9cTAiCvhVrGX09mpZmoHtNgKXf6RoG5gXEtluuJ63EqfuT1QoPw$
> >
> SEARCH the MAILING-LIST at:
> http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html<
> https://urldefense.com/v3/__http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html__;!!Dq0X2DkFhyF93HkjWTBQKhk!CZnbK8WwpYpOc2mUYWb9cTAiCvhVrGX09mpZmoHtNgKXf6RoG5gXEtluuJ63Eqdv4aqpLg$
> >
> _______________________________________________
> Wien mailing list
> Wien at zeus.theochem.tuwien.ac.at<mailto:Wien at zeus.theochem.tuwien.ac.at>
>
> https://urldefense.com/v3/__http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien__;!!Dq0X2DkFhyF93HkjWTBQKhk!CZnbK8WwpYpOc2mUYWb9cTAiCvhVrGX09mpZmoHtNgKXf6RoG5gXEtluuJ63EqfuT1QoPw$
> SEARCH the MAILING-LIST at:
> https://urldefense.com/v3/__http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html__;!!Dq0X2DkFhyF93HkjWTBQKhk!CZnbK8WwpYpOc2mUYWb9cTAiCvhVrGX09mpZmoHtNgKXf6RoG5gXEtluuJ63Eqdv4aqpLg$
> _______________________________________________
> Wien mailing list
> Wien at zeus.theochem.tuwien.ac.at<mailto:Wien at zeus.theochem.tuwien.ac.at>
> http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien
> SEARCH the MAILING-LIST at:
> http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html
> _______________________________________________
> Wien mailing list
> Wien at zeus.theochem.tuwien.ac.at
> http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien
> SEARCH the MAILING-LIST at:
> http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://zeus.theochem.tuwien.ac.at/pipermail/wien/attachments/20210505/fdac0b44/attachment.htm>