<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";
mso-fareast-language:EN-US;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:70.85pt 70.85pt 70.85pt 70.85pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="NL" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span lang="EN-GB">Dear WIEN2k users,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">I recently updated from WIEN2k.18.1 to WIEN2k.19.1.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">Both versions were compiled with the same intel compilers, intel_xe_2016.<br>
The FFTW3 (fftw-3.3.4) and LIBXC(libxc-4.3.4) packages were also compiled with the same intel compilers as the WIEN2k versions and<br>
WIEN2k.19.1 compiles without any error. Each individual node is a shared memory machine, and runs <a href="http://www.centos.org/" target="_blank"><span style="color:windowtext;text-decoration:none">CentOS 7</span></a>, where PBS/torque is used to schedule
jobs. I have the following compiler options for WIEN2k.19.1 <o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">M OpenMP switch: -qopenmp<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"> O Compiler options: -O -FR -mp1 -w -prec_div -pc80 -pad -ip -DINTEL_VML -traceback -assume buffered_io -I$(MKLROOT)/include<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"> L Linker Flags: $(FOPT) -L$(MKLROOT)/lib/$(MKL_TARGET_ARCH) -lpthread -lm -ldl -liomp5<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"> P Preprocessor flags '-DParallel'<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"> R R_LIBS (LAPACK+BLAS): -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"> F FFTW options: -DFFTW3 -DFFTW_OMP -I/home/lamaduro/2FFTW3/FFTW/include<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"> FFTW-LIBS: -L/home/lamaduro/2FFTW3/FFTW/lib -lfftw3 -lfftw3_omp<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"> FFTW-PLIBS: -lfftw3_mpi<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"> X LIBX options: -DLIBXC -I/home/lamaduro/LIBXC_4_3_4/LIBXC/LIBXC/include<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"> LIBXC-LIBS: -L/home/lamaduro/LIBXC_4_3_4/LIBXC/LIBXC/lib/ -lxcf03 –lxc<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">C Parallel Compiler: mpiifort<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"> FP Parallel Compiler Options: -O -FR -mp1 -w -prec_div -pc80 -pad -ip -DINTEL_VML -traceback -assume buffered_io -I$(MKLROOT)/include<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"> MP MPIRUN command: /opt/ud/openmpi-1.8.8/bin/mpirun<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">Sp SCALAPACK: -L$(MKLROOT)/lib/intel64<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"> -lmkl_scalapack_lp64<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"> -L$(MKLROOT)/lib/intel64 -lmkl_blacs_intelmpi_lp64<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">When running the code in serial mode on a unit cell of MoS2 everything runs fine on one node and one cpu. However, when I use more than one cpu then lapw1 gives the following error(in the case below 1 node and 2 cpus
were used as a test) <o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">NLVDW END<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">[1] Done mpirun -np 2 -machinefile .machinenlvdw /home/lamaduro/WIEN2k_19_2/nlvdw_mpi nlvdw.def >> .time00<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">LAPW0 END<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">[1] Done mpirun -np 2 -machinefile .machine0 /home/lamaduro/WIEN2k_19_2/lapw0_mpi lapw0.def >> .time00<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">w2k_dispatch_signal(): received: Segmentation fault<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">w2k_dispatch_signal(): received: Segmentation fault<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">application called MPI_Abort(MPI_COMM_WORLD, 0) - process 0<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">application called MPI_Abort(MPI_COMM_WORLD, 0) - process 1<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">[1] + Done ( cd $PWD; $t $ttt; rm -f .lock_$lockfile[$p] ) >> .time1_$loop<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">grep: lapw2*.error: No such file or directory<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">When I change to the WIEN2k.18.1 version lapw1 runs fine.
<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">Following the advice giving in this link <a href="http://zeus.theochem.tuwien.ac.at/pipermail/wien/2012-July/017209.html">
<span style="color:windowtext;text-decoration:none">http://zeus.theochem.tuwien.ac.at/pipermail/wien/2012-July/017209.html</span></a>
<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">I changed the blacs library to mkl_blacs_openmpi_lp64 and got the following errors when recompiling lapw1<br>
<br>
<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">/opt/ud/intel_xe_2016/compilers_and_libraries_2016.3.210/linux/mkl/lib/intel64/libmkl_blacs_openmpi_lp64.so: undefined reference to `ompi_mpi_comm_null'<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">/opt/ud/intel_xe_2016/compilers_and_libraries_2016.3.210/linux/mkl/lib/intel64/libmkl_blacs_openmpi_lp64.so: undefined reference to `MPI_Comm_f2c'<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">/opt/ud/intel_xe_2016/compilers_and_libraries_2016.3.210/linux/mkl/lib/intel64/libmkl_blacs_openmpi_lp64.so: undefined reference to `ompi_mpi_double'<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">/opt/ud/intel_xe_2016/compilers_and_libraries_2016.3.210/linux/mkl/lib/intel64/libmkl_blacs_openmpi_lp64.so: undefined reference to `ompi_mpi_dblcplex'<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">/opt/ud/intel_xe_2016/compilers_and_libraries_2016.3.210/linux/mkl/lib/intel64/libmkl_blacs_openmpi_lp64.so: undefined reference to `ompi_mpi_packed'<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">/opt/ud/intel_xe_2016/compilers_and_libraries_2016.3.210/linux/mkl/lib/intel64/libmkl_blacs_openmpi_lp64.so: undefined reference to `ompi_mpi_op_max'<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">/opt/ud/intel_xe_2016/compilers_and_libraries_2016.3.210/linux/mkl/lib/intel64/libmkl_blacs_openmpi_lp64.so: undefined reference to `ompi_mpi_op_maxloc'<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">/opt/ud/intel_xe_2016/compilers_and_libraries_2016.3.210/linux/mkl/lib/intel64/libmkl_blacs_openmpi_lp64.so: undefined reference to `ompi_mpi_op_min'<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">/opt/ud/intel_xe_2016/compilers_and_libraries_2016.3.210/linux/mkl/lib/intel64/libmkl_blacs_openmpi_lp64.so: undefined reference to `ompi_mpi_byte'<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">/opt/ud/intel_xe_2016/compilers_and_libraries_2016.3.210/linux/mkl/lib/intel64/libmkl_blacs_openmpi_lp64.so: undefined reference to `ompi_mpi_comm_world'<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">/opt/ud/intel_xe_2016/compilers_and_libraries_2016.3.210/linux/mkl/lib/intel64/libmkl_blacs_openmpi_lp64.so: undefined reference to `ompi_mpi_float'<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">/opt/ud/intel_xe_2016/compilers_and_libraries_2016.3.210/linux/mkl/lib/intel64/libmkl_blacs_openmpi_lp64.so: undefined reference to `ompi_request_null'<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">/opt/ud/intel_xe_2016/compilers_and_libraries_2016.3.210/linux/mkl/lib/intel64/libmkl_blacs_openmpi_lp64.so: undefined reference to `ompi_mpi_cplex'<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">/opt/ud/intel_xe_2016/compilers_and_libraries_2016.3.210/linux/mkl/lib/intel64/libmkl_blacs_openmpi_lp64.so: undefined reference to `ompi_mpi_unsigned_short'<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">/opt/ud/intel_xe_2016/compilers_and_libraries_2016.3.210/linux/mkl/lib/intel64/libmkl_blacs_openmpi_lp64.so: undefined reference to `MPI_Comm_c2f'<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">/opt/ud/intel_xe_2016/compilers_and_libraries_2016.3.210/linux/mkl/lib/intel64/libmkl_blacs_openmpi_lp64.so: undefined reference to `ompi_mpi_double_int'<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">/opt/ud/intel_xe_2016/compilers_and_libraries_2016.3.210/linux/mkl/lib/intel64/libmkl_blacs_openmpi_lp64.so: undefined reference to `ompi_mpi_int'<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">/opt/ud/intel_xe_2016/compilers_and_libraries_2016.3.210/linux/mkl/lib/intel64/libmkl_blacs_openmpi_lp64.so: undefined reference to `ompi_mpi_long_long_int'<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">/opt/ud/intel_xe_2016/compilers_and_libraries_2016.3.210/linux/mkl/lib/intel64/libmkl_blacs_openmpi_lp64.so: undefined reference to `ompi_mpi_integer'<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">/opt/ud/intel_xe_2016/compilers_and_libraries_2016.3.210/linux/mkl/lib/intel64/libmkl_blacs_openmpi_lp64.so: undefined reference to `ompi_mpi_dblprec'<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">/opt/ud/intel_xe_2016/compilers_and_libraries_2016.3.210/linux/mkl/lib/intel64/libmkl_blacs_openmpi_lp64.so: undefined reference to `MPI_Type_f2c'<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">/opt/ud/intel_xe_2016/compilers_and_libraries_2016.3.210/linux/mkl/lib/intel64/libmkl_blacs_openmpi_lp64.so: undefined reference to `MPI_Op_f2c'<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">/opt/ud/intel_xe_2016/compilers_and_libraries_2016.3.210/linux/mkl/lib/intel64/libmkl_blacs_openmpi_lp64.so: undefined reference to `ompi_mpi_op_sum'<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">/opt/ud/intel_xe_2016/compilers_and_libraries_2016.3.210/linux/mkl/lib/intel64/libmkl_blacs_openmpi_lp64.so: undefined reference to `MPI_Op_c2f'<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">/opt/ud/intel_xe_2016/compilers_and_libraries_2016.3.210/linux/mkl/lib/intel64/libmkl_blacs_openmpi_lp64.so: undefined reference to `ompi_mpi_real'<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">/opt/ud/intel_xe_2016/compilers_and_libraries_2016.3.210/linux/mkl/lib/intel64/libmkl_blacs_openmpi_lp64.so: undefined reference to `MPI_Type_c2f'<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">I’m not sure how to proceed. Any help would be appreciated.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:8.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:8.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:8.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">Kind regards,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">Luigi Maduro<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="mso-fareast-language:NL">PhD candidate<br>
Kavli Institute of Nanoscience<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="mso-fareast-language:NL">Department of Quantum Nanoscience<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="mso-fareast-language:NL">Faculty of Applied Sciences<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="mso-fareast-language:NL">Delft University of Technology<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</body>
</html>