<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=gb18030">
</head>
<body>
<p>Those undefined references, such as _intel_fast_memcpy, are
defined in an Intel compiler library [1]. When using Intel
compilers (ifort/icc/mpiifort), typically mkl is used instead of
OpenBLAS as well as the Intel MPI Library [2]. Since you have
used the gnu compilers (gfortran/gcc) to compile OpenBLAS, WIEN2k,
and your flavor of mpi (whether that be Open-MPI [3], MVAPICH [4],
or another one [5]), most likely you need to recompile FFTW3 with
your mpi compiler (e.g. [6]) based on gfortran/gcc too.<br>
</p>
<p><br>
</p>
[1]
<a class="moz-txt-link-freetext" href="https://software.intel.com/en-us/forums/intel-c-compiler/topic/815696">https://software.intel.com/en-us/forums/intel-c-compiler/topic/815696</a><br>
[2]
<a class="moz-txt-link-freetext" href="https://software.intel.com/content/www/us/en/develop/tools/mpi-library.html">https://software.intel.com/content/www/us/en/develop/tools/mpi-library.html</a><br>
[3] <a class="moz-txt-link-freetext" href="https://www.open-mpi.org/">https://www.open-mpi.org/</a><br>
[4] <a class="moz-txt-link-freetext" href="http://mvapich.cse.ohio-state.edu/">http://mvapich.cse.ohio-state.edu/</a><br>
[5] <a class="moz-txt-link-freetext" href="https://en.wikipedia.org/wiki/MPICH#MPICH_derivatives">https://en.wikipedia.org/wiki/MPICH#MPICH_derivatives</a><br>
[6] <a class="moz-txt-link-freetext" href="https://www.open-mpi.org/faq/?category=mpi-apps#general-build">https://www.open-mpi.org/faq/?category=mpi-apps#general-build</a><br>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">On 6/15/2020 7:36 PM, 晨晨 wrote:<br>
</div>
<blockquote type="cite"
cite="mid:tencent_1B4DCBFBB87A06082DEE8F2193EA6F626105@qq.com">
<meta http-equiv="Content-Type" content="text/html;
charset=gb18030">
<div>
<p class="MsoNormal"><span lang="EN-US">Thank you for your
details.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Now, there is another
problem. In order to
generate the lapw0_mpi command, when I run “make para” in
the SRC_lapw0, a part
of the errors is shown as the following and the whole
information is attached:</span> </p>
<p class="MsoNormal"><span lang="EN-US">mpif90 -o ./lapw0_mpi
cputim.o
modules.o W2kinit.o fft_modules.o
reallocate.o acggac.o ……<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">ylm.o -ffree-form -O2
-ftree-vectorize
-march=native -ffree-line-length-none -ffpe-summary=none
-DFFTW3
-I/home-yw/Soft/fftw-3.3.8/include
-fopenmp -L../SRC_lib
-lfftw3_mpi
-L/home-yw/Soft/fftw-3.3.8/lib -lfftw3
/home-yw/soft/packagedir/Openblas-0.39/gcc820/lib/libopenblas.so<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">/home-yw/Soft/fftw-3.3.8/lib/libfftw3_mpi.so:
undefined reference to `_intel_fast_memcpy'<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">/home-yw/Soft/fftw-3.3.8/lib/libfftw3_mpi.so:
undefined reference to `_intel_fast_memset'<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">/home-yw/Soft/fftw-3.3.8/lib/libfftw3_mpi.so:
undefined reference to `_intel_fast_memmove'<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">/home-yw/Soft/fftw-3.3.8/lib/libfftw3_mpi.so:
undefined reference to `__intel_sse2_strlen'<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">collect2: error: ld
returned 1 exit status<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">make[1]: *** [lapw0_mpi]
Error 1<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">make[1]: Leaving
directory `/home-yg/users/YG_cheny/Soft/19.2/SRC_lapw0'<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">make: *** [para] Error 2<o:p></o:p></span></p>
</div>
<div>
<div><br>
</div>
<div><br>
</div>
<div style="font-size: 12px;font-family: Arial
Narrow;padding:2px 0 2px 0;">------------------ 原始邮件 ------------------</div>
<div style="font-size: 12px;background:#efefef;padding:8px;">
<div><b>发件人:</b> "Gavin Abo"<a class="moz-txt-link-rfc2396E" href="mailto:gsabo@crimson.ua.edu"><gsabo@crimson.ua.edu></a>;</div>
<div><b>发送时间:</b> 2020年6月8日(星期一) 晚上8:18</div>
<div><b>收件人:</b> "wien"<a class="moz-txt-link-rfc2396E" href="mailto:wien@zeus.theochem.tuwien.ac.at"><wien@zeus.theochem.tuwien.ac.at></a>;<wbr></div>
<div><b>主题:</b> Re: [Wien] Wien Installation</div>
</div>
<div><br>
</div>
<span lang="EN-US"><o:p></o:p></span>
<blockquote type="cite"
cite="mid:tencent_09FE5216BCB4928EAF6660D180FE4C337909@qq.com">
<div><span lang="EN-US">In terms of OpenBLAS-0.3.9 libraries,
in my OpenBLAS-0.3.9/lib, there are several files as the
below: libopenblas_nehalemp-r0.3.9.a,
libopenblas_nehalemp-r0.3.9.so, libopenblas.so,
libopenblas.so.0, libopenblas.a. Are they as same as
yours?</span></div>
</blockquote>
<font color="#0000ff">I have the following, but you have the
same one named "libopenblas.so" that I use.<br>
</font><br>
<font color="#0000ff">username@computername:~/WIEN2k$ ls
~/OpenBLAS-0.3.9/*.a ~/OpenBLAS-0.3.9/*.so<br>
/home/username/OpenBLAS-0.3.9/libopenblas.a<br>
/home/username/OpenBLAS-0.3.9/libopenblas_barcelonap-r0.3.9.a<br>
/home/username/OpenBLAS-0.3.9/libopenblas_barcelonap-r0.3.9.so<br>
/home/username/OpenBLAS-0.3.9/libopenblas.so</font><br>
<blockquote type="cite"><span lang="EN-US">In addition, are
there commands such as lapw0_mpi, lapw1_mpi and lapw1c_mpi
generated by installing with gfortran+OpenBLAS, just like by
installing with ifort?</span></blockquote>
<font color="#0000ff">No, the Intel Parallel Studio XE 2020
Cluster Edition has the Intel MPI Library [1] that provides
intelmpi making it easy to build the WIEN2k *_mpi files. For
gfortran, it can be done but it is extra work as one has to
install scalapack [2] and MPI (such as OpenMPI [3]) separately
and then include them in the parallel compiler settings of
WIEN2k in order to build the files.<br>
</font><br>
<font color="#0000ff">[1]
<a class="moz-txt-link-freetext"
href="https://software.intel.com/content/www/us/en/develop/tools/parallel-studio-xe.html"
moz-do-not-send="true">https://software.intel.com/content/www/us/en/develop/tools/parallel-studio-xe.html</a></font><br>
<font color="#0000ff">[2] <a class="moz-txt-link-freetext"
href="http://www.netlib.org/scalapack/"
moz-do-not-send="true">http://www.netlib.org/scalapack/</a></font><br>
<font color="#0000ff">[3] <a class="moz-txt-link-freetext"
href="https://www.open-mpi.org/software/ompi/v4.0/"
moz-do-not-send="true">https://www.open-mpi.org/software/ompi/v4.0/</a></font><br>
<blockquote type="cite"
cite="mid:tencent_09FE5216BCB4928EAF6660D180FE4C337909@qq.com">
<div><span lang="EN-US"><o:p></o:p></span><br>
<span lang="EN-US">Then during ./siteconfig_lapw, I choose O
to set libraries “R_LIBS(LAPACK+BLAS)”, then how to set
the “Real libraries”? Would you like to tell me how set
the “Real libraries” parameter so that could obtain the
result just like your path for R_LIBS as
“-L/home/username/OpenBLAS-0.3.9 -lopenblas -llapack_lapw
–lpthread”?<o:p></o:p></span><br>
<br>
<span lang="EN-US">In my setting, I did a couple of tries
and obtain three kinds of results as below:<o:p></o:p></span><br>
<br>
<span lang="EN-US">1.<span style="font-variant-numeric:
normal; font-variant-east-asian: normal; font-stretch:
normal; font-size: 7pt; line-height: normal;"> </span></span><span
lang="EN-US">When I set the R_LIBS such as
/home/username/OpenBLAS-0.3.9/lib/libopenblas.so (or
libopenblas.so.0, or libopenblas_nehalemp-r0.3.9.so), then
the massage is: /usr/bin/ld: warning: libgfortran.so.3,
needed by /home-yw/Soft/OpenBLAS-0.3.9/lib/libopenblas.so,
may conflict with libgfortran.so.5.<o:p></o:p></span><br>
<br>
<span lang="EN-US">2.<span style="font-variant-numeric:
normal; font-variant-east-asian: normal; font-stretch:
normal; font-size: 7pt; line-height: normal;"> </span></span><span
lang="EN-US">When I set the R_LIBS such as
/home/username/OpenBLAS-0.3.9/lib or
/home/username/OpenBLAS-0.3.9, then the massage is: file
not recognized: Is a directory.<o:p></o:p></span><br>
<br>
<span lang="EN-US">3.<span style="font-variant-numeric:
normal; font-variant-east-asian: normal; font-stretch:
normal; font-size: 7pt; line-height: normal;"> </span></span><span
lang="EN-US">When I set he R_LIBS such as
/home/username/OpenBLAS-0.3.9/lib/
libopenblas_nehalemp-r0.3.9.a (or libopenblas.a), then the
error massage is much, including:
/home-yw/Soft/OpenBLAS-0.3.9/lib/libopenblas_nehalemp-r0.3.9.a(blas_server.o):
In function `goto_set_num_threads':
blas_server.c:(.text+0x7f3): undefined reference to
`pthread_create'.<o:p></o:p></span><br>
<br>
<span lang="EN-US">All above relating files were attached.<o:p></o:p></span><br>
</div>
</blockquote>
<font color="#0000ff">Attached is copy of my compile.msg from
SRC_aim that you can compare to yours.</font><br>
<font color="#0000ff">Below is what I used for a basic WIEN2k
19.2 install that should work for serial and k-point parallel
calculations (but won't work for mpi calculations). Of note,
it looks like you will need to replace
"/home/username/OpenBLAS-0.3.9" with
"/home-yw/Soft/OpenBLAS-0.3.9/lib" on your system. Also, you
might notice below that the "-llapack_lapw" of WIEN2k is not
needed since -lopenblas contains it own lapack library.<br>
</font><br>
<font color="#0000ff"><br>
username@computername:~/WIEN2k$ ls WIEN2k_INSTALLDATE <br>
WIEN2k_INSTALLDATE<br>
username@computername:~/WIEN2k$ rm WIEN2k_INSTALLDATE<br>
username@computername:~/WIEN2k$ ./siteconfig<br>
...<br>
continue or stop (c/s) c<br>
...<br>
Selection: LG<br>
...<br>
Press RETURN to continue<br>
...<br>
Your compiler: gfortran<br>
...<br>
Your compiler: gcc<br>
...<br>
Would you like to use LIBXC (that you have installed -
usually not needed)? (y,N): <br>
N<br>
...<br>
Do you want to use FFTW (recommended, but for sequential code
not required)? (Y,n): n<br>
...<br>
Selection: R
<br>
Real libraries=-L/home/username/OpenBLAS-0.3.9 -lopenblas
-lpthread<br>
...<br>
Current settings:<br>
M OpenMP switch: -fopenmp<br>
O Compiler options: -ffree-form -O2
-ftree-vectorize -march=native -ffree-line-length-none
-ffpe-summary=none<br>
L Linker Flags: $(FOPT) -L../SRC_lib<br>
P Preprocessor flags '-DParallel'<br>
R R_LIBS (LAPACK+BLAS): -L/home/username/OpenBLAS-0.3.9
-lopenblas -lpthread<br>
F FFTW options:<br>
FFTW-LIBS:<br>
X LIBX options:<br>
LIBXC-LIBS:<br>
<br>
S Save and Quit<br>
<br>
To change an item select option.<br>
Selection: S<br>
...<br>
Shared Memory Architecture? (y/N):y<br>
Do you know/need a command to bind your jobs to specific
nodes?<br>
(like taskset -c). Enter N / your_specific_command: N<br>
...<br>
(y/N) N<br>
...<br>
Selection: Q<br>
...<br>
Selection: A<br>
...</font><br>
<font color="#0000ff">Compile time errors (if any) were:<br>
<br>
<br>
Check file compile.msg in the corresponding SRC_*
directory for the <br>
compilation log and more info on any compilation problem.<br>
...<br>
Please enter the full path of the perl program:
/usr/bin/perl<br>
...<br>
Please enter the full path to your temporary directory: /tmp<br>
...<br>
username@computername:~/WIEN2k$ userconfig<br>
...<br>
username@computername:~/WIEN2k$ gedit ~/.bashrc<br>
username@computername:~/WIEN2k$ grep LD_LIBRARY ~/.bashrc<br>
export
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/username/OpenBLAS-0.3.9</font><font
color="#0000ff"><br>
</font></div>
</blockquote>
<blockquote type="cite"
cite="mid:tencent_1B4DCBFBB87A06082DEE8F2193EA6F626105@qq.com">
</blockquote>
</body>
</html>