<meta http-equiv="Content-Type" content="text/html; charset=GB18030"><div><p class="MsoNormal"><span lang="EN-US">Thank you for your reply.<o:p></o:p></span></p>

<p class="MsoNormal">I am installing wien version 19.2 on Linux
with gfortran and OpenBlas.</p>

<p class="MsoNormal">In the “Specify ScaLAPACK settings”, I
would like to know how to set the part of “Please specify the target
architecture of your ScaLAPACK libraries (e.g. intel64)!:”.</p>

<p class="MsoNormal"><span lang="EN-US">In addition, when the settings are as the
following, there are some errors.<o:p></o:p></span></p>

<p class="MsoNormal"><span lang="EN-US">Settings:<o:p></o:p></span></p>

<p class="MsoNormal"><span lang="EN-US">SCALAPACKROOT:/home/username/scalapack-2.1.0/<o:p></o:p></span></p>

<p class="MsoNormal"><span lang="EN-US">SCALAPACK_LIBNAME: libscalapack.a<o:p></o:p></span></p>

<p class="MsoNormal"><span lang="EN-US">BLACSROOT: /home/username/BLACS/LIB<o:p></o:p></span></p>

<p class="MsoNormal"><span lang="EN-US">BLACS_LIBNAME: blacs_MPI-LINUX-0.a<o:p></o:p></span></p>

<p class="MsoNormal"><span lang="EN-US">MKL_TARGET_ARCH: /home/username/scalapack-2.1.0/libscalapack.a<o:p></o:p></span></p>

<p class="MsoNormal"><span lang="EN-US">Some errors:<o:p></o:p></span></p>

<p class="MsoNormal"><span lang="EN-US">-ftree-vectorize -march=native
-ffree-line-length-none -ffpe-summary=none -DFFTW3
-I/home-yg/users/YG_cheny/Soft/fftw-3.3.6-pl2/gcc820/include -fopenmp
-L../SRC_lib -L/home-yg/users/YG_cheny/Soft/fftw-3.3.6-pl2/gcc820/lib -lfftw3</span>  /home-yw/soft/packagedir/Openblas-0.39/gcc820/lib/libopenblas.so
-L/home-yg/users/YG_cheny/Soft/scalapack-2.1.0//home-yg/users/YG_cheny/Soft/scalapack-2.1.0/libscalapack.a
-llibscalapack.a -L/home-yg/users/YG_cheny/Soft/BLACS/LIB//home-yg/users/YG_cheny/Soft/scalapack-2.1.0/libscalapack.a
-lblacs_MPI-LINUX-0.a  <br>
/usr/bin/ld: cannot find -llibscalapack.a<br>
collect2: error: ld returned 1 exit status<br>
make[1]: *** [hfc_mpi] Error 1<br>
make[1]: Leaving directory `/home-yg/users/YG_cheny/Soft/19.2/SRC_hf'<br>
make: *** [cp] Error 2<o:p></o:p></p></div><div><div><br></div><div><br></div><div style="font-size: 12px;font-family: Arial Narrow;padding:2px 0 2px 0;">------------------ 原始邮件 ------------------</div><div style="font-size: 12px;background:#efefef;padding:8px;"><div><b>发件人:</b> "Gavin Abo"<gsabo@crimson.ua.edu>;</div><div><b>发送时间:</b> 2020年6月16日(星期二) 上午10:38</div><div><b>收件人:</b> "wien"<wien@zeus.theochem.tuwien.ac.at>;<wbr></div><div></div><div><b>主题:</b> Re: [Wien] Wien Installation</div></div><div><br></div>
  
    
  
  
    <p>Those undefined references, such as _intel_fast_memcpy, are
      defined in an Intel compiler library [1].  When using Intel
      compilers (ifort/icc/mpiifort), typically mkl is used instead of
      OpenBLAS as well as the Intel MPI Library [2].  Since you have
      used the gnu compilers (gfortran/gcc) to compile OpenBLAS, WIEN2k,
      and your flavor of mpi (whether that be Open-MPI [3], MVAPICH [4],
      or another one [5]), most likely you need to recompile FFTW3 with
      your mpi compiler (e.g. [6]) based on gfortran/gcc too.<br>
    </p>
    <p><br>
    </p>
    [1]
    <a class="moz-txt-link-freetext" href="https://software.intel.com/en-us/forums/intel-c-compiler/topic/815696">https://software.intel.com/en-us/forums/intel-c-compiler/topic/815696</a><br>
    [2]
<a class="moz-txt-link-freetext" href="https://software.intel.com/content/www/us/en/develop/tools/mpi-library.html">https://software.intel.com/content/www/us/en/develop/tools/mpi-library.html</a><br>
    [3] <a class="moz-txt-link-freetext" href="https://www.open-mpi.org/">https://www.open-mpi.org/</a><br>
    [4] <a class="moz-txt-link-freetext" href="http://mvapich.cse.ohio-state.edu/">http://mvapich.cse.ohio-state.edu/</a><br>
    [5] <a class="moz-txt-link-freetext" href="https://en.wikipedia.org/wiki/MPICH#MPICH_derivatives">https://en.wikipedia.org/wiki/MPICH#MPICH_derivatives</a><br>
    [6] <a class="moz-txt-link-freetext" href="https://www.open-mpi.org/faq/?category=mpi-apps#general-build">https://www.open-mpi.org/faq/?category=mpi-apps#general-build</a><br>
    <div class="moz-cite-prefix"><br>
    </div>
    <div class="moz-cite-prefix">On 6/15/2020 7:36 PM, 晨晨 wrote:<br>
    </div>
    <blockquote type="cite" cite="mid:tencent_1B4DCBFBB87A06082DEE8F2193EA6F626105@qq.com">
      
      <div>
        <p class="MsoNormal"><span lang="EN-US">Thank you for your
            details.<o:p></o:p></span></p>
        <p class="MsoNormal"><span lang="EN-US">Now, there is another
            problem. In order to
            generate the lapw0_mpi command, when I run “make para” in
            the SRC_lapw0, a part
            of the errors is shown as the following and the whole
            information is attached:</span> </p>
        <p class="MsoNormal"><span lang="EN-US">mpif90 -o ./lapw0_mpi
            cputim.o
            modules.o  W2kinit.o fft_modules.o
            reallocate.o acggac.o ……<o:p></o:p></span></p>
        <p class="MsoNormal"><span lang="EN-US">ylm.o -ffree-form -O2
            -ftree-vectorize
            -march=native -ffree-line-length-none -ffpe-summary=none 
            -DFFTW3
            -I/home-yw/Soft/fftw-3.3.8/include 
            -fopenmp -L../SRC_lib 
            -lfftw3_mpi 
            -L/home-yw/Soft/fftw-3.3.8/lib -lfftw3 
/home-yw/soft/packagedir/Openblas-0.39/gcc820/lib/libopenblas.so<o:p></o:p></span></p>
        <p class="MsoNormal"><span lang="EN-US">/home-yw/Soft/fftw-3.3.8/lib/libfftw3_mpi.so:
            undefined reference to `_intel_fast_memcpy'<o:p></o:p></span></p>
        <p class="MsoNormal"><span lang="EN-US">/home-yw/Soft/fftw-3.3.8/lib/libfftw3_mpi.so:
            undefined reference to `_intel_fast_memset'<o:p></o:p></span></p>
        <p class="MsoNormal"><span lang="EN-US">/home-yw/Soft/fftw-3.3.8/lib/libfftw3_mpi.so:
            undefined reference to `_intel_fast_memmove'<o:p></o:p></span></p>
        <p class="MsoNormal"><span lang="EN-US">/home-yw/Soft/fftw-3.3.8/lib/libfftw3_mpi.so:
            undefined reference to `__intel_sse2_strlen'<o:p></o:p></span></p>
        <p class="MsoNormal"><span lang="EN-US">collect2: error: ld
            returned 1 exit status<o:p></o:p></span></p>
        <p class="MsoNormal"><span lang="EN-US">make[1]: *** [lapw0_mpi]
            Error 1<o:p></o:p></span></p>
        <p class="MsoNormal"><span lang="EN-US">make[1]: Leaving
            directory `/home-yg/users/YG_cheny/Soft/19.2/SRC_lapw0'<o:p></o:p></span></p>
        <p class="MsoNormal"><span lang="EN-US">make: *** [para] Error 2<o:p></o:p></span></p>
      </div>
      <div>
        <div><br>
        </div>
        <div><br>
        </div>
        <div style="font-size: 12px;font-family: Arial
          Narrow;padding:2px 0 2px 0;">------------------ 原始邮件 ------------------</div>
        <div style="font-size: 12px;background:#efefef;padding:8px;">
          <div><b>发件人:</b> "Gavin Abo"<a class="moz-txt-link-rfc2396E" href="mailto:gsabo@crimson.ua.edu"><gsabo@crimson.ua.edu></a>;</div>
          <div><b>发送时间:</b> 2020年6月8日(星期一) 晚上8:18</div>
          <div><b>收件人:</b> "wien"<a class="moz-txt-link-rfc2396E" href="mailto:wien@zeus.theochem.tuwien.ac.at"><wien@zeus.theochem.tuwien.ac.at></a>;<wbr></div>
          <div><b>主题:</b> Re: [Wien] Wien Installation</div>
        </div>
        <div><br>
        </div>
        <span lang="EN-US"><o:p></o:p></span>
        <blockquote type="cite" cite="mid:tencent_09FE5216BCB4928EAF6660D180FE4C337909@qq.com">
          <div><span lang="EN-US">In terms of OpenBLAS-0.3.9 libraries,
              in my OpenBLAS-0.3.9/lib, there are several files as the
              below: libopenblas_nehalemp-r0.3.9.a,
              libopenblas_nehalemp-r0.3.9.so, libopenblas.so,
              libopenblas.so.0, libopenblas.a. Are they as same as
              yours?</span></div>
        </blockquote>
        <font color="#0000ff">I have the following, but you have the
          same one named "libopenblas.so" that I use.<br>
        </font><br>
        <font color="#0000ff">username@computername:~/WIEN2k$ ls
          ~/OpenBLAS-0.3.9/*.a ~/OpenBLAS-0.3.9/*.so<br>
          /home/username/OpenBLAS-0.3.9/libopenblas.a<br>
          /home/username/OpenBLAS-0.3.9/libopenblas_barcelonap-r0.3.9.a<br>
          /home/username/OpenBLAS-0.3.9/libopenblas_barcelonap-r0.3.9.so<br>
          /home/username/OpenBLAS-0.3.9/libopenblas.so</font><br>
        <blockquote type="cite"><span lang="EN-US">In addition, are
            there commands such as lapw0_mpi, lapw1_mpi and lapw1c_mpi
            generated by installing with gfortran+OpenBLAS, just like by
            installing with ifort?</span></blockquote>
        <font color="#0000ff">No, the Intel Parallel Studio XE 2020
          Cluster Edition has the Intel MPI Library [1] that provides
          intelmpi making it easy to build the WIEN2k *_mpi files.  For
          gfortran, it can be done but it is extra work as one has to
          install scalapack [2] and MPI (such as OpenMPI [3]) separately
          and then include them in the parallel compiler settings of
          WIEN2k in order to build the files.<br>
        </font><br>
        <font color="#0000ff">[1] 
          <a class="moz-txt-link-freetext" href="https://software.intel.com/content/www/us/en/develop/tools/parallel-studio-xe.html" moz-do-not-send="true">https://software.intel.com/content/www/us/en/develop/tools/parallel-studio-xe.html</a></font><br>
        <font color="#0000ff">[2] <a class="moz-txt-link-freetext" href="http://www.netlib.org/scalapack/" moz-do-not-send="true">http://www.netlib.org/scalapack/</a></font><br>
        <font color="#0000ff">[3] <a class="moz-txt-link-freetext" href="https://www.open-mpi.org/software/ompi/v4.0/" moz-do-not-send="true">https://www.open-mpi.org/software/ompi/v4.0/</a></font><br>
        <blockquote type="cite" cite="mid:tencent_09FE5216BCB4928EAF6660D180FE4C337909@qq.com">
          <div><span lang="EN-US"><o:p></o:p></span><br>
            <span lang="EN-US">Then during ./siteconfig_lapw, I choose O
              to set libraries “R_LIBS(LAPACK+BLAS)”, then how to set
              the “Real libraries”? Would you like to tell me how set
              the “Real libraries” parameter so that could obtain the
              result just like your path for R_LIBS as
              “-L/home/username/OpenBLAS-0.3.9 -lopenblas -llapack_lapw
              –lpthread”?<o:p></o:p></span><br>
            <br>
            <span lang="EN-US">In my setting, I did a couple of tries
              and obtain three kinds of results as below:<o:p></o:p></span><br>
            <br>
            <span lang="EN-US">1.<span style="font-variant-numeric:
                normal; font-variant-east-asian: normal; font-stretch:
                normal; font-size: 7pt; line-height: normal;">       </span></span><span lang="EN-US">When I set the R_LIBS such as
              /home/username/OpenBLAS-0.3.9/lib/libopenblas.so (or
              libopenblas.so.0, or libopenblas_nehalemp-r0.3.9.so), then
              the massage is: /usr/bin/ld: warning: libgfortran.so.3,
              needed by /home-yw/Soft/OpenBLAS-0.3.9/lib/libopenblas.so,
              may conflict with libgfortran.so.5.<o:p></o:p></span><br>
            <br>
            <span lang="EN-US">2.<span style="font-variant-numeric:
                normal; font-variant-east-asian: normal; font-stretch:
                normal; font-size: 7pt; line-height: normal;">       </span></span><span lang="EN-US">When I set the R_LIBS such as
              /home/username/OpenBLAS-0.3.9/lib or
              /home/username/OpenBLAS-0.3.9, then the massage is: file
              not recognized: Is a directory.<o:p></o:p></span><br>
            <br>
            <span lang="EN-US">3.<span style="font-variant-numeric:
                normal; font-variant-east-asian: normal; font-stretch:
                normal; font-size: 7pt; line-height: normal;">       </span></span><span lang="EN-US">When I set he R_LIBS such as
               /home/username/OpenBLAS-0.3.9/lib/
              libopenblas_nehalemp-r0.3.9.a (or libopenblas.a), then the
              error massage is much, including:
/home-yw/Soft/OpenBLAS-0.3.9/lib/libopenblas_nehalemp-r0.3.9.a(blas_server.o):
              In function `goto_set_num_threads':
              blas_server.c:(.text+0x7f3): undefined reference to
              `pthread_create'.<o:p></o:p></span><br>
            <br>
            <span lang="EN-US">All above relating files were attached.<o:p></o:p></span><br>
          </div>
        </blockquote>
        <font color="#0000ff">Attached is copy of my compile.msg from
          SRC_aim that you can compare to yours.</font><br>
        <font color="#0000ff">Below is what I used for a basic WIEN2k
          19.2 install that should work for serial and k-point parallel
          calculations (but won't work for mpi calculations).  Of note,
          it looks like you will need to replace
          "/home/username/OpenBLAS-0.3.9" with
          "/home-yw/Soft/OpenBLAS-0.3.9/lib" on your system.  Also, you
          might notice below that the "-llapack_lapw" of WIEN2k is not
          needed since -lopenblas contains it own lapack library.<br>
        </font><br>
        <font color="#0000ff"><br>
          username@computername:~/WIEN2k$ ls WIEN2k_INSTALLDATE <br>
          WIEN2k_INSTALLDATE<br>
          username@computername:~/WIEN2k$ rm WIEN2k_INSTALLDATE<br>
          username@computername:~/WIEN2k$ ./siteconfig<br>
          ...<br>
          continue or stop (c/s) c<br>
          ...<br>
            Selection: LG<br>
          ...<br>
               Press RETURN to continue<br>
          ...<br>
               Your compiler: gfortran<br>
          ...<br>
               Your compiler: gcc<br>
          ...<br>
           Would you like to use LIBXC (that you have installed -
          usually not needed)? (y,N): <br>
          N<br>
          ...<br>
           Do you want to use FFTW (recommended, but for sequential code
          not required)? (Y,n): n<br>
          ...<br>
          Selection: R                                                  
          <br>
               Real libraries=-L/home/username/OpenBLAS-0.3.9 -lopenblas
          -lpthread<br>
          ...<br>
           Current settings:<br>
            M   OpenMP switch:           -fopenmp<br>
            O   Compiler options:        -ffree-form -O2
          -ftree-vectorize -march=native -ffree-line-length-none
          -ffpe-summary=none<br>
            L   Linker Flags:            $(FOPT) -L../SRC_lib<br>
            P   Preprocessor flags       '-DParallel'<br>
            R   R_LIBS (LAPACK+BLAS):    -L/home/username/OpenBLAS-0.3.9
          -lopenblas -lpthread<br>
            F   FFTW options:<br>
                FFTW-LIBS:<br>
            X   LIBX options:<br>
                LIBXC-LIBS:<br>
          <br>
            S   Save and Quit<br>
          <br>
                To change an item select option.<br>
          Selection: S<br>
          ...<br>
            Shared Memory Architecture? (y/N):y<br>
            Do you know/need a command to bind your jobs to specific
          nodes?<br>
            (like taskset -c). Enter N / your_specific_command: N<br>
          ...<br>
             (y/N) N<br>
          ...<br>
               Selection: Q<br>
          ...<br>
               Selection: A<br>
          ...</font><br>
        <font color="#0000ff">Compile time errors (if any) were:<br>
          <br>
          <br>
          Check file   compile.msg   in the corresponding SRC_*
          directory for the <br>
          compilation log and more info on any compilation problem.<br>
          ...<br>
             Please enter the full path of the perl program:
          /usr/bin/perl<br>
          ...<br>
           Please enter the full path to your temporary directory: /tmp<br>
          ...<br>
          username@computername:~/WIEN2k$ userconfig<br>
          ...<br>
          username@computername:~/WIEN2k$ gedit ~/.bashrc<br>
          username@computername:~/WIEN2k$ grep LD_LIBRARY ~/.bashrc<br>
          export
          LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/username/OpenBLAS-0.3.9</font><font color="#0000ff"><br>
        </font></div>
    </blockquote>
    <blockquote type="cite" cite="mid:tencent_1B4DCBFBB87A06082DEE8F2193EA6F626105@qq.com">
    </blockquote></div>