[Wien] Installation with MPI and GNU compilers

[Gavin Abo]

Some comments:

I haven't seen many mailing list posts about using a gfortran-based 
mpi.  That is probably because the clusters used for mpi are likely 
systems that cost something like $100k to $1 millon.  Those systems 
usually seem to be running Intel MPI.  So companies, computing centers, 
and universities for example likely have no problem paying say $1,499 
for Cluster Edition for C/C++ and Fortran [ 
https://software.intel.com/en-us/articles/academic-pricing ].

How much I get paid to help you: $0

How much the IT Support Technician at your organization likely gets paid 
to help you: average pay of about $52k per year according to glassdoor [ 
https://www.glassdoor.com/Salaries/it-support-technician-salary-SRCH_KO0,21.htm 
]

Meaning, please try asking your clusters IT department or helpdesk first 
for help with this if you have one.

I assume your using at least a GB-network cluster [ 
https://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/msg13632.html 
, 
https://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/msg09334.html 
] as mpi is usually useless with a single multi-core personal computer [ 
https://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/msg05470.html 
, 
https://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/msg07371.html 
].

If you plan to use Bandstructure in w2web of WIEN2k 17.1, a "At line 75 
of file modules_tmp_.F (unit = 5, file = 'ubuntu.in1c')" error can occur 
with gfortran when the "x lapw1 -band" button is clicked, so you may 
want to apply band.pl and scf.pl [ 
https://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/msg16069.html 
] or band.patch and scf.patch [ 
https://github.com/gsabo/WIEN2k-Patches/tree/master/17.1 ]. There are 
some other patches that might also be helpful for gfortran [ 
https://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/msg17175.html 
].  However, I currently don't have files (SRC_symmetry and x_lapw) to 
fix the gfortran error with "x dstart" if you use the init_lapw command, 
but you could probably get them from Prof. Blaha or make the changes 
yourself following the instructions given before [ 
https://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/msg16674.html 
].

Yes, it usually better to have only one MPI implementation and one BLAS 
library.  However, it should not be a problem as long as you can keep 
them from mixing and conflicting with each other.  As I recall, all mpi 
is somewhat equal [ 
https://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/msg09557.html 
], though openmpi might be easy to compile [ 
https://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/msg07343.html 
].

The libfftw3 and libfftw3_mpi of FFTW 3.x.x [ 
http://www.fftw.org/download.html ] are what you need to use.  The 
liblfftw3xf and libfftw3xf_gnu (or libfftw3x_cdft.a) files if I recall 
correctly would be located or generated in the interfaces directory of 
an Intel ifort/mkl installation [ 
https://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/msg07333.html 
].  The mkl interface to fftw3xf I think still does not work with mpi 
(only for serial compilation) [ 
https://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/msg06959.html 
].

Sorry, I currently don't have answers to your other questions, but I 
think you are headed in the right direction.

On 4/3/2018 2:02 PM, Rui Costa wrote:
> Dear wien2k users,
>
> I'm trying to install wien2k_17.1 with mpi, fftw and elpa using the 
> GNU compilers. I must say that I'm not an expert in linking and 
> compiling the packages, so probably some things will be wrong.
>
> I'm using Ubuntu 16.04 LTS and have installed:
> - BLAS: OpenBlas-0.2.20 which is in /opt/OpenBLAS and the libblas3 
> (shared version) and libblas-dev (static version) of netlib using 
> synaptic package manager which is in /usr/lib;
> - LAPACK: liblapack3 (shared version) and libpalack-dev (static 
> version) of netlib using synaptic package manager which is in /usr/lib;
> - BLACS: libblacs-mpi-dev and libblacs-openmpi1 of netlib using 
> synaptic package manager which is in /usr/lib
> - ScaLAPACK: libscalapack-mpi-dev where I don't find the shared or 
> static libraries, i.e., the libscalapack-mpi-dev.a or 
> libscalapack-mpi-dev.so files, and libscalapack-openmpi1 
> using synaptic package manager which is in /usr/lib;
> - MPI: mpich-3.2.1 and it is installed in /usr/local/bin, 
> /usr/local/lib and /usr/local/include and since I installed the 
> openmpi version of BLACS and ScaLAPACK, I also have Openmpi and the 
> binary files are at /usr/bin;
> - FFTW: fftw-3.3.7 and I installed this one from source files with the 
> option ./configure --enable-mpi, and installed it in /usr/local/lib/ 
> and /usr/lib/x86_64-linux-gnu;
> - ELPA: libelpa-dev and libelpa3 and installed these using the 
> synaptic manager again.
>
>
> Questions:
> 1) I'm concerned that having two MPI implementations and two BLAS 
> libraries might cause things to be compiled incorrectly. My idea was 
> to install wien2k with MPICH since it seems to be the recommended one 
> because there is almost no reference to OpenMPI, but on the other 
> hand, the ScaLAPACK and BLACS libraries that I have use OpenMPI and I 
> have tried installing these packages from source with MPICH but don't 
> know how. Do you have any recommendation on which libraries and MPI to 
> use?
>
> 2) In section 11.1.1 of the userguide it says:
>
> "in case you do not have icc installed, but use GNU-C (gcc) you must:
> – edit makefile, and remove -D_GNU from the line “ CC=gcc -D GNU” (to 
> remove additional from the object names)
> – make libintel64 compiler=gnu"
>
> which makefile is this? None of the Makefiles in SRC_lapw0, 1 and 2 
> have that -D_GNU flag. It also says to add, in my case,  -DFFTW3 to 
> FOPT and -lfftw3xf and -lfftw3xf_gnu to R_LIBS in the Makefiles of 
> lapw0 and lapw2, but I don't have any library named liblfftw3xf or 
> libfftw3xf_gnu, are these supposed to be libfftw3 or libfftw3_mpi that 
> I do have?
>
> 3) In the configuration of the parallel options I gave the locations 
> of the directories, I had to edit the variables ELPA_OPT and ELPA_LIBS 
> in the Makefile of SRC_lapw1 because they were not pointing to the 
> correct directories, but in the RP_LIBS it warns that since I'm using 
> gfortran, I might need additional libraries (-lredist -ltools -lfblacs 
> -lblacs -lmpi), but I don't have the first three libraries and have 
> tried searching them but I can't find them. What libraries are these? 
> Do they have different names?
>
>
> I'm sorry for the long post.
>
> Best regards,
> Rui Costa.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://zeus.theochem.tuwien.ac.at/pipermail/wien/attachments/20180403/ed4819c9/attachment.html>