[Wien] Compiling Wien2k 21.1 on Ubuntu 20.04 with gfortran
David Holec
david.holec at unileoben.ac.at
Wed Nov 24 11:09:56 CET 2021
Hi Pavel,
Many thanks for your insights. As you know, I am not an expert on how to
compile codes, for me, this is sadly a trial and error adventure.
I tried to compile it against the openblas library, but although the
compilation ends without any errors, I get a segmentation fault when
running lapw1 (on the test case from
http://www.wien2k.at/reg_user/benchmark/). The current setting are:
L Linker Flags: $(FOPT)
-L/usr/lib/x86_64-linux-gnu/openblas64-openmp
R R_LIBS (LAPACK+BLAS):
/usr/lib/x86_64-linux-gnu/openblas64-openmp/libopenblas64.so.0 -lpthread
(The rest is as I wrote in my first email.) Here is the list of linked
libraries:
$ ldd lapw1
linux-vdso.so.1 (0x00007ffea57d6000)
*libopenblas64.so.0 => /lib/x86_64-linux-gnu/libopenblas64.so.0
(0x000014fe2b2e5000) *
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0
(0x000014fe2b2c2000)
libgfortran.so.5 => /lib/x86_64-linux-gnu/libgfortran.so.5
(0x000014fe2affa000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x000014fe2aeab000)
libmvec.so.1 => /lib/x86_64-linux-gnu/libmvec.so.1
(0x000014fe2ae7f000)
libgomp.so.1 => /lib/x86_64-linux-gnu/libgomp.so.1
(0x000014fe2ae3d000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x000014fe2ac49000)
/lib64/ld-linux-x86-64.so.2 (0x000014fe2d4d3000)
libquadmath.so.0 => /lib/x86_64-linux-gnu/libquadmath.so.0
(0x000014fe2abff000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1
(0x000014fe2abe4000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x000014fe2abde000)
And here is the stacking fault (it doesn't tell me anything):
$x lapw1
Program received signal SIGSEGV: Segmentation fault - invalid memory
reference.
Backtrace for this error:
#0 0x148fb48b8d21 in ???
#1 0x148fb48b7ef5 in ???
#2 0x148fb452d20f in ???
#3 0x148fb54495fb in ???
Segmentation fault
0.949u 0.472s 0:00.84 167.8% 0+0k 14400+10992io 31pf+0w
Any idea?
With the settings I reported yesterday, everything works just fine (though
probably not very efficiently - but this is not a problem for me as this
binary is not a "production" binary on any HPC):
$ ldd lapw1
linux-vdso.so.1 (0x00007ffc60765000)
*libblas.so.3 => /lib/x86_64-linux-gnu/libblas.so.3 (0x0000150cec90f000)
liblapack.so.3 => /lib/x86_64-linux-gnu/liblapack.so.3
(0x0000150cec26b000) *
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0
(0x0000150cec248000)
libgfortran.so.5 => /lib/x86_64-linux-gnu/libgfortran.so.5
(0x0000150cebf80000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x0000150cebe31000)
libmvec.so.1 => /lib/x86_64-linux-gnu/libmvec.so.1
(0x0000150cebe05000)
libgomp.so.1 => /lib/x86_64-linux-gnu/libgomp.so.1
(0x0000150cebdc1000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x0000150cebbcf000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1
(0x0000150cebbb4000)
/lib64/ld-linux-x86-64.so.2 (0x0000150cec9fc000)
libquadmath.so.0 => /lib/x86_64-linux-gnu/libquadmath.so.0
(0x0000150cebb6a000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x0000150cebb64000)
and:
$ x lapw1
STOP LAPW1 END
162.045u 0.918s 2:31.25 107.7% 0+0k 8976+37856io 35pf+0w
$ grep HORB *output1*
test_case.output1: TIME HAMILT (CPU) = 16.3, HNS = 20.0, HORB
= 0.0, DIAG = 125.9, SYNC = 0.0
test_case.output1: TIME HAMILT (WALL) = 4.4, HNS = 20.0, HORB
= 0.0, DIAG = 126.0, SYNC = 0.0
(I am using it on my 10-year old "type writer":
$lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
Vendor ID: GenuineIntel
CPU family: 6
Model: 26
Model name: Intel(R) Xeon(R) CPU W3550 @
3.07GHz
)
---
*Dr David Holec*
*Computational Materials Science group*
Department of Materials Science
Montanuniversität Leoben
Franz-Josef-Strasse 18, A-8700 Leoben, Austria
tel. +43-(0)3842-4024211
fax. +43-(0)3842-4024202
materials.unileoben.ac.at
cms.unileoben.ac.at
________________________________
WHERE RESEARCH MEETS FUTURE
On Wed, 24 Nov 2021 at 08:27, Pavel Ondračka <pavel.ondracka at email.cz>
wrote:
> Hi David,
>
> as you said it works for you, so feel free to ignore, but I have some
> further tips if you are interested. Ubuntu switches between the
> different blas and lapack using the "alternatives", so its difficult to
> say if you actually link with the correct one.
>
> "ldd lapw1" in WIENROOT should show which one is actually linked, what
> you want to have is the openmp openblas
> /usr/lib/x86_64-linux-gnu/openblas-openmp/libblas.so
> /usr/lib/x86_64-linux-gnu/openblas-openmp/liblapack.so
> or alternatively
> /usr/lib/x86_64-linux-gnu/openblas-openmp/libopenblas.so
> It looks like you linked with the pthread one. This is not a problem
> when running at single thread but at higher thread number this might
> lead to oversubscription and slowdowns as the pthreaded openblas
> doesn't respect the OMP_NUM_THREADS set by Wien2k. So I would recommend
> to relink with the openmp OpenBLAS. BTW it is usually safer to link
> with OpenBLAS explicitly using the -lopenblas instead of the -llapack -
> lblas to be sure you don't accidentally link the netlib one
> (libopenblas is just the libblas and libblapack provided by OpenBLAS
> merged together).
>
> In general easy way how to check performance is to run the serial
> test_case from http://www.wien2k.at/reg_user/benchmark/ On modern CPUs
> (at least avx2) the runtime should be around 15-25 seconds at single
> thread.
>
> I see total runtime of ~18seconds on Fedora 35 with gfortran 11.2.1
> OpenBLAS and AMD Ryzen 9 3900X 12-Core Processor.
> Also look for the following line in test_case.output1, this is what I
> have:
> TIME HAMILT (WALL) = 2.2, HNS = 1.7, HORB = 0.0, DIAG =
> 14.0, SYNC = 0.0
> The time in HAMILT mostly depends on you compiler and vectorizing
> settings, while the DIAG is 99% lapack/blas related, so this can help
> with the diagnostics if things are slow.
>
> You might also get extra speedup of the HAMILT part by adding "-
> DHAVE_LIBMVEC" to the Compiler options.
>
> Best regards
> Pavel
>
> On Tue, 2021-11-23 at 11:07 +0100, David Holec wrote:
> > Dear all,
> >
> > I have just spent some time making Wien2k run on my single machine
> > running Ubuntu 20.04 with gfortran/gcc. Since I am not an expert, it
> > was a trial and error, but it seems that I found a working combination
> > (sadly, the default parameters didn't work for me). Maybe this will
> > help someone. Here are the settings that did the job for me:
> >
> > M OpenMP switch: -fopenmp
> > O Compiler options: -ffree-form -O2 -ftree-vectorize -
> > march=native -ffree-line-length-none -ffpe-summary=none
> > L Linker Flags: $(FOPT) -L/usr/lib/x86_64-linux-gnu
> > P Preprocessor flags '-DParallel'
> > R R_LIBS (LAPACK+BLAS): -lblas -llapack -lpthread
> > F FFTW options: -DFFTW3 -DFFTW_OMP -I/usr/include
> > FFTW-LIBS: -L/usr/lib/x86_64-linux-gnu -lfftw3 -
> > lfftw3_omp
> >
> > where the FFTW options were:
> >
> > R FFTWROOT: /usr/
> > V FFTW_VERSION: FFTW3
> > L FFTW_LIB: lib/x86_64-linux-gnu
> > N FFTW_LIBNAME: fftw3
> >
> > Compiler versions:
> > $ gcc --version
> > gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0
> > gfortran --version
> > GNU Fortran (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0
> >
> > And I used the generic lapack and openblas packages provides by Ubuntu
> > repos:
> > liblapack-dev/focal,now 3.9.0-1build1 amd64 [installed]
> > liblapack3/focal,now 3.9.0-1build1 amd64 [installed,automatic]
> >
> > liblapack64-3/focal,now 3.9.0-1build1 amd64 [installed,automatic]
> > liblapack64-dev/focal,now 3.9.0-1build1 amd64 [installed]
> >
> > libblas-dev/focal,now 3.9.0-1build1 amd64 [installed]
> > libblas3/focal,now 3.9.0-1build1 amd64 [installed,automatic]
> > libblas64-3/focal,now 3.9.0-1build1 amd64 [installed,automatic]
> > libblas64-dev/focal,now 3.9.0-1build1 amd64 [installed,automatic]
> >
> > libopenblas64-0/focal-updates,now 0.3.8+ds-1ubuntu0.20.04.1 amd64
> > [installed]
> > libopenblas64-0-openmp/focal-updates,now 0.3.8+ds-1ubuntu0.20.04.1
> > amd64 [installed]
> > libopenblas64-0-pthread/focal-updates,now 0.3.8+ds-1ubuntu0.20.04.1
> > amd64 [installed,automatic]
> >
> > (I am not totally sure if I need all the libraries above, but
> > certainly, with these, the compilation seems to work and I am able to
> > run SCF cycles & Telnes calculations without errors :-)
> >
> > All the best,
> > David
> > ---
> > Dr David Holec
> > Computational Materials Science group
> > Department of Materials Science
> > Montanuniversität Leoben
> >
> >
> >
> > Franz-Josef-Strasse 18, A-8700 Leoben, Austria
> > tel. +43-(0)3842-4024211
> > fax. +43-(0)3842-4024202
> > materials.unileoben.ac.at
> > cms.unileoben.ac.at
> > ________________________________
> > WHERE RESEARCH MEETS FUTURE
> > _______________________________________________
> > Wien mailing list
> > Wien at zeus.theochem.tuwien.ac.at
> > http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien
> > SEARCH the MAILING-LIST at:
> > http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html
>
> _______________________________________________
> Wien mailing list
> Wien at zeus.theochem.tuwien.ac.at
> http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien
> SEARCH the MAILING-LIST at:
> http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://zeus.theochem.tuwien.ac.at/pipermail/wien/attachments/20211124/c65aeae1/attachment.htm>
More information about the Wien
mailing list