[Wien] parallel compilation
Peter Blaha
pblaha at theochem.tuwien.ac.at
Tue Sep 15 10:33:09 CEST 2009
> if you do not select fine grained parallel, siteconfig_lapw does not
> give you options to tune parallel parameters. does that mean that it is
> not compiling parallel at all? if i do not do fine tuning, i can get a
> complete compilation.
If one does NOT select "fine grain parallel"; siteconfig will not produce any
mpi-executables. This is clearly the proper choice
i) for beginners
ii) on "small" computers, where you may not have installed mpi + fftw ;
or have only a slow network between them.
iii) when you intend to limit your unitcells to about 50 atoms/cell.
(Still k-point parallelization is possible, and this is much more efficient
than mpi for small systems).
If you select fine grain parallelization, also the mpi-executables will be
produced. But you need:
i) a properly installed mpi (+ mpi-fortran compiler, which is usually "created"
when mpi is installed. (Usually MPF is NOT ifort, but mpif90 !!)
Note: mpi2 needs some small code changes in lapw0.F
ii) a corresponding Scalapack+Blacs (also suitable for your compiler)
(usually it comes together with the mkl)
iii) FFTW routines (compiled with the mpi option.
iv) you need to properly specify the "installation place" (-Lxxxx) and the
name of the libraries (-lxxxx). Also make sure you understand 32-bit vs
64bit ("emt") issues.
In addition, this code is useful ONLY if you have unitcells with more than
about 50 atoms !!!! and you have hardware with fast network (Infiniband),
i.e. Gbit ethernet may work, but will be slow.
On a SINGLE Quadcore-computer, mpi-parallelization is probably NOT worth the effort
(use OMP_NUM_THREAD 2 (or 4))
linuxif9:MPF:ifort ???? unlikely! (see above)
> # Linux PC system with IFC 10 compiler + mkl 10 (-ip is broken; -static
> does not give traceback-lines)
> linuxif9:FC:ifortlinuxif9:MPF:ifort
> linuxif9:CC:cclinuxif9:FOPT:-FR -mp1 -w -prec_div -pc80 -pad -align
> -DINTEL_VML -traceback -O
> 3 -xW
> linuxif9:FPOPT:$(FOPT) -FR -mp1 -w -prec_div -pc80 -pad -align
> -DINTEL_VML -traceback -I/opt/mpich2/includelinuxif9:LDFLAGS: $(FOPT)
> -L/opt/intel/mkl/10.2.1.017/lib/em64t <http://10.2.1.017/lib/em64t>
> -pthread -i-static
> linuxif9:R_LIBS:-L/opt/intel/mkl/10.2.1.017/lib/em64t
> <http://10.2.1.017/lib/em64t> -lmkl_lapack -lmkl -liomp5
> -lguide -lmkl_core
> linuxif9:DPARALLEL:'-DParallel'
> linuxif9:RP_LIBS: -L/opt/intel/mkl/10.2.1.017/lib/em64t
> <http://10.2.1.017/lib/em64t> -lmkl_lapack -lmkl_intel
> _lp64 -lmkl_scalapack_lp64 -lmkl_blacs_lp64 -lmkl_sequential
> -lmkl_intel_ilp64 -
> lmkl_scalapack_ilp64 -L/opt/fftw-2.1.5/lib/ -lfftw_mpi -lfftw
> -L/opt/mpich2/lib
> -lmpich -lfmpich
> linuxif9:MPIRUN:mpiexec _EXEC_
>
> i am running on a xeon, CentOS5 machine. intel noncommercial.
>
> Any information would be appreciated.
>
> Thank you,
> JD
>
> On Mon, Sep 14, 2009 at 10:31 AM, Jeff DeReus <jdereus at gmail.com
> <mailto:jdereus at gmail.com>> wrote:
>
> fftw was compiled into /opt/fftw-2.1.5 with flags
>
> $ ./configure --prefix=/opt/fftw-2.1.5/ --enable-mpi --enable-threads
>
> Thank you,
> JD
>
>
> On Mon, Sep 14, 2009 at 10:27 AM, Laurence Marks
> <L-marks at northwestern.edu <mailto:L-marks at northwestern.edu>> wrote:
>
> It looks like you did not compile fftw (or it is somewhere else).
>
> 2009/9/14 Jeff DeReus <jdereus at gmail.com
> <mailto:jdereus at gmail.com>>:
> > Hello again. i am having some issues compiling lapw0/1/2
> modules in
> > wein2k_09. i am running on a CentOS 5.3 box. intel
> non-commercial
> > compilers and mkl.
> >
> >
> >
> > here are my current parallel settings from siteconfig.
> >
> > Current settings:
> > RP RP_LIB(SCALAPACK+PBLAS):
> -L/opt/intel/mkl/10.2.1.017/lib/em64t <http://10.2.1.017/lib/em64t>
> > -lmkl_lapack -lmkl_intel_lp64 -lmkl_scalapack_lp64
> -lmkl_blacs_lp64
> > -lmkl_sequential -L/opt/fftw-2.1.5/lib/ -lfftw_mpi -lfftw
> -L/opt/mpich2/lib
> > -lmpich
> > FP FPOPT(par.comp.options): $(FOPT) -FR -mp1 -w
> -prec_div -pc80 -pad
> > -align -DINTEL_VML -traceback -I/opt/mpich2/include
> > -I/opt/fftw-2.1.5/fortran
> > MP MPIRUN commando : mpiexec _EXEC_
> >
> > all other modules compile correctly. if i do not enable parallel
> > functionality, the compilation completes with no errors.
> >
> > when compiling /opt/wien2k/SRC_lapw0 it ends with these
> errors which i have
> > not been able to track down.
> >
> > fftw_para.o: In function `exec_fftw_para_':
> > fftw_para.F:(.text+0x77): undefined reference to
> `fftwnd_f77_mpi_'
> > fftw_para.F:(.text+0xb2): undefined reference to
> `fftwnd_f77_mpi_'
> > fftw_para.o: In function `init_fftw_para_':
> > fftw_para.F:(.text+0x101): undefined reference to
> > `fftw3d_f77_mpi_create_plan_'
> > fftw_para.F:(.text+0x129): undefined reference to
> > `fftw3d_f77_mpi_create_plan_'
> > fftw_para.F:(.text+0x14d): undefined reference to
> > `fftwnd_f77_mpi_local_sizes_'
> > make[1]: *** [lapw0_mpi] Error 1
> > make[1]: Leaving directory `/opt/wien2k/SRC_lapw0'
> > make: *** [para] Error 2
> >
> > however, /opt/wien2k/SRC_lapw1/2 end with these errors.
> >
> > <snip>
> > /opt/intel/mkl/10.2.1.017/lib/em64t/libmkl_intel_lp64.so
> <http://10.2.1.017/lib/em64t/libmkl_intel_lp64.so>: undefined
> > reference to `mkl_pdett__d_forward_trig_transform'
> > /opt/intel/mkl/10.2.1.017/lib/em64t/libmkl_intel_lp64.so
> <http://10.2.1.017/lib/em64t/libmkl_intel_lp64.so>: undefined
> > reference to `mkl_trans_mkl_comatcopy'
> > /opt/intel/mkl/10.2.1.017/lib/em64t/libmkl_intel_lp64.so
> <http://10.2.1.017/lib/em64t/libmkl_intel_lp64.so>: undefined
> > reference to `mkl_pdett_s_init_trig_transform'
> > /opt/intel/mkl/10.2.1.017/lib/em64t/libmkl_intel_lp64.so
> <http://10.2.1.017/lib/em64t/libmkl_intel_lp64.so>: undefined
> > reference to `mkl_pdepl_s_commit_helmholtz_3d'
> > /opt/intel/mkl/10.2.1.017/lib/em64t/libmkl_intel_lp64.so
> <http://10.2.1.017/lib/em64t/libmkl_intel_lp64.so>: undefined
> > reference to `mkl_pdepl_d_commit_helmholtz_2d'
> > /opt/intel/mkl/10.2.1.017/lib/em64t/libmkl_intel_lp64.so
> <http://10.2.1.017/lib/em64t/libmkl_intel_lp64.so>: undefined
> > reference to `mkl_pdett__d_commit_trig_transform'
> > /opt/intel/mkl/10.2.1.017/lib/em64t/libmkl_intel_lp64.so
> <http://10.2.1.017/lib/em64t/libmkl_intel_lp64.so>: undefined
> > reference to `mkl_trans_mkl_comatadd'
> > /opt/intel/mkl/10.2.1.017/lib/em64t/libmkl_intel_lp64.so
> <http://10.2.1.017/lib/em64t/libmkl_intel_lp64.so>: undefined
> > reference to `mkl_serv_getversionstring_c'
> > /opt/intel/mkl/10.2.1.017/lib/em64t/libmkl_intel_lp64.so
> <http://10.2.1.017/lib/em64t/libmkl_intel_lp64.so>: undefined
> > reference to `mkl_spblas_lp64_mkl_dcsrsky'
> > /opt/intel/mkl/10.2.1.017/lib/em64t/libmkl_intel_lp64.so
> <http://10.2.1.017/lib/em64t/libmkl_intel_lp64.so>: undefined
> > reference to `mkl_pdett__d_init_trig_transform'
> > make[1]: *** [lapw2c_mpi] Error 1
> > make[1]: Leaving directory `/opt/wien2k/SRC_lapw2'
> > make: *** [cp] Error 2
> >
> >
> > i can find no mention of the fftw3d errors anywhere.
> >
> > my complete appropriate section of siteconfig
> >
> > # Linux PC system with IFC 10 compiler + mkl 10 (-ip is
> broken; -static does
> > not give traceback-lines)
> > linuxif9:FC:ifort
> > linuxif9:MPF:ifort
> > linuxif9:CC:cc
> > linuxif9:FOPT:-FR -mp1 -w -prec_div -pc80 -pad -align
> -DINTEL_VML
> > -traceback -O3 -xW
> > linuxif9:FPOPT:$(FOPT) -FR -mp1 -w -prec_div -pc80 -pad
> -align -DINTEL_VML
> > -traceback -I/opt/mpich2/include
> > linuxif9:LDFLAGS: $(FOPT)
> -L/opt/intel/mkl/10.2.1.017/lib/em64t
> <http://10.2.1.017/lib/em64t> -pthread
> > -i-static
> > linuxif9:R_LIBS:-L/opt/intel/mkl/10.2.1.017/lib/em64t
> <http://10.2.1.017/lib/em64t> -lmkl_lapack -lmkl
> > -liomp5 -lguide -lmkl_core
> > linuxif9:DPARALLEL:'-DParallel'
> > linuxif9:RP_LIBS: -L/opt/intel/mkl/10.2.1.017/lib/em64t
> <http://10.2.1.017/lib/em64t> -lmkl_lapack
> > -lmkl_intel_lp64 -lmkl_scalapack_lp64 -lmkl_blacs_lp64
> -lmkl_sequential
> > -L/opt/fftw-2.1.5/lib/ -lfftw_mpi -lfftw
> > linuxif9:MPIRUN:mpiexec _EXEC_
> >
> >
> > any assistance in this matter would be appreciated.
> >
> > Thank you,
> > JD
> >
> > _______________________________________________
> > Wien mailing list
> > Wien at zeus.theochem.tuwien.ac.at
> <mailto:Wien at zeus.theochem.tuwien.ac.at>
> > http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien
> >
> >
>
>
>
> --
> Laurence Marks
> Department of Materials Science and Engineering
> MSE Rm 2036 Cook Hall
> 2220 N Campus Drive
> Northwestern University
> Evanston, IL 60208, USA
> Tel: (847) 491-3996 Fax: (847) 491-7820
> email: L-marks at northwestern dot edu
> Web: www.numis.northwestern.edu <http://www.numis.northwestern.edu>
> Chair, Commission on Electron Crystallography of IUCR
> www.numis.northwestern.edu/ <http://www.numis.northwestern.edu/>
> Electron crystallography is the branch of science that uses electron
> scattering and imaging to study the structure of matter.
> _______________________________________________
> Wien mailing list
> Wien at zeus.theochem.tuwien.ac.at
> <mailto:Wien at zeus.theochem.tuwien.ac.at>
> http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien
>
>
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> Wien mailing list
> Wien at zeus.theochem.tuwien.ac.at
> http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien
--
P.Blaha
--------------------------------------------------------------------------
Peter BLAHA, Inst.f. Materials Chemistry, TU Vienna, A-1060 Vienna
Phone: +43-1-58801-15671 FAX: +43-1-58801-15698
Email: blaha at theochem.tuwien.ac.at WWW: http://info.tuwien.ac.at/theochem/
--------------------------------------------------------------------------
More information about the Wien
mailing list