[Wien] parallel compilation
Jonas Baltrusaitis
jasius_1 at yahoo.com
Tue Sep 15 14:51:11 CEST 2009
Peter,
are you implying that it is possible to run one job on 2-4 cores in 'parallel' without any mpi and granularization? Could you elaborate on that?
Jonas
--- On Tue, 9/15/09, Peter Blaha <pblaha at theochem.tuwien.ac.at> wrote:
> From: Peter Blaha <pblaha at theochem.tuwien.ac.at>
> Subject: Re: [Wien] parallel compilation
> To: "A Mailing list for WIEN2k users" <wien at zeus.theochem.tuwien.ac.at>
> Date: Tuesday, September 15, 2009, 1:33 AM
> > if you do not select fine
> grained parallel, siteconfig_lapw does not give you options
> to tune parallel parameters. does that mean that it is
> not compiling parallel at all? if i do not do fine
> tuning, i can get a complete compilation.
>
> If one does NOT select "fine grain parallel"; siteconfig
> will not produce any
> mpi-executables. This is clearly the proper choice
>
> i) for beginners
> ii) on "small" computers, where you may not have
> installed mpi + fftw ;
> or have only a slow network between them.
> iii) when you intend to limit your unitcells to about 50
> atoms/cell.
>
> (Still k-point parallelization is possible, and this is
> much more efficient
> than mpi for small systems).
>
> If you select fine grain parallelization, also the
> mpi-executables will be
> produced. But you need:
>
> i) a properly installed mpi (+ mpi-fortran compiler, which
> is usually "created"
> when mpi is installed. (Usually MPF is
> NOT ifort, but mpif90 !!)
> Note: mpi2 needs some small code changes
> in lapw0.F
> ii) a corresponding Scalapack+Blacs (also suitable for your
> compiler)
> (usually it comes together with the mkl)
> iii) FFTW routines (compiled with the mpi option.
> iv) you need to properly specify the "installation place"
> (-Lxxxx) and the
> name of the libraries (-lxxxx). Also make
> sure you understand 32-bit vs
> 64bit ("emt") issues.
>
> In addition, this code is useful ONLY if you have unitcells
> with more than
> about 50 atoms !!!! and you have hardware with fast network
> (Infiniband),
> i.e. Gbit ethernet may work, but will be slow.
>
> On a SINGLE Quadcore-computer, mpi-parallelization is
> probably NOT worth the effort
> (use OMP_NUM_THREAD 2 (or 4))
>
>
> linuxif9:MPF:ifort ????
> unlikely! (see above)
>
>
>
> > # Linux PC system with IFC 10 compiler + mkl 10 (-ip
> is broken; -static does not give traceback-lines)
> > linuxif9:FC:ifortlinuxif9:MPF:ifort
> > linuxif9:CC:cclinuxif9:FOPT:-FR -mp1 -w
> -prec_div -pc80 -pad -align -DINTEL_VML -traceback -O
> > 3 -xW
> > linuxif9:FPOPT:$(FOPT) -FR -mp1 -w -prec_div -pc80
> -pad -align -DINTEL_VML -traceback
> -I/opt/mpich2/includelinuxif9:LDFLAGS: $(FOPT)
> -L/opt/intel/mkl/10.2.1.017/lib/em64t <http://10.2.1.017/lib/em64t> -pthread -i-static
> > linuxif9:R_LIBS:-L/opt/intel/mkl/10.2.1.017/lib/em64t
> <http://10.2.1.017/lib/em64t> -lmkl_lapack -lmkl
> -liomp5
> > -lguide -lmkl_core
> > linuxif9:DPARALLEL:'-DParallel'
> > linuxif9:RP_LIBS:
> -L/opt/intel/mkl/10.2.1.017/lib/em64t <http://10.2.1.017/lib/em64t> -lmkl_lapack
> -lmkl_intel
> > _lp64 -lmkl_scalapack_lp64 -lmkl_blacs_lp64
> -lmkl_sequential -lmkl_intel_ilp64 -
> > lmkl_scalapack_ilp64 -L/opt/fftw-2.1.5/lib/
> -lfftw_mpi -lfftw -L/opt/mpich2/lib
> > -lmpich -lfmpich
>
> linuxif9:MPIRUN:mpiexec _EXEC_
> >
> > i am running on a xeon, CentOS5 machine. intel
> noncommercial.
> >
> > Any information would be appreciated.
> >
> > Thank you,
> > JD
> >
> > On Mon, Sep 14, 2009 at 10:31 AM, Jeff DeReus <jdereus at gmail.com
> <mailto:jdereus at gmail.com>>
> wrote:
> >
> > fftw was compiled into
> /opt/fftw-2.1.5 with flags
> >
> > $ ./configure
> --prefix=/opt/fftw-2.1.5/ --enable-mpi --enable-threads
> >
> > Thank you,
> > JD
> >
> >
> > On Mon, Sep 14, 2009 at 10:27
> AM, Laurence Marks
> > <L-marks at northwestern.edu
> <mailto:L-marks at northwestern.edu>>
> wrote:
> >
> > It looks like
> you did not compile fftw (or it is somewhere else).
> >
> > 2009/9/14 Jeff
> DeReus <jdereus at gmail.com
> > <mailto:jdereus at gmail.com>>:
> > > Hello
> again. i am having some issues compiling lapw0/1/2
> > modules in
> > >
> wein2k_09. i am running on a CentOS 5.3 box.
> intel
> > non-commercial
> > > compilers and
> mkl.
> > >
> > >
> > >
> > > here are my
> current parallel settings from siteconfig.
> > >
> > >
> Current settings:
> > >
> RP RP_LIB(SCALAPACK+PBLAS):
> >
> -L/opt/intel/mkl/10.2.1.017/lib/em64t
> <http://10.2.1.017/lib/em64t>
> > > -lmkl_lapack
> -lmkl_intel_lp64 -lmkl_scalapack_lp64
> >
> -lmkl_blacs_lp64
> > >
> -lmkl_sequential -L/opt/fftw-2.1.5/lib/ -lfftw_mpi -lfftw
> >
> -L/opt/mpich2/lib
> > > -lmpich
> > >
> FP FPOPT(par.comp.options): $(FOPT) -FR -mp1
> -w
> > -prec_div -pc80
> -pad
> > > -align
> -DINTEL_VML -traceback -I/opt/mpich2/include
> > >
> -I/opt/fftw-2.1.5/fortran
> > >
> MP MPIRUN commando :
> mpiexec _EXEC_
> > >
> > > all other
> modules compile correctly. if i do not enable
> parallel
> > > functionality,
> the compilation completes with no errors.
> > >
> > > when compiling
> /opt/wien2k/SRC_lapw0 it ends with these
> > errors which i
> have
> > > not been able
> to track down.
> > >
> > > fftw_para.o: In
> function `exec_fftw_para_':
> > >
> fftw_para.F:(.text+0x77): undefined reference to
> >
> `fftwnd_f77_mpi_'
> > >
> fftw_para.F:(.text+0xb2): undefined reference to
> >
> `fftwnd_f77_mpi_'
> > > fftw_para.o: In
> function `init_fftw_para_':
> > >
> fftw_para.F:(.text+0x101): undefined reference to
> > >
> `fftw3d_f77_mpi_create_plan_'
> > >
> fftw_para.F:(.text+0x129): undefined reference to
> > >
> `fftw3d_f77_mpi_create_plan_'
> > >
> fftw_para.F:(.text+0x14d): undefined reference to
> > >
> `fftwnd_f77_mpi_local_sizes_'
> > > make[1]: ***
> [lapw0_mpi] Error 1
> > > make[1]:
> Leaving directory `/opt/wien2k/SRC_lapw0'
> > > make: ***
> [para] Error 2
> > >
> > > however,
> /opt/wien2k/SRC_lapw1/2 end with these errors.
> > >
> > > <snip>
> > >
> /opt/intel/mkl/10.2.1.017/lib/em64t/libmkl_intel_lp64.so
> > <http://10.2.1.017/lib/em64t/libmkl_intel_lp64.so>:
> undefined
> > > reference to
> `mkl_pdett__d_forward_trig_transform'
> > >
> /opt/intel/mkl/10.2.1.017/lib/em64t/libmkl_intel_lp64.so
> > <http://10.2.1.017/lib/em64t/libmkl_intel_lp64.so>:
> undefined
> > > reference to
> `mkl_trans_mkl_comatcopy'
> > >
> /opt/intel/mkl/10.2.1.017/lib/em64t/libmkl_intel_lp64.so
> > <http://10.2.1.017/lib/em64t/libmkl_intel_lp64.so>:
> undefined
> > > reference to
> `mkl_pdett_s_init_trig_transform'
> > >
> /opt/intel/mkl/10.2.1.017/lib/em64t/libmkl_intel_lp64.so
> > <http://10.2.1.017/lib/em64t/libmkl_intel_lp64.so>:
> undefined
> > > reference to
> `mkl_pdepl_s_commit_helmholtz_3d'
> > >
> /opt/intel/mkl/10.2.1.017/lib/em64t/libmkl_intel_lp64.so
> > <http://10.2.1.017/lib/em64t/libmkl_intel_lp64.so>:
> undefined
> > > reference to
> `mkl_pdepl_d_commit_helmholtz_2d'
> > >
> /opt/intel/mkl/10.2.1.017/lib/em64t/libmkl_intel_lp64.so
> > <http://10.2.1.017/lib/em64t/libmkl_intel_lp64.so>:
> undefined
> > > reference to
> `mkl_pdett__d_commit_trig_transform'
> > >
> /opt/intel/mkl/10.2.1.017/lib/em64t/libmkl_intel_lp64.so
> > <http://10.2.1.017/lib/em64t/libmkl_intel_lp64.so>:
> undefined
> > > reference to
> `mkl_trans_mkl_comatadd'
> > >
> /opt/intel/mkl/10.2.1.017/lib/em64t/libmkl_intel_lp64.so
> > <http://10.2.1.017/lib/em64t/libmkl_intel_lp64.so>:
> undefined
> > > reference to
> `mkl_serv_getversionstring_c'
> > >
> /opt/intel/mkl/10.2.1.017/lib/em64t/libmkl_intel_lp64.so
> > <http://10.2.1.017/lib/em64t/libmkl_intel_lp64.so>:
> undefined
> > > reference to
> `mkl_spblas_lp64_mkl_dcsrsky'
> > >
> /opt/intel/mkl/10.2.1.017/lib/em64t/libmkl_intel_lp64.so
> > <http://10.2.1.017/lib/em64t/libmkl_intel_lp64.so>:
> undefined
> > > reference to
> `mkl_pdett__d_init_trig_transform'
> > > make[1]: ***
> [lapw2c_mpi] Error 1
> > > make[1]:
> Leaving directory `/opt/wien2k/SRC_lapw2'
> > > make: *** [cp]
> Error 2
> > >
> > >
> > > i can find no
> mention of the fftw3d errors anywhere.
> > >
> > > my complete
> appropriate section of siteconfig
> > >
> > > # Linux PC
> system with IFC 10 compiler + mkl 10 (-ip is
> > broken; -static
> does
> > > not give
> traceback-lines)
> > >
> linuxif9:FC:ifort
> > >
> linuxif9:MPF:ifort
> > > linuxif9:CC:cc
> > >
> linuxif9:FOPT:-FR -mp1 -w -prec_div -pc80 -pad -align
> > -DINTEL_VML
> > > -traceback -O3
> -xW
> > >
> linuxif9:FPOPT:$(FOPT) -FR -mp1 -w -prec_div -pc80 -pad
> > -align
> -DINTEL_VML
> > > -traceback
> -I/opt/mpich2/include
> > >
> linuxif9:LDFLAGS: $(FOPT)
> >
> -L/opt/intel/mkl/10.2.1.017/lib/em64t
> > <http://10.2.1.017/lib/em64t> -pthread
> > > -i-static
> > >
> linuxif9:R_LIBS:-L/opt/intel/mkl/10.2.1.017/lib/em64t
> > <http://10.2.1.017/lib/em64t> -lmkl_lapack -lmkl
> > > -liomp5 -lguide
> -lmkl_core
> > >
> linuxif9:DPARALLEL:'-DParallel'
> > >
> linuxif9:RP_LIBS: -L/opt/intel/mkl/10.2.1.017/lib/em64t
> > <http://10.2.1.017/lib/em64t> -lmkl_lapack
> > >
> -lmkl_intel_lp64 -lmkl_scalapack_lp64 -lmkl_blacs_lp64
> >
> -lmkl_sequential
> > >
> -L/opt/fftw-2.1.5/lib/ -lfftw_mpi -lfftw
> > >
> linuxif9:MPIRUN:mpiexec _EXEC_
> > >
> > >
> > > any assistance
> in this matter would be appreciated.
> > >
> > > Thank you,
> > > JD
> > >
> > >
> _______________________________________________
> > > Wien mailing
> list
> > > Wien at zeus.theochem.tuwien.ac.at
> > <mailto:Wien at zeus.theochem.tuwien.ac.at>
> > > http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien
> > >
> > >
> >
> >
> >
> > --
> > Laurence Marks
> > Department of
> Materials Science and Engineering
> > MSE Rm 2036 Cook
> Hall
> > 2220 N Campus
> Drive
> > Northwestern
> University
> > Evanston, IL
> 60208, USA
> > Tel: (847)
> 491-3996 Fax: (847) 491-7820
> > email: L-marks
> at northwestern dot edu
> > Web:
> www.numis.northwestern.edu <http://www.numis.northwestern.edu>
> > Chair,
> Commission on Electron Crystallography of IUCR
> >
> www.numis.northwestern.edu/ <http://www.numis.northwestern.edu/>
> > Electron
> crystallography is the branch of science that uses electron
> > scattering and
> imaging to study the structure of matter.
> >
> _______________________________________________
> > Wien mailing
> list
> > Wien at zeus.theochem.tuwien.ac.at
> > <mailto:Wien at zeus.theochem.tuwien.ac.at>
> > http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien
> >
> >
> >
> >
> >
> ------------------------------------------------------------------------
> >
> > _______________________________________________
> > Wien mailing list
> > Wien at zeus.theochem.tuwien.ac.at
> > http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien
>
> --
>
>
> P.Blaha
> --------------------------------------------------------------------------
> Peter BLAHA, Inst.f. Materials Chemistry, TU Vienna, A-1060
> Vienna
> Phone: +43-1-58801-15671
> FAX: +43-1-58801-15698
> Email: blaha at theochem.tuwien.ac.at
> WWW: http://info.tuwien.ac.at/theochem/
> --------------------------------------------------------------------------
> _______________________________________________
> Wien mailing list
> Wien at zeus.theochem.tuwien.ac.at
> http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien
>
More information about the Wien
mailing list