[Wien] Pathscale+OpenMPI support

Scott Beardsley scott at cse.ucdavis.edu
Wed Nov 19 00:41:21 CET 2008


I have WIEN2k 8.3 working with the Pathscale 3.2 compiler and OpenMPI 
1.2.6. I'm new to WIEN so I think it is working at least. I did have to 
make a few changes. First there appears to be a bug in SRC_tetra/tetra.f:

$ diff ../orig/SRC_tetra/tetra.f SRC_tetra/tetra.f
265,266c265,266
<                   " for Atom",i4,"  col=",i3,"  Energy=",2f8.4,/)') &
<                   IDOS(1,1),IDOS(1,2),emin,emax
---
 >                   & " for Atom",i4,"  col=",i3,"  Energy=",2f8.4,/)') &
 >                   & IDOS(1,1),IDOS(1,2),emin,emax
$

Then I had to make the following changes to siteconfig_lapw:

$ diff ../orig/siteconfig_lapw siteconfig_lapw
258a259
 >      P    Linux (Pathscale compiler)
313a315,318
 >     case [p|P]:
 > 	set system = linuxpath
 > 	set cpfile = generic
 > 	breaksw
1572a1578,1589
 > # Linux system with Pathscale compiler
 > linuxpath:FC:pathf90
 > linuxpath:MPF:mpif90
 > linuxpath:CC:pathcc
 > linuxpath:FOPT:-O2 -OPT:Ofast -freeform
 > linuxpath:FPOPT:-O2 -OPT:Ofast -freeform
 > linuxpath:LDFLAGS:-L../SRC_lib -L/share/apps/acml-4.1.0/pathscale64/lib
 > linuxpath:R_LIBS:-lacml
 > linuxpath:DPARALLEL:'-DParallel'
 > linuxpath:RP_LIBS:-L/share/apps/openmpi-1.2.6/pathscale-3.2/lib64 
-L/share/apps/acml-4.1.0/pathscale64/lib -lacml 
/share/apps/SCALAPACK/openmpi-1.2.6/pathscale-3.2/lib/libscalapack.a 
/share/apps/SCALAPACK/openmpi-1.2.6/pathscale-3.2/lib/blacsF77.a 
/share/apps/SCALAPACK/openmpi-1.2.6/pathscale-3.2/lib/blacs.a 
/share/apps/SCALAPACK/openmpi-1.2.6/pathscale-3.2/lib/blacsF77.a -lmpi
 > linuxpath:MPIRUN:mpirun _EXEC_
 >
$

For some reason I still had to answer the questions when running 
siteconfig (perhaps I didn't set the defaults properly?).

Of course these changes aren't very generic (I doubt others have 
identical paths) but it should point others in the right direction.

If you're curious I've posted some docs/benchmarks on our wiki[1].

Our nodes are dual-socket, quad-core opterons (2.3Ghz) with 12G memory 
and 20Gbps DDR Infinipath (QLogic). I haven't tried using the QLogic MPI 
but I expect it is slightly faster than OpenMPI. I may try seeing how 
far out it will scale when the machine becomes less busy.

Scott


More information about the Wien mailing list