<p>In my experience the SIGSEV normally comes from mixing different flavors of mpif90 and mpirun. Openmpi, mpich2 and Intels mpi all need different versions of blacs. You can also have problems if you choose the wrong model for integers in the linking advisor page. I would check using ldd that lapw0_mpi is linked to the right version, and that the default versions are correct (e.g. which mpirun). Often you can minimize problems by using static linking for mpi.</p>
<p>N.B. The "contact developers" message is a relic of when some code was added for fault handlers and to eliminate issues with limits that used to be pervasive. It should probably be removed.</p>
<p>---------------------------<br>
Professor Laurence Marks<br>
Department of Materials Science and Engineering<br>
Northwestern University<br>
<a href="http://www.numis.northwestern.edu">www.numis.northwestern.edu</a> 1-847-491-3996<br>
"Research is to see what everybody else has seen, and to think what nobody else has thought"<br>
Albert Szent-Gyorgi<br>
</p>
<div class="gmail_quote">On Aug 24, 2012 8:22 AM, "Paul Fons" <<a href="mailto:paul-fons@aist.go.jp">paul-fons@aist.go.jp</a>> wrote:<br type="attribution"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word">
Dear Prof. Blaha,
<div>Thank you for your earlier email. Running the command manually gives the following output (for a GaAs structure that works fine in serial or k-point parallel form). I am still not sure what to try next. Any suggestions?
<div>
<div><br>
</div>
<div>
<div>matstud@ursa:~/WienDisk/Fons/GaAs> mpirun -np 4 ${WIENROOT}/lapw0_mpi lapw0.def</div>
<div>w2k_dispatch_signal(): received: Segmentation fault</div>
<div>w2k_dispatch_signal(): received: Segmentation fault</div>
<div>w2k_dispatch_signal(): received: Segmentation fault</div>
<div>w2k_dispatch_signal(): received: Segmentation fault</div>
<div> Child id 0 SIGSEGV, contact developers</div>
<div> Child id 1 SIGSEGV, contact developers</div>
<div> Child id 3 SIGSEGV, contact developers</div>
<div> Child id 2 SIGSEGV, contact developers</div>
<div>application called MPI_Abort(MPI_COMM_WORLD, 1) - process 3</div>
<div>application called MPI_Abort(MPI_COMM_WORLD, 1) - process 2</div>
<div>application called MPI_Abort(MPI_COMM_WORLD, 1) - process 1</div>
<div>application called MPI_Abort(MPI_COMM_WORLD, 1) - process 0</div>
<div>APPLICATION TERMINATED WITH THE EXIT STRING: Hangup (signal 1)</div>
<div><br>
</div>
<div><br>
</div>
<div>The MPI compilation options from siteconfig are as follows: (the settings are from the Intel MKL link advisor plus the fftw3 library)</div>
<div><br>
</div>
<div>
<div> Current settings:</div>
<div> RP RP_LIB(SCALAPACK+PBLAS): -L$(MKLROOT)/lib/intel64 $(MKLROOT)/lib/intel64/libmkl_blas95_lp64.a $(MKLROOT)/lib/intel64/libmkl_lapack95_lp64.a -lmkl_scalapack_lp64 -lmkl_cdft_core -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -lmkl_blacs_intelmpi_lp64
-openmp -lpthread -lm -L/opt/local/fftw3/lib/ -lfftw3_mpi -lfftw3 $(R_LIBS)</div>
<div> FP FPOPT(par.comp.options): -I$(MKLROOT)/include/intel64/lp64 -I$(MKLROOT)/include -FR -mp1 -w -prec_div -pc80 -pad -ip -DINTEL_VML -DFFTW3 -traceback</div>
<div> MP MPIRUN commando : mpirun -np _NP_ -machinefile _HOSTS_ _EXEC_</div>
<div><br>
</div>
</div>
<div>The file parallel_options now reads</div>
<div>
<div>setenv USE_REMOTE 1</div>
<div>setenv MPI_REMOTE 0</div>
<div>setenv WIEN_GRANULARITY 1</div>
<div>setenv WIEN_MPIRUN "mpirun -np _NP_ -machinefile _HOSTS_ _EXEC_"</div>
</div>
<div><br>
</div>
<div><br>
</div>
<div>I changed the MPI_REMOTE to 0 as suggested (I was not sure this applied to the Intel MPI environment as the siteconfig prompt only mentioned mich2.</div>
<div><br>
</div>
<div>As I mentioned the mpirun command seems to work fine. For example, the fftw3 benchmark program gives with 24 processes</div>
<div><br>
</div>
<div>
<div>mpirun -np 24 ./mpi-bench 1024x1024</div>
<div>Problem: 1024x1024, setup: 126.32 ms, time: 15.98 ms, ``mflops'': 6562.2</div>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div>
<div>On Aug 24, 2012, at 3:05 PM, Peter Blaha wrote:</div>
<br>
<blockquote type="cite">
<div>Hard to say.<br>
<br>
What is in $WIENROOT/parallel_options ?<br>
MPI_REMOTE should be 0 !<br>
<br>
Otherwise run lapw0_mpi by "hand":<br>
<br>
mpirun -np 4 $WIENROOT/lapw0_mpi lapw0.def (or including .machinefile .machine0)<br>
<br>
<br>
Am 24.08.2012 02:24, schrieb Paul Fons:<br>
<blockquote type="cite">Greetings all,<br>
</blockquote>
<blockquote type="cite"> I have compiled Wien2K 12.1 under OpenSuse 11.4 (and OpenSuse 12.1)<br>
</blockquote>
<blockquote type="cite">and the latest Intel compilers with identical mpi launch problems and I<br>
</blockquote>
<blockquote type="cite">am hoping for some suggestions as to where to look to fix things. Note<br>
</blockquote>
<blockquote type="cite">that the serial and k-point parallel versions of the code run fine (I<br>
</blockquote>
<blockquote type="cite">have optimized GaAs a lot in my troubleshooting!).<br>
</blockquote>
<blockquote type="cite"><br>
</blockquote>
<blockquote type="cite">Environment.<br>
</blockquote>
<blockquote type="cite"><br>
</blockquote>
<blockquote type="cite">I am using the latest intel fort, icc, and impi libraries for linux.<br>
</blockquote>
<blockquote type="cite"><br>
</blockquote>
<blockquote type="cite">matstud@pyxis:~/Wien2K> ifort --version<br>
</blockquote>
<blockquote type="cite">ifort (IFORT) 12.1.5 20120612<br>
</blockquote>
<blockquote type="cite">Copyright (C) 1985-2012 Intel Corporation. All rights reserved.<br>
</blockquote>
<blockquote type="cite"><br>
</blockquote>
<blockquote type="cite">matstud@pyxis:~/Wien2K> mpirun --version<br>
</blockquote>
<blockquote type="cite">Intel(R) MPI Library for Linux* OS, Version 4.0 Update 3 Build 20110824<br>
</blockquote>
<blockquote type="cite">Copyright (C) 2003-2011, Intel Corporation. All rights reserved.<br>
</blockquote>
<blockquote type="cite"><br>
</blockquote>
<blockquote type="cite">matstud@pyxis:~/Wien2K> icc --version<br>
</blockquote>
<blockquote type="cite">icc (ICC) 12.1.5 20120612<br>
</blockquote>
<blockquote type="cite">Copyright (C) 1985-2012 Intel Corporation. All rights reserved.<br>
</blockquote>
<blockquote type="cite"><br>
</blockquote>
<blockquote type="cite"><br>
</blockquote>
<blockquote type="cite">My OPTIONS files from /siteconfig_lapw<br>
</blockquote>
<blockquote type="cite"><br>
</blockquote>
<blockquote type="cite">current:FOPT:-FR -mp1 -w -prec_div -pc80 -pad -ip -DINTEL_VML -traceback<br>
</blockquote>
<blockquote type="cite">current:FPOPT:-I$(MKLROOT)/include/intel64/lp64 -I$(MKLROOT)/include -FR<br>
</blockquote>
<blockquote type="cite">-mp1 -w -prec_div -pc80 -pad -ip -DINTEL_VML -DFFTW3 -traceback<br>
</blockquote>
<blockquote type="cite">current:LDFLAGS:$(FOPT) -L$(MKLROOT)/lib/$(MKL_TARGET_ARCH) -pthread<br>
</blockquote>
<blockquote type="cite">current:DPARALLEL:'-DParallel'<br>
</blockquote>
<blockquote type="cite">current:R_LIBS:-lmkl_lapack95_lp64 -lmkl_intel_lp64 -lmkl_intel_thread<br>
</blockquote>
<blockquote type="cite">-lmkl_core -openmp -lpthread<br>
</blockquote>
<blockquote type="cite">current:RP_LIBS:-L$(MKLROOT)/lib/intel64<br>
</blockquote>
<blockquote type="cite">$(MKLROOT)/lib/intel64/libmkl_blas95_lp64.a<br>
</blockquote>
<blockquote type="cite">$(MKLROOT)/lib/intel64/libmkl_lapack95_lp64.a -lmkl_scalapack_lp64<br>
</blockquote>
<blockquote type="cite">-lmkl_cdft_core -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core<br>
</blockquote>
<blockquote type="cite">-lmkl_blacs_intelmpi_lp64 -openmp -lpthread -lm -L/opt/local/fftw3/lib/<br>
</blockquote>
<blockquote type="cite">-lfftw3_mpi -lfftw3 $(R_LIBS)<br>
</blockquote>
<blockquote type="cite">current:MPIRUN:mpirun -np _NP_ -machinefile _HOSTS_ _EXEC_<br>
</blockquote>
<blockquote type="cite"><br>
</blockquote>
<blockquote type="cite"><br>
</blockquote>
<blockquote type="cite"><br>
</blockquote>
<blockquote type="cite"><br>
</blockquote>
<blockquote type="cite">The code compiles and links without error. It runs fine in serial mode<br>
</blockquote>
<blockquote type="cite">and in k-point parallel mode, e.g.<br>
</blockquote>
<blockquote type="cite"><br>
</blockquote>
<blockquote type="cite">.machines with<br>
</blockquote>
<blockquote type="cite"><br>
</blockquote>
<blockquote type="cite">1:localhost<br>
</blockquote>
<blockquote type="cite">1:localhost<br>
</blockquote>
<blockquote type="cite">1:localhost<br>
</blockquote>
<blockquote type="cite">granularity:1<br>
</blockquote>
<blockquote type="cite">extrafine:1<br>
</blockquote>
<blockquote type="cite"><br>
</blockquote>
<blockquote type="cite">This runs fine. When I attempt to run a mpi process with 12 processes<br>
</blockquote>
<blockquote type="cite">(on a 12 core machine), I crash and burn (see below) with a SIGSEV error<br>
</blockquote>
<blockquote type="cite">with instructions to contact the developers.<br>
</blockquote>
<blockquote type="cite"><br>
</blockquote>
<blockquote type="cite">The linking options were derived from Intel's mkl link advisor (the<br>
</blockquote>
<blockquote type="cite">version on the intel site. I should add that the mpi-bench in fftw3<br>
</blockquote>
<blockquote type="cite">works fine using the intel mpi as do commands like hostname or even<br>
</blockquote>
<blockquote type="cite">abinit so it would appear that that the Intel MPI environment itself is<br>
</blockquote>
<blockquote type="cite">fine. I have wasted a lot of time trying to figure out how to fix this<br>
</blockquote>
<blockquote type="cite">before writing to the list, but at this point, I feel like a monkey at a<br>
</blockquote>
<blockquote type="cite">keyboard attempting to duplicate Shakesphere -- if you know what I mean.<br>
</blockquote>
<blockquote type="cite"> Thanks in advance for any heads up that you can offer.<br>
</blockquote>
<blockquote type="cite"><br>
</blockquote>
<blockquote type="cite"><br>
</blockquote>
<blockquote type="cite"><br>
</blockquote>
<blockquote type="cite">.machines<br>
</blockquote>
<blockquote type="cite"><br>
</blockquote>
<blockquote type="cite">lapw0:localhost:12<br>
</blockquote>
<blockquote type="cite">1:localhost:12<br>
</blockquote>
<blockquote type="cite">granularity:1<br>
</blockquote>
<blockquote type="cite">extrafine:1<br>
</blockquote>
<blockquote type="cite"><br>
</blockquote>
<blockquote type="cite">
<blockquote type="cite"> stop error<br>
</blockquote>
</blockquote>
<blockquote type="cite"><br>
</blockquote>
<blockquote type="cite">error: command /home/matstud/Wien2K/lapw0para -c lapw0.def failed<br>
</blockquote>
<blockquote type="cite">0.029u 0.046s 0:00.93 6.4%<span style="white-space:pre-wrap">
</span>0+0k 0+176io 0pf+0w<br>
</blockquote>
<blockquote type="cite"> Child id 2 SIGSEGV, contact developers<br>
</blockquote>
<blockquote type="cite"> Child id 8 SIGSEGV, contact developers<br>
</blockquote>
<blockquote type="cite"> Child id 7 SIGSEGV, contact developers<br>
</blockquote>
<blockquote type="cite"> Child id 11 SIGSEGV, contact developers<br>
</blockquote>
<blockquote type="cite"> Child id 10 SIGSEGV, contact developers<br>
</blockquote>
<blockquote type="cite"> Child id 9 SIGSEGV, contact developers<br>
</blockquote>
<blockquote type="cite"> Child id 6 SIGSEGV, contact developers<br>
</blockquote>
<blockquote type="cite"> Child id 5 SIGSEGV, contact developers<br>
</blockquote>
<blockquote type="cite"> Child id 4 SIGSEGV, contact developers<br>
</blockquote>
<blockquote type="cite"> Child id 3 SIGSEGV, contact developers<br>
</blockquote>
<blockquote type="cite"> Child id 1 SIGSEGV, contact developers<br>
</blockquote>
<blockquote type="cite"> Child id 0 SIGSEGV, contact developers<br>
</blockquote>
<blockquote type="cite">-------- .machine0 : 12 processors<br>
</blockquote>
<blockquote type="cite">
<blockquote type="cite"> lapw0 -p<span style="white-space:pre-wrap">
</span>(09:04:45) starting parallel lapw0 at Fri Aug 24 09:04:45 JST 2012<br>
</blockquote>
</blockquote>
<blockquote type="cite"><br>
</blockquote>
<blockquote type="cite"> cycle 1 <span style="white-space:pre-wrap">
</span>(Fri Aug 24 09:04:45 JST 2012) <span style="white-space:pre-wrap">
</span>(40/99 to go)<br>
</blockquote>
<blockquote type="cite"><br>
</blockquote>
<blockquote type="cite"> start <span style="white-space:pre-wrap">
</span>(Fri Aug 24 09:04:45 JST 2012) with lapw0 (40/99 to go)<br>
</blockquote>
<blockquote type="cite"><br>
</blockquote>
<blockquote type="cite"><br>
</blockquote>
<blockquote type="cite">using WIEN2k_12.1 (Release 22/7/2012) in /home/matstud/Wien2K<br>
</blockquote>
<blockquote type="cite">on pyxis with PID 15375<br>
</blockquote>
<blockquote type="cite">Calculating GaAs in /usr/local/share/Wien2K/Fons/GaAs<br>
</blockquote>
<blockquote type="cite"><br>
</blockquote>
<blockquote type="cite"><br>
</blockquote>
<blockquote type="cite"><br>
</blockquote>
<blockquote type="cite"><br>
</blockquote>
<blockquote type="cite">_______________________________________________<br>
</blockquote>
<blockquote type="cite">Wien mailing list<br>
</blockquote>
<blockquote type="cite"><a href="mailto:Wien@zeus.theochem.tuwien.ac.at" target="_blank">Wien@zeus.theochem.tuwien.ac.at</a><br>
</blockquote>
<blockquote type="cite"><a href="http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien" target="_blank">http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien</a><br>
</blockquote>
<blockquote type="cite"><br>
</blockquote>
<br>
-- <br>
Peter Blaha<br>
Inst.Materials Chemistry<br>
TU Vienna<br>
Getreidemarkt 9<br>
A-1060 Vienna<br>
Austria<br>
+43-1-5880115671<br>
_______________________________________________<br>
Wien mailing list<br>
<a href="mailto:Wien@zeus.theochem.tuwien.ac.at" target="_blank">Wien@zeus.theochem.tuwien.ac.at</a><br>
<a href="http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien" target="_blank">http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien</a><br>
</div>
</blockquote>
</div>
<br>
<div>
<div>
<div style="font-family:Helvetica;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:12px">
<font face="Helvetica" size="3" style="font:normal normal normal 12px/normal Helvetica">Dr. Paul Fons</font></div>
<div style="font-family:Helvetica;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:12px">
Senior Research Scientist</div>
<div style="font-family:Helvetica;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:12px">
Functional Nano-phase-change Research Team</div>
<div style="font-family:Helvetica;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:12px">
Nanoelectronics Research Institute</div>
<div style="font-family:Helvetica;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:12px">
<font face="Helvetica" size="3" style="font:normal normal normal 12px/normal Helvetica">National Institute for Advanced Industrial Science & Technology</font></div>
<div style="font-family:Helvetica;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:12px">
<font face="Helvetica" size="3" style="font:normal normal normal 12px/normal Helvetica">METI</font></div>
<div style="font-family:Helvetica;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:12px">
<br>
</div>
<div style="font-family:Helvetica;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:12px">
<font face="Helvetica" size="3" style="font:normal normal normal 12px/normal Helvetica">AIST Central 4, Higashi 1-1-1</font></div>
<div style="font-family:Helvetica;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:12px">
<font face="Helvetica" size="3" style="font:normal normal normal 12px/normal Helvetica">Tsukuba, Ibaraki JAPAN 305-8568</font></div>
<div style="font-family:Helvetica;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font:normal normal normal 12px/normal Helvetica;min-height:14px;font-size:12px">
<br>
</div>
<div style="font-family:Helvetica;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:12px">
<font face="Helvetica" size="3" style="font:normal normal normal 12px/normal Helvetica">tel. +81-298-61-5636</font></div>
<div style="font-family:Helvetica;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:12px">
<font face="Helvetica" size="3" style="font:normal normal normal 12px/normal Helvetica">fax. +81-298-61-2939</font></div>
<div style="font-family:Helvetica;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font:normal normal normal 12px/normal Helvetica;min-height:14px;font-size:12px">
<br>
</div>
<div style="font-family:Helvetica;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:12px">
<font face="Helvetica" size="3" style="font:normal normal normal 12px/normal Helvetica">email: </font><font face="Helvetica" size="3" color="#1919ff" style="font:normal normal normal 12px/normal Helvetica;color:rgb(25,25,255)"><u style="color:rgb(25,25,255)"><a href="mailto:paul-fons@aist.go.jp" target="_blank">paul-fons@aist.go.jp</a></u></font></div>
<div style="font-family:Helvetica;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font:normal normal normal 12px/normal Helvetica;min-height:14px;font-size:12px">
<br>
</div>
<div style="font-family:Helvetica;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:12px">
<font face="Hiragino Kaku Gothic Pro">The following lines are in a Japanese font</font></div>
<div style="font-family:Helvetica;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:12px">
<font face="Hiragino Kaku Gothic Pro"><br>
</font></div>
<div style="font-family:Helvetica;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:12px">
<font face="Hiragino Kaku Gothic Pro">〒305-8562 茨城県つくば市つくば中央東 1-1-1</font></div>
<div style="font-family:Helvetica;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:12px">
<font face="Hiragino Kaku Gothic Pro"><span style="font-family:'Hiragino Kaku Gothic Pro'">産業技術総合研究所</span></font></div>
<div style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:12px">
<font face="'Hiragino Kaku Gothic Pro'">ナノエレクトロニクス研究部門</font></div>
<div style="font-family:Helvetica;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:12px">
<font face="'Hiragino Kaku Gothic Pro'">相変化新規機能デバイス研究チーム</font></div>
<div style="font-family:Helvetica;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:12px">
<font face="'Hiragino Kaku Gothic Pro'">主任研究員</font></div>
<div style="font-family:Helvetica;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;font-size:12px">
<span style="font-family:'Hiragino Kaku Gothic Pro'">ポール・フォンス</span></div>
<span style="text-indent:0px;letter-spacing:normal;font-variant:normal;font-style:normal;font-weight:normal;line-height:normal;border-collapse:separate;text-transform:none;font-size:medium;white-space:normal;font-family:Helvetica;word-spacing:0px">
<div style="word-wrap:break-word">
<span style="text-indent:0px;letter-spacing:normal;font-variant:normal;font-style:normal;font-weight:normal;line-height:normal;border-collapse:separate;text-transform:none;font-size:medium;white-space:normal;font-family:Helvetica;word-spacing:0px">
<div style="word-wrap:break-word">
<div><span style="font-size:12px">
<div style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px">
<br>
</div>
</span></div>
</div>
</span><br>
</div>
</span><br>
</div>
<br>
</div>
<br>
</div>
</div>
</div>
</div>
</blockquote></div>