<div dir="ltr"><p style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="background-color:transparent;color:rgb(0,0,0);font-family:"Times New Roman";font-size:12pt;white-space:pre-wrap">Here are some of my compiler options for WIEN2k:</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap"><font face="monospace">FOPT:-mkl -O -FR -mp1 -w -prec_div -pc80 -pad -ip -g -DINTEL_VML -DMKL_LP64 -traceback -assume buffered_io -I$(TACC_MKL_INC)</font></span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap"><font face="monospace">FPOPT:-mkl -O -FR -mp1 -w -prec_div -pc80 -pad -ip -g -DINTEL_VML -DMKL_LP64 -traceback -assume buffered_io -I$(TACC_MKL_INC)</font></span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap"><font face="monospace">LDFLAGS:$(FOPT) -Wl,-rpath,/scratch/tacc/apps/intel19/impi19_0/fftw3/3.3.10/lib,-rpath,/opt/intel/compilers_and_libraries_2020.1.217/linux/mkl/lib/intel64,-rpath,/opt/intel/compilers_and_libraries_2020.1.217/linux/compiler/lib/intel64,-rpath,/usr/lib64 -L/usr/lib64 -lm -ldl -lpthread -L/opt/intel/compilers_and_libraries_2020.1.217/linux/compiler/lib/intel64 -liomp5</font></span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap"><font face="monospace">R_LIBS:-L/opt/intel/compilers_and_libraries_2020.1.217/linux/mkl/lib/intel64 -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core</font></span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap"><font face="monospace">RP_LIBS:$(R_LIBS)</font></span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap"><font face="monospace">FFTWROOT:/scratch/tacc/apps/intel19/impi19_0/fftw3/3.3.10/</font></span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap"><font face="monospace">FFTW_LIBNAME:fftw3</font></span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap"><font face="monospace">SCALAPACKROOT:/opt/intel/compilers_and_libraries_2020.1.217/linux/mkl/lib/</font></span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap"><font face="monospace">SCALAPACK_LIBNAME:mkl_scalapack_lp64</font></span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap"><font face="monospace">BLACSROOT:/opt/intel/compilers_and_libraries_2020.1.217/linux/mkl/lib/</font></span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap"><font face="monospace">BLACS_LIBNAME:mkl_blacs_intelmpi_lp64</font></span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap"><font face="monospace">MPIRUN:srun -N _nodes_ -n _NP_ _PINNING_ _EXEC_</font></span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap"><font face="monospace">CORES_PER_NODE:64</font></span></p></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sun, Mar 26, 2023 at 8:21 PM Brian Lee <<a href="mailto:brianhlee@utexas.edu">brianhlee@utexas.edu</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:12pt;font-family:"Times New Roman";color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap">Hi, thank you for the responses</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:12pt;font-family:"Times New Roman";color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap">Yes, sorry the dayfile was from a different test run. The run using "./wien2k_tasks_v4.sh 2 4" shows it as:</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap"><font face="monospace" size="1"><br></font></span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap"><font face="monospace">> lapw0 -p (12:51:21) starting parallel lapw0 at Thu Mar 23 12:51:21 CD$</font></span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap"><font face="monospace">-------- .machine0 : 2 processors</font></span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap"><font face="monospace">** lapw0 crashed!</font></span></p><div><br></div>.machines file was generated using:<div><br><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><font face="monospace"><span style="background-color:transparent;color:rgb(0,0,0);white-space:pre-wrap"># create hostfile_tacc from a batch</span><br></font></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap"><font face="monospace">mpiexec.hydra hostname|cut -d \. -f 1 | sort -n > hostlist_wien2k</font></span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap"><font face="monospace"># head of machines_kpoint</font></span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap"><font face="monospace">#</font></span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap"><font face="monospace">rm .machines</font></span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap"><font face="monospace">echo '#' > .machines</font></span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap"><font face="monospace">echo 'granularity:1' >> .machines</font></span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap"><font face="monospace"># list the hosts in rows for k-point parallelism</font></span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap"><font face="monospace">awk -v div=$1 '{_=int(NR/(div+1.0e-10))} {a[_]=((a[_])?a[_]FS:x)$1;l=(_>l)?_:l}END{for(i=0;i<=0;++i)print "lapw0:"a[i]":1"}' hostlist_wien2k >>.machines</font></span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap"><font face="monospace">awk -v div=$2 '{_=int(NR/(div+1.0e-10))} {a[_]=((a[_])?a[_]FS:x)$1;l=(_>l)?_:l}END{for(i=0;i<=l;++i)print "1:"a[i]":1"}' hostlist_wien2k >>.machines</font></span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap"><font face="monospace">#</font></span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap"><font face="monospace"># tail of machines_kpoint: allocate remaining k points one by one over all tasks</font></span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap"><font face="monospace">#</font></span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap"><font face="monospace">echo 'extrafine:1' >>.machines</font></span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap"><font face="monospace"># machines_kpoint is end</font></span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap"><font face="monospace"># cleanup</font></span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap"><font face="monospace">rm hostlist_wien2k</font></span></p><br><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:12pt;font-family:"Times New Roman";color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap">I believe both of fftw and WIEN2k were compiled with the same intel compilers, but I've attached my WIEN2k options in the second email. I’ve tried using different “CORES_PER_NODE” settings (16, 64) to either match the number of cores per node I request or the number of total cores per node, but the error is still the same, and r</span><span style="font-family:"times new roman",serif;background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-size:12pt;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap">unning </span><span style="font-family:"times new roman",serif;background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-size:11pt;vertical-align:baseline;white-space:pre-wrap">x lapw0 followed by x lapw1 -p in my job script leads to:</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-family:"times new roman",serif;background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-size:11pt;vertical-align:baseline;white-space:pre-wrap"><br></span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap"><font face="monospace"> LAPW0 END</font></span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap"><font face="monospace">forrtl: No such file or directory</font></span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap"><font face="monospace">forrtl: severe (28): CLOSE error, unit 200, file "Unknown"</font></span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap"><font face="monospace">Image PC Routine Line Source</font></span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap"><font face="monospace">lapw1_mpi 00000000004DCBAB Unknown Unknown Unknown</font></span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap"><font face="monospace">lapw1_mpi 00000000004CED9F Unknown Unknown Unknown</font></span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap"><font face="monospace">lapw1_mpi 000000000045DEE3 inilpw_ 264 inilpw.f</font></span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap"><font face="monospace">lapw1_mpi 0000000000462050 MAIN__ 48 lapw1_tmp_.F</font></span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap"><font face="monospace">lapw1_mpi 0000000000408362 Unknown Unknown Unknown</font></span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap"><font face="monospace"><a href="http://libc-2.28.so/" target="_blank">libc-2.28.so</a> 0000147E06BC9CF3 __libc_start_main Unknown Unknown</font></span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap"><font face="monospace">lapw1_mpi 000000000040826E Unknown Unknown Unknown</font></span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap"><font face="monospace">srun: error: c306-005: task 0: Exited with exit code 28</font></span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap"><font face="monospace">forrtl: No such file or directory</font></span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap"><font face="monospace">forrtl: severe (28): CLOSE error, unit 200, file "Unknown"</font></span></p><br><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:12pt;font-family:"Times New Roman";background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap">Any additional help/information would be greatly appreciated</span></p><span style="color:rgb(80,0,80)"><br><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:12pt;font-family:"Times New Roman";background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap">Regards,</span></p><p dir="ltr" style="line-height:1.656;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;font-family:Arial;color:rgb(51,63,72);background-color:transparent;font-weight:700;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap">Brian Lee</span><span style="font-size:11pt;font-family:Arial;color:rgb(51,63,72);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap"> </span><span style="font-size:11pt;font-family:Arial;color:rgb(190,91,23);background-color:transparent;font-weight:700;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap"> </span><span style="font-size:11pt;font-family:Arial;color:rgb(51,51,51);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap">| </span><span style="font-size:11pt;font-family:Arial;color:rgb(190,91,23);background-color:transparent;font-weight:700;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap">Graduate Student</span></p><p dir="ltr" style="line-height:1.656;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;font-family:Arial;color:rgb(190,91,23);background-color:transparent;font-weight:700;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap">The University of Texas at Austin </span><span style="font-size:11pt;font-family:Arial;color:rgb(51,51,51);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap">| </span><span style="font-size:11pt;font-family:Arial;color:rgb(190,91,23);background-color:transparent;font-weight:700;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap">Texas Materials Institute</span><span style="font-size:11pt;font-family:Arial;color:rgb(51,51,51);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap"> </span></p><p dir="ltr" style="line-height:1.656;margin-top:0pt;margin-bottom:0pt"><span style="font-size:12pt;font-family:"Times New Roman";color:rgb(51,63,72);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap">(he/him/his)</span></p></span></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Mar 23, 2023 at 3:51 PM Peter Blaha <<a href="mailto:peter.blaha@tuwien.ac.at" target="_blank">peter.blaha@tuwien.ac.at</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<p>My guess would be that you link with a fftw which is compiled
with gfortran, while wien2k is compiled with ifort (of the
opposite or different compiler versions.....). <br>
</p>
<p>Or it was compiled with proper compilers, but the mpi was mixed
(openmpi vs intelmpi, ...</p>
<p><br>
</p>
<p>You can also try to run only</p>
<p>x lapw0 (serial, so that you get proper vsp and vns files for
lapw1)<br>
</p>
<p>x lapw1 -p in mpi-mode. lapw1 does not link fftw (but
scalapack and hopefully elpa).</p>
<p><br>
</p>
<p>Otherwise your report cannot be fully correct:</p>
<p> You claim that you requested 2 cores for lapw0 and part of your
email supports this .</p>
<p>However, I do not understand why the dayfile claims to have 4
cores in .machine0 ???</p>
<p>About the way wien2k launches mpi jobs: You can "see" how it
does it in the error logs:</p>
<p><span id="m_-3484269451566230681m_-5252055818911981467gmail-docs-internal-guid-c648bbfe-7fff-6f38-200d-a1756f3cf2ed"><span style="color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap"><font face="monospace"> srun -K -N1 -n2 -r0 /home1/08844/leebrian/wien2k/lapw0_mpi lapw0.def >> .time00</font></span></span></p>
<p><span id="m_-3484269451566230681m_-5252055818911981467gmail-docs-internal-guid-c648bbfe-7fff-6f38-200d-a1756f3cf2ed"><span style="color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap"><font face="monospace">Your sysadmins can check this command and you can put this line in your submit script and test it.</font></span></span></p>
<p><span id="m_-3484269451566230681m_-5252055818911981467gmail-docs-internal-guid-c648bbfe-7fff-6f38-200d-a1756f3cf2ed"><span style="color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap"><font face="monospace">PS: In any case, you request 4 nodes and in total 64 cores.</font></span></span></p>
<p><span id="m_-3484269451566230681m_-5252055818911981467gmail-docs-internal-guid-c648bbfe-7fff-6f38-200d-a1756f3cf2ed"><span style="color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap"><font face="monospace">But with this .machines file you use only 2 cores in lapw0 and 16 in lapw1/2. This waists your cpu-hours.</font></span></span></p>
<p><span id="m_-3484269451566230681m_-5252055818911981467gmail-docs-internal-guid-c648bbfe-7fff-6f38-200d-a1756f3cf2ed"><span style="color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap"><font face="monospace">Check the part of your script (wien2k_tasks... ????) that generates the .machines file.</font></span></span></p>
<p><span id="m_-3484269451566230681m_-5252055818911981467gmail-docs-internal-guid-c648bbfe-7fff-6f38-200d-a1756f3cf2ed"><span style="color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap"><font face="monospace">PS: What is your CORES_PER_NODE setting ?
</font></span></span></p>
<p>PPS: The message from L.Marks that you need a ":number" in the
.machines file is not true. It is perfectly ok and the same to use
node:1 or only node<br>
</p>
<p><br>
</p>
<div>Am 23.03.2023 um 19:14 schrieb Brian
Lee:<br>
</div>
<blockquote type="cite">
<div dir="ltr"><span id="m_-3484269451566230681m_-5252055818911981467gmail-docs-internal-guid-c648bbfe-7fff-6f38-200d-a1756f3cf2ed">
<p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:12pt;font-family:"Times New Roman";color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap">Hello WIEN2k users/developers,</span></p>
<br>
<p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:12pt;font-family:"Times New Roman";background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap">I am a graduate student at UT Austin in the MS&E program and would like to test </span></p>
<p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:12pt;font-family:"Times New Roman";background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline;white-space:pre-wrap">WIEN2k_23.2 using various parallelization schemes. When I try to run “run_lapw -p” with the default MPI run command suggested during siteconfig along with a .machines file/job script that requests 2 processors per lapw0 and/or 2 processors per kpt, I receive the following error:</span></p>
</span>/index.html
</div>
<pre></pre>
</blockquote>
<pre cols="72">--
-----------------------------------------------------------------------
Peter Blaha, Inst. f. Materials Chemistry, TU Vienna, A-1060 Vienna
Phone: +43-158801165300
Email: <a href="mailto:peter.blaha@tuwien.ac.at" target="_blank">peter.blaha@tuwien.ac.at</a>
WWW: <a href="http://www.imc.tuwien.ac.at" target="_blank">http://www.imc.tuwien.ac.at</a> WIEN2k: <a href="http://www.wien2k.at" target="_blank">http://www.wien2k.at</a>
-------------------------------------------------------------------------</pre>
</div>
_______________________________________________<br>
Wien mailing list<br>
<a href="mailto:Wien@zeus.theochem.tuwien.ac.at" target="_blank">Wien@zeus.theochem.tuwien.ac.at</a><br>
<a href="http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien" rel="noreferrer" target="_blank">http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien</a><br>
SEARCH the MAILING-LIST at: <a href="http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html" rel="noreferrer" target="_blank">http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html</a><br>
</blockquote></div>
</blockquote></div>