<div dir="auto"><div><div>grep "Matrix size" *output1* -A18</div><div dir="auto"><br></div><div dir="auto">Somehow the "A" was lost in a cut & paste</div><div dir="auto"><br></div><div dir="auto">You should also look at the end of case.scf1* and case.output1* for messages, and check the error files.</div><div><br></div><div data-smartmail="gmail_signature">---<br>Emeritus Professor Laurence Marks (Laurie)<br><a href="http://www.numis.northwestern.edu">www.numis.northwestern.edu</a><br><a href="https://scholar.google.com/citations?user=zmHhI9gAAAAJ&hl=en">https://scholar.google.com/citations?user=zmHhI9gAAAAJ&hl=en</a><br>"Research is to see what everybody else has seen, and to think what nobody else has thought" Albert Szent-Györgyi</div><br><div class="gmail_quote gmail_quote_container"><div dir="ltr" class="gmail_attr">On Fri, Jan 24, 2025, 09:40 Laurence Marks <<a href="mailto:laurence.marks@gmail.com">laurence.marks@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="auto"><div><div>Sorry, but you have not provided enough information for more than a guess.</div><div dir="auto"><br></div><div dir="auto">Exit code 9 is when the OS kills the task, often from out of memory (oom) buy it does not have to be. The larger calculation will require about 8*8 more memory (perhaps more) than your simple calculation: do "grep "Matrix size" *output1* -18". You probably ran out of memory, and will need to use more mpi/kpt for the larger calculation.</div><div dir="auto"><br></div><div dir="auto">N.B., using 2 ompi per task is also useful in reducing the total memory useage. Combine this with mpi.</div><div dir="auto"><br></div><div><br></div><div data-smartmail="gmail_signature">---<br>Emeritus Professor Laurence Marks (Laurie)<br><a href="http://www.numis.northwestern.edu" target="_blank" rel="noreferrer">www.numis.northwestern.edu</a><br><a href="https://scholar.google.com/citations?user=zmHhI9gAAAAJ&hl=en" target="_blank" rel="noreferrer">https://scholar.google.com/citations?user=zmHhI9gAAAAJ&hl=en</a><br>"Research is to see what everybody else has seen, and to think what nobody else has thought" Albert Szent-Györgyi</div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Jan 24, 2025, 07:46 Sergeev Gregory <<a href="mailto:sgregory@live.ru" target="_blank" rel="noreferrer">sgregory@live.ru</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
Dear developers,</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
I do my calculations on hpc with slurm system and I have strange behaviour of parallel wien2k jobs:</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
I have two structures:</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
1. Structure with 8 atoms in unitcell (simple structure)</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
2. Supercell structure with 64 atoms (2*2*2 supercell structure) based on cell from simple structure</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
I try to do Wien2k calculations on parallel mode with two configs:</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
1. Calculations on 1 node (1 node has 48 processors) with 12 parallel jobs with 4 processors per each job (one node job)</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
2. Calculations on 2 nodes (2 node has 48*2=96 processors) with 24 parallel jobs with 4 processors per each job (two node job)</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
For "simple structure" "one node job" and "two node job" work without problems.</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
For "supercell structure" "one node job" works well, but "two node job" crashs with errors in .time1_* files (I use Intel MPI):</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
-----------------</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
n053 n053 n053 n053(21) </div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
===================================================================================</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
= BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
= PID 21859 RUNNING AT n053</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
= EXIT CODE: 9</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
= CLEANING UP REMAINING PROCESSES</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
= YOU CAN IGNORE THE BELOW CLEANUP MESSAGES</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
===================================================================================</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
===================================================================================</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
= BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
= PID 21859 RUNNING AT n053</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
= EXIT CODE: 9</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
= CLEANING UP REMAINING PROCESSES</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
= YOU CAN IGNORE THE BELOW CLEANUP MESSAGES</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
===================================================================================</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
Intel(R) MPI Library troubleshooting guide:</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<a href="https://software.intel.com/node/561764" rel="noreferrer noreferrer" target="_blank">https://software.intel.com/node/561764</a></div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
===================================================================================</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
0.042u 0.144s 2:45.42 0.1% 0+0k 4064+8io 60pf+0w </div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
-----------------</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
First I thinked, that there are problems with unufficial memory on "2 node job" (but why, if "1 node job" works with same processors per one parallel job?). I tried to twice increaced used memory per task (#SBATCH --cpus-per-task 2), but this fix haven't solve
problem. Same error.</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
Any ideas why such strange behavior?</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
Does Wien2k have problems scaling to multiple nodes?</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
I would appreciate your help. I want to speed up calculations for complex structures, I have the resources, but I can't do it.</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
</div>
_______________________________________________<br>
Wien mailing list<br>
<a href="mailto:Wien@zeus.theochem.tuwien.ac.at" rel="noreferrer noreferrer" target="_blank">Wien@zeus.theochem.tuwien.ac.at</a><br>
<a href="http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien" rel="noreferrer noreferrer noreferrer" target="_blank">http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien</a><br>
SEARCH the MAILING-LIST at: <a href="http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html" rel="noreferrer noreferrer noreferrer" target="_blank">http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html</a><br>
</blockquote></div></div></div>
</blockquote></div></div></div>