<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div class="elementToProof" style="text-align: left; text-indent: 0px; background-color: rgb(255, 255, 255); margin: 0px; font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Dear Prof. Blaha, Prof. Marks, Gavin,</div>
<div style="text-align: left; text-indent: 0px; background-color: rgb(255, 255, 255); margin: 0px; font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
I have carefully studied all the advices you gave me, and finally solved my problem.</div>
<div style="text-align: left; text-indent: 0px; background-color: rgb(255, 255, 255); margin: 0px; font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="text-align: left; text-indent: 0px; background-color: rgb(255, 255, 255); margin: 0px; font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Professor Blaha's advice to check the parallel_options file was especially valuable.</div>
<div style="text-align: left; text-indent: 0px; background-color: rgb(255, 255, 255); margin: 0px; font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
In this file, for variable WIEN_MPIRUN I used dafault value: "mpirun -np _NP_ _EXEC_"</div>
<div style="text-align: left; text-indent: 0px; background-color: rgb(255, 255, 255); margin: 0px; font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
After I replaced it to "srun --mpi=pmi2 -K -N_nodes_ -n_NP_ -r_offset_ _PINNING_ _EXEC_" all works fine.</div>
<div style="text-align: left; text-indent: 0px; background-color: rgb(255, 255, 255); margin: 0px; font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
I don't know, why with "mpirun -np _NP_ _EXEC_" simple case works on 2 nodes unlike supercell case, but problem solved now.</div>
<div style="text-align: left; text-indent: 0px; background-color: rgb(255, 255, 255); margin: 0px; font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="text-align: left; text-indent: 0px; background-color: rgb(255, 255, 255); margin: 0px; font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Thanks again for your help</div>
<div style="text-align: left; text-indent: 0px; background-color: rgb(255, 255, 255); margin: 0px; font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="text-align: left; text-indent: 0px; background-color: rgb(255, 255, 255); margin: 0px; font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
- Gregory Sergeev</div>
<div class="elementToProof" style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div id="appendonsend"></div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>От:</b> Wien <wien-bounces@zeus.theochem.tuwien.ac.at> от имени Peter Blaha <peter.blaha@tuwien.ac.at><br>
<b>Отправлено:</b> 24 января 2025 г. 20:52<br>
<b>Кому:</b> wien@zeus.theochem.tuwien.ac.at <wien@zeus.theochem.tuwien.ac.at><br>
<b>Тема:</b> Re: [Wien] Problem with parallel jobs of comlex structures (supercells) on hpc</font>
<div> </div>
</div>
<div class="BodyFragment"><font size="2"><span style="font-size:11pt">
<div class="PlainText">Check<br>
$WIENROOT/WIEN2k_parallel_options<br>
<br>
setenv TASKSET "no"<br>
if ( ! $?USE_REMOTE ) setenv USE_REMOTE 1<br>
if ( ! $?MPI_REMOTE ) setenv MPI_REMOTE 0<br>
setenv WIEN_GRANULARITY 1<br>
setenv DELAY 0.1<br>
setenv SLEEPY 1<br>
setenv WIEN_MPIRUN "mpirun -np _NP_ -machinefile _HOSTS_ _EXEC_"<br>
<br>
Is your MPI_REMOTE set to zero or one ?<br>
and USE_REMOTE ??<br>
<br>
Can you do k-parallel only (no mpi) on 2 nodes ?<br>
<br>
You did not show the .machines file. Is it ok ?<br>
<br>
And maybe the beginning of your job scipt, maybe some slurm parameters <br>
are not set properly for the 2 node job ?<br>
<br>
Am 24.01.2025 um 14:36 schrieb Sergeev Gregory:<br>
> Dear developers,<br>
> I do my calculations on hpc with slurm system and I have strange <br>
> behaviour of parallel wien2k jobs:<br>
> <br>
> I have two structures:<br>
> 1. Structure with 8 atoms in unitcell (simple structure)<br>
> 2. Supercell structure with 64 atoms (2*2*2 supercell structure) based <br>
> on cell from simple structure<br>
> <br>
> I try to do Wien2k calculations on parallel mode with two configs:<br>
> 1. Calculations on 1 node (1 node has 48 processors) with 12 parallel <br>
> jobs with 4 processors per each job (one node job)<br>
> 2. Calculations on 2 nodes (2 node has 48*2=96 processors) with 24 <br>
> parallel jobs with 4 processors per each job (two node job)<br>
> <br>
> For "simple structure" "one node job" and "two node job" work without <br>
> problems.<br>
> <br>
> For "supercell structure" "one node job" works well, but "two node job" <br>
> crashs with errors in .time1_* files (I use Intel MPI):<br>
> <br>
> -----------------<br>
> n053 n053 n053 n053(21)<br>
> ===================================================================================<br>
> = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES<br>
> = PID 21859 RUNNING AT n053<br>
> = EXIT CODE: 9<br>
> = CLEANING UP REMAINING PROCESSES<br>
> = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES<br>
> ===================================================================================<br>
> <br>
> ===================================================================================<br>
> = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES<br>
> = PID 21859 RUNNING AT n053<br>
> = EXIT CODE: 9<br>
> = CLEANING UP REMAINING PROCESSES<br>
> = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES<br>
> ===================================================================================<br>
> Intel(R) MPI Library troubleshooting guide:<br>
> <a href="https://software.intel.com/node/561764" data-auth="NotApplicable">
https://software.intel.com/node/561764</a><br>
> ===================================================================================<br>
> 0.042u 0.144s 2:45.42 0.1% 0+0k 4064+8io 60pf+0w<br>
> -----------------<br>
> <br>
> First I thinked, that there are problems with unufficial memory on "2 <br>
> node job" (but why, if "1 node job" works with same processors per one <br>
> parallel job?). I tried to twice increaced used memory per task (#SBATCH <br>
> --cpus-per-task 2), but this fix haven't solve problem. Same error.<br>
> <br>
> Any ideas why such strange behavior?<br>
> Does Wien2k have problems scaling to multiple nodes?<br>
> <br>
> I would appreciate your help. I want to speed up calculations for <br>
> complex structures, I have the resources, but I can't do it.<br>
> <br>
> <br>
> <br>
> _______________________________________________<br>
> Wien mailing list<br>
> Wien@zeus.theochem.tuwien.ac.at<br>
> <a href="http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien" data-auth="NotApplicable">
http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien</a><br>
> SEARCH the MAILING-LIST at: <a href="http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html" data-auth="NotApplicable">
http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html</a><br>
<br>
-- <br>
-----------------------------------------------------------------------<br>
Peter Blaha, Inst. f. Materials Chemistry, TU Vienna, A-1060 Vienna<br>
Phone: +43-158801165300<br>
Email: peter.blaha@tuwien.ac.at<br>
WWW: <a href="http://www.imc.tuwien.ac.at" data-auth="NotApplicable">http://www.imc.tuwien.ac.at</a> WIEN2k:
<a href="http://www.wien2k.at" data-auth="NotApplicable">http://www.wien2k.at</a><br>
-------------------------------------------------------------------------<br>
<br>
_______________________________________________<br>
Wien mailing list<br>
Wien@zeus.theochem.tuwien.ac.at<br>
<a href="http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien" data-auth="NotApplicable">http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien</a><br>
SEARCH the MAILING-LIST at: <a href="http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html" data-auth="NotApplicable">
http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html</a><br>
</div>
</span></font></div>
</body>
</html>