<div dir="auto">It would be good to have a brief summary of what matters with hf:<div dir="auto">a) For speed</div><div dir="auto">b) For accuracy</div><div dir="auto"><br></div><div dir="auto">It is probably somewhere in the docu, but another cite for the list would be useful (to me as well as others).<br><br><div data-smartmail="gmail_signature" dir="auto">_____<br>Professor Laurence Marks<br>"Research is to see what everybody else has seen, and to think what nobody else has thought", Albert Szent-Gyorgi<br><a href="http://www.numis.northwestern.edu">www.numis.northwestern.edu</a></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sat, Aug 15, 2020, 07:36 Tran, Fabien <<a href="mailto:fabien.tran@tuwien.ac.at">fabien.tran@tuwien.ac.at</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr" style="font-size:12pt;color:#000000;background-color:#ffffff;font-family:Calibri,Arial,Helvetica,sans-serif">
<p>Since hybrids are expensive, you should do the SCF calculation with let's say a 6x6x6 k-mesh (from experience such mesh should be enough for converged electron density), and then do just one iteration on a denser k-mesh if needed (e.g., for DOS or<span>
transport properties). To do this one iteration with hybrids, you have to use the options "-newklist -i 1" (see user's guide).</span><br>
</p>
<p><br>
</p>
<p>Concerning case.inhf, you should check the convergence of the results with nband in particular.<br>
</p>
<p><br>
</p>
<p><br>
</p>
<div style="word-wrap:break-word;line-break:after-white-space">
<hr style="display:inline-block;width:98%">
<div id="m_2000452282176789355divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" face="Calibri, sans-serif" color="#000000"><b>From:</b> Wien <<a href="mailto:wien-bounces@zeus.theochem.tuwien.ac.at" target="_blank" rel="noreferrer">wien-bounces@zeus.theochem.tuwien.ac.at</a>> on behalf of pboulet <<a href="mailto:pascal.boulet@univ-amu.fr" target="_blank" rel="noreferrer">pascal.boulet@univ-amu.fr</a>><br>
<b>Sent:</b> Saturday, August 15, 2020 2:22 PM<br>
<b>To:</b> A Mailing list for WIEN2k users<br>
<b>Subject:</b> Re: [Wien] Wien post from <a href="mailto:pascal.boulet@univ-amu.fr" target="_blank" rel="noreferrer">pascal.boulet@univ-amu.fr</a> (Errors with DGEMM and hybrid calculations)</font>
<div> </div>
</div>
<div>Hi Fabien,
<div><br>
</div>
<div>Mg2Si is a small structure with 2 irreducible positions in the IBZ. Here is the struct file:</div>
<div>
<div>Mg2Si cubic 225 Fm-3m</div>
<div>F LATTICE,NONEQUIV.ATOMS 2 225 Fm-3m</div>
<div>MODE OF CALC=RELA unit=bohr</div>
<div> 11.999761 11.999761 11.999761 90.000000 90.000000 90.000000</div>
<div>ATOM 1: X=0.00000000 Y=0.00000000 Z=0.00000000</div>
<div> MULT= 1 ISPLIT= 2</div>
<div>Si NPT= 781 R0=.000050000 RMT= 2.39 Z: 14.00000</div>
<div>LOCAL ROT MATRIX: 1.0000000 0.0000000 0.0000000</div>
<div> 0.0000000 1.0000000 0.0000000</div>
<div> 0.0000000 0.0000000 1.0000000</div>
<div>ATOM 2: X=0.25000000 Y=0.25000000 Z=0.25000000</div>
<div> MULT= 2 ISPLIT= 2</div>
<div> 2: X=0.25000000 Y=0.25000000 Z=0.75000000</div>
<div>Mg NPT= 781 R0=.000050000 RMT= 2.50000 Z: 12.00000</div>
<div>LOCAL ROT MATRIX: 1.0000000 0.0000000 0.0000000</div>
<div> 0.0000000 1.0000000 0.0000000</div>
<div> 0.0000000 0.0000000 1.0000000</div>
<div> 48 NUMBER OF SYMMETRY OPERATIONS</div>
<div><br>
</div>
<div>The number of k-points is 72 (12x12x12), RmtKmax is 7, GMAX=12 (for test purpose, usually I set it to 24).</div>
<div><br>
</div>
<div>Best,</div>
<div>Pascal</div>
<div><br>
</div>
<div><br>
</div>
<div>
<div dir="auto" style="color:rgb(0,0,0);letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none;word-wrap:break-word;line-break:after-white-space">
<div dir="auto" style="color:rgb(0,0,0);letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none;word-wrap:break-word;line-break:after-white-space">
<div dir="auto" style="color:rgb(0,0,0);letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none;word-wrap:break-word;line-break:after-white-space">
<div dir="auto" style="color:rgb(0,0,0);letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none;word-wrap:break-word;line-break:after-white-space">
<div dir="auto" style="color:rgb(0,0,0);letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none;word-wrap:break-word;line-break:after-white-space">
<div dir="auto" style="color:rgb(0,0,0);letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none;word-wrap:break-word;line-break:after-white-space">
<div dir="auto" style="text-align:start;text-indent:0px;word-wrap:break-word;line-break:after-white-space">
<div style="color:rgb(0,0,0);letter-spacing:normal;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none">
<span style="font-size:16pt;font-family:Mistral">Pascal Boulet</span></div>
<div style="color:rgb(0,0,0);letter-spacing:normal;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none">
<font face="Lucida Handwriting"><span style="font-size:21px">—</span></font></div>
<div style="color:rgb(0,0,0);letter-spacing:normal;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none">
<em style="color:rgb(34,187,234);font-family:verdana;font-size:11px;line-height:22px">Professor in computational materials - DEPARTMENT OF CHEMISTRY</em></div>
<div>
<div style="color:rgb(0,0,0);letter-spacing:normal;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none;font-family:Tahoma,Verdana,Arial,Helvetica,sans-serif;font-size:15px;line-height:22px">
<span style="font-size:11px;font-family:verdana">University of Aix-Marseille </span><span style="font-size:11px;font-family:verdana">- Avenue Escadrille Normandie Niemen - F-13013 Marseille - FRANCE</span></div>
<div style="color:rgb(0,0,0);letter-spacing:normal;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none;font-family:Tahoma,Verdana,Arial,Helvetica,sans-serif;font-size:15px;line-height:22px">
<span style="font-size:11px;font-family:verdana">Tél: +33(0)4 13 55 18 10 - Fax : +33(0)4 13 55 18 50</span></div>
<div style="color:rgb(0,0,0);letter-spacing:normal;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none">
<font face="verdana"><span style="font-size:12px;line-height:22px">Email : </span></font><font face="verdana" color="#22bbea"><span style="font-size:12px;line-height:20px"><a href="mailto:pascal.boulet@univ-amu.fr" target="_blank" rel="noreferrer">pascal.boulet@univ-amu.fr</a></span></font></div>
<div style="font-size:14px"><br>
</div>
</div>
</div>
</div>
</div>
<br>
</div>
<br>
</div>
<br>
</div>
<br>
</div>
<br>
<br>
</div>
<div><br>
<blockquote type="cite">
<div>Le 15 août 2020 à 11:40, Tran, Fabien <<a href="mailto:fabien.tran@tuwien.ac.at" target="_blank" rel="noreferrer">fabien.tran@tuwien.ac.at</a>> a écrit :</div>
<br>
<div>
<div>Hi,<br>
<br>
For calculations on small cells with many k-points it is more preferable (for speediness) to use k-point paralllelization instead of MPI parallelization. And, as mentioned by PB, MPI applied to small matrices may not work.<br>
<br>
Of course, if the number of cores that you have at disposal is larger (twice for instance) than the number of k-points in the IBZ, then you can combine the k-point and MPI parallelizations (two cores for each k-point).<br>
<br>
An example of .machines file for k-point parallelization is (supposing that you have 6 k-points in the IBZ and want to use one machine having 6 cores):<br>
<br>
lapw0: n1071 n1071 n1071 n1071 n1071 n1071<br>
dstart: n1071 n1071 n1071 n1071 n1071 n1071<br>
nlvdw: n1071 n1071 n1071 n1071 n1071 n1071<br>
1:1071<br>
1:1071<br>
1:1071<br>
1:1071<br>
1:1071<br>
1:1071<br>
granularity:1<br>
extrafine:1 <br>
<br>
The six lines "1:1071" mean that lapw1, lapw2 and hf are k-point (no MPI) parallelized (one line fore each core). lapw0, dstart and nlvdw are MPI parallelized. In this example, the omp parallelization is ignored by supposing that OMP_NUM_THREADS is set to 1.<br>
<br>
Besides, I find your computational time with HF as very large. What is the number of atoms in the cell, the number of k-points (plz, specify th n1xn2nx3 k-mesh), RKmax, etc.?<br>
<br>
Best,<br>
FT<br>
<br>
From: Wien <<a href="mailto:wien-bounces@zeus.theochem.tuwien.ac.at" target="_blank" rel="noreferrer">wien-bounces@zeus.theochem.tuwien.ac.at</a>> on behalf of pboulet <<a href="mailto:pascal.boulet@univ-amu.fr" target="_blank" rel="noreferrer">pascal.boulet@univ-amu.fr</a>><br>
Sent: Saturday, August 15, 2020 10:35 AM<br>
To: A Mailing list for WIEN2k users<br>
Subject: Re: [Wien] Wien post from <a href="mailto:pascal.boulet@univ-amu.fr" target="_blank" rel="noreferrer">
pascal.boulet@univ-amu.fr</a> (Errors with DGEMM and hybrid calculations)<br>
<br>
Dear Peter,<br>
<br>
Thank you for your response. It clarifies some points for me.<br>
<br>
I have run another calculation for Mg2Si, which is a small system, on 12 cores (no MKL errors!). The job ran for 12 hours (CPU time limit I set) and made only 3 SCF cycles without converging. <br>
<br>
The .machines file I use looks like this:<br>
<br>
1:1071:12<br>
lapw0: n1071 n1071<br>
dstart: n1071 n1071<br>
nlvdw: n1071 n1071<br>
granularity:1<br>
extrafine:1 <br>
<br>
I guess I am not optimising the number of cores w.r.t. the size of the problem (72 k-points, 14 HF bands, 12 occupied +2 unoccupied).<br>
<br>
I changed the number of processors for 72, hoping for a 1 k-point/core parallelisation and commenting all the lines of .machines except granularity and extrafine. I got less than 1 cycle in 12 hours.<br>
<br>
What should I do to run the HF part on k-points parallelisation only (no mpi)? This point that is not clear for me from the manual.<br>
<br>
Thank you<br>
Best regards<br>
Pascal<br>
<br>
Pascal Boulet<br>
—<br>
Professor in computational materials - DEPARTMENT OF CHEMISTRY<br>
<br>
University of Aix-Marseille - Avenue Escadrille Normandie Niemen - F-13013 Marseille - FRANCE<br>
Tél: +33(0)4 13 55 18 10 - Fax : +33(0)4 13 55 18 50<br>
<a href="mailto:pascal.boulet@univ-amu.fr" target="_blank" rel="noreferrer">Email : pascal.boulet@univ-amu.fr</a><br>
<br>
Le 12 août 2020 à 14:12, Peter Blaha <<a href="mailto:pblaha@theochem.tuwien.ac.at" target="_blank" rel="noreferrer">pblaha@theochem.tuwien.ac.at</a>> a écrit :<br>
<br>
Your message is too big to be accepted.<br>
<br>
Anyway, the DGEMM messages seem to be a relict of the mkl you are using, and most likely is related to the use of too many mpi-cores for such a small matrix. At least when I continue your Mg2Si calculations (in k-parallel mode) the :DIS and :ENE are continuous,
which means that the previous results are ok.<br>
<br>
Concerning hf, I don't know. Again, running this in sequential (k-point parallel) mode is no problems and converges quickly.<br>
<br>
I suggest that you change your setup to a k-parallel run for such small systems.<br>
<br>
Best regards<br>
Peter Blaha<br>
<br>
---------------------------------------------------------------------<br>
Subject:<br>
Errors with DGEMM and hybrid calculations<br>
From:<br>
pboulet <<a href="mailto:pascal.boulet@univ-amu.fr" target="_blank" rel="noreferrer">pascal.boulet@univ-amu.fr</a>><br>
Date:<br>
8/11/20, 6:31 PM<br>
To:<br>
A Mailing list for WIEN2k users <<a href="mailto:wien@zeus.theochem.tuwien.ac.at" target="_blank" rel="noreferrer">wien@zeus.theochem.tuwien.ac.at</a>><br>
<br>
Dear all,<br>
<br>
I have a strange problem with LAPACK. I get an error message with wrong parameters sent to DGEMM, but still wien2k (19.2) seems to converge the scf. Is that possible? What could be the "problem"?<br>
<br>
I have attached an archive containing the summary of the SCF + compilation options + SLURM output file. The error message is in the dayfile.<br>
The same error shows up with Wien2k 18.1.<br>
<br>
Actually this case is a test case for testing hybrid calculations as I have problems with my real case, which is found to be metallic with PBE. At least Mg2Si is a small band gap semiconductor.<br>
<br>
When I go ahead with the HSE06 functional and Mg2Si I get a different error: segmentation fault during the hf run. As Mg2Si is a small system I guess this is not a memory problem: the node is 128GB.<br>
Note that the same problem occurs for my real case file, but the LAPACK problem does not occur.<br>
<br>
The second archive contains some files related to the hybrid calculation.<br>
<br>
Some hints would be welcome as I am completely lost in these (unrelated) errors!<br>
<br>
Thank you.<br>
Best regards,<br>
Pascal<br>
<br>
<br>
Pascal Boulet<br>
-- <br>
<br>
P.Blaha<br>
--------------------------------------------------------------------------<br>
Peter BLAHA, Inst.f. Materials Chemistry, TU Vienna, A-1060 Vienna<br>
Phone: +43-1-58801-165300 FAX: +43-1-58801-165982<br>
Email: <a href="mailto:blaha@theochem.tuwien.ac.at" target="_blank" rel="noreferrer">blaha@theochem.tuwien.ac.at</a> WIEN2k: <a href="http://www.wien2k.at" target="_blank" rel="noreferrer">http://www.wien2k.at</a><br>
WWW: <a href="http://www.imc.tuwien.ac.at/TC_Blaha" target="_blank" rel="noreferrer">http://www.imc.tuwien.ac.at/TC_Blaha</a><br>
--------------------------------------------------------------------------<br>
_______________________________________________<br>
Wien mailing list<br>
<a href="mailto:Wien@zeus.theochem.tuwien.ac.at" target="_blank" rel="noreferrer">Wien@zeus.theochem.tuwien.ac.at</a><br>
<a href="http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien" target="_blank" rel="noreferrer">http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien</a><br>
SEARCH the MAILING-LIST at: <a href="http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html" target="_blank" rel="noreferrer">http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html</a><br>
_______________________________________________<br>
Wien mailing list<br>
<a href="mailto:Wien@zeus.theochem.tuwien.ac.at" target="_blank" rel="noreferrer">Wien@zeus.theochem.tuwien.ac.at</a><br>
<a href="http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien" target="_blank" rel="noreferrer">http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien</a><br>
SEARCH the MAILING-LIST at: <a href="http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html" target="_blank" rel="noreferrer">http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html</a><br>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</div>
_______________________________________________<br>
Wien mailing list<br>
<a href="mailto:Wien@zeus.theochem.tuwien.ac.at" target="_blank" rel="noreferrer">Wien@zeus.theochem.tuwien.ac.at</a><br>
<a href="https://urldefense.com/v3/__http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien__;!!Dq0X2DkFhyF93HkjWTBQKhk!D_jOLQXK_C3fGFJQYomOw3JDdIajaXrWeuomuO8f1qRIDbOqVHcsGyb0iJXYHhWJae5mEg$" rel="noreferrer noreferrer" target="_blank">https://urldefense.com/v3/__http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien__;!!Dq0X2DkFhyF93HkjWTBQKhk!D_jOLQXK_C3fGFJQYomOw3JDdIajaXrWeuomuO8f1qRIDbOqVHcsGyb0iJXYHhWJae5mEg$</a> <br>
SEARCH the MAILING-LIST at: <a href="https://urldefense.com/v3/__http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html__;!!Dq0X2DkFhyF93HkjWTBQKhk!D_jOLQXK_C3fGFJQYomOw3JDdIajaXrWeuomuO8f1qRIDbOqVHcsGyb0iJXYHhU3ic6GGg$" rel="noreferrer noreferrer" target="_blank">https://urldefense.com/v3/__http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html__;!!Dq0X2DkFhyF93HkjWTBQKhk!D_jOLQXK_C3fGFJQYomOw3JDdIajaXrWeuomuO8f1qRIDbOqVHcsGyb0iJXYHhU3ic6GGg$</a> <br>
</blockquote></div>