<div dir="auto"><div>Since you used my name in vain....</div><div dir="auto"><br></div><div dir="auto">The part I added was the Kahan summation. I don't see any reason that should lead to issues.</div><div dir="auto"><br></div><div dir="auto">I have a guess what is going on, particularly as the code already has a 2020 comment about a possible sigsev. My understanding (from a quick search of information) is that each thread keeps a local copy of the sum until the end. I will speculate that with "schedule(guided)" it is splitting the sums into more chunks. It could be that then that there is not enough space for these sub-sums. This would then be consistent with no problem after the first iteration.</div><div dir="auto"><br></div><div dir="auto">Possible solutions are:</div><div dir="auto">1. Change the schedule(guided).</div><div dir="auto">2. Add, borrowed from the top of lapw1.F </div><div dir="auto">#if defined(INTEL_VML) && defined(_OPENMP)</div><div dir="auto"> call KMP_SET_STACKSIZE_S(100000000)</div><div dir="auto">#endif</div><div dir="auto">I am not sure about this one as the size seems large, it is a guess. See <a href="https://www.cita.utoronto.ca/~merz/intel_c10b/main_cls/mergedProjects/optaps_cls/common/optaps_par_exrt.htm">https://www.cita.utoronto.ca/~merz/intel_c10b/main_cls/mergedProjects/optaps_cls/common/optaps_par_exrt.htm</a> for more, and you might be able to use an environmental variable instead.</div><div dir="auto">3. First trim the non-zero values at the larger k from the loop first without omp. Requires a little coding.</div><div><br></div><div data-smartmail="gmail_signature">___<br>Emeritus Professor Laurence Marks (Laurie)<br>Department of Materials Science and Engineering, Northwestern University<br><a href="http://www.numis.northwestern.edu">www.numis.northwestern.edu</a><br>"Research is to see what everybody else has seen, and to think what nobody else has thought" Albert Szent-Györgyi</div></div><br><div class="gmail_quote gmail_quote_container"><div dir="ltr" class="gmail_attr">On Sun, Jun 8, 2025, 13:05 Fecher, Gerhard <<a href="mailto:fecher@uni-mainz.de">fecher@uni-mainz.de</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hallo Michael,<br>
<br>
>From the comments (*** LDM CHANGES ***), it seems that Laurence added or changed the omp part, I guess he may know what is potentially going wrong<br>
<br>
The code following line 1356 is very roughly similar to that at 1644 ff but no segmentation fault appears, but has the same comment by jdoumont.<br>
<br>
The segmentation fault appears with ifx (from 2025.2) but not with ifort (last Version 2025.2) as far as I remember.<br>
<br>
Note:<br>
If you check from your github link the files bad.txt, fix.txt, and good.txt then you see that they just removed the bad line in good.txt by commenting it for test, thats all.<br>
in fix.txt they removed the $omp directives, what is the same that I did already at the beginning of the year (see my post "ifx OMP Problem in lapw0 ...." from 2nd January)<br>
However, this just prevents the problem but does not heal it !<br>
<br>
I guess good.txt is just to locate the errorneous file and is not a suggestion to remove this line !!<br>
<br>
Ciao<br>
Gerhard<br>
<br>
DEEP THOUGHT in D. Adams; Hitchhikers Guide to the Galaxy:<br>
"I think the problem, to be quite honest with you,<br>
is that you have never actually known what the question is."<br>
<br>
====================================<br>
Dr. Gerhard H. Fecher<br>
Institut of Physics<br>
Johannes Gutenberg - University<br>
55099 Mainz<br>
________________________________________<br>
Von: Wien [<a href="mailto:wien-bounces@zeus.theochem.tuwien.ac.at" target="_blank" rel="noreferrer">wien-bounces@zeus.theochem.tuwien.ac.at</a>] im Auftrag von Michael Fechtelkord via Wien [<a href="mailto:wien@zeus.theochem.tuwien.ac.at" target="_blank" rel="noreferrer">wien@zeus.theochem.tuwien.ac.at</a>]<br>
Gesendet: Sonntag, 8. Juni 2025 13:13<br>
An: A Mailing list for WIEN2k users<br>
Cc: Michael Fechtelkord<br>
Betreff: Re: [Wien] New findings on the lapw0 seg fault core dump error<br>
<br>
Hello Gerhard and Peter,<br>
<br>
<br>
I am using ifx 2025.1.1 and I also read that OpenMP reductions cause a<br>
segfault using Intel compilers. They recommend serializing the loops or<br>
removing the line that performs the reduction eliminate the segfault.<br>
<br>
<a href="https://github.com/flang-compiler/flang/issues/56" rel="noreferrer noreferrer" target="_blank">https://github.com/flang-compiler/flang/issues/56</a><br>
<br>
<br>
I have answered Peter's question below inserted between his comments.<br>
<br>
So can I comment the reduction procedure out (it is not needed?).<br>
Serializing in the first cycle I did already by setting omp_lapw0:1.<br>
After the first cycle lapw0 runs smooth even with 8 omp_threads.<br>
<br>
<br>
Best regards,<br>
<br>
Michael<br>
<br>
<br>
Am 08.06.2025 um 10:27 schrieb Fecher, Gerhard:<br>
> Dear Peter and Michael,<br>
> I receive the segmentation fault with OneAPI 2024.2 and OneAPI 2025.1<br>
> it appears already with -O1<br>
><br>
> I mentioned already some time ago: when I comment the $omp directives at lines 1649 ff. then the program runs smooth.<br>
><br>
> It seems that this is an old unresolved problem, as it is mentioned in a comment by jdoumont 30/7/20<br>
> (however, it seems not to depend on the size of the calculation)<br>
><br>
> Ciao<br>
> Gerhard<br>
><br>
> DEEP THOUGHT in D. Adams; Hitchhikers Guide to the Galaxy:<br>
> "I think the problem, to be quite honest with you,<br>
> is that you have never actually known what the question is."<br>
><br>
> ====================================<br>
> Dr. Gerhard H. Fecher<br>
> Institut of Physics<br>
> Johannes Gutenberg - University<br>
> 55099 Mainz<br>
> ________________________________________<br>
> Von: Wien [<a href="mailto:wien-bounces@zeus.theochem.tuwien.ac.at" target="_blank" rel="noreferrer">wien-bounces@zeus.theochem.tuwien.ac.at</a>] im Auftrag von Peter Blaha [<a href="mailto:peter.blaha@tuwien.ac.at" target="_blank" rel="noreferrer">peter.blaha@tuwien.ac.at</a>]<br>
> Gesendet: Samstag, 7. Juni 2025 20:40<br>
> An: <a href="mailto:wien@zeus.theochem.tuwien.ac.at" target="_blank" rel="noreferrer">wien@zeus.theochem.tuwien.ac.at</a><br>
> Betreff: Re: [Wien] New findings on the lapw0 seg fault core dump error<br>
><br>
> Very curious.<br>
><br>
> Is "number of PW" in case.clmsum after init_lapw and after the<br>
> first cycle identical ?<br>
Number of PW is 2239 in the starting case.clmsum as well as in the<br>
case.clmsum after the first cycle<br>
><br>
> Since this is a small case: Can you manually look at the<br>
> Fouriercoefficients in clmsum. Any "huge" numbers ? Any *** numbers,<br>
No big numbers, no ****<br>
><br>
> After dstart, I guess none of the FK are zero. After mixer (after 1st<br>
> iteration) the later ones should be zero.<br>
><br>
> My guess is a problem in the libthread library of your compiler version<br>
> (ifx 2025.xxx ?). The problems did not show up with previous compilers ?<br>
I am using ifx 2025.1.1<br>
><br>
><br>
> Am 07.06.2025 um 18:18 schrieb Michael Fechtelkord via Wien:<br>
>> smiles .. no it is MgF2.. Just two atoms in a cubic cell. and it is not<br>
>> dependent on the structure. It crashes for all in the first cycle using<br>
>> the clmsum from the init_lapw<br>
>><br>
>> Am 07.06.2025 um 17:34 schrieb Peter Blaha:<br>
>>> Is this a big supercell ?<br>
>>><br>
>>> The only thing I could imagine is that the number of PWs is bigger<br>
>>> after dstart than after the 1st cycle.<br>
>>> grep for "PW" in the clmsum files from dstart and after the 1st cycle.<br>
>>> Eventually reduce number of PW until it works as a temporary fix.<br>
>>> It might be a "stack" problem and I think one can increase this<br>
>>> somehow, but I can't remember how.<br>
>>><br>
>>> Am 06.06.2025 um 22:25 schrieb Michael Fechtelkord via Wien:<br>
>>>> and a additional comment.<br>
>>>><br>
>>>><br>
>>>> lapw0 crashes only in the first cycle with OMP_NUM_THREADS higher<br>
>>>> than 1. When I set lapw0:1 for the first cycle (using -i 1 in<br>
>>>> run_lapw) and then after the first run set it back to lapw0:8 it runs<br>
>>>> without a problem for the complete scf cycle. It seems that is a<br>
>>>> problem with the initial case.clmsum file (init_lapw -b -prec 1).<br>
>>>><br>
>>>><br>
>>>> Am 06.06.2025 um 22:07 schrieb Michael Fechtelkord via Wien:<br>
>>>>> Hello Peter,<br>
>>>>><br>
>>>>><br>
>>>>> omp_lapw0 in .machines was 8. I reduced it from 8 to 4, then to 2<br>
>>>>> and finally to 1. Only in the case of omp_lapw0:1 lapw0 does not crash.<br>
>>>>><br>
>>>>> omp_global:2<br>
>>>>><br>
>>>>><br>
>>>>> Best regards,<br>
>>>>><br>
>>>>> Michael<br>
>>>>><br>
>>>>><br>
>>>>> Am 06.06.2025 um 17:59 schrieb Peter Blaha:<br>
>>>>>> What was your OMP_NUM_THREADS variable ?<br>
>>>>>><br>
>>>>>> Set it to 1, 2, ... and check if the error occurs again.<br>
>>>>>><br>
>>>>>> Am 06.06.2025 um 14:07 schrieb Michael Fechtelkord via Wien:<br>
>>>>>>> I debugged the core-dump file with gdb and using debugging symbols<br>
>>>>>>> in compilation of lapw0.<br>
>>>>>>><br>
>>>>>>> The debugger gave me the line which causes the coredump<br>
>>>>>>><br>
>>>>>>> _----------------------------------------<br>
>>>>>>><br>
>>>>>>> Debuginfod has been enabled.<br>
>>>>>>> To make this setting permanent, add 'set debuginfod enabled on'<br>
>>>>>>> to .gdbinit.<br>
>>>>>>> [Thread debugging using libthread_db enabled]<br>
>>>>>>> Using host libthread_db library "/lib64/libthread_db.so.1".<br>
>>>>>>> Core was generated by `/usr/local/WIEN2k/lapw0 lapw0.def'.<br>
>>>>>>> Program terminated with signal SIGSEGV, Segmentation fault.<br>
>>>>>>><br>
>>>>>>> #0 0x000000000048b89b in<br>
>>>>>>> MAIN__.DIR.OMP.PARALLEL.LOOP.12.split63842.split63939 ()*at<br>
>>>>>>> lapw0.F:1649*<br>
>>>>>>><br>
>>>>>>> *1649 !$omp parallel do reduction(+:rhopw00,cwk,cvout) &*<br>
>>>>>>><br>
>>>>>>><br>
>>>>>>> [Current thread is 1 (Thread 0x14823edbe740 (LWP 339344))]<br>
>>>>>>><br>
>>>>>>> ------------------------------------<br>
>>>>>>><br>
>>>>>>> Maybe somebody has an idea how to fix it..<br>
>>>>>>><br>
>>>>>>><br>
>>>>>>> Best regards<br>
>>>>>>><br>
>>>>>>> Michael<br>
>>>>>>><br>
>>>>>>><br>
>>>>>>> Am 17.05.2025 um 13:48 schrieb Michael Fechtelkord via Wien:<br>
>>>>>>>> Hello everybody,<br>
>>>>>>>><br>
>>>>>>>><br>
>>>>>>>> I have new results considering the lapw0 crash which happens<br>
>>>>>>>> partially (segmentation fault error - core dump).<br>
>>>>>>>><br>
>>>>>>>> It seems that the crucial thing is the case.clmsum file. (I am no<br>
>>>>>>>> expert here) But if this is somehow the key. It can produce the<br>
>>>>>>>> lapw0 so it might be that it is sometimes triggering the lapw0.<br>
>>>>>>>><br>
>>>>>>>> I cal<a href="https://www.google.com/maps/search/culated+MgF2+and+subs?entry=gmail&source=g">culated MgF2 and subs</a>tituted the new generated clmsum by an<br>
>>>>>>>> older one and then there was no crash. I cannot attach them<br>
>>>>>>>> because the file size is too large.<br>
>>>>>>>><br>
>>>>>>>><br>
>>>>>>>> I am not so into debugging, to find out why and where it happens.<br>
>>>>>>>><br>
>>>>>>>><br>
>>>>>>>> Best regards,<br>
>>>>>>>><br>
>>>>>>>> Michael<br>
>>>>>>>><br>
>>>>>>>><br>
>>>>>>> --<br>
>>>>>>> Dr. Michael Fechtelkord<br>
>>>>>>><br>
>>>>>>> Institut für Geologie, Mineralogie und Geophysik<br>
>>>>>>> Ruhr-Universität Bochum<br>
>>>>>>> Universitätsstr. 150<br>
>>>>>>> D-44780 Bochum<br>
>>>>>>><br>
>>>>>>> Phone: +49 (234) 32-24380<br>
>>>>>>> Fax: +49 (234) 32-04380<br>
>>>>>>> <a href="mailto:Email%3AMichael.Fechtelkord@ruhr-uni-bochum.de" target="_blank" rel="noreferrer">Email:Michael.Fechtelkord@ruhr-uni-bochum.de</a><br>
>>>>>>> Web Page:<a href="https://www.ruhr-uni-bochum.de/kristallographie/kc/" rel="noreferrer noreferrer" target="_blank">https://www.ruhr-uni-bochum.de/kristallographie/kc/</a><br>
>>>>>>> mitarbeiter/fechtelkord/<br>
>>>>>>><br>
>>>>>>><br>
>>>>>>> _______________________________________________<br>
>>>>>>> Wien mailing list<br>
>>>>>>> <a href="mailto:Wien@zeus.theochem.tuwien.ac.at" target="_blank" rel="noreferrer">Wien@zeus.theochem.tuwien.ac.at</a><br>
>>>>>>> <a href="http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien" rel="noreferrer noreferrer" target="_blank">http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien</a><br>
>>>>>>> SEARCH the MAILING-LIST at: <a href="http://www.mail-archive.com/" rel="noreferrer noreferrer" target="_blank">http://www.mail-archive.com/</a><br>
>>>>>>> <a href="http://wien@zeus.theochem.tuwien.ac.at/index.html" rel="noreferrer noreferrer" target="_blank">wien@zeus.theochem.tuwien.ac.at/index.html</a><br>
> --<br>
> -----------------------------------------------------------------------<br>
> Peter Blaha, Inst. f. Materials Chemistry, TU Vienna, A-1060 Vienna<br>
> Phone: +43-158801165300<br>
> Email: <a href="mailto:peter.blaha@tuwien.ac.at" target="_blank" rel="noreferrer">peter.blaha@tuwien.ac.at</a><br>
> WWW: <a href="http://www.imc.tuwien.ac.at" rel="noreferrer noreferrer" target="_blank">http://www.imc.tuwien.ac.at</a> WIEN2k: <a href="http://www.wien2k.at" rel="noreferrer noreferrer" target="_blank">http://www.wien2k.at</a><br>
> -------------------------------------------------------------------------<br>
><br>
> _______________________________________________<br>
> Wien mailing list<br>
> <a href="mailto:Wien@zeus.theochem.tuwien.ac.at" target="_blank" rel="noreferrer">Wien@zeus.theochem.tuwien.ac.at</a><br>
> <a href="http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien" rel="noreferrer noreferrer" target="_blank">http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien</a><br>
> SEARCH the MAILING-LIST at: <a href="http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html" rel="noreferrer noreferrer" target="_blank">http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html</a><br>
> _______________________________________________<br>
> Wien mailing list<br>
> <a href="mailto:Wien@zeus.theochem.tuwien.ac.at" target="_blank" rel="noreferrer">Wien@zeus.theochem.tuwien.ac.at</a><br>
> <a href="http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien" rel="noreferrer noreferrer" target="_blank">http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien</a><br>
> SEARCH the MAILING-LIST at: <a href="http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html" rel="noreferrer noreferrer" target="_blank">http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html</a><br>
<br>
--<br>
Dr. Michael Fechtelkord<br>
<br>
Institut für Geologie, Mineralogie und Geophysik<br>
Ruhr-Universität Bochum<br>
Universitätsstr. 150<br>
D-44780 Bochum<br>
<br>
Phone: +49 (234) 32-24380<br>
Fax: +49 (234) 32-04380<br>
Email: <a href="mailto:Michael.Fechtelkord@ruhr-uni-bochum.de" target="_blank" rel="noreferrer">Michael.Fechtelkord@ruhr-uni-bochum.de</a><br>
Web Page: <a href="https://www.ruhr-uni-bochum.de/kristallographie/kc/mitarbeiter/fechtelkord/" rel="noreferrer noreferrer" target="_blank">https://www.ruhr-uni-bochum.de/kristallographie/kc/mitarbeiter/fechtelkord/</a><br>
<br>
_______________________________________________<br>
Wien mailing list<br>
<a href="mailto:Wien@zeus.theochem.tuwien.ac.at" target="_blank" rel="noreferrer">Wien@zeus.theochem.tuwien.ac.at</a><br>
<a href="http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien" rel="noreferrer noreferrer" target="_blank">http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien</a><br>
SEARCH the MAILING-LIST at: <a href="http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html" rel="noreferrer noreferrer" target="_blank">http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html</a><br>
_______________________________________________<br>
Wien mailing list<br>
<a href="mailto:Wien@zeus.theochem.tuwien.ac.at" target="_blank" rel="noreferrer">Wien@zeus.theochem.tuwien.ac.at</a><br>
<a href="http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien" rel="noreferrer noreferrer" target="_blank">http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien</a><br>
SEARCH the MAILING-LIST at: <a href="http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html" rel="noreferrer noreferrer" target="_blank">http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html</a><br>
</blockquote></div>