<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>Dear Laurence,</p>
<p><br>
</p>
<p>I used 40 k-points. <br>
</p>
<p><br>
</p>
<p>The integration part makes no problems (-mode integ), the memory
consuming part is the current part (-mode current). </p>
<p>Your hint for lapw1 shows even more that it would be safer to use
4 parallel calculations instead of eight without loosing much
perfomance (the 14900k has only 8 performance cores, the other 16
(efficient cores) are slower.<br>
</p>
<p><br>
</p>
<p>Best regards,</p>
<p>Michael</p>
<p><br>
</p>
<div class="moz-cite-prefix">Am 13.05.2024 um 10:14 schrieb Laurence
Marks:<br>
</div>
<blockquote type="cite"
cite="mid:CANkSMZC3iqGUdjSZs0FMMsGT7dfLH=1zj8AsDLfOvGpWois1UQ@mail.gmail.com">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<div dir="ltr">
<div class="gmail_default"
style="font-family:verdana,sans-serif;color:#000000">For my
own curiosity, is it 40,000 k-points or 40 k-points?</div>
<div class="gmail_default"
style="font-family:verdana,sans-serif;color:#000000"><br>
</div>
<div class="gmail_default"
style="font-family:verdana,sans-serif;color:#000000">N.B., as
Peter suggested, did you try using mpi, which would be four
of <span
style="font-family:Arial,Helvetica,sans-serif;color:rgb(34,34,34)">nmr_integ:localhost:2</span></div>
<div class="gmail_default" style="">I suspect (but might be
wrong) that this will reduce you memory useage by a factor of
2, and will only be slightly slower than what you have. If
needed you can also go to 4 mpi. Of course you have to have
compiled it...</div>
<div class="gmail_default" style=""><br>
</div>
<div class="gmail_default" style="">N.N.B., you presumably
realise that you are using 16 cores for lapw1, as each k-point
has 2 cores.</div>
<div class="gmail_default"
style="font-family:verdana,sans-serif;color:#000000"><br>
</div>
<div class="gmail_default"
style="font-family:verdana,sans-serif;color:#000000"><br>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Mon, May 13, 2024 at
4:00 PM Michael Fechtelkord via Wien <<a
href="mailto:wien@zeus.theochem.tuwien.ac.at"
moz-do-not-send="true" class="moz-txt-link-freetext">wien@zeus.theochem.tuwien.ac.at</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote"
style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hello
all,<br>
<br>
<br>
as far as I can see it, a job with 8 cores may be faster, but
uses <br>
double of the space on scratch (8 partial nmr vectors with
size <br>
depending on the kmesh per direction eg. nmr_mqx instead of 4
partial <br>
vectors) and that also doubles the RAM usage of the NMR
current <br>
calculation because 8 partial vectors per direction are used.<br>
<br>
I will try the -quota 8 option, but currently it seems that
calculations <br>
on eight cores are at high risk to crash because of the
memory and <br>
scratch space it needs and that already for 40k points. I
never had <br>
problems with calculations on 4 cores even with only 64 GB RAM
and 1000k <br>
points.<br>
<br>
<br>
Best regards,<br>
<br>
Michael<br>
<br>
<br>
Am 12.05.2024 um 18:02 schrieb Michael Fechtelkord via Wien:<br>
> It shows EXECUTING: /usr/local/WIEN2k/nmr_mpi -case
MS_2M1_Al2 <br>
> -mode current -green -scratch /scratch/WIEN2k/
-noco<br>
><br>
> in all cases and in htop the values I provided below.<br>
><br>
><br>
> Best regards,<br>
><br>
> Michael<br>
><br>
><br>
> Am 12.05.2024 um 16:01 schrieb Peter Blaha:<br>
>> This makes sense.<br>
>> Please let me know if it shows<br>
>><br>
>> EXECUTING: /usr/local/WIEN2k/nmr_mpi -case
MS_2M1_Al2 -mode <br>
>> current -green -scratch /scratch/WIEN2k/
-noco<br>
>><br>
>> or only nmr -case ...<br>
>><br>
>> In any case, it is running correctly.<br>
>><br>
>> PS: I know that also the current step needs a lot of
memory, after <br>
>> all it needs to read the eigenvectors of all
eigenvalues, ...<br>
>><br>
>> PPS: -quota 8 (or 24) might help and still
utilizing all cores, <br>
>> but I'm not sure if it would save enough memory in
the current steps.<br>
>><br>
>><br>
>><br>
>> Am 12.05.2024 um 10:09 schrieb Michael Fechtelkord
via Wien:<br>
>>> Hello all, hello Peter,<br>
>>><br>
>>><br>
>>> That is what is really running in the background
(from htop: this is <br>
>>> a new job with 4 nodes but it was the same with 8
nodes -p 1 - 8), <br>
>>> so no nmr_mpi.<br>
>>><br>
>>><br>
>>> TIME+ Command<br>
>>><br>
>>> 96.0 14.9 19h06:05 /usr/local/WIEN2k/nmr -case
MS_2M1_A12 -mode <br>
>>> current -green -scratch /scratch/WIEN2k/ -noco -p
3<br>
>>><br>
>>> 95.8 14.9 19h05:10 /usr/local/WIEN2k/nmr -case
MS_2M1_A12 -mode <br>
>>> current -green -scratch /scratch/WIEN2k/ -noco -p
1<br>
>>><br>
>>> 95.1 14.9 19h06:00 /usr/local/WIEN2k/nmr -case
MS_2M1_A12 -mode <br>
>>> current -green -scratch /scratch/WIEN2K/ -noco -p
2<br>
>>><br>
>>> 95.5 15.4 19h08:10 /usr/local/WIEN2k/nmr -case
MS_2M1_A12 -mode <br>
>>> current -green -scratch /scratch/WIEN2k/ -noco -p
4<br>
>>><br>
>>> 94.6 14.9 18h35:33 /usr/local/WIEN2k/nmr -case
MS_2M1_A12 -mode <br>
>>> current -green -scratch /scratch/WIEN2k/ -noco -p
3<br>
>>><br>
>>> 93.3 15.4 18h36:24 /usr/local/WIEN2k/nmr-case
MS_2M1_Al2 -mode <br>
>>> current -green -scratch /scratch/WIEN2k/ -noco -p
4<br>
>>><br>
>>> 93.3 14.9 18h33:02 /usr/local/WIEN2k/nmr-case
MS_2M1_A12 -mode <br>
>>> current -green -scratch/scratch/WIEN2k/ -noco -p2<br>
>>><br>
>>> 94.0 14.9 18h38:44 /usr/local/WIEN2k/nmr -case
MS_2M1_A12 -mode <br>
>>> current -green -scratch /scratch/WIEN2k/ -noco -p
1<br>
>>><br>
>>><br>
>>> Regards,<br>
>>><br>
>>> Michael<br>
>>><br>
>>><br>
>>> Am 11.05.2024 um 20:10 schrieb Michael
Fechtelkord via Wien:<br>
>>>> Hello Peter,<br>
>>>><br>
>>>><br>
>>>> I just use "x_nmr_lapw -p" and the rest is
initiated by the nmr <br>
>>>> script. The Line "/usr/local/WIEN2k/nmr_mpi
-case MS_2M1_Al2 -mode <br>
>>>> current -green -scratch
/scratch/WIEN2k/ -noco " is just <br>
>>>> part of the whole procedure and not initiated
by me manually.. (I <br>
>>>> only copied the last lines of the
calculation).<br>
>>>><br>
>>>><br>
>>>> Best regards,<br>
>>>><br>
>>>> Michael<br>
>>>><br>
>>>><br>
>>>> Am 11.05.2024 um 18:08 schrieb Peter Blaha:<br>
>>>>> Hallo Michael,<br>
>>>>><br>
>>>>> I don't understand the line:<br>
>>>>><br>
>>>>> /usr/local/WIEN2k/nmr_mpi -case
MS_2M1_Al2 -mode current <br>
>>>>> -green -scratch /scratch/WIEN2k/
-noco<br>
>>>>><br>
>>>>> The mode current should run only
k-parallel, not in mpi ??<br>
>>>>><br>
>>>>> PS: The repetition of<br>
>>>>><br>
>>>>> nmr_integ:localhost is useless.<br>
>>>>><br>
>>>>> nmr mode integ runs only once (not
k-parallel, sumpara has already <br>
>>>>> summed up the currents)<br>
>>>>><br>
>>>>> But one can use
nmr_integ:localhost:8<br>
>>>>><br>
>>>>><br>
>>>>> Best regards<br>
>>>>><br>
>>>>> Am 11.05.2024 um 16:19 schrieb Michael
Fechtelkord via Wien:<br>
>>>>>> Hello Peter,<br>
>>>>>><br>
>>>>>> this is the .machines file content:<br>
>>>>>><br>
>>>>>> granulartity:1<br>
>>>>>> omp_lapw0:8<br>
>>>>>> omp_global:2<br>
>>>>>> 1:localhost<br>
>>>>>> 1:localhost<br>
>>>>>> 1:localhost<br>
>>>>>> 1:localhost<br>
>>>>>> 1:localhost<br>
>>>>>> 1:localhost<br>
>>>>>> 1:localhost<br>
>>>>>> 1:localhost<br>
>>>>>> nmr_integ:localhost<br>
>>>>>> nmr_integ:localhost<br>
>>>>>> nmr_integ:localhost<br>
>>>>>> nmr_integ:localhost<br>
>>>>>> nmr_integ:localhost<br>
>>>>>> nmr_integ:localhost<br>
>>>>>> nmr_integ:localhost<br>
>>>>>> nmr_integ:localhost<br>
>>>>>><br>
>>>>>><br>
>>>>>> Best regards,<br>
>>>>>><br>
>>>>>> Michael<br>
>>>>>><br>
>>>>>><br>
>>>>>> Am 11.05.2024 um 14:58 schrieb Peter
Blaha:<br>
>>>>>>> Hmm. ?<br>
>>>>>>><br>
>>>>>>> Are you using k-parallel AND
mpi-parallel ?? This could <br>
>>>>>>> overload the machine.<br>
>>>>>>><br>
>>>>>>> How does the .machines file look
like ?<br>
>>>>>>><br>
>>>>>>><br>
>>>>>>> Am 10.05.2024 um 18:15 schrieb
Michael Fechtelkord via Wien:<br>
>>>>>>>> Dear all,<br>
>>>>>>>><br>
>>>>>>>><br>
>>>>>>>> the following problem occurs
to me using the NMR part of WIEN2k <br>
>>>>>>>> (23.2) on a opensuse LEAP
15.5 Intel platform. WIEN2k was <br>
>>>>>>>> compiled using one-api 2024.1
ifort and gcc 13.2.1. I am using <br>
>>>>>>>> ELPA 2024.03.01, Libxc 6.22,
fftw 3.3.10 and MPICH 4.2.1 and <br>
>>>>>>>> the one-api 2024.1 MKL
libraries. The CPU is a I9 14900k with <br>
>>>>>>>> 24 cores where I use eight
for the calculations. The RAM is 130 <br>
>>>>>>>> Gb and a swap file of 16 GB
on a Samsung PCIE 4.0 NVME SSD. The <br>
>>>>>>>> BUS width is 5600 MT / s.<br>
>>>>>>>><br>
>>>>>>>> The structure is a
layersilicate and to simulate the ratio of <br>
>>>>>>>> Si:Al = 3:1 I use a 1:1:2
supercell currently. The monoclinic <br>
>>>>>>>> symmetry of the new structure
(original is C 2/c) is P 2/c and <br>
>>>>>>>> contains 40 atoms (K, Al, Si,
O, and F).<br>
>>>>>>>><br>
>>>>>>>> I use 3 NMR LOs for K and O
and 10 for Si, Al, and F (where I <br>
>>>>>>>> need the chemical shifts).
The k mesh is 40k points.<br>
>>>>>>>><br>
>>>>>>>> The interesting thing is that
the RAM is sufficient during NMR <br>
>>>>>>>> vector calculations (always
under 100 Gb RAM occupied) and at <br>
>>>>>>>> the beginning of the electron
current calculation. However, the <br>
>>>>>>>> RAM use increases to a
critical point in the calculation and <br>
>>>>>>>> more and more data is
outsourced into the SWAP File which is <br>
>>>>>>>> sometimes 80% occupied.<br>
>>>>>>>><br>
>>>>>>>> As you see this time only one
core failed because of memory <br>
>>>>>>>> overflow. But using 48k
points 3 cores crashed and so the whole <br>
>>>>>>>> current calculation. The
reason is of the crash clear to me. <br>
>>>>>>>> But I do not understand, why
the current calculation reacts so <br>
>>>>>>>> sensitive with so few atoms
and a small k mesh. I made <br>
>>>>>>>> calculations with more atoms
and a 1000K point mesh on 4 cores <br>
>>>>>>>> .. they worked fine. So can
it be that the Intel MKL library is <br>
>>>>>>>> the source of failure? So I
better get back to 4 cores, even <br>
>>>>>>>> with longer calculation
times?<br>
>>>>>>>><br>
>>>>>>>> Have all a nice weekend!<br>
>>>>>>>><br>
>>>>>>>><br>
>>>>>>>> Best wishes from<br>
>>>>>>>><br>
>>>>>>>> Michael Fechtelkord<br>
>>>>>>>><br>
>>>>>>>>
-----------------------------------------------<br>
>>>>>>>><br>
>>>>>>>> cd ./ ... x lcore -f
MS_2M1_Al2<br>
>>>>>>>> CORE END<br>
>>>>>>>> 0.685u 0.028s 0:00.71
98.5% 0+0k 2336+16168io 5pf+0w<br>
>>>>>>>><br>
>>>>>>>> lcore .... ready<br>
>>>>>>>><br>
>>>>>>>><br>
>>>>>>>> EXECUTING:
/usr/local/WIEN2k/nmr_mpi -case MS_2M1_Al2 <br>
>>>>>>>> -mode current
-green -scratch /scratch/WIEN2k/ -noco<br>
>>>>>>>><br>
>>>>>>>> [1] 20253<br>
>>>>>>>> [2] 20257<br>
>>>>>>>> [3] 20261<br>
>>>>>>>> [4] 20265<br>
>>>>>>>> [5] 20269<br>
>>>>>>>> [6] 20273<br>
>>>>>>>> [7] 20277<br>
>>>>>>>> [8] 20281<br>
>>>>>>>> [8] +
Abgebrochen ( cd $dir; $exec2 >> <br>
>>>>>>>> nmr.out.${loop} ) >&
nmr.err.$loop<br>
>>>>>>>> [7] +
Fertig ( cd $dir; $exec2 >> <br>
>>>>>>>> nmr.out.${loop} ) >&
nmr.err.$loop<br>
>>>>>>>> [6] +
Fertig ( cd $dir; $exec2 >> <br>
>>>>>>>> nmr.out.${loop} ) >&
nmr.err.$loop<br>
>>>>>>>> [5] +
Fertig ( cd $dir; $exec2 >> <br>
>>>>>>>> nmr.out.${loop} ) >&
nmr.err.$loop<br>
>>>>>>>> [4] +
Fertig ( cd $dir; $exec2 >> <br>
>>>>>>>> nmr.out.${loop} ) >&
nmr.err.$loop<br>
>>>>>>>> [3] +
Fertig ( cd $dir; $exec2 >> <br>
>>>>>>>> nmr.out.${loop} ) >&
nmr.err.$loop<br>
>>>>>>>> [2] +
Fertig ( cd $dir; $exec2 >> <br>
>>>>>>>> nmr.out.${loop} ) >&
nmr.err.$loop<br>
>>>>>>>> [1] +
Fertig ( cd $dir; $exec2 >> <br>
>>>>>>>> nmr.out.${loop} ) >&
nmr.err.$loop<br>
>>>>>>>><br>
>>>>>>>> EXECUTING:
/usr/local/WIEN2k/nmr -case MS_2M1_Al2 -mode <br>
>>>>>>>> sumpara -p 8 -green
-scratch /scratch/WIEN2k/<br>
>>>>>>>><br>
>>>>>>>><br>
>>>>>>>> current .... ready<br>
>>>>>>>><br>
>>>>>>>><br>
>>>>>>>> EXECUTING: mpirun -np 1
-machinefile .machine_nmrinteg <br>
>>>>>>>> /usr/local/WIEN2k/nmr_mpi
-case MS_2M1_Al2 -mode integ -green<br>
>>>>>>>><br>
>>>>>>>><br>
>>>>>>>> nmr: integration ... done
in 4032.3s<br>
>>>>>>>><br>
>>>>>>>><br>
>>>>>>>> stop<br>
>>>>>>>><br>
>><br>
> _______________________________________________<br>
> Wien mailing list<br>
> <a href="mailto:Wien@zeus.theochem.tuwien.ac.at"
target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">Wien@zeus.theochem.tuwien.ac.at</a><br>
> <a
href="http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien"
rel="noreferrer" target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien</a><br>
> SEARCH the MAILING-LIST at: <br>
> <a
href="http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html"
rel="noreferrer" target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html</a><br>
<br>
-- <br>
Dr. Michael Fechtelkord<br>
<br>
Institut für Geologie, Mineralogie und Geophysik<br>
Ruhr-Universität Bochum<br>
Universitätsstr. 150<br>
D-44780 Bochum<br>
<br>
Phone: +49 (234) 32-24380<br>
Fax: +49 (234) 32-04380<br>
Email: <a
href="mailto:Michael.Fechtelkord@ruhr-uni-bochum.de"
target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">Michael.Fechtelkord@ruhr-uni-bochum.de</a><br>
Web Page: <a
href="https://www.ruhr-uni-bochum.de/kristallographie/kc/mitarbeiter/fechtelkord/"
rel="noreferrer" target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">https://www.ruhr-uni-bochum.de/kristallographie/kc/mitarbeiter/fechtelkord/</a><br>
<br>
_______________________________________________<br>
Wien mailing list<br>
<a href="mailto:Wien@zeus.theochem.tuwien.ac.at"
target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">Wien@zeus.theochem.tuwien.ac.at</a><br>
<a
href="http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien"
rel="noreferrer" target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien</a><br>
SEARCH the MAILING-LIST at: <a
href="http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html"
rel="noreferrer" target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html</a><br>
</blockquote>
</div>
<br clear="all">
<div><br>
</div>
<span class="gmail_signature_prefix">-- </span><br>
<div dir="ltr" class="gmail_signature">
<div dir="ltr">Professor Laurence Marks (Laurie)
<div>Northwestern University<br>
<div><a href="http://www.numis.northwestern.edu"
target="_blank" moz-do-not-send="true">Webpage</a> and <a
href="http://scholar.google.com/citations?user=zmHhI9gAAAAJ&hl=en"
target="_blank" moz-do-not-send="true">Google Scholar
link</a></div>
<div>"Research is to see what everybody else has seen, and
to think what nobody else has thought", Albert
Szent-Györgyi</div>
</div>
</div>
</div>
<br>
<fieldset class="moz-mime-attachment-header"></fieldset>
<pre class="moz-quote-pre" wrap="">_______________________________________________
Wien mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Wien@zeus.theochem.tuwien.ac.at">Wien@zeus.theochem.tuwien.ac.at</a>
<a class="moz-txt-link-freetext" href="http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien">http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien</a>
SEARCH the MAILING-LIST at: <a class="moz-txt-link-freetext" href="http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html">http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html</a>
</pre>
</blockquote>
<pre class="moz-signature" cols="72">--
Dr. Michael Fechtelkord
Institut für Geologie, Mineralogie und Geophysik
Ruhr-Universität Bochum
Universitätsstr. 150
D-44780 Bochum
Phone: +49 (234) 32-24380
Fax: +49 (234) 32-04380
Email: <a class="moz-txt-link-abbreviated" href="mailto:Michael.Fechtelkord@ruhr-uni-bochum.de">Michael.Fechtelkord@ruhr-uni-bochum.de</a>
Web Page: <a class="moz-txt-link-freetext" href="https://www.ruhr-uni-bochum.de/kristallographie/kc/mitarbeiter/fechtelkord/">https://www.ruhr-uni-bochum.de/kristallographie/kc/mitarbeiter/fechtelkord/</a>
</pre>
</body>
</html>