[Wien] xxmr2d:out of memory / estimate memory consumption

[Peter Blaha]

x lapw1  -nmat_only


can be run quickly after   init_lapw. It runs in fractions of seconds on 
the frontend and just determines the matrix size and prints it into   
case.nmat_only .


 From the matrix size and the knowledge if you have inversion or not 
(case.in1 or in1c) you can estimate the total memory, and also estimate 
how many cores you should use.


You have  NMAT~84000 and no inversion. Memory: 10^5 * 10^5 *16  ~ 10^11 
bytes ~ 100 GB for one matrix.


One needs approximately 3-4 such matrices, so I would request so many 
nodes that you have  at least 400 GB available. Usually you don't have 
this on one node.In addition, one node is much too slow for this.

As I said before, you want sub-matrices bigger than ~2000, but also not 
larger than something like 8000.

Thus you need at least   10x10=100 cores (if they have enough memory; 
and adapt the number to a multiple of your cores/node (36 * 3 ??), but 
much better 20x20 or even 30x30 cores.



Am 04.07.2023 um 17:01 schrieb Ilias, Miroslav:
>
> Greetings,
>
>
> I have the big system of 
> https://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/msg22609.html 
> <https://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/msg22609.html> 
> .
>
>
> After fixing the proper openmpi compilation of wien2k I proceed 
> further into the lapw1_mpi module. But here I got the error 
> "xxmr2d:out of memory" for SBATCH parameters  N=1,  n=36 (that is 36 
> MPI threads), and --mem=450GB.
>
>
> Now I am looking for a method for estimating the total memory 
> consumption, --mem.
>
>
> In the TsOOQg.output1_1 file I see some info about memory spending in 
> MB...Would it be please possible to estimate the SBATCH memory need 
> from this info ?
>
>
>
>
> MPI-parallel calculation using    36 processors
> Scalapack processors array (row,col):   6   6
> Matrix size        84052
> Optimum Blocksize for setup 124 Excess %  0.428D-01
> Optimum Blocksize for diag  30 Excess %  0.143D-01
> Base Blocksize   64 Diagonalization   32
>          allocate H      2997.6 MB          dimensions 14016 14016
>          allocate S      2997.6 MB          dimensions 14016 14016
>     allocate spanel        13.7 MB          dimensions 14016    64
>     allocate hpanel        13.7 MB          dimensions 14016    64
>   allocate spanelus        13.7 MB          dimensions 14016    64
>       allocate slen         6.8 MB          dimensions 14016    64
>         allocate x2         6.8 MB          dimensions 14016    64
>   allocate legendre        89.0 MB          dimensions 14016    13    64
> allocate al,bl (row)         4.7 MB          dimensions 14016    11
> allocate al,bl (col)         0.0 MB          dimensions    64    11
>         allocate YL         3.4 MB          dimensions    15 14016     1
> number of local orbitals, nlo (hamilt)     1401
>       allocate YL          20.5 MB          dimensions    15 84052     1
>       allocate phsc         1.3 MB          dimensions 84052
>
> Time for al,bl    (hamilt, cpu/wall) :        24.49       24.82
> Time for legendre (hamilt, cpu/wall) :         5.19        5.21
>
>
> And lapw1.error :
>
> **  Error in Parallel LAPW1
> **  LAPW1 STOPPED at Tue 04 Jul 2023 10:36:17 AM CEST
> **  check ERROR FILES!
> Error in LAPW1
>
> Best,
>
>
> Miro
>
>
>
>
>
>
>
> _______________________________________________
> Wien mailing list
> Wien at zeus.theochem.tuwien.ac.at
> http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien
> SEARCH the MAILING-LIST at:http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html

-- 
-----------------------------------------------------------------------
Peter Blaha,  Inst. f. Materials Chemistry, TU Vienna, A-1060 Vienna
Phone: +43-158801165300
Email:peter.blaha at tuwien.ac.at           
WWW:http://www.imc.tuwien.ac.at       WIEN2k:http://www.wien2k.at
-------------------------------------------------------------------------
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://zeus.theochem.tuwien.ac.at/pipermail/wien/attachments/20230704/dd84a787/attachment.htm>