[Wien] slurm script

[Peter Blaha]

> Just to make sure, I have two questions:
> 
> - This script will allow running WIEN2k on slurm cluster without 
> passwordless ssh login?

No, I don't think so.

Yes, this script should create the   .machines file on the fly in CASE.
In your example, probably 32 lines (I doubt that this will be very 
efficient).

run_lapw takes this .machines file and spans parallel jobs according to 
you installation in siteconfig (usually ssh unless you configured this 
differently). And for this you need of course passwordless ssh.


> 
> - Should this script generate .machines file which is present in the 
> CASE directory? Should this .machines file contain a list_of_all_nodes 
> times the tasks_per_node? For N=4 and tasks_per_node=8 this should be 32 
> lines, 8 lines per each node (like the localhost lines when running on 
> desktop machine).
> 
> Best,
> Lukasz
> 
> 
> On 2024-06-17 20:57, Peter Blaha wrote:
>> Below some comments:
>>
>>> I have gotten the following error:
>>>
>>> [lplucin at iffslurm Au-bulk-test]$ more slurm-69063.out
>>> iffcluster0414: Using Ethernet for MPI communication.
>>> SBATCH: Command not found.
>>
>> It looks as if you have a line with
>> SBATCH ...      in your script instead of
>> #SBATCH ....
>>
>> Probably the definition of number of nodes:
>> #SBATCH -N 4        # define number of nodes
>>
>> PS: For slurm
>> #command      are slurm directives,
>> ##command     is a comment
>> command       would be a Linux command, but not a SLURM directive.
>>
>>> DIRECTORY = /local/th1/topo/lplucin/Au-bulk-test
>>> WIENROOT = /Users/lplucin/WIEN2k
>>> SCRATCH = /SCRATCH
>>> SLURM_NTASKS: Undefined variable.
>>
>>>
>>> Then I tried to define:
>>>
>>> setenv SLURM_NTASKS_PER_NODE 8
>>> setenv SLURM_NTASKS 32
>>
>> These variables should be set automatically by the
>> #SBATCH --tasks-per-node=8
>> #SBATCH -N 4        # define number of nodes
>>
>> directives.
>>
>>
>> PS: Your cluster support should have some information/documents how
>> one submits a slurm job and which variables should be set/used.
>>
>> I could imagine that different slurm versions/setups work slightly 
>> different.
>>
>>>
>>> but this did not help much, something started on one node, but 
>>> crashed immediately.
>>>
>>> Any advice would be appreciated!
>>>
>>> Best,
>>> Lukasz
>>>
>>>
>>>
>>>
>>> #!/bin/tcsh
>>> #SBATCH -J compound
>>> #SBATCH --tasks-per-node=8
>>> ##SBATCH --mail-user=xxx at yyy.zzz
>>> ##SBATCH --mail-type=END
>>>
>>> ## The next 3 lines may allow some the definition of specific queues 
>>> depending on your cluster
>>> #SBATCH --partition topo
>>> ##SBATCH --qos=XXX
>>> ##SBATCH --account=XXX
>>> #
>>> #########################
>>> #SBATCH -N 4 # define number of nodes
>>> set mpijob=1 # define number of cores for one lapw1/2 mpi-job
>>> ##
>>> ## The lines above need to be adapted depending our your case (size 
>>> of the
>>> ## problem and number of k-points).
>>> ## The example requests 64 cores on 4 nodes (because core_per_nodes 
>>> is 16),
>>> ## and configures WIEN2k such that 4 k-point-parallel jobs will run 
>>> on 16-cores
>>> ## (one node) in mpi-mode, i.e. it assumes that you have 4, 8, 12 ... 
>>> k-points
>>> ## and the case is of a size that it runs reasonably on 16 cores.
>>> ## For a bigger case and only 1 k-point, you would set mpijob=64 
>>> instead....
>>> #########################
>>>
>>> set cores_per_node=8
>>> setenv OMP_NUM_THREADS 1
>>> set lapw2_vector_split=4
>>> setenv PSM_SHAREDCONTEXTS_MAX 4
>>> setenv SCRATCH /SCRATCH                           # set it to 
>>> recommended value
>>>
>>> _______________________________________________
>>> Wien mailing list
>>> Wien at zeus.theochem.tuwien.ac.at
>>> http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien
>>> SEARCH the MAILING-LIST at: 
>>> http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html
> _______________________________________________
> Wien mailing list
> Wien at zeus.theochem.tuwien.ac.at
> http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien
> SEARCH the MAILING-LIST at:  
> http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html

-- 
--------------------------------------------------------------------------
Peter BLAHA, Inst.f. Materials Chemistry, TU Vienna, A-1060 Vienna
Phone: +43-1-58801-165300
Email: peter.blaha at tuwien.ac.at    WIEN2k: http://www.wien2k.at
WWW:   http://www.imc.tuwien.ac.at
-------------------------------------------------------------------------