[Wien] slurm script

[pluto]

Dear Prof. Blaha,

Thank you for the quick answer. I will be trying to adapt the script 
with the help of our IT experts.

Just to make sure, I have two questions:

- This script will allow running WIEN2k on slurm cluster without 
passwordless ssh login?

- Should this script generate .machines file which is present in the 
CASE directory? Should this .machines file contain a list_of_all_nodes 
times the tasks_per_node? For N=4 and tasks_per_node=8 this should be 32 
lines, 8 lines per each node (like the localhost lines when running on 
desktop machine).

Best,
Lukasz


On 2024-06-17 20:57, Peter Blaha wrote:
> Below some comments:
> 
>> I have gotten the following error:
>> 
>> [lplucin at iffslurm Au-bulk-test]$ more slurm-69063.out
>> iffcluster0414: Using Ethernet for MPI communication.
>> SBATCH: Command not found.
> 
> It looks as if you have a line with
> SBATCH ...      in your script instead of
> #SBATCH ....
> 
> Probably the definition of number of nodes:
> #SBATCH -N 4        # define number of nodes
> 
> PS: For slurm
> #command      are slurm directives,
> ##command     is a comment
> command       would be a Linux command, but not a SLURM directive.
> 
>> DIRECTORY = /local/th1/topo/lplucin/Au-bulk-test
>> WIENROOT = /Users/lplucin/WIEN2k
>> SCRATCH = /SCRATCH
>> SLURM_NTASKS: Undefined variable.
> 
>> 
>> Then I tried to define:
>> 
>> setenv SLURM_NTASKS_PER_NODE 8
>> setenv SLURM_NTASKS 32
> 
> These variables should be set automatically by the
> #SBATCH --tasks-per-node=8
> #SBATCH -N 4        # define number of nodes
> 
> directives.
> 
> 
> PS: Your cluster support should have some information/documents how
> one submits a slurm job and which variables should be set/used.
> 
> I could imagine that different slurm versions/setups work slightly 
> different.
> 
>> 
>> but this did not help much, something started on one node, but crashed 
>> immediately.
>> 
>> Any advice would be appreciated!
>> 
>> Best,
>> Lukasz
>> 
>> 
>> 
>> 
>> #!/bin/tcsh
>> #SBATCH -J compound
>> #SBATCH --tasks-per-node=8
>> ##SBATCH --mail-user=xxx at yyy.zzz
>> ##SBATCH --mail-type=END
>> 
>> ## The next 3 lines may allow some the definition of specific queues 
>> depending on your cluster
>> #SBATCH --partition topo
>> ##SBATCH --qos=XXX
>> ##SBATCH --account=XXX
>> #
>> #########################
>> #SBATCH -N 4 # define number of nodes
>> set mpijob=1 # define number of cores for one lapw1/2 mpi-job
>> ##
>> ## The lines above need to be adapted depending our your case (size of 
>> the
>> ## problem and number of k-points).
>> ## The example requests 64 cores on 4 nodes (because core_per_nodes is 
>> 16),
>> ## and configures WIEN2k such that 4 k-point-parallel jobs will run on 
>> 16-cores
>> ## (one node) in mpi-mode, i.e. it assumes that you have 4, 8, 12 ... 
>> k-points
>> ## and the case is of a size that it runs reasonably on 16 cores.
>> ## For a bigger case and only 1 k-point, you would set mpijob=64 
>> instead....
>> #########################
>> 
>> set cores_per_node=8
>> setenv OMP_NUM_THREADS 1
>> set lapw2_vector_split=4
>> setenv PSM_SHAREDCONTEXTS_MAX 4
>> setenv SCRATCH /SCRATCH                           # set it to 
>> recommended value
>> 
>> _______________________________________________
>> Wien mailing list
>> Wien at zeus.theochem.tuwien.ac.at
>> http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien
>> SEARCH the MAILING-LIST at:  
>> http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html