[Wien] k-point parallel job in distributed file system

[B. Yanchitsky]

I had a problem with NFS - large files (~500Mb) not copied correctly and small transfer
rate (something like 100Kb/sec instead of 10Mb/sec). I tried various linux distributions and spent
much time in attempts to locate the problem. After a month it appeared that problem is either
due to on-board ethernet card or, most probably, broken driver for ethernet card.
With another driver and ethernet card i don't have problems.
Check at least, that large files are copied correctly and transfer rate is good.

Regards,
Bogdan

XU ZUO wrote:
> Unfortunately, I am suffered from the instability of the k-point
> parallelization. I understand that this problem is caused by the bad NFS
> performance (problems on read/write latency and synchronization) and that
> adjusting $delay and $sleepy may solve the problem. However, as the cluster
> load and traffic are dynamic, it is better to design adaptive code, which
> can handle this problem dynamically. 
> 
> -----Original Message-----
> From: wien-bounces at zeus.theochem.tuwien.ac.at
> [mailto:wien-bounces at zeus.theochem.tuwien.ac.at] On Behalf Of Stefaan
> Cottenier
> Sent: Thursday, August 17, 2006 7:57 PM
> To: wien at zeus.theochem.tuwien.ac.at
> Subject: Re: [Wien] k-point parallel job in distributed file system
> 
> You say you were able to do a k-point parallel run, but it was slow.  
> This means that all your nodes can access a common place (where your
> case.struct etc. are). Your problem probably is that you have put $SCRATCH
> also in that same directory, which indeed causes a lot of network traffic.
> The solution is easy: either you assign to $SCRATCH a directory that exists
> on all your nodes (often this is the case for /tmp), or -- if that is not
> possible -- you assign on-the-fly the correct workspace directory for the
> node(s) you are submittin to (like in the PBS script from the other reply).
> 
> Stefaan
> 
> 
> 
>>Hello,
>>
>> 	We are trying to do k-point parallel wien2k job in a linux cluster 
>>which has distributed file system. Though we are able to do k-point 
>>parallel calculation, we have a problem in assigning a common work 
>>space ($SCRATCH) to read/write all input/output files. This means 
>>that, for example, if we do a 10 kpoint calculation in 10 nodes, all 
>>the 10 nodes should communicate to the common working area through ssh 
>>to read/write files. This slows down the performance and also the network.
>>So far we have done k-point parallel calculations in supercomputers 
>>with shared memory and hence we never had such a problem.  Is it 
>>possible to do k-point parallel calculations in distributed file 
>>system without any common working area?
>>
>>I have received the following from the system expert here.
>>
>>###
>>Hmm, I've been looking through the jungle of scipts which constitutes 
>>wien2k, and it is clear to me that this way of paralellizing isn't 
>>meant for distributed filesystems (local disks on nodes). Unless the 
>>wien2k people have a solution, I don't think we will get around this 
>>without some major reprogramming. At least it seems so to me, but I 
>>must admit that I don't have the complete overview of todo tasks.
>>
>>Also a quick google of the proble, did not provide a solution.
>>This is very efficient for SMP types of machines, but is a bit ad-hoc 
>>for cluster type computers.
>>On the bright side, it doesn't seem taht the program does a lot of 
>>disk read/write in the long run. Only 10-20 min bursts of 10 megs/sek.
>>####
>>
>>Looking forward your responses to do the computation more efficently.
>>
>>Best regards
>>Ravi
> 
> 
> Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm
> 
> _______________________________________________
> Wien mailing list
> Wien at zeus.theochem.tuwien.ac.at
> http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien
> 
> 
> _______________________________________________
> Wien mailing list
> Wien at zeus.theochem.tuwien.ac.at
> http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien
> 
>