How to use the BIRD (Batch Infrastucture Ressource at DESY)
Read http://www-flc.desy.de/flc/work/group/groupmeetingTalks/2008/080128-BL_computingFLC.pdf (especially the BIRD part, but the rest is also very good to know!)
- Ask your admin to register you for BIRD and space on the pool disks
Read http://www-it.desy.de/systems/services/batch/sge/ and maybe even http://www-it.desy.de/systems/services/batch/sge/DOCS/user_guide.pdf
There is a bug in the setting.sh which leads to problems if you use a modified Z Shell color scheme. See here for help.
If you encounter the following using grid commands
t2145220288:p28936: Fatal error: [Thread System] GLOBUSTHREAD: pthread_create() failed [Thread System] insufficient memory (ENOMEM)zsh: 28936 abort (core dumped
Try to limit your stack size (zsh):
$ ulimit -s 100000
Anyhow submitting a job is very easy. You write a shell script which contains all the commands to be executed on the worker node. They are processed one after the other, like a batch (dt. Stapel), this is where the name comes from.
If you need more help have a look at the Batch Cluster For Dummies page.
Beside the commands there are a few switches, for example if you only want to run on 32-bit machines. You can either supply these switches as command line arguments, or you can include them inside your submitted shell script in the form of special comments that begin with the #$ character sequence.
# job-test.sh #$ -l arch=x86 #$ -l distro=sld4 echo "job test" > job-test.log
Submit this job with
qsub job-test.sh
and figure out where the job-test.log appears.
monitoring jobs
If you're not satisfied with the output of qmon or qstat, you might want to visit http://www.desy.de/~blutz/cluster/cluster.xml to see a list of the current job queues. The data is updated every 5 minutes.
excluding hosts
If you have the feeling that your jobs fail on a particular host, you can exclude it. (But don’t forget to inform the administrators, so that they can fix the problem for the sake of those who will come after you.)
-l h=\!fishyhost
See man 5 complex for more options. Note that this will only give you “complex – Sun Grid Engine complexes configuration file format” after sourcing the setup file /usr/sge/default/common/settings.sh. On a normal host, you may get “complex – basics of complex mathematics” instead.
dCache problems
If you have problems accessing your files see: