Project

General

Profile

Introduction to Condor for users » History » Version 3

Miguel Dias Costa, 29/02/2012 15:16

1 1 Miguel Dias Costa
h1. Introduction to Condor for users
2 1 Miguel Dias Costa
3 1 Miguel Dias Costa
Condor is a job submission system that creates a high throughput computing environments.
4 1 Miguel Dias Costa
5 2 Miguel Dias Costa
h2. Getting started
6 2 Miguel Dias Costa
7 1 Miguel Dias Costa
After gaining ssh accesh to a condor frontend, a user can see what resources are available by running
8 1 Miguel Dias Costa
9 3 Miguel Dias Costa
<pre>
10 1 Miguel Dias Costa
condor_status
11 3 Miguel Dias Costa
</pre>
12 2 Miguel Dias Costa
13 2 Miguel Dias Costa
and check the status of the queue by running
14 2 Miguel Dias Costa
15 3 Miguel Dias Costa
<pre>
16 2 Miguel Dias Costa
condor_q
17 3 Miguel Dias Costa
</pre>
18 2 Miguel Dias Costa
19 2 Miguel Dias Costa
In order to submit jobs, one needs to create a submit script that defines the requirements and a few options. A simple example to get started would be
20 2 Miguel Dias Costa
21 2 Miguel Dias Costa
h3. Example job submission script helloworld.submit
22 2 Miguel Dias Costa
23 3 Miguel Dias Costa
<pre>
24 2 Miguel Dias Costa
executable     = helloworld.sh
25 2 Miguel Dias Costa
universe       = vanilla 
26 2 Miguel Dias Costa
27 2 Miguel Dias Costa
should_transfer_files = YES
28 2 Miguel Dias Costa
when_to_transfer_output = ON_EXIT
29 2 Miguel Dias Costa
stream_output = true
30 2 Miguel Dias Costa
31 2 Miguel Dias Costa
transfer_input_files = helloworld.dat
32 2 Miguel Dias Costa
33 2 Miguel Dias Costa
request_cpus  = 1
34 2 Miguel Dias Costa
request_memory  = 8000
35 2 Miguel Dias Costa
36 2 Miguel Dias Costa
requirements   = (target.Arch == "X86_64")
37 2 Miguel Dias Costa
38 2 Miguel Dias Costa
input          = /dev/null
39 2 Miguel Dias Costa
output         = out                
40 2 Miguel Dias Costa
error          = error             
41 2 Miguel Dias Costa
log            = log                                                    
42 2 Miguel Dias Costa
			  
43 2 Miguel Dias Costa
queue                  
44 3 Miguel Dias Costa
</pre>
45 2 Miguel Dias Costa
46 2 Miguel Dias Costa
47 2 Miguel Dias Costa
h3. Example additional file helloworld.dat
48 2 Miguel Dias Costa
49 3 Miguel Dias Costa
<pre>
50 2 Miguel Dias Costa
Hello World!
51 3 Miguel Dias Costa
</pre>
52 2 Miguel Dias Costa
53 2 Miguel Dias Costa
h3. Example executable script helloworld.sh
54 2 Miguel Dias Costa
55 3 Miguel Dias Costa
<pre>
56 2 Miguel Dias Costa
#!/bin/bash
57 2 Miguel Dias Costa
echo "----------------------"
58 2 Miguel Dias Costa
hostname
59 2 Miguel Dias Costa
echo "----------------------"
60 2 Miguel Dias Costa
date
61 2 Miguel Dias Costa
echo "----------------------"
62 2 Miguel Dias Costa
echo "Sleeping 20s"
63 2 Miguel Dias Costa
sleep 20
64 2 Miguel Dias Costa
echo "----------------------"
65 2 Miguel Dias Costa
cat helloworld.dat
66 3 Miguel Dias Costa
</pre>
67 2 Miguel Dias Costa
68 2 Miguel Dias Costa
After creating these 3 files, the job would be submitted by running
69 2 Miguel Dias Costa
70 2 Miguel Dias Costa
condor_submit helloworld.submit
71 2 Miguel Dias Costa
72 2 Miguel Dias Costa
h2. Checking status
73 2 Miguel Dias Costa
74 2 Miguel Dias Costa
To see details about the status of each job in the queue, one would run
75 2 Miguel Dias Costa
76 3 Miguel Dias Costa
<pre>
77 2 Miguel Dias Costa
condor_q -b
78 3 Miguel Dias Costa
</pre>
79 2 Miguel Dias Costa
80 2 Miguel Dias Costa
When a job is running, it is also possible to access the node it is using, by identifying the jobid with condor_q and then running
81 2 Miguel Dias Costa
82 3 Miguel Dias Costa
<pre>
83 2 Miguel Dias Costa
condor_ssh_to_job jobid
84 3 Miguel Dias Costa
</pre>
85 2 Miguel Dias Costa
86 2 Miguel Dias Costa
h2. More Information
87 2 Miguel Dias Costa
88 2 Miguel Dias Costa
http://research.cs.wisc.edu/condor/manual/v7.4/ref.html