Batch Processing on Unix Systems

Okay – indeed cobra will allow you to use the batch command. So here is how you can use it.

Lets say that you have 4 programs that you want to run sequentially for timing purposes. They have the names normal, optimized, unrolled, and unrolled_optimized. They are all located in the “testing” directory under my cobra account. Create a file, we will name our example “job”, with the contents looking like the following

Contents of “job”:

cd ~pounds/testing

./normal >normal_output

./optimized >optimized_output

./unrolled >unrolled_output

./unrolled_optimized >unrolled_optimized_output

Make sure that job is executable (chmod u+x job). And issue the command

batch <job

If you use the “at -l” command, you will see what other jobs are waiting to execute. Look at the man pages on “batch” and “at” for more assistance. You can use atrm to remove a job. You can verify your commands are running using top. Typically “batch” will only allow jobs to run when the system is relatively quiet.

Be careful – you really don't want to abuse this. I have seen sysadmins get violent with users who take advantage of systems using background processing. I can imagine some of the CS faculty would also not be too happy.

You can also use the “nohup” command to run in background – but it is so easily abused that I would discourage you using this on cobra. The problem with nohup is that you can start as many background processes as you want and they will all timeslice with the processor. The only way to use nohup on a large multiprocessor system is to invoke the “nice” command. This makes your processes run at a lower priority so interactive users don't experience a slowdown in performance when they are doing things like editing code! Here is how you use nohup and nice. Lets say you have an executable named “unrolled”. To run it in background issue the following commands

nohup nice ./unrolled >unrolled_output &

bg

You will have to explicitly kill these from the shell if you need to stop them. Even logging off won't kill them. Type “jobs” to see what you have running in the current shell and use “top” to view any processes you have running from previous shells. Do not abuse this – use batch if at all possible. If you start running many jobs, even “niced”, there could start to be memory contention issues and everybody's performance will hit the floor.