RE: The cron problem and a solution

[George Bonser <grep@oriole.sbay.org>]

On 01-Feb-98 jfm2@club-internet.fr wrote:
> 
> The best we can do is to perform maintenance tasks when the load is low.

Another alternative is to run a process at regular intervals from cron that
checks to see if certain housekeeping chores have been dine recently.  If not,
it can do thise things (roll logs, update the databases, etc) at a very low
priority using nice.  In this way the system performance is not adversely
affected since the housekeeping process only runs when nothing else is
requesting CPU time.

atrun might also be useful since you can specify several different queues with
different nice levels.  You put the lowest priority tasks in a queue with the
highest numeric nice level. This also has the advantage of making it easier to
identify tasks since you only assign certain tasks to certain queues.  Tasks in
the w queue might be log rollers, x queue might be database updaters, etc.

> 
> The solution I propose is to use self-resubmitting batch jobs.  Batch
> will not start a job when the load is high and in addition it will not
> start it until the term is expired.

By using nice levels of the jobs, you can pretty much ignore system load.  The
submitted jobs will just take a back seat to higher priority jobs when they are
running.  This will cause the submitted job to take longer to complete but
would not affect the completion time of user jobs.

> A problem is what happens if a
> computer crashes at the wrong momenyt so the job cannot resubmit
> itself.  The solution is to check at boot time than all maintence jobs
> are queued and resubmit if necessary.

Sounds good. It could also be cron using nice to actually execute the job.

> However the Unix "batch"
> program is very primitive: to begin with it does not support the
> jobname concept.  The only solution I see for knowing if a job is
> present in the queue is to grep for a magic comment in the files
> present in the batch queue.  Of course we should have to check
> ownership to avoid security problems.

Put different types of jobs into different job queues. You get a-z and A-Z or
52 different queues to work with.
 
> Because the traditional scheme is better for a continuously powered on
> machine the user should be asked at install time if the computer will
> be supposed to run 24h a day.

Yes.  This is important.  Another method would be to do the housekeeping at
shutdown.


George Bonser 
If NT is the answer, you didn't understand the question. (NOTE: Stolen sig)
http://www.debian.org
Debian/GNU Linux ... the maintainable operating system.