Re: The cron problem and a solution

[jfm2@club-internet.fr]

> X-Priority: 3 (Normal)
> Content-Type: text/plain; charset=us-ascii
> Date: Sun, 01 Feb 1998 19:06:48 -0800 (PST)
> Reply-To: grep@shorelink.com
> Sender: grep@calvin.shorelink.com
> From: George Bonser <grep@oriole.sbay.org>
> Cc: seul-dev-install@seul.org, seul-dev-admin@seul.org
> X-UIDL: a824504d9e177f07b7bee10a59272308
> 
> 
> On 01-Feb-98 jfm2@club-internet.fr wrote:
> > 
> > The best we can do is to perform maintenance tasks when the load is low.
> 
> Another alternative is to run a process at regular intervals from cron that
> checks to see if certain housekeeping chores have been dine recently.  If not,
> it can do thise things (roll logs, update the databases, etc) at a very low
> priority using nice.  In this way the system performance is not adversely
> affected since the housekeeping process only runs when nothing else is
> requesting CPU time.
> 
> atrun might also be useful since you can specify several different queues with
> different nice levels.  You put the lowest priority tasks in a queue with the
> highest numeric nice level. This also has the advantage of making it easier to
> identify tasks since you only assign certain tasks to certain queues.  Tasks in
> the w queue might be log rollers, x queue might be database updaters, etc.
> 
> > 
> > The solution I propose is to use self-resubmitting batch jobs.  Batch
> > will not start a job when the load is high and in addition it will not
> > start it until the term is expired.
> 
> By using nice levels of the jobs, you can pretty much ignore system load.  The
> submitted jobs will just take a back seat to higher priority jobs when they are
> running.  This will cause the submitted job to take longer to complete but
> would not affect the completion time of user jobs.
> 
> > A problem is what happens if a
> > computer crashes at the wrong momenyt so the job cannot resubmit
> > itself.  The solution is to check at boot time than all maintence jobs
> > are queued and resubmit if necessary.
> 
> Sounds good. It could also be cron using nice to actually execute the job.
> 

Nicing a job is not enough.  Suppose the job is IO intensive (like
makewhatis) and the user is doing something IO intensive.  The user
task gets priority and issues an IO request.  The CPU is now free so
makewhatis is allowed to run and will issue an IO request.  Both jobs
will be very slow because the disk heads will be constantly moving
from where the foreground job is reading to where the makewhatis is
rezading.  So the best solution is trying to avoiding starting the
cron job until there is no other activity.

Another case where nicing falls short is in case we are short of
memory.

> > However the Unix "batch"
> > program is very primitive: to begin with it does not support the
> > jobname concept.  The only solution I see for knowing if a job is
> > present in the queue is to grep for a magic comment in the files
> > present in the batch queue.  Of course we should have to check
> > ownership to avoid security problems.
> 
> Put different types of jobs into different job queues. You get a-z and A-Z or
> 52 different queues to work with.
>  

The security problem is easy to deal with.  My idea was checking who
owned the batch job.

> > Because the traditional scheme is better for a continuously powered on
> > machine the user should be asked at install time if the computer will
> > be supposed to run 24h a day.
> 
> Yes.  This is important.  Another method would be to do the housekeeping at
> shutdown.
> 

I don't like it.  People will not want to wait 10 minutes until the
computer shuts down properly.

-- 
			Jean Francois Martinez

==================== The Linux.  Use the Linux, Luke! =======================