Re: File Access: cache-on-HDD, more

[Nicholas Lee <N.J.Lee@statslab.cam.ac.uk>]

On Mon, 16 Nov 1998, Adrian Ratnapala wrote:

> hdd cache: should we be doing it or the app?

Leave it to the kernal in most cases.

File caching I think that be should handled (via mmaps) by the kernal,
with maybe some method to provide it hints (again thru the mmap
functions).

mmap (I think) loads 64kb pages on demand, and caches theses
automagically.


If a developer is seriously interested in high performance file access
he's likely to design a system to their specific needs.

So we could provide a default caching via mmaps and let the developer
optimise/extend the system in need.


> How is our pakfile going to help us with speed?  As I see it, it
> will mostly be acting as a fast directory cache.  We read the pakfile
> and store an in memory map of path<->offset.  Does this make a really
> big difference?  Should we be worrying more about things like the cache.

I guess there is two levels here, the external file/directory system and
the internal archive system.

It might be good to split the indexes for each up.  So there is a
pakdirectory file, which essentially acts as an alias <-> file location
index.   And a pakfile index which is stored in each directory indexing
the objects in that directory.


An addition feature we could add to the file services, is some form of
patching interface for modifing specific files.


Got to run, (need to think a bit more about it)
Nicholas

--
      "I reserve the right to contradict myself"
Nicholas Lee (Li Peng Ming)  n.j.lee at statslab.cam.ac in uk
Somewhere Out There