admin:initial brainstorming

[Luka (Peter) <luka@mit.edu>]

Here's some initial info to get this group started.  I'll describe the
group's purpose a bit first, then present a topic that was brought up on
seul-dev-install, but belongs here, and then throw out a new idea which I
think is worth pursuing immediately.

I. Group Purpose
  Ok, the name sucks.  You can probably see the historical origins of the
label "admin" for this group (especially if you look at the initial group
definition), but it's pretty misleading, and the info on the web is outdate
too, so ignore it and read this instead.
  What we're considering is adding some addition system-level abstraction
to Linux.  This does not (necessarily) mean kernel changes, but more
importantly abstractions above or improvents to things like the init process,
the configuration files, and basically anything that "makes the OS go," you
might say.  This group may also develop administrative utilities that sit
on top of the new system layer (hence the source of the name), but those are
really secondary to building a system layer which anyone can interface to.
The only reason this group would only be developing the most direct admin apps
is for efficiency, and quality reasons, since the people who just wrote the
system layer are probably going to have the easiest time writing the apps
based on it (though any other coders should technically be able to do that
task just as well).
  And if anyone has suggestions for a better name for this group, let me know,
and we might change it...

II. Some general discussion
  This responds to some mail exchanged on the seul-dev-install list under
the Subject: I think It was Peter Luka's install abstraction proposal... 
  Well, kinda - it was more than an install abstraction.

here's what jmunoz wrote:
>I think the abstraction is critical to simplifying the System. Higher
>Level data will also accomodate different filesystem standards and Un*x
>versions.  Conceptually, /etc/rc and /etc/rc.d/rc.xxx are very similar,
>but are structurally different.  This sort of higher level abstraction
>could allow us to say "Right now, we will modify Red Hat's config tool
>and base the 
>distribution upon .rpm, and later add a high level model interpreter to
>support Debian without too much trouble.  If we opted for the low level
>storage (current state of affairs) we will have to rewrite debian's
>package manager to support .rpm, or reverse engineer alien to convert
>dpkg's to rpm's.  I am not insisting we use Red Hat, just suggesting we
>support as many package types as transparently as possible.
>
>I like Your proposal for a "High Level" (abstract) representation that
>is later interpreted into the low level config (I have two HD's (high
>level) create fstab with sda and sdb (low level)).  I missed the
>outright rejection you mentioned Peter (Luka), and you may need to asses
>whether the "Silent Majority" can be coaxed to vote or something.  The
>petty nature of many of the discussions and the incredible volume of
>rambling* on this group really limits my ability and desire to reply to
>the cogent, well thought out, organized plans (like the Registry
>proposal).  I think this may be true for many others too.
>
>There are many positive features in the W95 registry. I like the
>"Registry corrupted, restoring from old copy" feature, even though it is
>a bit trigger happy and erases all the config I just did, sometimes
>right, sometimes wrong. I like it's structure better than the old w3.11
>registry.  I liked w3.11 ini files (relatively comprehensible), but the
>registry is a different beast, much more like a database than a flat
>file.  A registry inplemented with flat files will be *VERY* bad anytime
>more than one process needs to access it, and don't try to tell me that
>the FS handles locking or relational integrity well.  The problem with a
>DB is that your OS becomes dependent on the operation of the DB( I think
>this is obvious and the main reason against it), so the question is "can
>we implement a bombproof DB that will be one of the mandatory OS
>components like the scheduler or the root shell?" A second option might
>be to design it so that in single user mode there will be no concurrency
>issues, have the application handle relational integrety, and allow
>certain processes text mode access, untill we switch info "full db
>working, multi-user mode".  Might be tricky...

response from kai wetzel:
>John Munoz wrote:
>[...]
>> There are many positive features in the W95 registry.
>
>> A registry inplemented with flat files will be *VERY* bad anytime
>> [...]
>> working, multi-user mode".  Might be tricky...

>When I thought abou this isuue the last time, I came to
>the preliminary conclusion that it's the best idea to store
>the master preference settings in editable text files (*.rc)
>and have a "db" process which builds up a cached copy from
>this in memory.  Applications would handle read/write access
>through this "db" process.  I don't know whether it's a good
>idea or not, though ;)

I haven't had a chance to think about the above too deeply yet, so I won't
respond directly in much detail until I do.
Regarding the rejection of the comprehensive registry layer, tho, you're
right that it wasn't really an outright rejection I suppose.  More that
it's a particularly large undertaking, and it really wasn't getting discussed
very productively on the seul-project list.  So there's nothing wrong with
reviving the idea and seeing where it goes.  We'll have to be careful before
going to actually implement that our ideas are of the same scale as our group,
and we can check that when we get there...

III. Concept for config files.
  This is a recent idea I've been thinking about quite a bit, which should
lend itself to a pretty simple but very useful implementation.
  It starts with the fact that you've got all these config files (dotfiles,
files in /etc, etc...), which are pretty evil to deal with because you can
edit them all by hand... AFTER you spend enough time figuring out how to
do so... and AFTER you do some tests to make sure you really read the docs
right and that what you think will work really does work, and so on...
But wait a minute: most of these files are really basic line/space-delimited
text, with a really really simple (though underdocumented and maybe
non-intuitive) format.  Why can't the machine deal with them for you and
present a better editing interface than vi or emacs?  Perhaps even...
point and click? (*gasp*)
  Well, the reason it can't do it is because there's a lot of them, the
specs are in human language (if they're even written down completely at all),
and they sometimes vary with release.
  So, here's the ridiculously simple idea on how to deal with this.
Establish a standard (machine-parsable) format for specifying the contents of
a config file.  This should not be difficult given the simple format most
of the files have (line/space-delimited, etc, as I said).  This format info
can then be written by us for a certain amount of standard things.  We can
ask the distributors nicely to include the format info we wrote with their
distribs of those things.  And if it catches on, ppl writing software with
config files will start including a file of this format with the distributions
to describe the config files.  This config-descriptor-file can be either added
literally or as a reference in what would otherwise be a comment in the config
file it describes (or otherwise installed in a special place so the machine
can associate description-files with config files easily).  And then all we
need to write is ONE generic but spiffy (probably dual graphical/text, plus
command line) utility, and possibly a programming library, that can read
the descriptor-file, thus automatically know the format for the config file,
and then present an intuitive interface for editing the /information/ in
the config file, rather than the file itself.
  Really the only remotely difficult part of this project would be figuring
out an optimal descriptor-file format.  So a few things we should look at:
 o SGML Document-Type-Definitions (DTD's) : they do stuff like this
   (specifying the format of a document in a machine-parseable way).
 o Linuxconf : (which as of a recent beta, no longer needs to modify SysV init,
   and /might/ not munge your machine - but be careful if you try to install
   it please)  Linuxconf has the end effect we're going for here, of allowing
   graphical, user-friendly configuration of programs (which in the unix
   world is mainly config text-file tweaking).  We should see how it does
   stuff (whether it implements any sensible abstraction like this), see if
   it has code we can reuse, and gather ideas on how to write/not write the
   interface as well.
 o Other suggestions to study?  Name them...
One basic requirement I can think of right now is that the descriptor file
will have to contain some human-readable text that gets associated with
pieces of info in the config file it describes, so that when the interactive
configuration program runs, it can display these labels for the user.  We
might also want to have a shorter name, or numerical id, for each of these
items, to make a library that interfaces to this system more sensible.
  As I said, this should be really simple to implement and very useful, so
if no one has done it already, I think we should look at the above sources to
make sure we're smart about how we do this, and then just DO IT.
  Comments?  Volunteers to work on this?

=====-----------------------------------------=====
 Peter Luka            ...            luka@mit.edu
 SEUL Project development leader/systems architect
               http://www.seul.org
-----=========================================-----