Re: user surveys for the Seul project

[Roger Dingledine <arma@mit.edu>]

In message <3588F526.1EDD84A4@ix.netcom.com>, kmself@ix.netcom.com writes:
>Roger wrote:
>
>> I think this is a neat idea, separate from the survey idea. I have no
>> idea how I'd start quantifying the process of sitting there watching
>> somebody work, especially without disrupting their activity. On a different
>> note tho, it would be pretty straightforward to write a little background
>> daemon that keeps track of the use of the computer, which windows are
>> used and how much, etc. We would write the app, then go find a couple
>> hundred volunteers who would grab it, run it in the background, and forget
>> about it for a while. It would periodically (manually or automatically)
>> submit reports to us, along with the user's description of him/herself
>> (experienced, cad worker, irc junkie, whatever) so we can categorize it.
>
>Several hundred is probably off by a factor of ten.  Someone's got to
>review this stuff, and no, it's not multiple response (especially not at

Actually, I was picturing that it would be very quantitative. I have no
idea how to actually implement the video thing in a fair and good way, so
I was ignoring it. :} My idea would record things things like how many
windows were open and how often the windows got switched between, what the
load average was, how much cpu usage there was and what was using it, how
much usage each of the various servers (sendmail, httpd) got, how much mouse
usage there was vs keyboard usage vs idleness, number of jobs in a window,
number of processes active, memory usage, drive usage, interrupt count.

This is all very quantitative: we can figure out how hard people use their
computers, and what they use them for. This is also very Unix-centric:
implementing such a thing in windows would suck.
(Unless it wouldn't. I'm just speculating.)

There are no videos to look at, and no question of how to get the hardware
in place and set up the experiment and stuff. You just get people who want
to help out to run this daemon in the background, and voila, it keeps track
of things.

>I'll think a bit more about just what sorts of useful info might come
>from this (other than late-night surfing habits).  I'm thinking stuff
>like:
> - what apps are used for what percent of time?  cpu?
> - what accellerator keys are use?
> - what mousing sequences are used (menu traversing)
> - how much KB to mouse?
> - system active/idle time
> - what leads to application errors (any way to watch what's going on in
>background that causes Netscape to hang?)
> - network connection usage, esp. PPP.  Methods, intervals.
> - fs mounts/umounts
> - whatever.

A lot of this can be done in a non-interactive way (that is, without
a human having to parse an analog image). Granted we won't get the
"most important stuff" (how does the user actually navigate through
stuff, what's his response time to a new window, etc?) unless we're
really clever with the code, but it's definitely a start.

I think both ideas are useful. One is simpler to implement than the
other, but one has the potential to give us a lot more data.

--Roger