Re: [seul-edu] tutor-web -- quizzes and grades

[Gunnar Stefansson <gunnar@hafro.is>]

Happy New Year and welcome to the new millenium - thanks for the
reply. 

> > A unified system to replace my ...and a few other things.
> 
> My first thought is that there are already projects out there working
> towards this sort of thing. None of them has quite the focus or intent

I'm sure there are - would love to study a few.  Nature magazine had
an edititorial a few months ago focussed on the "Grid" but also
mentioned quite a few interesting web methodologies of somewhat
related nature.

My main problem was that almost everything I have looked at is closed,
proprietary, only useful to students who have paid tuition to a
specific school and usually course specific.  There seems to be no
standard in how to present educational material consistently from the
lecturer's presentation through the book format, self-study material
and on to the exams.

> > The quiz: Each page should have a "Q"uiz button, intended to whip up a
> > question on the topic.  Now, since these should be *used* rather than
> > just be for fun, two students sitting beside each other should not get
> > the exact same questions.  Three methods should be implemented (at
> 
> Because you're worried about collaboration?

I should have been more specific and not jumped the gun.

On-line quizzes have at least two purposes.  First, and most
importantly they provide the student with a means to practice and find
out how they stand.  But if there is no feedback into the grading
system, then the quiz will not be taken seriously.  Thus, the quiz in
this system will also provide some sort of grade.  The use of the
grade depends on the person giving the corresponding real-world
course, if that course exists.  More generally the grade should be an
indicator for the student.

> > Requirements: A student is required to obtain 75% (7.5 out of 10 or
> > GPA of 3, if you like) in order to pass a course on the tutor-web.

> It seems to me that you've got your threat model wrong. You're worried
> about trying to 'enforce' that people play fair. I don't think that will
> work (certainly not easily)

I guess my views on the above are biassed from my own surroundings and
the way I need to use the quiz - if others decide to come in, then
there may be other uses. 

I will use the quiz as a requirement for students to be allowed to
take an exam in the course.  At present the requirement is for
students to have handed in homework assignments - and I would like the
quiz to replace these.  Homework assignments are routinely copied by
some students, so there is nothing new in that.

In actual fact there really is no big problem in omitting the security 
and copycat issue altogether since such individuals will simply fail
the exam -- and initially these issues will not be handled seriously
except possibly through a mild deterrant.

For the system to be more generally useful we need the quiz to provide
both practice and a grade.  Thus a student should be encouraged to try
his/her way but these trial-and-error studies should not play a role
in the final grade.  For this reason I was assuming that it would make
sense to allow the student to work on a given small topic until the
grade from the most recent questions was high.

In order to provide enough combinations of questions, I propose using
randomized questions and random numbers within e.g. geometry and
algebra questions.  This will also take care of the
peek-over-thy-neighbor's-shoulder issue in most cases.  I don't really
see this as a major breakthrough in security - rather as a simple
deterrant (cf unconnected cameras in stores) and almost a consequence
of other requirements.

> You talk above of modelling it to determine
> how easy it is to pass the class based on randomly selecting answers;
> but that's not the way I would attack if I were trying to pass the class
> without learning the material. For nearly all grading schemes you propose,
> I could write a script to gather answers (either by guessing until I
> knew all the questions and then getting 10 right as you describe above,
> or by creating multiple anonymous accounts until I knew all the questions,
> etc) and automate the task of passing a class. If I am determined enough,
> I will beat your system.
> 
Yes, of course we can cheat on this if we want to.  I guess my
inclination is to do something initially which the average student
feels is not worth the hassle of breaking, i.e. it takes more time
than simply answering the questions.  Later on we can address
security...

In the meantime, even without any formal security, we need to make
sure in a multiple choice (abc) quiz that it is not enough to try the
three possible answers in a row and get the correct one thus without
any work whatsoever.

> I'd be happy to talk to you more about your grading scheme as it gets
> closer to being needed. I'm not so experienced with this teaching thing
> (I've only taught two terms of one class each), but security is my
> specialty. (And come to think of it, I've taken classes for 17 years.
> I must have *some* perspective by now on how they should work. :)

Thanks - I'll get back to you when we get a bit further with the system.

> 
> > Funding:  Currently out of my pocket.  Depending on response, we may
> > have to move to other computing equipment, allow an advertisement in a 
> > bottom corner of each page, or whatever.
>  
> Just work on making the system work and be useful. Once it's useful,
> I'll take care of making sure hosting happens.
> 
> (Rule of thumb: you should *never* let "i don't have a machine to run it
> from" stop you from building a service which is useful to the community.
> If it's useful, I'll take care of finding it a home.)

Yeah - that's my model exactly.  It may still become useful to include 
an ad someplace just to fund hardware, but that's a long way off still.


Cheers

G
-------------------------------------------------------------------------------
URL     http://www.hafro.is/~gunnar          http://www.hi.is/~gunnar
E-mail  gunnar@hafro.is                      gunnar@hi.is
        Marine Research Institute            Univ. of Iceland
        P.O. Box 1390                        Science Institute
        121 Reykjavik, Iceland               Dunhaga 7, 101 Reykjavik, Iceland
Phone   +354-552-0240                        +354-525-5915
FAX     +354-562-3790
Motto: Don't call, don't knock, use e-mail