Re: The first comment is...

[Roger Dingledine <arma@mit.edu>]

In message <35CDB9D2.78BD9869@appliedtheory.com>, blizzard@appliedtheory.com writes:
>> I'll try to think of some more. For now I'm reading about nntp and looking
>> at various news servers (particularly innd) to try to figure out which one
>> might be most appropriate for us. I think perhaps the most difficult part
>> of the initial setup for LNN will be configuring an innd to be secure and
>> small-footprinted, and then convincing everybody that they want to run it
>> on their machines. News servers are not known for their security. Hmm.
>> 
>
>Has anyone considered just using the web to transfer this around?  I
>don't know about you but I don't want the overhead of running a news
>server.  There are also security implications with running one that I
>don't want to deal with.  There aren't a lot of security problems with
>inn that I'm aware of, it's just another thing to keep track of.
>
>News servers do have a nice property; They are very good at figuring out
>what articles they have in comparison with what other news servers have
>and just transferring the ones that they need.  We can probably come up
>with another scheme with a web based scheme.  I may be a bit late to
>this discussion.  Can someone outline what the points are of NNTP/news
>that make it the transport mechanism of choice?
>
>--Chris

I wrote out a brief description earlier of the functionality I want from the
system (http://linuxunited.org/projects/news/reqspec.html). I tried to
write it in a way that doesn't dictate what protocol or programs we should
use.

When I started thinking about the protocol to use for exchanging messages,
I realized that NNTP was exactly what we wanted: it has neat stuff like
a perl module designed for it, many C programs already out there, etc. If
you check out the NNTP protocol
(http://www.w3.org/Protocols/rfc977/rfc977.html), you will find that it is
exactly what we want: it defines a protocol for exchanging news messages
in a fast and reliable way, as well as handling distribution and hierarchical
categories for articles. Indeed, RFC 977 sounds like what I would write up
describing our situation with Linux news, if I had the inclination to write
something that long.

However, this doesn't at all mean that we need to use a large klunky news
server to get the functionality we need. While INN will do what we want, it
*will* be klunky and it *will* be obnoxious to configure, and we'll never
really be confident in its security.

But there are other options for nntp implementations that are much much
simpler and more lightweight. For instance, perl and python each have
an nntplib that looks pretty good (from a brief glance).

There appears to be an original reference implementation of the nntp
protocol, described at http://trout.tarugo.com/man.shtml/nntpd(8)

There are alternative news servers to INN, such as C News:
http://www.li.org/Resources/LDP/nag/node259.html#SECTION001900000000000000000

So here's the summary, for now:
* I still claim that NNTP is the correct protocol to be using here
* It has the huge advantage that many programs and utilities are already
  written, tested, and documented. And there are many other users.
* but I agree that it would be really cool to find a backend news server
  that is lightweight and security-friendly. INN (and probably C News,
  but I haven't looked at it indepth yet) are unacceptable.

What did you have in mind for a "web-based scheme"? web servers are
definitely not set up to deal well with push, which is how we need to
be distributing the news (pull would be painful at best, given how
frequently we'd need to query). But I'd be up for any sort of ideas at
this point, just in case they're right. ;)

Thanks,
--Roger