SV: SV: SV: SV: SV: Progress Update

["Bjarke Hammersholt Roune" <asmild@post6.tele.dk>]

> > > I see, and this is a good reason. Personally, I'd have named
> that class
> > > something more like StaticHashTable, to reflect its immutability and
> > > leave the name free for a "normal" hash table. But maybe I'm
> nitpicking
> > > here.
> >
> > Well, then I'd end up with names like ConstStaticHashTableIterator. I
> > decided ConstHashTableIterator was long enough... ;)
> >
> > Not a bad point. Fixing it would be only a matter of a
> search-and-replace
> > operation, though.
>
> Such a specialized class with such a general name as "HashTable" is not
> a very good idea...
>
Well, its a special implementation, not a special interface. I'm planning on
adding AddEntry() and RemoveEntry() methods to it.

> > > They *cannot* all hash to the same location, since I there is a load
> > > limit. When an entry has to be added to a full bucket, I grow the hash
> > > table. Of course, pathogen hash patterns (many keys hashing to similar
> > > values) could cause excessive growing of the table, but that life with
> > > hash tables.
> >
> > That gives you more space, but still, if every entry in your
> table hashes to
> > the value 1, then I don't see what you can do about that,
> no matter how large
> > your table is: the rest of the positions in the table would
> just be empty.
>
> Then, you have a problem with your hashing algorithm. Your hash table
> would also have a problem, wouldn't it? It would have a huge number of
> entries in one of the hash position. Yours would be way faster than
> mine because of the binary searchable sorted list, but it wouldn't be
> anything like a correctly hashed table.
>
Any hashing algorithm can potentially generate such a situation. Its just
not very likely. however as the number of entries in a hashtable grow, the
more potential you have for getting a large amount of entries in the same
position (or "bucket", as you call it). With a very good hashing algorithm,
it doesn't happen so often, but eventually it will.

Anyways, this way, you can specify a very very large load factor, and still
get decent performance.

> > Well, is this much worse? Again, the requirements to the
> HashTable is quite
> > specific.
>
> Not *much* worse, but the point isn't only programmer-time for users,
> but for your time.
>
> You must apply a similar development approach to the rest of
> PenguinPlay, right? If doing the same thing takes me 10% of the time it
> took you and performs 90% of the way your implementation would (and
> would be "fixable" if I invested more time, but I could do when I choose
> to), it would still take me considerable time to finish a usable version
> of a project of the scope of PenguinPlay.
>
It took me at most half an hour to get the DynHashTable up and running as
well as debugged. Probably less, I don't remember.

> Do you think that it'll get any easier by taking 10 times as much effort
> to get that last 10% right from the start?
>
Well, I *wanted* to implement a hashtable. I've never done it before. I've
implemented PLENTY of linked lists. Anyways, it would have to be fixed at
one time or another, and I'd end up having implemented both a linked list as
well as a hashtable. I wouldn't have spent time doing something else on
PFile even if I had finished the hashtable in 10 minutes, as it was the only
thing on PFile I could do at the time.

> I said 10 files opening because I counted the resource file as one
> opening. The resource file contains a few hundreds files. I guess I was
> a bit misleading with this. Anyway, we don't "open" files all that often
> anyway, from time to time during game play, we do (level ups that change
> the "theme", for example), that's about it.
>
> The point is that improving the color conversion process for example
> yields results that are multiple orders of magnitude greater than the
> best resource file system could give us.
>
Spend half a minute on doing a global search-and-replace. Nothing else.

Then, at release, spend half a minute adding mount and unmount calls in your
initialization and deinitialization parts of you program. After that, spend
perhaps 1 and a half minute creating the needed pak files.

Now, if you can get just 1 percent, or even 0.1 percent, performance
increase for your 2 and a half minute time investment, I think you've made a
pretty good deal... :)

As a side note, I tried opening and closing 10000 files in a directory with
10000 files (ie, all was opened once each). I just did a fclose(fopen(path,
"rb")). Nothing else. All paths were allocated and created before-hand. It
took two minutes to run. The amount of files was a bit extreme, but: two
minutes!!!

I also tried with smaller amounts of opened files, aswell as smaller amounts
of files in the directory. For all but the smallest amounts, it took more
than 10 seconds (its a while ago, so I don't remember exactly the times).

Windows is the most prevalent game platform. Even if you think FAT32 sucks,
that's what the platform most gamers use use.

Funny thing: what took by *FAR* the most time was actually closing the
files. Does anyone have any idea why this is so?

> I wish the PenguinPlay system
> was strongly implanted, and I just feel the efforts should be wisely
> invested for this to happen.
>
Power to you! :)

> > I'd say it isn't fixable if it takes an external codebase to
> fix it. That's
> > called a work-around. Like when you don't do something specific
> in a program
> > because you know that it'll die if you do, or only do it in a
> specific way.
>
> This is all documented, and is a design choice made for the simplicity
> of the design, which is a valid goal. The fact that this was fixable in
> a user-space library rather than making it "properly" in kernel-space
> just prompted them to go for it and leave this to user-space, keeping
> the system-specific kernel-space clean of such complexity.
>
I must say I perfer to being able to trust the kernel to fix its OWN
problems that trusting code in a completely different code library to do it.

If that was all so simple as you say, why doesn't the kernel just do what
libc does?

> This made Unix stronger rather than weaker, as it would seem at first
> glance!
>
Hmm... It forces people to think of the shortcomings of UNIX each time they
make a call to it. Is there something wird going on here I should be
compensating for?

This rather reminds of the of the way Microsoft makes their software, and
that is exactly the reason people do not like Microsoft's software.

Think like this: if you computer crashes and mess up its internal state, you
just try again! Problem solved! Much easier on the developers... I mean,
adding autosave is alot easier than to never crash.

I think the iss

> Does it really matter to the user doing an fread() that the libc
> actually is doing a loop of read() calls? Nope. Don't worry, everything
> is fine!
>
libc is what exactly? Sounds like it works only with C/C++.

I do, though, admit its a nice workaround to a bug. You hardly notice it.

I remember Microsoft calling stuff like this "features".

Ok, sorry, I'm being a bit unreasonable, I know, but I just don't think
leaving it to your users to fix your own problems is a very good aim.

> > > Yep, but they did have at least a much better solution and
> people didn't
> > > care. NeXT also had a much better system and look where they are now
> > > (OpenStep programming is a dream come true by the way, simply
> > > fantastic).
> >
> > In some way, they must've failed, it seems... :)
>
> Their problem was cost it seems. They had fantastic machines, operating
> system, programming tools and user interfaces (look at Window Maker or
> AfterStep today for a taste of it).
>
Then it would seem they were killed by the high cost of hardware, not their
way of making software...

> > Well, it doesn't matter what OS they choose, PenguinFile will
> still work.
> > And PenguinFile doesn't have any competetion. That's why it was started.
> > There's a gap there, and if there is only one lib to fill it...
>
> Doom, Quake, Quadra and numerous others have archive support. What
> boggles me about PenguinFile and PenguinPlay is that archive support is
> often done as an afterthought and still done nearly as good as it could
> be, good enough anyway. What is an *afterthought* in most project is a
> major undertaking in PenguinPlay, is that expected?
>
PFile will do alot more than what normal games do. But we need a solid
foundation to build that functionality on. Building things solidly takes a
while.

> > Not in all aspects. There are things Windows currenlty does
> much better than
> > any Unix. Like aid for the disabled or an easy GUI (for
> beginners, not for
> > people who are actually serious).
>
> Aid for the disabled and easy GUI can also be done in Unix.
>
GUI: Not as well as on Windows (yet!)

Aid for the disabled: What about blind people? People without hands, or
unusable fingers? I might be wrong, but I really don't think they would get
much out of using any Unix.

On windows, you have voice synthesis programs that will read anything on
screen, even in programs that doesn't support it (I think. Its not sometihng
I've spent alot of time reading about...), and you have alot of voice
recognision programs that are certainly better on Windows than on Unix (are
there even any?)

There are also ALOT of aid for the disabled built into Windows itself.
Atleat, that's what I've been told by people who are actually disabled.

> This isn't the point anyway. The point is that technical superiority
> isn't the key to success. It is *part* of it, and is sure a very
> desirable feature to have, but if you don't get *something* out soon,
> there will be a system that will cover this area, and it could be of the
> "fundamentally flawed, unfixable" kind. I do not care for "perfection"
> right now, I only care for not getting invaded by stupid stuff! Give me
> something *fixable* eventually, NOW!
>
Hmm... There's a few build errors still. Other than that, the thing compiles
and builds nicely. My guess is that Christian will get it to run by today.

Of course, actually getting it to work is another matter.

Bug hunt:

There's a mosqito! Shoot it! Shoot it! Yes! That'll look good on the dinner
table!