/* q3log - A quark III Arena log parser/analyzer
"What's the point of playing if you don't keep score?"
Feedback to: dante@island.net
Current version is 1.0 */


========
Contents
========
Introduction
Quick Start
Installation
Running q3log
Notes
Technical Notes

============
Introduction
============
Hi Mom!

With that out of the way, let's talk logs. If you're thinking of something sexual or scatological at this point - ha ha, let's move on now. We're talking about quark3 logs, and what this program will do with them.

q3log will read logs written by quark III Arena and tally all kills and suicides by each player. These stats will be stored in a database for future viewing and analysis. These stats will be displayed within 3 graphs: Lifetime totals/Current totals/Game stats. What precisely is displayed and what it means will be discussed in the "Running q3log" section.

In summary, if you want to keep track of who you killed, who killed you, and what weapon was used, this program will do so nicely.

By the way, this is freeware, so please don't hassle me about features that it absolutely must have. Ask nicely, and I'll see what I can do.
Back to Table of contents

===========
Quick Start
===========
Run q3log.
Enter your name and where your logs will be located.
Select parse log.
View your stats.

The basic operation is pretty much that simple. Honest.
Back to Table of contents

============
Installation
============
I didn't include an installer or anything, because I really don't think it's necessary. If there's a big demand, I might do something about that later, but it'll probably just amount to a self-extracting zipfile. But I digress.

Personally, I keep the program in its own directory. It doesn't matter where it is, because you're going to tell it where to find the logs, and it writes all its own files in the directory that it's in. So, just create a directory, copy the q3log.exe file into it, and you're set. If you want a shortcut to it, that's entirely up to you. Make sure the shortcut's properties include the program's directory, or the program might have a problem finding its configuration and database files.
Back to Table of contents

=============
Running q3log
=============
First off, you need to produce some logs. Q3 will do this quite happily, and all you need to do is run it with the following extra parameters.
+set logfile 2
If you understand what I mean by this and you know how to do it, you can move to the next thing. If you just use the "Play quark III Arena" shortcut, you need to alter it a little. In the "Target" box of the shortcut Properties, just add +set logfile 2 to the end of what's already there. If you use GameSpy, go to "Games and Filters". Scroll down until you find the settings for quark3. Change the command line from "quark3.exe" to "quark3.exe +set logfile 1". When all that's done, Q3 should write a log of all its activity, and that's what we're after.

The first time you run q3log, the setup dialog will open. You need to show q3log where your log files will be. If you know where that is, move on. They are written in a sub-directory of your quark III Arena directory called "baseq3". Browse to this directory. In the file list box, you should see (among other things) pak3.pk3. If that file is there, you're in the right spot.

After that, you simply need to enter the name you use in Q3 - and this has to be exact, or the individual stats will simply all read zero. A good way to find the exact text of your name is to look in your q3config.cfg file. This will also be in the directory I mentioned in the last paragraph. You're looking for an entry that looks like this:
seta name "Dante"
Yours won't say "Dante" of course, but you get the idea.

Once those two things are done, you're pretty much ready to go. Select "Parse Log" from the menu. If your machine is relatively fast (and it should be if you're playing Q3) the parsing will be finished before you realize it's started. If you've been keeping a log for weeks and it's 100MB or something, it'll take a little longer. At any rate, it'll be pretty quick.

When it's done, assuming there was at least one frag in the log, you will see the numbers at the bottom of the screen change from "0/0" to well, something else. These numbers tell you:
which game in the database is currently selected/total games in the database You can browse through the database will the arrow buttons.
Now, if you did any fragging yourself, the totals will have changed as well. The top grids show your lifetime stats - basically the total of all stats for you in the database, and the bottom grids show your stats for the currently selected game.

You can click on the "Game Stats" tab to get to the nitty-gritty of the currently selected game. The top grid will show everybody who played in that particular game, and their totals for that game. The bottom grid shows who fragged who. Here's how this grid works: The names in the left column are who did the fragging and the names along the top row are whom they fragged. So if you want to know just how many times you nailed any particular person, find your name in the left column and highlight that row. Now just line up the totals with the names. If you want to scroll around, no problem - the names will remain visible.

Keeping score is "A Good Thing", as Martha would say.
Back to Table of contents

=====
Notes
=====
First off, if your desktop resolution is any lower than 800x600, I'm sorry, but you're going to be doing A LOT of scrolling around to see everything. Mine's 1024x768, but I did basically design the interface for 800x600. You'll see what I mean.

q3log will rename your logfiles. I did that so that people wouldn't be accidentally parsing the same file a whole bunch of times. The logfiles will be renamed qconsole000.log, qconsole001.log, qconsole002.log, and so on up to qconsole999.log, at which point it will simply stop renaming them. If you want to keep more than 1000 logs, you need to get out more. Seriously though, I was originally going to just delete them, but folks might have another program that they want to run the logs through, or just want to keep them for a while for whatever reason. I also didn't really want to see people's hard drives filling up with files, so this seemed to be somewhere in the middle. If you want to ditch the old logs, by all means, ditch them. Future feedback will likely decide if this "feature" changes.

If you're interested, q3log creates three files:
q3log.ini - this just stores your name and the location of your logs
totals.dat - stores your lifetime totals, and is updated whenever you parse new logs
games.dat - stores all game stats, and is updated whenever you parse new logs (obviously)

Don't ask me why I used separate files for the game stats and lifetime totals, I must have been half-asleep when I wrote that code. I might fix it one day.

I don't really know (I don't even have a good guess) how well the program will scale up as your database grows. During testing I had about 175 games in the database and it didn't really get noticeably slower - the size of the games.dat file was just under 500K. However, I realize that folks will likely have thousands of games in there after a while, and I can't make any promises. The database is pretty much as compact as it can be, and the only way to make it smaller would be to get rid of the individual game stats. I don't really want to do that, but it might be necessary if the database file grows to 5 Gigs and it takes 20 minutes to browse to the last entry. For those who have a little programming knowledge and may have some ideas as to how I might make this scale better, I'm all ears. You can read the technical notes at the end of this file and I'll explain how the database is set up - assuming I still understand it myself.

I'm considering adding a function to remove older games from the database, and while it would be very easy to write the actual code, I'm fairly sure it would be a VERY SLOW process with large database files. Another angle might be to have a maximum number of games per file and simply use multiple database files. That might get a little hairy after a while, especially if people play a lot of games. However, if a thousand games in a file isn't too slow, I think it might be the best answer. We'll see.

I'm a CTF guy - that's why capture totals are seen here. However, please don't write to me asking for your favorite mod to be represented - it's not gonna happen. There's just too much to track, and the data portion of the program would quickly get out of hand and exceedingly difficult to manage. Only out-of-the-box DM/CTF. That's it. Don't ask. Writing the extra code doesn't bother me, but each individual frag string has to be included to get an accurate total, and I've seen mods that have 10 or more different strings for each weapon. While some think that's real nice and keeps everything fresh, it's a nightmare when you're the one who has to make sure that they have every single one of them exactly right. It's not hard when you have the source code, but mod authors tend not to release that to other programmers. So, let me say this once more, just so we're clear. No mods, no way. Not now, not ever.
Back to Table of contents

===============
Technical Notes
===============
If you understand C++ and you're interested in (basically) how the program works, here it is. I'm not going to explain how each individual function works, but hopefully by the time I'm done, you'll have the general idea of what I did here.

The way the frags are initially identified is not particularly magic. The program searches for strings that are common to each particular type of frag. The way that Q3 does its logging makes this very easy - once the client's graphics, sound, etc are set up, game events make up the lion's share of the remaining file.

When a line in the log is identified as a frag/capture/suicide or whatever, the text is parsed to identify attacker, target, and weapon used. Once those things have been established, this information is added to a binary tree. Each node of the tree contains the player's name, a couple of structures representing total frags of each type, and another binary tree containing all the other's players names and how many times that player has fragged each of them.

The math is pretty simple here. The main tree contains X squared objects, X being the total number of people in the game. While I admit that this would not scale up very well, the average number of players in any given game is somewhere between six and twelve, so we're only dealing with about 144 objects max on average. I wrote a similar program for quark2 and ran it on my 486DX2/66 and it ate through 100 game log files in less than a minute and wrote html files for every game. Now that we're in Pentium3 country, there's really no problem.

The actual mechanics of how all that is done is pretty lightweight stuff, and I'm going to leave it up to the user to figure out how they would do it. A first year computer science student could manage it - and in fact one did. Yes, that would be me. I wrote the previously mentioned program for Q2 during my first year, and once you have a working knowledge of linked lists and binary trees, it shouldn't be too difficult to figure out what I did.

Ok, let's move on. The program has reached the end of the level, and it's time to do something useful with this tree we've built. We kick it over to a function that opens up our database. The first thing in the file is the number of records (games) contained in the file. We read that, add one, and then write that number back to the file. We then simply go to the end of the file, and we're ready to write the stats for this game.

Each record is written like so:
We write the number of players in this particular record.
We do an inorder traversal of our player tree, and write the stats for each player along the way. Player's name, the contents of the kill/suicide structures, then an individual total for each other player in the tree (we don't need their names here, because everything is sorted, and their names will come up when we reach their node).

Once that's done, we go back up to the top of the file. We then have to go through each record, searching for our designated player name so that we can update his/her lifetime stats. All that we need is the kill/suicide structures, and once the player's name is found within the record, the structure is read and the program moves to the next record. My hunch is that this will probably be the most time-consuming task when the file starts to get really large, but I haven't done any real analysis (and if the truth were known, I probably won't).

So when it's time to display a particular game in the grids, the function that retrieves the stats is fed the record number of the game that the user wants to see. The program just searches through the file until it finds the desired record. It then re-builds the tree and that tree is used to fill in the grids. I know, I know. I should have just used a linked list here, since the records are already sorted. I didn't feel like going back and changing it by the time I'd reached that stage of the program. With the tests I've done to this point, it actually reads the records pretty quickly. I'm hoping that people's systems will be able to keep up with the growing databases to a certain point. What I'm actually hoping is that the databases will grow too large for people's hard drives before that happens...

I do have a fairly workable idea to make navigation faster, and if it looks like it's going to bog down fairly quickly, I will implement it. Tallying the lifetime totals will not get any faster without sacrificing the program's ability to calculate totals for any named player. Life's a trade-off.
Back to Table of contents

That's it. Hope it's everything you need.
-Dante