Introduction
For over six years, I have been collecting QWTF match demo files, capturing them to video, and posting them on YouTube. In addition, I have been accumulating a large collection of match data thanks to Erik Selberg's (a.k.a. >V<SPEEDenator) excellent SPEEDStats script. Recently, I decided to start investigating this large and unique dataset to see what insights could be gleaned from it.
On this page, I present some preliminary findings. Specifically, the histograms below examine how players favored certain weapons over others for several of the classes. They compare total frags with one weapon to total frags with another weapon (or, in a few cases, groups of weapons). While not a perfect method, in this way, we can gain some insight into which weapons different classes tended to prefer. Some results may appear obvious (sniper rifle vs. any other sniper weapon or soldier rocket launcher vs. any other soldier weapon come to mind), but even these obvious results help confirm the veracity of the data.
My sincere thanks to SPEEDenator for his aforementioned SPEEDStats script, without which this study would not have happened. I also thank lunk[DJedi] for his willingness to host my site for several years now. Finally, I thank all of those who have contributed their QWTF match demos to my project, whether by direct donation of QWD files or simply by taking some step long ago to prevent their files from being lost on a broken hard drive. This project is driven by your data, and so I thank you.
Methodology
My starting dataset is the collection of 852 matches (as of this writing) that I have assembled over the years. For each match, I have data generated by SPEEDStats. For this study, I am using aggregate frags by weapon counts by class for each match. In other words, for each match, I count how many frags there are with each weapon by each class. For example, if a match had two demomen, I add up how many grenade launcher frags each demoman had and the total is the grenade launcher frag count for the match. In the case of a weapon shared by more than one class, such as the super shotgun, frags by each class with that weapon are tracked separately. For example, if five soldiers combine for 10 frags by super shotgun and two medics combine for 30 frags by super shotgun, the 10 soldier-super shotgun frags and the 30 medic-super shotgun frags are tracked separately.
I have written a Python script and a set of Python classes that parse each individual SPEEDStats page and extract all pertinent information. The script aggregates and organizes the quantitative information from each match. I then export the data to a CSV file. I read the CSV file into R, wherein I created each graph.
For each comparison, I extract the pertinent information about each of the weapons in the comparison according to the following rules:
- I only use matches that I have classified as type "Match", representing a competitive match between two clans. I disregard all matches that I have classified as type "Scrimmage", "Practice", "Pub", or "Pickup", as I am most interested in how the weapons were used in organized play.
- I require each match to have at least 8 players on both Red and Blue. Again, I am interested in high-level play, and though there were some small-sided leagues in the late 1990s and early 2000s, the highest level of QWTF play was usually done with 8v8 or 10v10 sides.
- In order for the comparison to have some validity, I require the sum total of the number of frags used by all weapons in each comparison to be at least 10. This is to prevent matches with minimal frag information about certain weapons from unfairly altering the comparison outcomes. For example, if in a match only one player played demoman and then only briefly, but recorded one grenade launcher frag and no pipe bomb frags, this would count as a match with 100% frags by grenade launcher and 0% frags by pipe bomb. By my current methodology, this match would be given equal weight with a match in which several players played demoman for the entire match.
For each comparison, I filter out the matches that do not meet all of the criteria above. With the remaining matches, I perform the calculations. To compare weapon1 with weapon2, I use the following formula:
(weapon1 - weapon2)/(weapon1 + weapon2)
With this formula, I can determine for which of the two weapons there were more frags and to what extent. The maximum value of this formula is 1, which means that 100% of the frags were caused by weapon1. The minimum value of this formula is -1, which means that 100% of the frags were caused by weapon2. A value of 0 for this formula means that 50% of the frags were caused by each weapon. A value of 0.5 means that twice as many frags were caused by weapon1 than by weapon2, and a value of -0.5 means that twice as many frags were caused by weapon2 than by weapon1.
The p-values are determined by running a two-sample t-test on the raw frag counts.
How to Read the Graphs
Each graph below depicts the histogram of the equation in the above section when applied to every valid match. In the example graph below, we see a comparison of hand grenade frags to grenade launcher frags by demomen. There are 463 data points (matches) contributing to this histogram.
As there are more and higher bars on the right side of the histogram, there are more matches where there are more grenade launcher frags than demoman hand grenade frags.
The Graphs
RESULT: In most cases, demomen scored frags with their pipe bombs far more often than with their grenade launchers.
INTERPRETATION: Unsurprisingly, demomen in high-level play usually played defense, and their pipe bombs were much more effective at this task than their grenade launcher.
RESULT: In most cases, demomen scored frags with their grenade launcher more often than with their hand grenades, but not by a huge margin.
INTERPRETATION: When demomen weren't using their pipe bombs (often due to a failed pipe bomb trap detonation), they usually would fight off the immediate threat with their grenade launcher and hand grenades. I suspect that the grenade launcher had many more frags than the hand grenade because even though a hand grenade could be detonated much more precisely, a demoman could launch several grenades from his launcher in the time it took to prime and throw a single hand grenade.
RESULT: Engineers scored frags with their hand grenades only slightly more often than with their EMP grenades.
INTERPRETATION: As the two types of grenades were both capable of inflicting similar amounts of damage on opponents, it is unsurprising that they would have similar frag rates. Perhaps hand grenades win out by a small margin because they were usually more effective against enemy scouts than EMP grenades were.
RESULT: In most cases, engineers scored many more frags with their super shotguns than with their railguns. In many cases, they used their shotguns to the exclusion of their railguns.
INTERPRETATION: The two main reasons that engineers would use the railgun are that it penetrates multiple enemies and it is the default weapon for the engineer upon respawn. It also does not require reloading and it has a novelty factor, being a weapon created specifically for Team Fortress. The super shotgun, however, is a far more powerful weapon, and most engineers favored it, sensibly.
RESULT: Engineers scored frags with both their sentry guns and their other means (weapons, grenades, dispenser), but they favored their non-sentry gun weapons by a small margin.
INTERPRETATION: In high-level play, enemies usually were on the lookout for enemy sentry guns in the usual location. Therefore, they would be more likely to try and stay out of the firing range of the sentry gun until they were ready to try and destroy it. In other words, it functioned more as a deterrent. Also, it did not always immediately lock onto opponents and start firing. Engineers often had to get out in front of their sentry guns and fight, rather than rely upon the sentry gun to do all of the work for them.
RESULT: Heavy Weapons Guys used their assault cannons far more for fragging than their super shotguns, usually exclusively.
INTERPRETATION: If you had the choice of using the boring super shotgun that lots of other classes can use or the huge honkin' assault cannon that can blow away enemies...which one are you choosing? Note that N = 74 in this case, and probably in many of those cases, a heavy weapons guy was used more as a joke than seriously.
RESULT: Medics used their super nailgun more than they used their hand grenades, but only by a small margin.
INTERPRETATION: The super nailgun is the default weapon for the medic, and is also exclusive to the medic. Though the nails take time to travel, they can do damage at long range (useful for assaulting enemy snipers at a distance) and if several hits can be scored with it quickly, any enemy can be brought down quickly. The medic also relies heavily on the hand grenade for fragging, so why would there be more super nailgun frags? My guess is that the medic can get more nails outside of respawn much more easily than the medic can get more hand grenades.
RESULT: The medic usually fragged with the super nailgun much more than with the super shotgun, in some cases almost exclusively so.
INTERPRETATION: Again, the medic defaults to the super nailgun on respawn. Though the super shotgun and super nailgun have similar fragging ability, most medics will favor the super nailgun as it is easier to damage enemies at a distance with the super nailgun than with the super shotgun, and medics typically did not want to get too close to the heavier classes.
RESULT: Scouts fragged with their nailguns much, much more than with their shotguns, usually exclusively so.
INTERPRETATION: Scouts were lousy at fragging, but at least they had the sense to choose their least lousy weapon for doing so.
RESULT: Snipers nearly always had significantly more frags with their sniper autorifles than with their hand grenades.
INTERPRETATION: Snipers only fought at close range as a last resort, and the sniper autorifle is a much more effective weapon at distance than the hand grenade. Also, many sniper autorifle frags come as a result of hitting an enemy with the sniper rifle but failing to frag the enemy. Knowing that the enemy was likely low on health, snipers typically switched to the sniper autorifle to finish the enemy off.
RESULT: Snipers nearly always had significantly more frags with their sniper autorifles than with their nailguns.
INTERPRETATION: Again, snipers prefer to fight at long range. The nailgun is not an effective weapon to begin with, and the sniper autorifle is right next to the sniper rifle in the list of weapons, making it the easier choice when the sniper is forced to fight at close range. Also, as stated above, many sniper autorifle frags are scored after non-fatal sniper rifle hits.
RESULT: Snipers always fragged more with their sniper rifles than with their sniper autorifles, usually by a wide margin, if not exclusively.
INTERPRETATION: Unsurprisingly, the weapon that defines the sniper is used far more than any other sniper weapon.
RESULT: Soldiers always used their rocket launchers more than their hand grenades. On average, they scored twice as many frags with the rocket launcher than with their hand grenades.
INTERPRETATION: Just as the sniper rifle defines the sniper, the rocket launcher defines the soldier. However, since the rocket launcher is not the instant kill weapon that the sniper rifle is, the soldier often needs an assist with his hand grenades for fragging. Still, the soldier can fire multiple rocket launchers in the time it takes to prime and throw a grenade, and the rocket launcher is useful at long range, while the hand grenade is not.
To Do
- Consider adding weights to each data point by number of frags in the match. Currently, a match with 10 combined frags for the weapons being compared has the same weight as a match with 500 combined frags for the weapons being compared.
- A few of the matches for which I have SPEEDStats are not parsed by my Python script properly. As the current number is 36 out of 857 matches (4.2%), the numbers would not change significantly with these additional matches, but I would like to add them to the full dataset.
- Assess statistics on a per-map basis. Some weapons excel on certain maps and fare poorly on others. Most obviously, the sniper rifle works well on maps with large open spaces, good sight lines, and areas with protection for the sniper (2fort5r, well6) and not so well on maps lacking such features (h4rdcore, 32smooth).
- Compare clans for choice of classes and weapons.
- Prove that Clan Marauder were, in fact, the worldwide leaders in pwnage.
Contact Information
Comments? Please feel free to contact me (Tickenest) at my Gmail address (it's easy to guess). Thank you for reading.