Over the years manufacturers competing in this market have been made and broken based on quite possibly one of the most important features a new graphics chip can boast: drivers. 

Back in the "early days" of competition in the 3D graphics market where Matrox, NVIDIA and S3 were still trying to make a name for themselves (ironically enough for the former two), drivers played a huge part in the success of their products.  The Matrox G200 was ostracized because it shipped without a fully working OpenGL ICD and the Savage 3D/Savage 4 parts were both plagued with horrible driver support.  It wasn't until NVIDIA got their act together with the Riva 128's drivers that they were taken seriously as well.  Now, with only two competitors left, it is a necessity that we put their driver quality and performance issues up to the utmost scrutiny. 

One of the most controversial topics when it comes to driver support is Windows 2000 support and performance.  Microsoft's original intention for Windows 2000 was for it to be a professional/corporate-only Operating System, leaving Windows 98 and later, Windows ME for the home user.  Unfortunately this strategy almost completely ignores a good portion of the market that happens to be quite a few AnandTech readers, the power user, or the hardware enthusiast.  Microsoft made it very clear to the graphics manufacturers that Windows 2000 wasn't to be a gaming OS, however the users definitely found it very well suited to their needs.

The launch date of Windows 2000 came and went, with the best driver support for the OS coming from Matrox.  Later on, 3dfx, ATI and NVIDIA took notice of the demand and put out driver sets that supported the new OS however very little could be said about the support.  3dfx's first Windows 2000 drivers were plagued with bugs and little could be said for the performance of the ATI and NVIDIA drivers either.  The worst by far was the message that ATI had to send all Rage Fury MAXX owners when they demanded Windows 2000 drivers that read as follows:

Dear Customer,

You have previously contacted our Microsoft Windows 2000 feedback email address to comment or inquire about RAGE FURY MAXX drivers and support for the Microsoft Windows 2000 operating system.

I regret to inform you that our plan to provide a RAGE FURY MAXX driver for the Windows 2000 operating system cannot be completed. Due to issues with enumerating the chip ID's, of the dual RAGE 128 PRO graphics chips, we are unable to complete the driver for our RAGE FURY MAXX card.

ATI sincerely apologizes for promising a driver to you and then being unable to complete it. This was an unforeseen technical issue during the development of the Windows 2000 RAGE FURY MAXX drivers.

For those customers who wish to contact me directly to comment on this situation, please email me at [email protected]. I welcome your comments and feedback

If you haven't gotten the hint by now, there were definitely some frustrations regarding the performance and reliability of graphics drivers under Windows 2000.  The same frustrations carried over into the game developer market, as quite a few games simply would not work under Windows 2000.  Even today, recently released titles such as EA Sport's NBA Live 2001 aren't fully supported under Windows 2000.  Whether it's a combination of Microsoft's unwillingness to realize that there is a group of enthusiasts out there that do play games and want the robustness that Windows 2000 offers, or it's a lack of developer/manufacturer support for the platform as a gaming solution, it is clear that there is a problem. 

Starting with our 3dfx Voodoo4 4500AGP Review, we began including a Windows 2000 Driver Performance section in all of our 3D graphics chip reviews.  Reviews of the ATI Radeon SDR and NVIDIA's GeForce2 Pro followed, both including this new Windows 2000 Driver Performance section. 

The results we found were definitely disappointing to say the least.

Can't you be more like your brother?

If we think of the former 3dfx, ATI and NVIDIA as brothers, we were definitely pushing 3dfx and ATI to be more like their "brother" when it came to Windows 2000 driver performance.  The reason was simple; if you look at the performance figures we derived from running Quake III Arena's built in timedemo under Windows 98 vs Windows 2000 it was clear that only NVIDIA's drivers did not result in a performance hit when moving to the more robust OS. 

ATI Radeon Windows 2000 Performance 4 months ago

While 3dfx's performance was disappointing, our first benchmark session with ATI's drivers was horrifying; the performance difference between the two driver sets was so great that Windows 98 was often times close to twice as fast as Windows 2000.  ATI did manage to correct the performance issues we noticed last October in a later build of the drivers, but the performance delta remained. 

We are on the verge of being witness to the announcement and release of the next-generation of 3D graphics accelerators from NVIDIA, and later, from ATI.  But before we all even contemplate shelling out the big bucks as our "must-upgrade" genes kick in, it is time to get a much bigger and clearer picture of how the drivers of the two major competitors stack up under Windows 2000 and what areas need work on.  This time around, for a much more in-depth look, we will be focusing our performance tests in much more than just Quake III Arena, just in case a manufacturer decides to optimize for a particular game that the enthusiast community likes to benchmark under.  Instead, we gathered a total of 8 games and compared their performance under Windows 98 and Windows 2000.  Now it's really time to see which drivers are indeed perfect and which ones need work. 

The Test

We used the latest drivers from ATI and NVIDIA as of February 19, 2001 and in order to keep the comparison fair we used a 64MB GeForce2 GTS and a 64MB Radeon DDR.  DirectX 8.0 was used in all tests.

Human Head Studios' Rune
http://www.humanhead.com/

Rune is a third person perspective game based on the UnrealTournament engine from Epic.  Since it made use of the same engine as UnrealTournament, we managed to record a quick demo and use the engine's built-in timedemo function to measure performance. 

The default 3D accelerated mode, like in UnrealTournament, is Direct3D, which is what we used to conduct the performance tests.  The settings were all left at default with the exception of resolution and color depth which were set to 1024 x 768 and 32-bits respectively.

You can see that both ATI and NVIDIA take a performance hit when moving to Windows 2000. 

The Radeon dropped in performance by 17%, falling a full 10 fps in the timedemo.  NVIDIA's deficit was slightly lower yet still noticeable with the GeForce2 GTS painfully enduring a 12% drop. 

Rage Software's Expendable
http://www.rage.co.uk

Expendable is another third person perspective game that employ's Rage's own custom engine.  The Direct3D game is aging however we still use it as a gauge for system memory bandwidth performance and it is a very useful benchmark for driver comparisons.

We ran Expendable with the built in timedemo feature by executing go.exe with the '- timedemo' switch.  The only settings we changed were resolution and color-depth.

The Radeon test bed exhibited a huge performance drop under Expendable from 87 fps down to 47 fps, a decline of 40 frames per second or 46%.  We ran the test numerous times and continually came up with the same results.  We originally thought that v-sync hadn't been disabled but the frame rate peaked at above 70 fps during the test while the refresh rate was held to 60 fps therefore ruling out that theory. 

In spite of this, NVIDIA's performance is relatively unchanged between 98 and 2000 with a small increase under the professional OS. 

It is worth noting that using ATI's officially supported release drivers (v5.0.3035) under Windows 2000 the game is completely unplayable.  The background textures are missing as are the textures for all of the characters and most of the objects although they do clip in randomly.  We did test with the latest special-purpose (unofficial) drivers (v5.13.1.3100) however we did run a quick compatibility test with the release drivers to check for any anomalies. 

This problem was not present in any of NVIDIA's driver candidates.

Bioware's MDK2
http://www.bioware.com

Our third title in this comparison is MDK2, another third person perspective game although this time, an OpenGL title.

The MDK2 demo comes with a built in benchmark that is executable from the game's control panel.  The only settings changes we made were resolution and color-depth.

Both ATI and NVIDIA take a performance hit here as they did in the first benchmark with Rune.  What is interesting in this comparison is that NVIDIA actually takes a larger performance hit than ATI does which is completely unexpected, especially judging by what we thought was representative's of NVIDIA's driver performance when we ran Quake III Arena in our earlier reviews.

In any case, the GeForce2 falls behind by approximately 12% under Windows 2000 while ATI drops 10%.  The two performance differences are very close to one another however the fact that a similar drop exists across two separate manufacturers indicates that maintaining performance improvement under Windows 2000 isn't as simple as we once thought.

Now that we have seen a performance drop occur under both a Direct3D and an OpenGL game we can't attribute the drop to a particular API being less optimized under one OS.  Let's continue on and see what else pops up.

American McGee's Alice
http://www.americanmcgee.com

American McGee's Alice offers a demented telling of Alice in Wonderland and unlike the original story, makes use of the Quake III Arena engine, which uses the OpenGL API if you're not already aware. 

Since the game had built in timedemo functionality but no demo, we simply recorded a quick demo at the very end of the game.  Other than the resolution and color depth, Alice's settings were left untouched.

We just mentioned that Alice was based on the Q3A engine, meaning that the same performance characteristics we saw under Q3A in our previous articles should appear here as well; and they do. 

The GeForce2 GTS actually enjoys a 10% increase in performance, which happens to be the first tangible performance increase we have seen thus far in the comparison.

With the latest drivers it seems as if ATI has definitely fixed the issues we noticed under Quake III Arena the first time around as the Radeon now offers Windows 2000 performance that is on-par with its performance under Windows 98.

idSoftware's Quake III Arena
http://www.idsoftware.com

We felt it very appropriate to follow up the Alice performance chart with one of Quake III Arena, since it is based on the Q3A engine after all.  This time around we simply used the demo001 in-game demo file to measure performance instead of having to record our own.

The game's settings were left on default with the exception of the resolution being set to 1024 x 768, the color depths being both set to 32-bit and the geometric detail set to medium.

Unlike Alice, there is no room for a performance boost here under Windows 2000 for ATI or NVIDIA.  So NVIDIA's 10% performance increase we saw with Alice does not translate over into the game that originally debuted the engine upon which "she" was based. 

Croteam's Serious Sam
http://www.croteam.com

The Croatian-based Croteam has recently released the second public test of their upcoming first-person shooter: Serious Sam.  This test actually has quite a powerful benchmarking engine in it.  The OpenGL game not only outputs the average frame rate but also outputs the sustained high and sustained low frame rate numbers during the course of the demo playback. 

The plethora of settings in the test were left at their defaults with the exception of resolution and color depth.  The built in 'Coop Party 04' demo was used to measure performance.

Regardless of what driver revision we used under Windows 2000, the Radeon would completely lock up during the benchmarking of the card.  It isn't a compatibility problem between the game and the Radeon because it worked just fine and performed quite well under Windows 98. 

It is worth noting that this is a very early test version of the game but in spite of that, NVIDIA's drivers perform just as well as they do under Windows 98 while ATI's prevent the card from even completing the benchmark under Windows 2000.

Let's hope that this problem is corrected before the shipping version of Serious Sam hits the streets. 

Epic Games' UnrealTournament
http://www.epicgames.com

UnrealTournament is also another benchmark that we are quite familiar with as it is a standard part of our benchmark suite here at AnandTech. 

This Direct3D test was conducted with all settings left at their defaults with the obvious exception of the resolution and color depth. We used Reverend's Thunder Demo for the test.

Since Rune was based on the UnrealTournament engine and both cards took a performance hit under that benchmark, we expected a similar showing here.  And although we did technically get similar results as in a performance drop among both candidates, the percentages were obviously much different.

The Radeon dropped a total of 46%, tying the performance drop that was noticed under Expendable. 

NVIDIA's performance drop was restricted to, at most, 3% unlike the 12% drop under Rune. 

Synetic/THQ's Mercedes-Benz Truck Racing
http://www.thq.com/

The final game we used was THQ's Mercedes-Benz Truck Racing.  As the name implies this is a racing simulator with Mercedes-Benz trucks.  The game makes use of the Direct3D API. 

We used the built-in benchmark to measure the frame rate, leaving all of the settings at the generic Direct3D adapter values other than the resolution and color depth.

ATI once again falls behind by a total of 21 fps or 39% while NVIDIA just nudges ahead as we move to Windows 2000. 

Final Words

Out of the eight games compared here, NVIDIA showed a performance drop when moving to Windows 2000 in three games; of those three only two were significant (more than 2%) deficits. 

ATI on the other hand lost ground under Windows 2000 in 6 out of the 8 games plus a seventh game would not even complete the benchmark under 2000.  Out of those 6 games where ATI dropped in performance, 5 of them exhibited more than a 2% drop.

Here's a quick summary of the results:

 

Today we proved that NVIDIA's Windows 2000 drivers aren't as perfect as they may seem as our test GeForce2 platform fell behind by 12 - 13% in two of the eight benchmarks when run under Windows 2000 when compared to its performance under Windows 98. 

We also managed to prove that ATI has some very serious issues that need to be tended to if they are to compete with NVIDIA going forward.  Windows 2000 is as much a power user's OS as it is a corporate OS and ATI's drivers under Windows 2000 are simply unacceptable for any gamer to tolerate.  They have definitely been getting better but they are still leaps behind NVIDIA.

It would be a great shame for the Radeon's successor to debut as a healthy alternative to the NV20 with ATI's drivers as its Achilles' heel yet again.  ATI will have around three months from the release of the NV20 to shape up their drivers for the launch of their next-generation graphics chip, let's just hope their Windows 2000 drivers are solid by then. 

As for NVIDIA, they aren't completely off the hook.  There are a few areas where performance improvement could be appreciated as we have shown by this small sample of games.  While their drivers are better than ATI's they still aren't perfect and could still use improvement.  However when building a Windows 2000-gaming system you're much better off with NVIDIA than you are with ATI simply because of drivers.  It is unfortunate since the Radeons are so aggressively priced right now.