You’ve been living too perfect of a life if you’ve never used the phrase “it’s been a long day,” and for NVIDIA it has most definitely been a very long day. Just over two weeks ago the graphics industry was shook by some very hard hitting comments from Gabe Newell of Valve, primarily relating to the poor performance of NVIDIA cards under Half Life 2. All of the sudden ATI had finally done what they had worked feverishly for years to do, they were finally, seemingly overnight, crowned the king of graphics and more importantly – drivers. There were no comments on Half Life 2 day about ATI having poor drivers, compatibility problems or anything even remotely resembling discussions about ATI from the Radeon 8500 days.

Half Life 2 day was quickly followed up with all sorts of accusations against NVIDIA and their driver team; more and more articles were published with new discoveries, shedding light on other areas where ATI trounced NVIDIA. Everything seemed to all make sense now; even 3DMark was given the credibility of being the “I told you so” benchmark that predicted Half Life 2 performance several months in advance of September 12, 2003. At the end of the day and by the end of the week, NVIDIA had experienced the longest day they’ve had in recent history.

Some of the more powerful accusations went far beyond NVIDIA skimping on image quality to improve performance; these accusations included things like NVIDIA not really being capable of running DirectX 9 titles at their full potential, and one of the more interesting ones – that NVIDIA only optimizes for benchmarks that sites like AnandTech uses. Part of the explanation behind the Half Life 2 fiasco was that even if NVIDIA improves performance through later driver revisions, the performance improvements are only there because the game is used as a benchmark – and not as an attempt to improve the overall quality of their customers’ gaming experience. If that were true, then NVIDIA’s “the way it’s meant to be played” slogan would have to go under some serious rethinking; the way it’s meant to be benchmarked comes to mind.

But rewind a little bit; quite a few of these accusations being thrown at NVIDIA were the same ones thrown at ATI. I seem to remember the launch of the Radeon 9700 Pro being tainted with one accusation in particular – that ATI only made sure their drivers worked on popular benchmarking titles, with the rest of the top 20 games out there hardly working on the new R300. As new as what we’re hearing these days about NVIDIA may seem, let us not be victim to the near sightedness of the graphics industry – this has all happened before with ATI and even good ol’ 3dfx.

So who are you to believe? These days it seems like the clear purchase is ATI, but on what data are we basing that? I won’t try to build up suspense senselessly, the clear recommendation today is ATI (how’s that for hype-less journalism), but not because of Half Life 2 or any other conspiracies we’ve seen floating around the web these days.

For entirely too long we’ve been basing GPU purchases on a small subset of tests, encouraging the hardware vendors to spend the majority of their time and resources optimizing for those games. We’re not just talking about NVIDIA, ATI does it too, and you would as well if you were running either of those two companies. We’ve complained about the lack of games with built-in benchmarks and cited that as a reason to sticking with the suite that we’ve used – but honestly, doing what’s easy isn’t a principle I founded AnandTech on 6+ years ago.

So today we bring you quite a few new things, some may surprise you, some may not. ATI has released their Fall refresh product – the Radeon 9800XT and they are announcing their Radeon 9600XT. NVIDIA has counterattacked by letting us publish benchmarks from their forthcoming NV38 GPU (the successor to the NV35 based GeForce FX 5900 Ultra). But quite possibly more important than any of those announcements is the suite of benchmarks we’re testing these cards in; how does a total of 15 popular games sound? This is the first installment of a multipart series that will help you decide what video card is best for you, and hopefully it will do a better job than we have ever in the past.

The extensive benchmarking we’ve undertaken has forced us to split this into multiple parts, so expect to see more coverage on higher resolutions, image quality, anti-aliasing, CPU scaling and budget card comparisons in the coming weeks. We’re working feverishly to bring it all to you as soon as possible and I’m sure there’s some sort of proverb about patience that I should be reciting from memory to end this sentence but I’ll leave it at that.

Now that the long-winded introduction is done with, let’s talk hardware before we dive into a whole lot of software.

The Newcomers

As we briefly mentioned, there are three new products to talk about today – the Radeon 9800 XT, the Radeon 9600 XT and then NVIDIA’s NV38.

The XT line of Radeon 9x00 cards is specifically targeted at the very high end of the gaming market. With AMD and their Athlon 64 FX, Intel and the Pentium 4 Extreme Edition, it’s not too surprising to see even more companies going this direction. With an ultra-premium part like the Radeon 9800 XT the profit margins are high and more importantly, the PR opportunities are huge – claiming the title of world’s fastest desktop GPU never hurts.

The effort required to produce a part like the Radeon 9800 XT is much lower than a serious redesign. When making any kind of chip (CPU, GPU, chipset, etc…) the design team is usually given a cutoff point where they cannot make any more changes to the design, and that is the design that will go into production. However, it is very rare that manufacturers get things right on the first try. Process improvements and optimizing of critical paths within a microprocessor are both time intensive tasks that require a good deal of experience.

Once ATI’s engineers had more experience with the R350 core and more time with it they began to see where the limitations of the GPU’s clock speed existed; remember that your processor can only run as fast as its slowest speed path so it makes a great deal of sense to change the layout and optimize the use of transistors, etc… to speed up the slow paths within your GPU. This oversimplified process is what ATI and their foundry engineers have been working on and the results are encompassed in the R360 – the core of the Radeon 9800 XT.

The Radeon 9800 XT is able to run at a slightly higher core frequency of 412MHz, quite impressive for ATI’s 0.15-micron chip (yes, this is the same process that the original R300 was based on). Keep in mind that the Radeon 9800 Pro ran at 380MHz and you’ll see that this 8% increase in clock speed is beginning to reach the limits of what ATI can do at 0.15-micron.

The Radeon 9800 XT does receive a boost in memory speed as well, now boasting a 365MHz DDR memory clock (730MHz effective) – an increase of 7% over the original Radeon 9800 Pro and an increase of 4% over the 256MB 9800 Pro. ATI was much more proud of their core clock improvements as we will begin to crave faster GPU speeds once more shader intensive games come out.

The Radeon 9800 XT does have a thermal diode (mounted on-package but not on-die) that has a driver interface that will allow the card to automatically increase its core speed if the thermal conditions are suitable. The GPU will never drop below its advertised 412MHz clock speed, but it can reach speeds of up to 440MHz as far as we know. The important thing to note here is that ATI fully warrantees this overclocking support, an interesting move indeed. Obviously they only guarantee the overclock when it is performed automatically in the drivers, as they do not rate the chips for running at the overclocked speed in all conditions.

The OverDrive feature, as ATI likes to call it, will be enabled through the Catalyst 3.8 drivers and we’ll be sure to look into its functionality once the final drivers are made available.

The Radeon 9800 XT will be available in the next month or so and it will be sold in 256MB configurations at a price of $499 – most likely taking the place of the Radeon 9800 Pro 256MB.

ATI is also announcing the Radeon 9600 XT; a much more exciting derivative of the Radeon 9600 Pro based on a low-k dielectric 0.13-micron process. The benefits of a low-k dielectric are mainly related to shielding from crosstalk in high transistor density chips; the benefit of a low-k process is mainly the ability to scale up clock speeds, which is why you will see that ATI is able to clock the 9600 XT at 500MHz. According to ATI, the Radeon 9600 XT should be the first mainstream part to outperform the Radeon 9700 Pro in all situations – not bad for a $199 card.

The Radeon 9600 XT will hit the streets sometime in November and we’ll be sure to bring you coverage of that card as soon as we get our hands on one.

Finally we have the NV38, NVIDIA’s Fall refresh part; we won’t see NV40 and R4x0 until next Spring so both companies are bringing out higher clocked versions of their current cards in order to compete during the holidays.

Just like the Radeon 9800 XT, the NV38 is basically a higher clocked version of the NV35 (GeForce FX 5900 Ultra) with a new cooling system. Now running at 475/475 (950MHz DDR), the NV38 boasts a 5% increase in core clock and an 11% increase in memory frequency.

The card will officially be launched next month but we’re going to be able to bring you a preview of the NV38’s performance today.

The New Test Suite

As we mentioned at the beginning of this article, we are introducing a brand new test suite with this review and we are also kicking off the first installment of a multipart series covering multiple aspects of current (and somewhat next) generation gaming performance.

By no means should you take the limited (yet extensive) tests we have here as all you will see from us, but rather something to whet your appetite for what is yet to come. The focus of this review is plain and simple – comparing the basic performance of the latest offerings from ATI and NVIDIA. In the future installments we will cover image quality, CPU scaling and other aspects of performance in greater detail. We will be making notes of noticeable visual differences between ATI and NVIDIA in this article, but a comparison with supporting images will be done in Part II of the series.

As far as the new test suite is concerned, here are the benchmarks that made it in:

AquaMark 3
Command & Conquer Generals: Zero Hour
F1 Challenge ’99-‘02
Final Fantasy XI Benchmark 2
Halo
Homeworld 2
Jedi Knight III: Jedi Academy
Microsoft Flight Simulator 2004
Neverwinter Nights: The Shadows of the Undrentide
Simcity 4
Splinter Cell
Unreal Tournament 2003
X2
Warcraft III: Frozen Throne
Wolfenstein: Enemy Territory

We are working on expanding the suite even further, but for now this is what we have. If you’d like to see more games added please feel free to let us know either by sending an email or even better, leaving a comment through the system at the bottom of the page.

We used ATI’s publicly available Catalyst 3.7 drivers and in order to support the NV38 we used NVIDIA’s forthcoming 52.14 drivers. The 52.14 drivers apparently have issues in two games, neither of which are featured in our test suite (Half Life 2 & Gunmetal).

Our test bed was configured as follows:

2.8GHz Intel Processor Prescott
512MB DDR400
Intel 875P Motherboard

AquaMark 3

Despite what some people would like everyone to think, Aquamark3 is really a test of how people developing software now envision DirectX 9 pixel and vertex shaders will be used in the future. The situation is very reminiscent of the first Sony PlayStation: the first games that used the technology were limited by the hardware until developers really learned to work with the hardware rather than on the hardware. As time progressed, we went from what were essentially ports of 16bit console games to amazingly complex and beautiful games like Gran Turismo 2. The same thing will happen with shader technology, and no amount of guessing and throwing functions at a gpu will tell you how its performance will really be in the future. Essentially, my advice is that any piece of software that claims it is a valid predictor of future performance should be taken lightly. We based our decision to include Aquamark3 on its popularity in the community. Aquamark3 is a cool piece of software, with some pretty neat tests, and a high score in any benchmark can still earn bragging rights in the forums. The only Aquamark3 test we ran was the publicly available 1024x768 4xAF noAA in order to maximize the usefulness of these numbers to the community. Our drivers were set to allow application control of AF and AA.

We can see almost a pairing off of the cards in direct competition with one another from each camp. ATI pulls ahead by an insignificant margin in the case of the top cards, but the 5600 Ultra falls way behind in this test. Image quality appears to have improved for NVIDIA in this benchmark over what has been reported of previous drivers, and the NV38 handled the massive overdraw portion of the test the smoothest of all the cards. We will be taking a much closer look at image quality very soon, but until then, it looks like ATI and NVIDIA have equal footing in the Aquamark3 arena and we are left to find more useful information about their differences elsewhere. We would also like to point out that the 9700 Pro held its own in this test inching out the standard 9800.

Command & Conquer Generals: Zero Hour

The recently released expansion to the very popular Real-Time Strategy (RTS) game Command & Conquer Generals seems to do a good job of living up to the standards set by its prestigious ancestry. RTS games usually get overlooked in GPU roundups and comparisons as they aren’t considered graphically intense. However, smoothness is very important to gameplay; goodness knows I’ve blamed plenty of lost armies on ill timed drops in framerate. For this benchmark, we created a multiplayer game consisting of 6 hard armies on one team with us, and one easy army. We then used the replay feature in conjunction with FRAPS to measure performance. This was done with and without 4xAA/8xAF.

In this first test we can see that all the ATI cards are huddled together at the top while the nvidia cards lag behind. Clearly this game favors the ATI architecture. One of NVIDIAs strong points, memory bandwidth, doesn't get a chance to shine in this game as its mostly small textures and low poly objects with some pretty cool particle effects. That kind of setup just doesn't tilt in NVIDIAs favor.

Even with AA and AF enabled neither camp is severely hampered; and the only card that really drops off significantly is the 9600 Pro. The fact that the FX 5900 and NV38 are neck and neck suggests that the reason for NVIDIAs performance in this benchmark has something to do with an aspect of the architecture that isn't directly (or significantly?) affected by GPU core clock or memory bus bandwidth/speed; more than likely we're talking about driver issues here.

F1 Challenge ’99-‘02

We’ve had plenty of requests to benchmark with a racing simulation. When we were designing the new testing suit this was one of the first games that came to mind. The game is faced paced, has lots of graphics options, and could keep someone who is into F1 racing busy for weeks at a time. Combine all of that with a nice replay feature and we have a very useful benchmark. We just ran a lap at Australia and counted the framerate of the replay via FRAPS while following one of the drivers in the middle of the pack.

In this bench, everyone seems to being doing really well with the exception of the two lowest end cards. It seems very clear that this test is CPU bound, and we are looking forward to benching some CPUs with this game (as well as trying to push the highend cards with some higher resolution tests). There really is no clear winner, but NV38 does come out at the top of the pile.

When we flip the filter switch, the 9800 XT drops the least in frame rate, and takes a clear lead over NV38 and the 9800 Pro. Usually NVIDIA is the camp gaining the most ground after AA and AF are enabled, but it is very much worth noting that in this benchmark (and others we will point out later) AA and AF didn't really seem to work as well on the NVIDIA cards as it did on the ATI cards. There was some difference between the two, but we will have to do more research into this area before we can bring forth anything conclusive.

Final Fantasy XI

The Final Fantasy series has always been popular, and will probably always be popular. I have a personal affinity towards the series, but I’m not so sure about the Massively Multiplayer Online style they are going for this time around. The benchmark is much more straightforward than the success of their future game: we set it to high-res benchmark mode (1024x768) and record the number of frames that get rendered (in the bottom right hand corner). Then we divide that by the length of time the benchmark runs to get an average frames per second.

The two most expensive ATI cards manage to nudge out the rest of the pack which are mostly tied. Again, the 9700 Pro does a good job of remaining competitive, and the 9600 Pro is still hanging on while doing a good job of clearly beating the 5600 Ultra. Unlike the Aquanox3 style of benchmark which has a set number of frames to render, this benchmark has a set time to move through a scene and render as many frames as possible while doing so. That makes image comparison a little more difficult. No matter what hardware this benchmark is run on, there seems to be a lot of motion blur and antialiasing is not an available option (and doesn't work even if its enabled at the driver level).

Halo

With Halo in stores today, we have welcomed it with open arms into our testing suite. The packaged benchmark that comes with the game is made up of all the cut scenes between levels (which are fairly graphics intensive). We ran the benchmark at 1024x768 @ 75Hz , which provided plenty of work for all our cards. We opted to simply include average framerate for this article, but there are some really interesting features of this benchmark (like percentage of time above a certain frame rate) that we may revisit later.

This benchmark clearly lets the 9800XT stretch its legs a little and pull away from the pack Of course, 51 fps is not really what you want to see for a first person shooter on the PC. We also don't get the benefit of AA (and therefore we don't get to see if the NVIDIA chips could make up some slack via their memory bandwidth), as AA is not currently supported in Halo.

Homeworld 2

This game is a space based RTS with plenty of particle effects and eye candy. Framerates can jump wildly from hundreds down to the teens depending on how close you want to get to the action. We decided to run two benchmarks (both with FRAPS), one on an in game cut scene (benchmark 1), and the second on a brief skirmish (benchmark 2) that was easily repeatable (the AI responded consistently to my inaction after the cut scene, so we could easily watch the same destruction occur over and over). These two measurements seemed to reflect the different aspects of gameplay well.

NVIDIA takes this test hands down. The ships of homeworld have some really nice textures, and there are some beautiful particle effects that can be seen just flying around in space watching things blow up. This first test is mostly just watching one big ship fly up past the camera.

We see that the trends remain the same with AA/AF enabled. The 9800XT gained some ground on the NVIDIA cards, but not enough to make a real difference in this test. This is another test where it was not apparent that NVIDIA's AA was doing as much as needed for the scene, so we will take a closer look at this benchmark when we do our image quality comparison.

In this battle scene (with more ships, more textures, and lots of missiles and laser fire), NVIDIA's lead is even more pronounced, could also be due to the cards memory bandwidth as there are more textures that need to be moved in and out of video RAM.

Again, we are not sure about the quality of the AA and filtering in this benchmark, but we will be looking into the matter very promptly. For now though, NVIDIA wears the Homeworld 2 crown.

Jedi Knight: Jedi Academy

Jedi Academy is another game in a series with a distinguished history. This game is a bit of a blend between first person shooter and third person (Tomb Raider style) action depending on your weapon of choice. In order to bench this game we ran FRAPS over an in game cut scene featuring Chewie, gunfire, and a nice explosion.

NV38 takes this benchmark, followed by its little brother the 5900 Ultra. And...

Barring any image quality problems, NVIDIA takes the AA/AF benchmark as well.

Microsoft Flight Simulator 2004

This was a difficult game to benchmark with. Anand struggled for a long time to work something out that got around the randomly generated weather. We finally got a benchmark together, but we had to turn off weather completely. As time allows, we may introduce another benchmark that incorporates the weather feature in a repeatable fashion. For this test, FRAPS was used to determine the average framerate while flying over a specified section of sea and land.

Clearly this test shows ATI as the leader in performance for this MS game. CPU scaling should also prove to be valuable as it seems like the ATI cards want to go further but they are just being held back by the rest of the system.

Neverwinter Nights: Shadows of Undrentide

This is the expansion to the popular RPG game Neverwinter Nights, and it features some of the most amazing particle effects we've seen in any game. The magic is all very well done and this is simply a beautiful game to watch. You may not like playing it unless you are already into RPGs or story driven adventure games, but anyone can enjoy watching a cut scene or two from Neverwinter. We happened to be able to benchmark the very first in game cut scene in the expansion as it had a nice sized battle with magic flying everywhere. FRAPS was again used to determine framerate.

NVIDIA takes this benchmark.

Surpisingly, even the 5600 Ultra is favored under Neverwinter, as it climbs out from under the bottom of the pile to just above the venerable 9700 Pro.

Simcity 4

Simcity is another game that was tough to benchmark; we finally decided that most of the time, people are scrolling around, so why not benchmark framerates while we scroll. This was done in the Big City Tutorial map; we essentially just moved side to side over a set path and recorded the framerate with FRAPS.

Without AA/AF, we see the entire ATI family climb on top of NVIDIA. Moving to AA/AF:

The gain becomes even more pronounced, and everything looks like its become CPU bound. This will definitely be another good CPU test.

Splinter Cell

Splinter Cell has been used before in our testing suites, and we are using the same procedure used for the Athlon 64 tests.

With this test we get another a stratification similar to that of Aquamark3: the cards seem more or less evenly matched with thier direct competition.

Unreal Tournament 2003

Everyone knows Unreal Tournament 2003 as a benchmark, and everyone has optimized the stuffing out of it. People play it like a religion, and we'll do the same from the testing side.

ATI's hardware takes the lead in the flyby benchmark but not by much.

The XT just pulls ahead to a greater percentage lead when AA/AF are enabled. Of course, NV38 moves ahead of everything else in the pack, but the XT is the clear leader.

X2

X2 is a game that is due out in November, but for now, they have a really extensive benchmark/demo out that pushes a lot of different aspects of the graphics engine. This benchmark is a simple download and run. We used the benchmark with everything but antialiasing enabled.

From these two graphs, it seems like NVIDIA is the clear winner, but in watching this demo run so many times, we noticed that the NVIDIA cards were running choppier than the ATI cards, and we again had some image quality questions we need to answer. We will also have to take a look at the variation between setting AA in the driver and in the application, if, indeed, there are any differences. This benchmark does say that NVIDIA would be the top choice for funning X2 right now, but it would be best to hold off on making that call until we can explore these results a little better.

Warcraft III: Frozen Throne

Warcraft III is a very popular game, and while its not the most stressful graphics test out there, the replay mode offers the ability to track a player and speed the game up to 8 times its original speed. This can really push some numbers through the CPU and GPU. We ran this test on a replay of a tournament match we found online.

Everything seems fairly CPU bound until...

Anisotropic Filtering and AA give a clear lead to the ATI cards in this case.

Wolfenstein: Enemy Territory

This entire game is actually downloadable off the internet. We downloaded the replay we used from 3dcenter.de (radar.dm_82), and ran it as a timedemo.

All the NVIDIA cards seemed to stick together and perform about the same, and with the exception of the leading 9800XT, ATI cards performed about the same as eachother as well.

In looking at these tests, Wolfenstein seems to be a little more CPU bound that GPU bound, but we will definitely find out in one of our coming articles just how true that is. In this case, turning on AA/AF shifts the leadership of the pack away from ATI to NVIDIA.

Final Words

We’ll limit the talk about the new batch of $500 cards to the first few lines of this conclusion. The Radeon 9800 XT offers a marginal performance improvement over the regular Radeon 9800 Pro, definitely not worth upgrading to for current 9800 Pro owners.

As far as people looking to upgrade once for the long run, with new architectures due out in 6 months, a $500 investment today would be significantly more out of date than if you purchase a card right before a refresh. We rarely recommend that you buy the fastest performing card on the market; in fact the last time we did that was with the Radeon 9700 Pro – the impact of which is clearly not equaled by the Radeon 9800 XT (nor were we expecting it to). Whether spending $500 is worth it today is your call, but you can definitely get very similar performance out of a used Radeon 9700 Pro or even a non-Pro Radeon 9800 at much better price points. If money is no object, then we’re sure that ATI wouldn’t mind shipping a few more XTs in your direction.

We’re quite wary of recommending any of the current NVIDIA cards at this point, for two major reasons. First, with NV38 coming right around the corner any FX 5900 Ultra purchases wouldn’t be wise investments. Also, given the marginal performance improvements you can expect out of a 5% core clock increase, don’t have incredibly high expectations for the NV38. We can’t recommend the GeForce FX 5600 Ultra because NVIDIA has already indicated that NV36 (the 5600 Ultra’s successor) will be here shortly to replace it and should offer significantly greater performance. So if you’re looking to buy a video card right now, ATI is the way to go.

Looking at the stats, ATI clearly wins in 6 games, NVIDIA wins in 4 games and the two come very close in 5 games. Games such as Command & Conquer Generals: Ground Zero and Simcity 4: Rush Hour are examples where ATI clearly has the lead over NVIDIA and the argument could be made that ATI holds the lead because they optimize for all games, while NVIDIA just optimizes for benchmark titles. However, looking at games like Homeworld 2 and Neverwinter Nights you could make the exact opposite argument.

What’s clear is that both manufacturers optimize for the more popular games and the focus of optimizations is obviously greater on more visible games. With that said, we’re hoping that by expanding our test suite we will be able to encourage optimizations to make more games run better. We’ll see how the picture we’ve depicted here today changes as time goes on.

Although we did provide some insight into the “next generation” of games with scores from Halo, the real question on everyone’s mind is still Half Life 2 as well as Doom3. The performance crown under Doom3 is still in NVIDIA’s camp apparently, and although the latest drivers have closed the gap significantly, ATI is still ahead in Half Life 2. The numbers we’ve seen indicate that in most tests ATI only holds single digit percentage leads (< 5%), although in some cases ATI manages to pull ahead by double digits.

There’s much more to come, but for now we’ve given you quite a bit to chew on…