It’s been a little more than a year and a half since AGEIA launched the PhysX PPU, and so far it’s fair to say that the product has been teetering on being a dud. As we’ve discussed in pervious articles, AGEIA has been battling both technical hurdles (extra PPU work dragging down the GPU) and software hurdles. When we discussed the issue over two years ago when AGEIA announced the PPU, we highlighted the likely problems that AGEIA would end up having getting developers to use their technology and unfortunately for AGEIA this has effectively come true: we can count the number of AAA titles released that support the PhysX hardware on one hand, in fact we’ve even benchmarked all of them. As the late Rodney Dangerfield would say, AGEIA just isn’t getting no respect.

As the average software development cycle is two-to-four years, the first products designed from scratch with PhysX support are just now emerging. It’s a short list. After a flurry of initial announcements, there aren’t a lot of well-known games on the horizon that are known to be supporting PhysX. And while this would probably be a swan song under any other circumstances, AGEIA has scored just a couple of significant wins as of late that will be keeping them in the game.

First and foremost is that for the time being their biggest competition is dead. GPU acceleration of physics, in spite of operating on a similar time table, has not panned out. The announcement of this technology and subsequent promise has certainly knocked some of the wind out of AGEIA’s operations, and the threat of a real GPU physics solution is always looming on the horizon (especially with AMD’s recent comments on DirectX 11). But with Intel’s acquisition of physics-leader Havok and the disappearance of their Havok FX package under mysterious circumstances, for the time being AGEIA can enjoy the fact that PhysX is the only game in town for hardware accelerated physics when it comes to gaming.

The other and far more important piece of news however is that in spite of drought of games supporting PhysX, AGEIA managed to get a single win some time back which is finally coming to fuition, and that win may very well make everything else irrelevant. That win? Epic’s Unreal Engine 3.

As game development costs have increased, studios have turned to licensing game engines rather than developing their own. For this generation of hardware and engines, there’s no bigger player in the market than Epic, whose Unreal Engine 3 has been licensed at a mind numbing rate. This has been fantastic news for AGEIA, who now is the default middleware provider for a significant percentage of first person shooters to be released over the next few years. With their troubles getting developers to adopt PhysX elsewhere, AGEIA needs PhysX support on Unreal Engine 3 games to be utilized to make or break the hardware.

Although several Unreal Engine 3 games have shipped since last year, Epic has still been hammering down the PC version of the engine and its PhysX hardware support. Only now has an Unreal Engine 3 game shipped with PhysX hardware support, Epic’s Unreal Tournament 3. With UT3 having shipped, AGEIA has reached a milestone: Unreal Engine 3 is finally shipping to developers with full PhysX hardware support, and a AAA game has finally shipped that can use PhysX for first-order physics, and a welcome change for all parties from previous AAA games that have only used second-order physics.

To get an idea of how PhysX will perform under the Unreal Engine 3, we’ll benchmark UT3 with and without PhysX hardware acceleration. While every game using the engine will be different and making strong predictions from a single datapoint isn’t possible, it will none the less give us an idea of what we can expect with future Unreal Engine 3 titles. Furthermore UT3 is a big enough title on its own that it can justify & drive PhysX sales if the performance is there, which with be the other major aspect we will be looking at today.

AGEIA’s Mod Pack & The Test

Unfortunately the PhysX situation in Unreal Tournament 3 is a little more convoluted than we would like. While the engine is capable of first-order physics using PhysX, because the game is not solely a technological showcase, there aren’t any maps that ship with it that heavily utilize physics interactions. The game does use the PhysX libraries to run some physical simulations, but other than a handful of situations such as cloth simulations, there isn’t much obvious physics work taking place. It’s not the eye-candy feasts that we’ve seen with other PhysX hardware enabled games, but this is also a benefit: UT3’s PhysX hardware support doesn’t enable any extra eye-candy, it solely offloads the physics work to the PPU. This will be an important distinction going forward as we’re going to be looking solely at issues of performance and not non-quantifiable (and performance-impairing) aspects of eye-candy.

AGEIA, apparently having taken charge of promoting PhysX on Unreal Engine 3, has been developing a mod kit to help modders/developers integrate physics situations in to their maps, and as a part of that kit has released a pair of maps that make heavy use of physical interactions as part of the gameplay. This makes UT3 straddle the line between being or not being a real/meaningful PhysX enabled game since as far as we know these are the only maps for the game that use first-order physics. This means that the biggest selling point for using a PhysX card with UT3 becomes the mod and not the game itself, but for the purpose of this article we’ll take the situation as it’s presented.

Along with our benchmarks of the shipping UT3 maps, we’ll include both of AGEIA’s mod maps in our benchmarks. The first map is CTF-Tornado, a map featuring a tornado that moves around unanchored objects, rips up some buildings, and sucks up unsuspecting players. As far as physical interactions goes this map is only moderately heavy, an intentional decision for allowing it to be playable without the PhysX PPU.

The second map is CTF-Lighthouse, AGEIA’s crown jewel out of their map pack, making very heavy use of physical interactions. This map features numerous destructible walls and floors, showcasing the direction AGEIA wants to push developers to go with physics. It’s also a pig of a map, as we’ll see it’s not playable at all without the PhysX PPU.

We’ve had to deviate a bit from our normal benchmarking routines, since UT3 does not support demo recording with AGEIA’s maps, presumably because of the first-order physics used. Instead we’ve gone with botmatches for all of our tests (including the normal UT3 maps) which offer variable results but also run the full client physics simulation process, ensuring that we’re capturing something fairly close to real world performance under repeatable circumstances. The use of bots means some CPU time is spent dealing with them, but given what we know about Unreal Tournament 2004’s player statistics, playing against bots turned out to be rather popular, so we don’t believe our results to be all that far fetched.

Every map we tested was run at 1600x1200 (our realistic scenario) and 800x600 (our anti-GPU bottlenecking scenario) both with the PhysX hardware enabled and disabled, on both a high-end QX6850 processor and a mid-range E6600. Varying all 3 processing components will allow us to reasonably isolate any bottlenecks in performance. All tests were run 3 times to reduce variability and the results averaged.

PhysX Performance Under UT3: AGEIA’s Maps

To start things off, we’ll begin with the physically demanding situations from AGEIA’s map pack before moving on to the stock UT3 maps. As AGEIA’s showpieces, these maps will trend to being PPU limited.

Although UT3 isn’t a particularly resource-hogging game, it does tend to be GPU limited more than any other situation. We see that on our 1600x1200 tests, where CTF-Tornado performs very similarly with both of our processors when PhysX enabled. What we end up with overall are very balanced results that show that everything can be a bottleneck at some point.

As we hinted at earlier, CTF-Tornado is generally playable with or without the PhysX PPU with any of our setups. Adding the PPU adds anywhere between 20-25% in our more realistic 1600x1200 situations, to 30-60% when we move away from the GPU bottleneck. However what the benchmarks don’t show is the performance of the map at its lowest; UT3 doesn’t give us the lowest frame rate values for these situations, but when staring directly in to the tornado – the most physically demanding action on this map – performance is certainly dropping in to the single digits. This is fine for bot matches, but this would be impractical for online play.

CTF-Lighthouse on the other hand shows massive performance improvements, and is completely unplayable without a PPU, where it’s literally a slideshow. Adding the PPU increases performance by a minimum of 280%, a welcome change from the PPU driving performance down in some of our past benchmarks. Even with the PPU however, CTF-Lighthouse struggles to be smooth at times, staying just on the side of playability. With no real way to vary the physics load we can’t test all possible options, but we believe it’s very likely that we’re PPU limited on this map, certainly at 800x600 when the GPU bottleneck is removed.

We also have a quick look at CPU usage for UT3 when running this map, to get an idea of just how well the CPU is being used; if it’s not being well used by the software PhysX simulations, then CTF-Lighthouse in particular is not a fair comparison. UT3 is designed around dual-core processors with some light helper threads to occupy cores 3 and 4, so if physics are the bottleneck then the multithreaded software PhysX simulations should be able to completely load the CPU.

However we are not seeing this. CPU usage is hovering at a little below 60% total usage on our QX6850. Given that we're not GPU limited, one thread in particular must be the CPU limited thread, and isn't capable of being further split. Our first thought is that the thread(s) that handle game logic also are doing the software physics, which would explain why we're not seeing a greater use of the other cores for the physics calculations. It's unfortunate if this is the case though, as it means that Unreal Tournament 3 (and possibly other Unreal Engine 3 games) is unable to fully utilize more than 2 cores and change. It also means that we potentially could be getting better software performance, which works against the PhysX hardware.

Never the less, this map is certainly designed to be a best-case scenario for the hardware, which is exactly what it delivers.

PhysX Performance Under UT3: Stock Maps

Moving on to performance with the stock UT3 maps, these do not involve any first-order physics to the best of our knowledge. However it’s still entirely possible to improve on second-order performance.

When we were originally benchmarking for this article, we started and intended to finish with WAR-OnyxCoast, an arctic-themed Warfare mode map that offers the full gamut of UT3 weapons and vehicles. What we found surprised us, so much in fact that we didn’t originally believe the results had we not been the ones doing benchmarking ourselves.

Against our initial expectations, WAR-OnyxCoast shows a performance improvement with the PhysX PPU installed, even with the light physics work being done. This is most obvious with our QX6850, where performance increases a respectable 15-20%. We ran this test several times more than the rest of our tests just to make sure this wasn’t a fluke or a product of botmatch variations and it’s not, there’s a definite performance improvement.

What’s driving this performance improvement we can only hypothesize on, compared to the more muted improvements with our E6600. Our best guess is that we’re CPU limited with the slower processor, with the limiting process being something other than physics (possibly bot AI), whereas that limit lifts with a faster processor. Of course we’d expect with more processing power the CPU would have no trouble keeping up with the PhysX PPU, but this doesn’t appear to be the case. It’s possible we’re still CPU and GPU limited on this map (explaining the change from 1600x1200 to 800x600), but we’re going to have to settle with the fact that we have a mystery on our hands for now.

Based on our results with WAR-OnyxCoast, we’ve also benchmarked CTF-Coret and WAR-Floodgate, to see if we can repeat those performance improvements, this time only with our QX6850.

Unlike WAR-OnyxCoast, neither of these maps showcases a large enough performance improvement with the PhysX hardware to positively confirm our earlier results. WAR-Floodgate is most like WAR-OnyxCoast out of these two additional maps, so it’s not particularly surprising that it peaks with an 8% performance improvement, but this is very close to the threshold for botmatch-induced variability, so it’s not conclusive. Meanwhile CTF-Coret flips between gaining and losing a marginal amount of performance, so we can be sure that it’s acting entirely within our experimental variability.

Looking at all of our results then, is the PhysX PPU improving performance under UT3’s stock maps? Probably. We can’t rule out other possibilities with the data we have, but our best explanation is that given a big enough map with enough players and vehicles, and enough of a computer to not be held down elsewhere, the PhysX PPU is giving us a measurable performance improvement of 10-20%. However we also have to keep in mind that with the frame rates we were already getting and the kinds of maps we believe this benefit is most pronounced on, that it’s not making a significant difference.

Closing Thoughts

Wrapping things up, we’re reminded again of AGEIA’s prowess at producing technical demos. Like the free CellFactor: Revolution and Warmonger games, these maps serve more as a demo of what developers can do with the PhysX hardware than anything else. But AGEIA is also closer to getting the PhysX PPU used in enough important games that it becomes something more than a piece of tech demo hardware.

The performance improvements offered by the PhysX PPU with AGEIA’s maps are substantial, which is where we would expect them to be, but CTF-Tornado also offers an interesting path for AGEIA to go down that we haven’t seen them go down yet. Out of the PhysX titles we’ve looked at so far, the addition of the PPU either adds performance-harming eye-candy (GRAW), effectively meaningless eye-candy (City of Heroes), or a game uses physics so heavily that it’ll never be a mass-market AAA title because it’s unplayable without the PPU (CellFactor).

We’ve been wondering if and when someone would produce a work using the PPU that showcases the PPU as a way to improve performance without the lack of said PPU killing performance, and we’ve found that work in CTF-Tornado. Perhaps it’s not an otherwise remarkable map, but we think AGEIA would certainly have an easier time selling the PPU if there were more games that could be playable with or without a PPU, while the PPU offered a tangible benefit.

Unreal Tournament 3 as measured by its stock maps however is not that game. There’s an interesting performance boost in what we believe are a fraction of the total maps, but it’s practically academic. The possible performance boost doesn’t occur in enough maps or occur as a big enough boost to clearly justify installing a PhysX PPU; either way the game is going to be completely playable. But it’s a start.

Unfortunately we find ourselves once again sitting on the fence when it comes to deciding if the PhysX PPU is a worthwhile piece of hardware or a dud. AGEIA continues to show us just enough in the way of games and performance that we can’t write it off, but we still don’t have the killer app that justifies the PPU either. It once again boils down to a matter of throwing it a new computer along with the kitchen sink, or playing exactly the right combination of games and enjoying exactly the right combination of eye-candy, that the PPU becomes worth it.

At the very least, AGEIA and its partners have brought down the price on PhysX cards from their heavy introductory prices. At least through the end of the year, AGEIA has BFG selling their PhysX card for $99, making the part a more practical purchase. This doesn’t change our fence-sitting position on the matter, but it does help to keep the PPU viable for now and keeps us from writing it off after so few games in the last year and a half.

Looking forward, it seems like it’s a matter of time however until AGEIA and its partners release a new PhysX PPU and cards based on said PPU. We’ve heard the rumors of a new, far more powerful part in production, and we know from AGEIA’s developer efforts that they intend to stay in the game a while longer yet. Coupled with the interesting situation on CTF-Lighthouse of being PPU limited, a new PhysX PPU seems imminent. When, how much, and how that will go over with still such few PhysX-supporting games out there remains to be seen. Now may not be the best time to throw in the kitchen sink however if AGEIA intends to release a new PhysX product within the next few months.