Way back at CES we saw our first glimpse of MSM8x60 and Qualcomm's Snapdragon Mobile Development Platform (MDP) with an MSM8660 SoC inside. Though we couldn't run any benchmarks on it, we got a sneak preview of dual core snapdragon. This year at MWC Qualcomm gave us a considerable amount of hands-on time with the same MDP hardware. We did a bit of testing, and then Qualcomm did something awesome - they let us take an MDP with the MSM8660 inside home with us.

In case you haven't been following, MSM8x60 is Qualcomm's first dual-core SoC. It's two Scorpion cores, an Adreno 220 GPU, and cellular modem all built on the 45 nm process. Like the rest of Qualcomm's lineup, the x in MSM8x60 can either be a 2, denoting GSM, UMTS, and HSPA+ support, or a 6, denoting all the same thing but with CDMA2000 and EVDO. Similarly, 9 will connote LTE support when we get to that point as well. MSM8x60 comes clocked at either 1.2 GHz or a higher 1.5 GHz. The MDP we were given was the higher 1.5 GHz variant. The MSM8x60 is the successor to MSM8x55, which is single core 1 GHz Snapdragon and Adreno 205, also built on 45 nm process. 

It's been a bit busy since we got the MDP home with us, but we've run and re-run tests on the MDP all along and are ready to talk about GPU performance. The MDP we have right now is optimized for evaluating graphics performance, which we'll take a look at in this article. We'll then go into architectural changes between Adreno 200, 205, and 220, and then do the same combo of performance numbers plus architectural deep-dive with the CPU side of things in a future article. 

The MDP itself is stark black and completely utilitiarian - it's a glossy, tall, extremely square package, but a smartphone nonetheless. It runs Android 2.3.2 (as an aside, isn't it ironic that prototype devices are running newer builds of Android than most smartphones you can buy retail right now) and has a number of interesting extras. There's a 13 MP rear facing camera with LED flash, front facing camera, 3.8" WVGA display, speakers, microHDMI port, microUSB, and microSD card slot. The MDP's battery is a comparatively tiny 3.3 Whr, but the entire package is totally mobile. 

The one we have is again optimized for GPU testing, and sadly lacked cellular connectivity or WiFi, but gets the job done nonetheless. 

The MDP's purpose is to act as a pre-commercial handset for software development rather than be something you'd ever carry around. Making reference hardware designed for developers isn't anything new in the SoC space - look no further than OMAP's Blaze platform. Qualcomm started making MDPs with the MSM8655 and sells them through bsquare, who sells them for $995. Though it isn't up yet, both these pages will soon have the MSM8660-based Mobile Development Platform. 

One of the more interesting things about the MDP is that because it's a development platform, not a consumer device sold by an OEM though a carrier, it can either have Vsync enabled or disabled. This is the framerate cap we've seen on virtually every other device, which usually is right around 60 Hz. Through adb, we can turn Vsync on or off on the fly, then reboot Android's surface flinger and run tests. Sadly this procedure, while simple on the MDP, doesn't work on any other devices that aren't running Qualcomm's Android builds.

Since all of the devices we've tested thus far have been commercial, they've shipped with Vsync on and thus are limited to around 60 FPS. Thus, more than a few platforms could have had instantaneous bursts during test runs which are clipped to Vsync. To be fair to those, we've run everything on the MDP with Vsync on and off for illustrative purposes. 

The MDP we tested was running all out, with maximum clocks and no power management thorottling CPU or GPU clocks down based on usage and load. As a result, testing was performed constantly plugged into a beefy USB AC power adapter. Though these benchmarks show off what MSM8660 is capable of, actual performance in shipping devices will obviously be different depending on UI, power management, and display resolution choices made by Qualcomm's OEM customers, like HTC.

GLBenchmark 2.0

GLBenchmark 2.0 – as its name implies – tests OpenGL ES 2.0 performance on compatible devices across multiple software platforms. The suite includes two long benchmarking scenarios with a demanding combination of OpenGL ES 2.0 effects - texture based and direct lighting, bump, environment, and radiance mapping, soft shadows, vertex shader based skinning, level of detail support, multi-pass deferred rendering, noise textures, and ETC1 texture compression. In addition, there's a large suite of subtests and feature tests.

GLBenchmark 2.0 is the best example of an even remotely current 3D game running on this class of hardware–even then this is a bit of a stretch. GLBenchmark 2.0 is still our current go-to test as it is our best best for guaging real world performance, even across different mobile OSes. Keep in mind that with GLBenchmark 2.0 we still cannot run at any resolution than native – in this case 800x480 (WVGA) – and the same applies for other devices in the suite, they're all at respective native resolutions. GLBenchmark 3.0 will fix this somewhat with the ability to render into an off-screen buffer of arbitrary size.

We never formally reviewed the T-Mobile MyTouch 4G, but have one nonetheless and have included it in our benchmark numbers a few times. Likewise, I purchased an HTC Inspire 4G for personal use which we'll review soon. The importance of these two devices is that they represent the current generation of single-core Snapdragon SoCs with Adreno 205 graphics. Comparatively, the 1.5 GHz MSM8660 with Adreno 220 is 2.2x faster than the 1 GHz MSM8655 with Adreno 205. 

Interestingly enough our run through Egypt came slightly higher with Vsync on than it did off - we're just showing the margin of error here. 

Pro is a less challenging test than Egypt, as it's simply the GLBenchmark 1.x main suite with OpenGL ES 2.0 features and shaders. Already we're at the framerate cap here on both MSM8660 and likely OMAP 4430. Pro likewise demonstrates huge gains from Adreno 205 to Adreno 220 - in this case 3.7x. 

Quake 3... er... kwaak3

Quake has been a regular fixture in our mobile GPU benchmarking suite, and likewise we ran it on the MSM8660. It's starting to show its age however, as we've quickly gone from virtually unplayable framerates a little over a year ago to hitting the 60 FPS framerate cap on virtually every newish SoC.

That said, it's still worth using to demonstrate performance gains, even if it isn't entirely representative of what game engines will look like going forward. 

Here we can see just how obvious that framerate cap is by looking at the clustering of results right around 55-60 FPS. The cap on Qualcomm's MDP is a precise 60 Hz, but the cap on Hummingbird and others is around 56.

There's a significant jump from Adreno 200 to 205 of 2.9x, though it's likely we're capped on the Inspire 4G. I'd have numbers from the myTouch 4G as well, but it's impossible to get console to come up since the search button has been repurposed as the "Genius" button. Now I remember why I got stuck on that phone. 

3DMark Mobile 2010

3DMark Mobile by Rightware is a new one for us. We've used Rightware's newer mobile GPU benchmark, Basemark, for some time now, but encountered instability on the MDP and MSM8660 and couldn't get test runs to complete and produce useable results. In addition there was some missing geometry and textures that we felt invalidated some runs. 

Qualcomm provided us an APK of 3DMark Mobile 2010 which I ran on the MDP and all the other Qualcomm SoC-packing devices I had on hand. I'm not sure whether this APK was Adreno specific like I've heard 3DMM can be, but for whatever reason it didn't want to run on any other devices on hand from any other vendors. It is useful for showing generational performance differences between Adreno 200, 205, and 220, however. 

The first test is Taiji and seems to be the more graphically demanding of the two, featuring some water pixel shading, normal mapping, and a complicated scene. 

In Taiji we can see truly generational performance differences between the three Adreno GPUs. Adreno 205 is 4.5x faster than Adreno 200, and then Adreno 220 is 2.0x faster than Adreno 205. Keep in mind we're also dealing with different clock speeds and memory architectures between those last two, however.

The second test is Hoverjet, and isn't quite as demanding. Interestingly enough, Adreno 205 doesn't do quite as well here. There's now a 4.5x performance gain from Adreno 205 to Adreno 220. 

3DMark Mobile 2010 feels a bit old to us, and Rightware has indicated that Basemark will be their flagship mobile GPU benchmarking suite going forward. Once those stability issues are addressed, we'll finish running the MDP – and thus MSM8660 – and update things. 

Quadrant 3D and 2D

Last and definitely least (at least in my mind) on the list is Quadrant, which has unfortunately become something of a de-facto one stop shop for benchmarking Android devices, famously spitting out one easy to digest score. Grab the pro version, and you get subtest scores. Its developers aren't very open about what all is being tested in the suite, the overall score is heavily skewed by I/O performance. Moreover the subtests themselves don't seem heavy enough for this current, much less upcoming, generation of SoCs. Benchmarking tools are rapidly evolving in the mobile space, and I expect to see far better tools quickly displace ones like Quadrant.

That said, we went ahead and ran the whole suite on the MDP just with Vsync off. I've seen some issues with Quadrant's 3D tests on Android 2.3 before, and the MDP was no exception. Again, we're going to do more CPU plus overall performance testing along with an architecture deep dive in an upcoming piece, but for now you can get a bit of that in the overall score. 

What all Quadrant is putting emphasis on with its 2D and 3D subtests is something of a mystery to me. There isn't a whole lot of documentation, but again it's become something of a standard. The 1.5 GHz MSM8660 leads in overall score and the 2D subtest, but trails Tegra 2 in the 3D subtest. If you notice the difference between Hummingbird (SGX540) from 2.1 to 2.3, you can see how Quadrant's strange 3D behavior on Android 2.3 seems to continually negatively impact performance. I saw the same odd missing texture and erratic performance back when I tested the Nexus S as I did on the MDP. Things like this and lack of updates are precisely why we need even better testing tools to effectively gauge performance.

As it is now, Adreno 220 shows anywhere from 2-5x performance gains over Adreno 205. Right now the reasons for those improvements aren't entirely clear, but we're going to do a GPU architecture deep dive relatively shortly and sort everything out. Likewise, we'll evaluate the 1.2 GHz MSM8660 and compare. Devices with the MSM8x60 aren't that far off - we're looking forward to getting our hands on the HTC Pyramid and EVO 3D which both will likely have 1.2 GHz MSM8x60 inside. 

Final Words

When we first started looking at Qualcomm's Snapdragon SoCs we were impressed by their CPU performance but largely put off by the performance of the Adreno 200 GPU. The 45nm Snapdragon with the Adreno 205 GPU changed things as it roughly doubled GPU performance. The Adreno 220 brings about another doubling in GPU performance.

We'll eventually see more conservative GPU performance gains in line with Moore's Law, however for the foreseeable future these sorts of yearly generational improvements are to be expected.