Testing Hardware, Revised

We're working on the Corsair Obsidian 550D review, which should post alongside this article, but we're also putting both a new testbed and new methods of testing into play. Over the past year I've found that while our testing was comparable in a fairly global sense, there were definitely shortcomings to it that I felt warranted revision. When you're dealing with things like thermal performance and acoustics, getting consistent results is never as easy as you'd like it to be. In a perfect world we could produce a temperature-controlled anechoic chamber, but that just isn't feasible for me right now.

Before I get into the specifics of how we're revising our testing, let me introduce you to our new testbed for ATX and Micro-ATX enclosures.

Why did we make these changes? The CPU is actually incidental; we just need something that produces a substantial amount of heat. On the other hand, we've been needing a motherboard with a built-in USB 3.0 header for a few months now, as routing a USB 3.0 connection out the back of the enclosure has become passe at this point. The GA-Z68MX-UD2H-B3 is also Micro-ATX as opposed to ATX; instead of stratifying between ATX and Micro-ATX/Mini-ITX, it makes more sense now to stratify ITX as a separate platform. The Micro-ATX form factor just isn't the limiting factor it once was, and I've tested enough desktops to know you can very easily build a high performance machine on a Micro-ATX platform.

The graphics card was a source of a bit more debate. I didn't like how our old platform only overclocked the CPU, while the GTX 580 felt like too much card for the kind of everyday system that we wanted our stock settings to represent. NVIDIA's GeForce GTX 560 Ti winds up being an excellent compromise; cards in this thermal envelope and at this power level tend to hit the sweet spot in the market, while the 560 Ti can also have its clocks and voltage pumped up to the point where it starts producing thermals on par with an enthusiast-class card. The ASUS model we chose to use features the kind of aftermarket cooling that's becoming increasingly common, but also has a substantial amount of both thermal headroom and latitude in fan speeds.

Speaking of thermal headroom, I found that the Zalman CNPS9900 cooler used for the CPU on our previous testbed wound up more often than not causing acoustic and thermal results to skew. It was entirely too easy to hit the limits of that cooler in terms of just pulling heat off of the CPU. The Cooler Master Hyper 212 Evo we've opted to use instead is both inexpensive and efficient; I've personally been using a Hyper 212 Plus in my desktop to cool an overclocked i7-990X and I have no complaints. This cooler is capable of running quietly under ideal conditions but also has headroom (both thermally and acoustically) for less than ideal ones.

Finally, the remainder of our testbed consists of a couple of new parts. Corsair's new Corsair Link kit is useful for monitoring and logging temperatures and fits in a 3.5" drive bay, making it ideal for assembly and for thermal testing. The SilverStone Strider Plus 750W power supply is 20mm shorter than our previous testbed power supply, coming in at 160mm and allowing for both easier assembly in testing and more latitude. Instead of having to jump to another PSU for cases that don't have a lot of room for the PSU, it's easier to just say "this case tops out at 160mm."

Testing Hardware (Mini-ITX), Revised

As I mentioned before, we're also changing the way we split the test platforms. As Micro-ATX systems can be competitive with full ATX systems in terms of performance, it makes more sense to lump those two together than to lump Micro-ATX enclosures with Mini-ITX enclosures as we've done in the past. The result is a Mini-ITX testbed that's a little bit more geared towards what one would expect to find in a Mini-ITX system proper.

The impetus behind the changes for the Mini-ITX testbed can seem a bit more nebulous compared to the ATX, but let's discuss our thoughts.

On the motherboard/CPU/HSF front, we wanted a motherboard that had a native USB 3.0 header, something still rarefied in the Mini-ITX market. Of all the Mini-ITX boards on the market, Zotac's was the one best equipped for our purposes and realistically is probably one of the most feature rich Mini-ITX boards available in general. The CPU is also fairly representative of how much power someone might try to cram into an enclosure, as Sandy Bridge continues to strike an excellent balance between performance and thermals. Finally, our old heatsink/fan just didn't really spin up or down in testing; this is a limitation of not only Zalman's implementation but also the old motherboard's fan control. Either way, the low profile heatsink from SilverStone should resolve the problem just fine.

Where graphics are concerned, the GeForce GTX 580 in our old testbed was simply too much card for a Mini-ITX review. Oftentimes it was too big for the enclosures we wanted to test in, and even if it could fit there are very few Mini-ITX users likely to use such a powerful and power hungry card. While a single-slot AMD Radeon HD 7750 would probably have been ideal for our revision, it was more difficult to obtain in the wild. Zotac offers a model of GeForce GTS 450 that does the trick, though: the 450 Eco uses DDR3 instead of GDDR5 and has reduced clocks, but it also employs a single-slot cooler, is a short card in general, and requires no external power lead.

The remainder of our testbed carries over from the previous page.

Update: We originally included an i5-2300 in the Mini-ITX testbed. However, before testing with it we opted to swap to an i3-2120 to allow for testing cases with dedicated power supplies that wouldn't be able to handle a 95W CPU but could handle a 65W CPU.

Before continuing on, we'd like to thank the following vendors for furnishing us with the hardware to revise our  testbeds.

• Thank you to Puget Systems for providing us with the Intel Core i7-2700K and i3-2120.
• Thank you to Gigabyte for providing us with the GA-Z68MX-UD2H-B3 motherboard.
• Thank you to Zotac for providing us with the Z68ITX-A-E motherboard and GeForce GTS 450 Eco.
• Thank you to Crucial for providing us with the Ballistix Smart Tracer memory.
• Thank you to Corsair for providing us with the Corsair Link kit.
• Thank you to Cooler Master for providing us with the Hyper 212 Evo heatsink and fan unit.
• Thank you to Kingston for providing us with the SSDNow V+ 100 SSD.
• Thank you to CyberPower for providing us with the Samsung BD-ROM/DVD+/-RW drive.
• And thank you to SilverStone for providing us with the power supply and NT07-1156 heatsink/fan combo.

Testing Methodology, Revised

For those of you who aren't familiar or don't remember, here's a brief primer on how we used to test cases to get you up to speed.

Acoustic testing is standardized at one foot from the front of the case, using the Extech SL10 with an ambient noise floor of ~32dB. For reference, that's what my silent apartment measures with nothing running, testing acoustics in the dead of night (usually between 1am and 3am). A lot of us sit about a foot away from our computers, so this should be a fairly accurate representation of the kind of noise the case generates, and it's close enough to get noise levels that should register above ambient.

Thermal testing is run with the computer having idled at the desktop for fifteen minutes, and again with the computer running both Furmark (where applicable) and Prime95 (less one thread when a GPU is being used) for fifteen minutes. I've found that leaving one thread open in Prime95 allows the processor to heat up enough while making sure Furmark isn't CPU-limited. We're using the thermal diodes included with the hardware to keep everything standardized, and ambient testing temperature is always between 71F and 74F. Processor temperatures reported are the average of the CPU cores.

That all seems fairly reasonable, but over time subtle issues have crept in that we're taking the opportunity to correct.

For starters, we've found that while the Extech SL10 is perfectly fine for testing sound levels at about 37dBA and up, it's downright lousy for handling anything designed for silent running. The meter only has an official noise floor of 40dB, which is frankly a bit loud. To produce more accurate results, we've switched over to a beefier Extech SL130. The SL130 is rated to go as low as 30dB (basically the lowest any reasonably priced sound meter will go). In addition, I've actually moved since I started doing case reviews, and my new apartment is much quieter than the old one, resulting in an ambient noise floor well below 30dB. Unlike the SL10, the SL130 won't make "an educated guess" about sound levels below its rated floor, either. I'm continuing to test acoustics with the microphone a foot directly in front of the top of the enclosure to ensure consistent readings on that front. Anything below 30dB still rates as "near silent", but this is a big step away from 40dB.

Meanwhile, thermal testing has proven to be a bit trickier than initially anticipated. Maintaining a consistent interior temperature of an apartment (or even just one room) is easier said than done. Even a variation in ambient temperatures that I mentioned before can color results. As a result, instead of using the absolute temperatures reported from the hardware's thermal diodes during testing, I'm reporting the delta over ambient temperature. Ambient temperature is also measured at the beginning of each test cycle (after fifteen minutes of idle, and before fifteen minutes of burn-in.)

Test cycles are also being ever so slightly modified. I'm continuing to use seven threads of Prime95 to stress the processor, but GPU stress is now being handled by eVGA's OC Scanner instead of Furmark. Furmark is an odd duck that I've found to be unreliable as a GPU stress testing tool; Furmark just consumes power, but doesn't actually simulate proper GPU stress the way something like OC Scanner will. We've also seen some driver tweaks by both AMD and NVIDIA over the years designed to prevent Furmark, so it's best to use something else.

As before, I'm continuing to use the thermal diodes of the internal components rather than separate thermal sensors, and CPU temperature is reported as an average of the four cores. SSD temperature will continue to be included as a representative of how well the enclosure cools installed drives, but chipset and RAM temperatures are no longer going to be included. RAM thermals are really only relevant in extreme cases, and modern chipsets just aren't the heat generators that old dogs like the X58 were.

One new wrinkle I'm including is fan speed, though. Since the CPU and GPU fans are both thermally controlled, it may be useful to see just how hard these fans have to work in any given enclosure. These results aren't going to be strictly comparable between enclosures due to variations in ambient temperatures, but should be a reasonable starting point.

Conclusion: More Reliable Comparisons

The ultimate goal of the revised testing hardware and methods is to prune the excess data that wasn't particularly useful before while substantially improving the reliability of the results produced during the testing process. Our Sandy Bridge testing platform may not be state of the art for too much longer, and NVIDIA is no doubt cooking up a suitable replacement for the GeForce GTX 560 Ti—outside of pricing, we could even argue that GTX 680 is that replacement—but these components should remain representative of the kinds of thermal and acoustic loads end users will be building their desktops off of for some time to come.

As always, we welcome any comments or suggestions on what you'd like to see from our case reviews. Keep in mind that our general goal is to review the cases as they ship from the manufacturer, so replacing all of the case fans with, for example, $20 Sanyo Denki fans isn't our intention—particularly when you're reviewing sub-$100 cases, adding $60 or more in fans is a bit extreme. If a manufacturer wants to market a case as being quiet or silent, then the onus is on them to provide acceptable fans for that goal.

In the meantime, I'm personally looking forward to accumulating fresh data sets using these new methods. With a year of testing from the previous platform, we have a better understanding of some of the weaknesses of our previous testbeds. While our earlier results are still useful in a broad sense, future case reviews (starting with the Corsair Obsidian 550D) should allow for better comparisons in a finer, more granular sense.