AnandTech Upgrade

To keep up with the AnandTech tradition, we'll open the covers on the latest upgrade performed at the AnandTech NOC Facility a few months ago. This latest upgrade focused on an enterprise database platform change, and a security infrastructure upgrade. We apologize for the delay, but there has been so much going on after the upgrade that this article took a bit of a back seat.

Planning for the upgrade started back in October of 2003. Our planning involved a review of the storage requirements of the databases, and growth predictions for the next 2-3 years. Next, we started research on a Firewall for the network. If you can believe it, we had been running Windows 2000 servers on the public internet with no firewall. Were we lucky? Probably, but we were meticulous in applying security updates as they came out and made good use of local security policies in Windows (especially after SQL Slammer hit us one morning).

It seems that all of our trips to the NOC to perform the various upgrades over the years have always been eventful to say the least. This upgrade actually went extremely well, which is attributed to some careful planning and the excellent staff at Elite Internet Communications (our ISP).

The Database Upgrade

For the past two years, we've been using two APPRO 1124 AMD Athlon MP 1U machines as our database servers; one ran the main site and the AD database, and the other ran the Forums database. They have worked very well for us, with an up-time of over 99%. Our only hardware failure with those machines was a couple of dead drives. We've been using two database servers in total to run the backend of AnandTech, mostly for redundancy. For this upgrade, we decided to go to one database server, which lessens the administration burden and keep one of the old APPRO servers as a cold backup if required. If you read our recent article on the Quad Opteron vs. Quad Xeon, you can probably guess which platform we chose.

We selected the Quad AMD Opteron 848 Server , and outfitted it with 8GB of memory, running Windows 2003 Enterprise. Why Windows? We chose Windows because we run Microsoft SQL Server. The choice of the Opteron wasn't difficult; it was the fastest platform in our review under 32 bit, and it is ready for 64 bit when we are.

Storage Requirements

Over the years, we've grown and so thus has our database size, especially the forums. We were nearly out of space prior to the upgrade - the old RAID array for the forums was 6 x 9GB IBM drives in RAID 10. It was time for a serious storage upgrade for our new database server. We hand-picked components to build a fast, reliable and large Raid 10 Array. We selected an LSI Logic 320 for the Raid controller and we used 8 Maxtor Atlas III 15K 36GB SCSI drives for the drive array. We spent a fair bit of time hunting down an enclosure for our new 140GB Raid 10 Array, and in the end, we selected CI Designs RMHR 9000. Most of the rack enclosures that we found were US$3,000+, but we picked up the CI Designs RMHR 9000 for just US$1300 (a steal in our opinion). This rack is outfitted with dual 300 watt redundant power supplies and 9 hot swappable drive bays.

Firewall

For the past few years, we've had our Windows 2000 servers sitting on the public internet. Most people would think that we're crazy, and we probably were! But, over those few years, we were only hit with 1 vulnerability (SQL Slammer). The main reasons for not implementing a firewall were cost and implementation time. With Anand in school and I, running FuseTalk, finding the time to implement was a challenge. So, after 4-5 years of hanging our network out there, we decided to protect the network with a firewall.

We spent a few weeks researching and pricing the various solutions for a network of our size. In the end, we chose a Netscreen 25 from Netscreen Technologies (recently acquired by Juniper Networks). The Netscreen 25 met our current needs with room to grow. The first thing that we had to look at was sessions, as most of the products out there are based on the number of simultaneous sessions that they will serve. The Netscreen 25 serves 16,000 simultaneous sessions and 4,000 new sessions per second. We serve anywhere from 3000 to 6000 simultaneous sessions, depending on the day. Throughput is probably secondary, since most of the firewalls in this range will handle more throughput than required. The Netscreen 25 is no exception, as it handles 100Mbit/sec of firewall throughput; we burst at 41Mbit/sec, depending on the day.

Conclusion

Overall, the upgrade went quite well. There were no major issues at all (unlike some of our previous upgrades). There was about a 3-hour outage while the work was being done, and a slight slowdown while we forced 100MB Full duplex on the Netscreen, as the Cisco 2948L3 that we use requires 100MB Full duplex to be forced or it starts to produce frame errors all over the place. Our next major change is upgrading the forums to the new FuseTalk .NET forum software, since we've had a few "issues" with ColdFusion under load on the forums recently. On the hardware side of things, we'll probably start looking at 64bit once the Windows 64bit platform is released. For now, we have a lot of headroom and a stable, secure and robust infrastructure.