Introduction and Testing Methodology

Small businesses and power users in a home setting have begun to face challenges with managing large amounts of data. These are generated either as part of day-to-day business operations or backing up of multimedia files from phones / tablets / TV recordings etc. One option is to use a dedicated COTS (commercial off-the-shelf) NAS from a vendor such as Synology or QNAP. Sometimes, it is also necessary to have a file server that is much more flexible with respect to programs that can be run on it. This is where storage servers based on Microsoft's offerings or even units based on Linux distributions such as Red Hat and Ubuntu come into play. These servers can either be bought as an appliance or assembled in a DIY fashion. Today, we will be looking at a system based on the latter approach.

A DIY approach involves selection of an appropriate motherboard and a chassis to place it in. Depending on the requirements and motherboard characteristics, one can opt for ECC or ordinary RAM. The platform choice and the number of drives would dictate the PSU capacity. The file server being discussed today uses the ASRock C2750D4I mini-ITX motherboard in a U-NAS NSC 800 chassis. 8 GB of ECC DRAM and a 400 W PSU round up the barebones components. The table below lists the components of the system.

ASRock C2750D4I + U-NAS NSC-800
Form Factor	8-bay mini-tower / mITX motherboard
Platform	Intel Avoton C2750
CPU Configuration	8C/8T Silvermont x86 Cores 4 MB L2, 20W TDP 2.4 GHz (Turbo: 2.6 GHz)
SoC SATA Ports	2x SATA III (for two hot-swap bays) 4x SATA II (for one OS drive)
Additional SATA Ports	Marvell SE9172 (2x) (for two hot-swap bays) Marvell SE9230 (4x) (for four hot-swap bays)
I/O Ports	3x USB 2.0 1x D-Sub 2x RJ-45 GbE LAN 1x RJ-45 IPMI LAN 1x COM1 Serial Port
Expansion Slots	1x PCIe 2.0 x8 (Unused)
Memory	2x 4GB DDR3-1333 ECC UDIMM Samsung M391B5273DH0-YH9
Data Drives	8x OCZ Vector 128 GB
Chassis Dimensions	316mm x 254mm x 180mm
Power Supply	400W Internal PSU
Diskless Price (when built)	USD 845

Evaluation Methodology

A file server can be used for multiple purposes, unlike a dedicated NAS. Evaluating a file server with our standard NAS testing methodology wouldn't do justice to the eventual use-cases and would tell only a part of the story to the reader. Hence, we adopt a hybrid approach in which the evaluation is divided into two parts - one, as a standalone computing system and another as a storage device on a network.

In order to get an idea of the performance of the file server as a standalone computing system, we boot up the unit with a USB key containing a Ubuntu-on-the-go installation. The drives in the bays are configured in a mdadm RAID-5 array. Selected benchmarks from the Phoronix Test Suite (i.e, those benchmarks relevant to the usage of a system as a file server) are processed after ensuring that any test utilizing local storage (disk benchmarks, in particular) point to the mdadm RAID-5 array. Usage of the Phoronix Test Suite allows readers to have comparison points for the file server against multiple systems (even those that haven't been benchmarked by us).

As a storage device on a network, there are multiple ways to determine the performance. One option would be to repeat all our NAS benchmarks on the system, but that would be take too much time to process for a given system that we are already testing as a standalone computer. On the other hand, it is also important to look beyond numbers from artificial benchmarks and see how a system performs in terms of business metrics. SPEC SFS 2014 comes to our help here. The benchmark tool is best used for evaluation of SANs. However, it also helps us here to see the effectiveness of the file server as a storage node in a network. The SPEC SFS 2014 has been developed by the IOZone folks, and covers evaluation of the filer in specific application scenarios like the number of virtual machines that can be run off the filer, number of simultaneous databases, number of video streams that can be simultaneously recorded and the number of simultaneous software builds that can be processed.

Our SPEC SFS 2014 setup consists of a SMB share on the file server under test connected over an Ethernet network to our NAS evaluation testbed outlined below. Further details about the SPEC SFS 2014 workloads will be provided in the appropriate section.

AnandTech NAS Testbed Configuration
Motherboard	Asus Z9PE-D8 WS Dual LGA2011 SSI-EEB
CPU	2 x Intel Xeon E5-2630L
Coolers	2 x Dynatron R17
Memory	G.Skill RipjawsZ F3-12800CL10Q2-64GBZL (8x8GB) CAS 10-10-10-30
OS Drive	OCZ Technology Vertex 4 128GB
Secondary Drive	OCZ Technology Vertex 4 128GB
Tertiary Drive	OCZ Z-Drive R4 CM88 (1.6TB PCIe SSD)
Other Drives	12 x OCZ Technology Vertex 4 64GB (Offline in the Host OS)
Network Cards	6 x Intel ESA I-340 Quad-GbE Port Network Adapter
Chassis	SilverStoneTek Raven RV03
PSU	SilverStoneTek Strider Plus Gold Evolution 850W
OS	Windows Server 2008 R2
Network Switch	Netgear ProSafe GSM7352S-200

The above testbed runs 10 Windows 7 VMs simultaneously, each with a dedicated 1 Gbps network interface. This simulates a real-life workload of up to 10 clients for the NAS being evaluated. All the VMs connect to the network switch to which the NAS is also connected (with link aggregation, as applicable). The VMs generate the NAS traffic for performance evaluation.

Thank You!

We thank the following companies for helping us out with our NAS testbed:

• Thanks to Intel for the Xeon E5-2630L CPUs and the ESA I-340 quad port network adapters
• Thanks to Asus for the Z9PE-D8 WS dual LGA 2011 workstation motherboard
• Thanks to Dynatron for the R17 coolers
• Thanks to G.Skill for the RipjawsZ 64GB DDR3 DRAM kit
• Thanks to OCZ Technology for the two 128GB Vertex 4 SSDs, twelve 64GB Vertex 4 SSDs and the OCZ Z-Drive R4 CM88
• Thanks to SilverStone for the Raven RV03 chassis and the 850W Strider Gold Evolution PSU
• Thanks to Netgear for the ProSafe GSM7352S-200 L3 48-port Gigabit Switch with 10 GbE capabilities.

Setup Impressions and Platform Analysis

The setup process for the file server included assembling the hardware as well as installing the operating system and configuring the shares. The install process on the motherboard side was quite simple - the ASRock Rack C2750D4I (already reviewed by Ian last year) is not a socketed motherboard and the SoC as well as the heat sink come pre-installed. I won't go too much into the motherboard itself, as it has been already covered in detail in our review. The gallery below presents some photos of the motherboard during the setup process.

We decided to use the U-NAS NSC-800 for our build. The U-NAS NSC-800 comes with eight drive bays in front of a hot-swappable back-plane (both SAS and SATA ones are available, and our chassis came equipped with the latter) and a compact size that can accommodate a mini-ITX motherboard. There are three internal 2.5" drive bays (which can be used for SSD cache drives) and / or for OS drives). There is space for a 1U power supply. There are two 120mm chassis fans just behind the drive bays. The drive sleds can accommodate both 3.5" and 2.5" disks, and the overall construction feels solid.

Compatible PSUs can be ordered from U-NAS itself. The 1U PSU that came bundled with our chassis was the 400W ASPOWER U1A-C20400-D. The PSU carries a 80 PLUS GOLD rating. It must be noted that the 5V and 3.3V outputs should not be subject to a load of more than 150W.

Pretty much the only downside of the chassis for a modern build is the absence of a front USB 3.0 port, but, given that the ASRock Rack C2750D4I doesn't have a USB 3.0 header, this is a non-issue in our build. The chassis comes with the cables from the backplane neatly laid out to enable easy connection to the motherboard. The challenge in assembling the build was actually routing all the wires from the PSU to the motherboard connectors while keeping it managed enough to place the chassis cover unhindered. Despite being a very compact 8-bay unit, the chassis supports a single slot PCIe card. The ASRock Rack C2750D4I does have a spare PCIe slot, and we populated it with an Intel ESA I340-T4 just for the purpose of the build (not used in the benchmarking). The gallery below presents some photographs taken during our assembly process.

The ASRock Rack C2750D4I platform has already been extensively analyzed in our dedicated review. However, I will reproduce the SoC diagram and how the motherboard components are laid out below.

It is important to have an idea of the platform during the assembly of the NAS. Instead of just choosing eight ports at random to connect the backplane to, the layout analysis allows us to select the proper ports. The configuration mentioned in the table in the previous page ensures that the Avoton SoC can talk to the drives of the system with maximum possible bandwidth.

Performance Metrics - Phoronix Test Suite

The file server's BIOS settings allow it boot off a USB key. We had no trouble doing the same with a portable installation of Ubuntu 14.04 (kernel version 3.16).

Database Benchmarks

The first test we look at involves determination of the time taken to perform 12500 record insertions into an indexed database. We use SQLite v3.7.3. SQLite performance depends to a large extent on the capabilities of the CPU. Benchmarks from other systems can be viewed on OpenBenchmarking.org.

The pgbench database benchmark involves recording the transaction rate for database operations using PostgreSQL. Unlike SQLite insertions-only test, pgbench is based on TPC-B, running five SELECT, UPDATE and INSERT commands per transaction.

Benchmarks from other systems can be viewed on OpenBenchmark.org

Web Server Benchmarks

The NGINX and Apache benchmarks record the number of static web page requests that can be serviced in a given time interval. It gives an idea of the load that can be handled if a given system were to be configured as a web server. The test load consists of a total of 500K requests (for NGINX) / 1M requests (for Apache) with 100 concurrent ones.

Benchmark numbers for both of these are available on OpenBenchmarking.org (NGINX, Apache).

TCP Loopback

The efficiency of the networking stack in the system (not to be confused with the hardware network adapter itself) can be determined by measures the loopback TCP performance. We record the time taken to transfer 10GB of data via loopback.

Given that we have the same networking stack for a given OS release across different hardware configurations, the efficiency is going to vary based solely on the CPU capabilities again. Benchmarks from other systems can be viewed on OpenBenchmarking.org.

CacheBench

CacheBench is an artificial benchmark to determine the performance of the cache and DRAM components in a system. It consists of three profiles - reads, writes and read/modify/writes. The bandwidth is recorded for each profile, with bigger numbers indicating better performance.

The numbers depend on the internal cache access speeds as well as the speed of the DRAM in the system. Benchmarks from other systems can be viewed on OpenBenchmarking.org

Stream

The system memory is tested out using the stream benchmark. The STREAM benchmark is a simple, synthetic benchmark designed to measure sustainable memory bandwidth (in MB/s) and a corresponding computation rate for four simple vector kernels (Copy, Scale, Add and Triad).

7-Zip Compression

The 7-Zip compression benchmark records the MIPS for the compression mode. This is the same benchmark that we use in the evaluation of mini-PCs, except that this is based on the Linux version. Higher MIPS ratings correspond to better performance, and the numbers are primarily based on the performance of the CPU in the system.

Benchmark numbers for other systems can be viewed on OpenBenchmarking.org

Linux Kernel Compilation

The timed Linux kernel compilation benchmark records the time taken to build the Linux 3.18 kernel. It is a good multi-discipline benchmark, stressing multiple aspects of the system including the DRAM, CPU and, to a certain extent, even the storage.

Benchmark numbers for other systems can be viewed on OpenBenchmarking.org

C-Ray

C-Ray is a simple raytracer designed to evaluate the floating point performance of a CPU. This is a multi-threaded test, and the time taken to complete the routine is recorded.

Benchmark numbers for other systems can be viewed on OpenBenchmarking.org

Performance Metrics - Storage Subsystem

In the previous section, we looked at various benchmarks for databases, web servers, general memory and CPU performance etc. For a file server, the storage performance is of paramount importance, since the main expectation from the system is one of writing to and reading from a disk volume protected against disk failure. In this section, we use Ubuntu 14.04 and mdadm to configure the disks in the hot-swap drive bays in a RAID-5 volume. Selected benchmarks from the Phoronix Test Suite are run with the RAID-5 volume as the target disk.

AIO Stress

Our first test in the storage benchmark is the AIO Stress PTS test profile. It is an asynchronous I/O benchmark, and our configuration tests random writes to a 2048MB test file using a 64KB record size, enabling apples-to-apples comparison with the other results reported to OpenBenchmarking.org

FS-Mark

FS-Mark is used to evaluate the performance of a system's file-system. The benchmark involves determination of the rate of processing files in a given volume. Different test profiles are used - processing 1000 files of 1MB each, processing 5000 files of 1MB each using four threads, processing 4000 files of 1MB each spread over 32 sub-directories and finally, 1000 files of 1MB each without using sync operations to the disk. The processing efficiencies are recorded in the graphs below.

PostMark

This benchmark simulates small-file testing similar to the tasks endured by web and mail servers. This test profile performs 25,000 transactions with 500 files simultaneously with the file sizes ranging between 5 and 512 kilobytes.

Numbers from the evaluation of other systems can be found on OpenBenchmarking.org

NAS Performance - SPEC SFS 2014

Evaluation of the system as a storage node on the network can be done using multiple approaches. As a simple NAS accessed from a single client, Intel NASPT would work very well. There are other artificial benchmarking programs such as IOMeter and IOZone (all of which are used in our dedicated NAS reviews). However, when it comes to file servers used in business scenarios, business metrics make more sense. For example, a database administrator might wonder how many simultaneous databases could be sustained on a given machine? An administrator in a software company might want to know how many simultaneous software builds could be processed on the machine if it were to be used as a storage node. SPEC SFS 2014 allows us to evaluate systems based on such business metrics.

Prior to discussing about the various business scenarios, let us take a look at the test setup (including details of the testbed and how the file server itself was configured.

Solution Under Test Bill of Materials

• ASRock Rack C2750D4I in a U-NAS NSC-800 (8GB RAM)
• AnandTech NAS Testbed (64GB RAM, 1GB to each member VM
• Netgear GSM7352S Ethernet Switch

Component Software

• ASRock Rack C2750D4I system running Windows Storage Server 2012 R2
• Load generators running on AnandTech NAS Testbed (10x Windows 7 VMs in a Windows Server 2008 R2 Hyper-V Installation)

Storage and File-Systems

• ASRock Rack C2750D4I - 8x OCZ Vector 128GB SSDs : Storage Spaces with Parity Space
• AnandTech NAS Testbed - NTFS partitions created at OS install time on OCZ Vertex 4 64GB SSDs

Transport Configuration

• ASRock Rack C2750D4I - 2x 1GbE LAN Ports in 802.11ad LACP to Netgear GSM7352S
• AnandTech NAS Testbed - 11x 1GbE LAN Ports to Netgear GSM7352S (1x management, 1x to each of 10 VMs)
• All SMB benchmark traffic flowed through the Netgear GSM7352S network switch

The four business metrics that we will be looking at today include:

• Database
• Software Build
• Video Data Acquisition (VDA)
• Virtual Desktop Infrastructure (VDI)

The database and software build categories are self-explanatory. The VDA profile refers to usage of a storage node as a recording target for streaming video (usually from IP cameras). The VDI profile refers to the number of virtual desktops / virtual machines that can be supported using the file server as a storage node for the virtualization infrastructure.

Database

The following graphs show the requested and achieved op rates for the database workload. Note that beyond four databases, the gap between them is more than 10% - this automatically means that the storage system is unable to support more than four databases concurrently. In all the workloads, it is the latency which decides the suitability and not the bandwidth available.

The SPEC SFS 2014 benchmark also provides a summary file for each workload which contains data additional to whatever is graphed above. The summary for the database workload is available here

Software Build

A similar analysis for the software build benchmark profile shows that the system is able to support up to 10 builds without any problems.

The report summary for the software build workload is available here

Video Data Acquisition

Video data acquisition for up to 10 streams is easily handled by our DIY solution.

The report summary for the VDA workload is available here

Virtual Desktop Infrastructure

VDI presents a very sorry story. The op rate achieved is not even close to the required rate, and the solution seems incapable of supporting any virtualization infrastructure.

The report summary for the VDI workload is available here

Miscellaneous Aspects and Concluding Remarks

Low power consumption and the ability to keep the drives cool under stress are some of the desirable characteristics of file servers. In order to evaluate this aspect, we subject the system with Windows Storage Server 2012 R2 to a stability test using AIDA64. For stressing the disks (storage spaces volume using the drive bays), we used Microsoft's DiskIO with the pre-defined worst-case SATA patterns issued randomly with verification.

We find that the CPU doesn't throttle and the cores stabilize around 87 C, with the motherboard at 75 C. During this process, we also recorded power consumption at the wall. We found that the maximum power consumption was around 70 W, while the idling power consumption was around 38 W.

The ASRock Rack C2750D4I with the U-NAS NSC-800 is the first of three file servers that we have evaluated with Ubuntu 14.04 and Windows Storage Server 2012 R2. While the Phoronix Test Suite benchmarks have been chosen for their relevant to file server workloads, it appears that the SPEC SFS 2014 benchmarks do not bring out what people use the Storage Spaces feature for. We are still optimizing the testbed for SPEC SFS 2014, and we do have plans to evaluate other NAS OSes used in DIY builds using the same approach.

Coming to the business end of the review, we have two things to discuss. The ASRock Rack C2750D4I is almost the ideal motherboard for power users wanting to build a dedicated low-power file server or NAS. Native USB 3.0 support would have been nice, but a add-on card solution is not a bad one for those desiring high-speed external I/O. Other than that, there is not much to complain - The layout of the motherboard's SATA ports ensure that most of the disks can talk at full speed to the Avoton SoC. This becomes an issue only if one decides to take maximum advantage by utilizing all the SATA ports on the board. However, in our evaluation scenario, as the platform for a 8-bay file server, it fit in perfectly.

This brings us to the second aspect - the 8-bay U-NAS NSC-800. There are plenty of storage-focused chassis designs to choose from. In fact, the Silverstone DS380 is an excellent choice for those looking to install the motherboard in a flexible system. On the other hand, the NSC-800 brings in the compactness factor into the equation. Simply put, it is not possible to get a 8-bay NAS chassis smaller than the NSC-800. Any smaller, and a mini-ITX board with a single-slot PCIe card is not going to fit in. Even though U-NAS tries its best to make cabling easier by providing the SATA cables in a bundle from the backplane, installing the motherboard in such a cramped space is definitely a challenge.

The build presented in this review has a diskless cost of around USD 845 - a completely reasonable number given that the unit contains 8GB of ECC RAM. A look at the prices of commercial off-the-shelf diskless file servers based on Avoton make the DIY approach very attractive.