Introduction and Testbed Setup

QNAP is one of the leading vendors in the COTS NAS (commercial off-the-shelf network attached storage) space. They have a wide variety of hardware platforms to choose from, ranging from ARM-based single-bay systems to Xeon-based rackmounts. Earlier this week, they launched the Bay Trail-based TS-x51+ series for home and SOHO users.

Despite Intel having launched the Braswell platform for storage applications, QNAP has opted to remain with Bay Trail for the TS-x51+ series. A look at the specifications reveals that the core SoC and memory capacity seem to be similar to the TS-x53 Pro launched last year. However, while the TS-x53 Pro targets the mid-end SMB market, the focus of the TS-x51 is more towards the home consumer side. The differences between the TS-453 Pro and the TS-451+ are listed below:

• 3x USB 3.0 ports on the TS-453 Pro vs. 2x USB 3.0 ports on the TS-451+
• 4x GbE LAN ports on the TS-453 Pro vs. 2x GbE LAN ports on the TS-451+
• No LCD display / local hands-on control on the TS-451+
• Internal 250W PSU in the TS-453 Pro vs. External 90W power brick in the TS-451+
• IR remote controller (MCE compatible) included in the TS-451+, optional with the TS-453 Pro
• Industrial design differences in the chassis and drive trays

The specifications of our review sample of the QNAP TS-451+ are provided in the table below

QNAP TS-451+ Specifications
Processor	Intel Celeron J1900 (4C/4T Silvermont x86 @ 2.0 GHz)
RAM	8 GB
Drive Bays	4x 3.5"/2.5" SATA II / III HDD / SSD (Hot-Swappable)
Network Links	2x 1 GbE
External I/O Peripherals	2x USB 3.0, 2x USB 2.0
Expansion Slots	None
VGA / Display Out	HDMI (with HD Audio Bitstreaming)
Full Specifications Link	QNAP TS-451+ Specifications
Price	USD 649

The various specifications of the NAS are backed up by the data gleaned via SSH access to the unit.

Having reviewed QNAP's SMB-targeted NAS units for the last couple of years, I was a bit put off by the industrial design of the unit. The drive caddies of the TS-451+ are plastic and flimsy. Despite the home consumer / SOHO focus, I would prefer that the caddies be the same as the ones used in the Pro series at this price point. Otherwise, the local control / LCD display panel is not going to be missed much (I rarely used that feature even in the Pro series). An in-built PSU would be good, but most SOHO NAS units come with power bricks and it is something difficult to get rid of in the units targeting this price point / market segment.

Our review unit initially shipped with QTS 4.1.4, but we upgraded to QTS 4.2 for our evaluation. The setup process itself is quite straightforward. Upon connection to the network, the QNAP TS-451+ receives a DHCP address even in a diskless state. The IP address can be determined either from the DHCP provider in the system or via the Qfinder utility. Accessing the IP address with the default admin/admin login credentials got us going with the setup process. We started off with one disk in the unit, and it was configured as a JBOD volume. Disks were added one by one, migrating in the process from JBOD to RAID-1 and on to RAID-5. The QTS OS handled the RAID migration and expansion without any issues. A detailed discussion of QTS 4.2 and the QNAP mobile apps will be done in an upcoming review.

In the rest of the review, we will take a look at the benchmark numbers for both single and multi-client scenarios across a number of different client platforms as well as access protocols. We also have a separate section devoted to the performance of the NAS with encrypted shared folders. Prior to all that, we will take a look at our testbed setup and testing methodology.

Testbed Setup and Testing Methodology

The QNAP TS-451+ can take up to 4 drives. Users can opt for either JBOD, RAID 0, RAID 1, RAID 5, RAID 6 or RAID 10 configurations. We expect typical usage to be with a single RAID-5 volume. To keep things consistent across different NAS units, we benchmarked a RAID-5 volume. Four Western Digital WD4000FYYZ RE drives were used as the test disks. Our testbed configuration is outlined below.

AnandTech NAS Testbed Configuration
Motherboard	Asus Z9PE-D8 WS Dual LGA2011 SSI-EEB
CPU	2 x Intel Xeon E5-2630L
Coolers	2 x Dynatron R17
Memory	G.Skill RipjawsZ F3-12800CL10Q2-64GBZL (8x8GB) CAS 10-10-10-30
OS Drive	OCZ Technology Vertex 4 128GB
Secondary Drive	OCZ Technology Vertex 4 128GB
Tertiary Drive	OCZ Z-Drive R4 CM88 (1.6TB PCIe SSD)
Other Drives	12 x OCZ Technology Vertex 4 64GB (Offline in the Host OS)
Network Cards	6 x Intel ESA I-340 Quad-GbE Port Network Adapter
Chassis	SilverStoneTek Raven RV03
PSU	SilverStoneTek Strider Plus Gold Evolution 850W
OS	Windows Server 2008 R2
Network Switch	Netgear ProSafe GSM7352S-200

The above testbed can run up to 25 Windows 7 or CentOS VMs simultaneously, each with a dedicated 1 Gbps network interface. This simulates a real-life workload of up to 25 clients for the NAS being evaluated. All the VMs connect to the network switch to which the NAS is also connected (with link aggregation, as applicable). The VMs generate the NAS traffic for performance evaluation. However, keeping in mind the nature of this unit, we restricted ourselves to a maximum of 10 simultaneous clients. A detailed explanation of our solution-based benchmarking approach is available here.

Thank You!

We thank the following companies for helping us out with our NAS testbed:

• Thanks to Intel for the Xeon E5-2630L CPUs and the ESA I-340 quad port network adapters
• Thanks to Asus for the Z9PE-D8 WS dual LGA 2011 workstation motherboard
• Thanks to Dynatron for the R17 coolers
• Thanks to G.Skill for the RipjawsZ 64GB DDR3 DRAM kit
• Thanks to OCZ Technology for the two 128GB Vertex 4 SSDs, twelve 64GB Vertex 4 SSDs and the OCZ Z-Drive R4 CM88
• Thanks to SilverStone for the Raven RV03 chassis and the 850W Strider Gold Evolution PSU
• Thanks to Netgear for the ProSafe GSM7352S-200 L3 48-port Gigabit Switch with 10 GbE capabilities.
• Thanks to Western Digital for the eight WD RE hard drives (WD4000FYYZ) to use in the NAS under test.

Single Client Performance - CIFS & iSCSI on Windows

The single client CIFS and iSCSI performance of the QNAP TS-451+ was evaluated on the Windows platforms using Intel NASPT and our standard robocopy benchmark. This was run from one of the virtual machines in our NAS testbed. All data for the robocopy benchmark on the client side was put in a RAM disk (created using OSFMount) to ensure that the client's storage system shortcomings wouldn't affect the benchmark results. It must be noted that all the shares / iSCSI LUNs are created in a RAID-5 volume.

The QNAP TS-451+ manages to be amongst the top three in almost every benchmark trace. If at all one could nit-pick, it is the write performance. In real-life situations (with one or two clients), this minor performance aspect will hardly be noticed by users.

We created a 250 GB iSCSI LUN / target and mapped it on to a Windows VM in our testbed. The same NASPT benchmarks were run and the results are presented below. The observations we had in the CIFS subsection above hold true here too.

A few iSCSI benchmarks exhibited extensive caching behavior in some of the NASPT test passes. One of the issues we found was that setting the unit to report a volatile write cache would actually prevent Windows from turning off the write cache for the iSCSI volume. In any case, the iSCSI performance comes in the top three of the list, just like what we saw in the CIFS case.

Encryption Support Evaluation

Consumers looking for encryption capabilities can opt to encrypt a iSCSI share with TrueCrypt or some in-built encryption mechanism in the client OS. However, if requirements dictate that the data must be shared across multiple users / computers, relying on encryption in the NAS is the best way to move forward. Most NAS vendors use the industry-standard 256-bit AES encryption algorithm. One approach is to encrypt only a particular shared folder while the other approach is to encrypt the full volume. Till recently, QNAP only supported volume-level encryption. However, with the launch of QTS 4.2, folder level encryption is also supported now. We evaluated the effects of enabling folder-level encryption.

On the hardware side, encryption support can be in the form of specialized hardware blocks in the SoC (common in ARM / PowerPC based NAS units). In x86-based systems, accelerated encryption support is dependent on whether the AES-NI instruction is available on the host CPU. The Celeron J1900 unfortunately doesn't have AES-NI support. That prepares us for the performance penalty.

Performance loss is around the 50% range. Writes, in particular, take a heavy hit with encryption enabled. The quad-core CPU helps a bit in taking up the cryptographic load, but our advice would be for users to not take advantage of the encryption capabilities of the TS-451+ unless performance is not a concern.

Multi-Client CIFS Performance for Consumer Workloads

The workloads experienced by a NAS unit in a typical home consumer setting have changed quite a bit over the last few years. Multiple mobile devices in a typical household raise the possibility that a NAS could be subject to the streaming out of multiple video files simultaneously. The popularity of IP cameras also make it necessary for NAS units to be able to record multiple video streams at the same time.

In our previous NAS reviews, we evaluated multi-client scenarios using synthetic workload traces and IOMeter. While there is nothing wrong in presenting numbers from such benchmarks, the reader is often left confused as to what those numbers might mean for his particular use-cases. Intel's NASPT benchmarking program gives us a good idea of the performance of the NAS unit when accessed by a single client. We took the source code of Intel's NASPT along with the supplied application traces and tweaked them to be able to run from more than one Windows client simultaneously in a co-ordinated manner. The graphs below present the results from tracking various metrics during the course of the benchmark runs. It must be noted that the average service times refer to what is obtained for all the traces when some of the data has already been cached in the client's memory. Unfortunately, NASPT doesn't provide any sort of guideline on what the optimal bandwidth and service times are for a good user experience.

Content Creation

The Content Creation workload seems to get acceptable performance for up to 6 clients. Beyond that, we have a noticeable drop in the per-client bandwidth numbers. The detailed table with a breakdown of all the throughput numbers as well as the service times is available here

Folder Copy from NAS

The Folder Copy from NAS workload seems to get acceptable performance for up to 10 clients (maximum that we tested). The detailed table with a breakdown of all the throughput numbers as well as the service times is available here

Folder Copy to NAS

The Folder Copy to NAS workload seems to get acceptable performance for up to 6 clients. Beyond that, we have a saturation of the total throughput despite being on a 802.3ad link. The detailed table with a breakdown of all the throughput numbers as well as the service times is available here

File Copy from NAS

The File Copy from NAS workload seems to get acceptable performance for up to 4 clients. Beyond that, the total available bandwidth plateaus well below the link-aggregation limit, indicating an IOPS issue. The detailed table with a breakdown of all the throughput numbers as well as the service times is available here

File Copy to NAS

The File Copy to NAS workload seems to get acceptable performance for up to 6 clients. Beyond that, we have a sudden spike in the average response time. The detailed table with a breakdown of all the throughput numbers as well as the service times is available here

HD Video (1x) Playback

The HD Video (1x) Playback workload seems to get acceptable performance for up to 3 clients. Beyond that, we have a plateau slightly below the link-aggregation limit. The detailed table with a breakdown of all the throughput numbers as well as the service times is available here

HD Video(1x) Playback and Record

The HD Video(1x) Playback and Record workload seems to get acceptable performance for up to 4 clients. Beyond that, we have a plateauing of the total bandwidth number. Thanks to the full-duplex nature of the NICs, we are able to cross the 250 MBps mark (thanks to the simultaneous reads and writes through the LAN). The detailed table with a breakdown of all the throughput numbers as well as the service times is available here

HD Video (1x) Record

The HD Video (1x) Record workload seems to get acceptable performance for up to 4 clients. Beyond that, we have a sudden spike in the average response time. The detailed table with a breakdown of all the throughput numbers as well as the service times is available here

HD Video (2x) Playback

The HD Video (2x) Playback workload seems to get acceptable performance for up to 3 clients. Beyond that, the total available bandwidth plateaus. The detailed table with a breakdown of all the throughput numbers as well as the service times is available here

HD Video (4x) Playback

The HD Video (4x) Playback workload seems to get acceptable performance for up to 3 clients. Beyond that, the total available bandwidth plateaus. The detailed table with a breakdown of all the throughput numbers as well as the service times is available here

Office Productivity

The Office Productivity workload seems to get acceptable performance for up to 10 clients (maximum that we tested). The detailed table with a breakdown of all the throughput numbers as well as the service times is available here

Photo Album

The Photo Album workload also seems to get acceptable performance for up to 10 clients (maximum we tested). The detailed table with a breakdown of all the throughput numbers as well as the service times is available here

The detailed logs from the processing of our benchmarks - inclusive of metrics such as the file open times for each workload on each of the clients - can be found here

Multi-Client CIFS Performance for Professional Workloads

NAS units used in SMBs / SMEs need to provide good performance under heavy load from multiple clients. The SPEC SFS 2014 benchmark uses real-life workloads (just like Intel NASPT), but makes it easier for users to understand the benchmark results. This is achieved by using the concept of business metrics. Given a particular NAS unit, how many concurrently accessed databases can reside in it? How many IP cameras or video streams can be simultaneously recorded? To determine this metric, each load point is associated with a target required op rate. If the NAS under test doesn't meet that op rate, it is deemed as an 'invalid run'. SPEC requires all published benchmarks to follow certain strict rules - such as presented results having no invalid runs for at least 10 load points. Unfortunately, small-scale NAS systems with 7200 RPM drives can't meet these requirements, Hence, we can't officially publish SPEC SFS 2014 benchmark results for the evaluation of the QNAP TS-451+.

Using a popular filer benchmarking program, we did play back multi-client real-world professional workload access traces on the NAS using up to 10 Windows 7 VMs. However, failing to meet the required op rate criteria at a particular load point made us stop the testing a couple of load points down the road. The QNAP TS-451+ with four 7200 RPM hard drives in RAID-5 can support recording of 10 or more video streams, 5 or less databases, 4 or less software builds and 2 or less virtual desktops.

Database Operations

The Database Operations workload seems to get / doesn't seem to get acceptable performance for up to 5 clients.The detailed metrics from our trace playback are available here

Software Builds

The Software Builds workload seems to get / doesn't seem to get acceptable performance for up to 4 clients.The detailed metrics from our trace playback are available here

Video Recording

The Video Recording workload seems to get / doesn't seem to get acceptable performance for up to 10 clients.The detailed metrics from our trace playback are available here

Virtual Desktops

The Virtual Desktops workload seems to get / doesn't seem to get acceptable performance for up to 2 clients.The detailed metrics from our trace playback are available here

Multi-Client NFS Performance for Professional Workloads

We looked at the multi-client performance of CIFS shares for professional workloads in the previous section. In a similar manner, we also evaluated the multi-client NFS performance of the QNAP TS-451+. Instead of the Windows 7 VMs, we used CentOS 7 VMs. The network configuration remained the same. The NFS share exported on the NAS was mounted with the following options.

<NAS_IP>:/PATH_TO_NFS_SHARE /PATH_TO_LOCAL_MOUNT_FOLDER nfs rw,relatime,vers=3,rsize=32768,wsize=32768,namlen=255,hard,proto=tcp,timeo=600,retrans=2, sec=sys,mountaddr <NAS_IP>,mountvers=3,mountproto=udp,local_lock=none,addr=<NAS_IP> 0 0

Using a popular filer benchmarking program, we did played back the same multi-client real-world professional workload access traces used in the previous section. Similar to the strategy for the CIFS performance evaluation, failing to meet the required op rate criteria at a particular load point made us stop the testing a couple of load points down the road. The QNAP TS-451+ with four 7200 RPM hard drives in RAID-5 can support recording of 10 or more video streams, 4 or less databases, 5 or less software builds and 2 or less virtual desktops when it comes to NFS performance.

Database Operations

The Database Operations workload seems to get / doesn't seem to get acceptable performance for up to 4 clients.The detailed metrics from our trace playback are available here

Software Builds

The Software Builds workload seems to get / doesn't seem to get acceptable performance for up to 5 clients.The detailed metrics from our trace playback are available here

Video Recording

The Video Recording workload seems to get / doesn't seem to get acceptable performance for up to 10 clients.The detailed metrics from our trace playback are available here

Virtual Desktops

The Virtual Desktops workload seems to get / doesn't seem to get acceptable performance for up to 2 clients.The detailed metrics from our trace playback are available here

Miscellaneous Aspects and Final Words

In order to keep testing consistent across all 4-bay units, we performed all our expansion / rebuild testing as well as power consumption evaluation with the unit configured in RAID-5. The disks used for benchmarking (Western Digital WD4000FYYZ) were also used in this section. The table below presents the average power consumption of the unit as well as time taken for various RAID-related activities. In the genera state with the hard disk powered on, but not actively reading or writing data, the power consumption was around 50 W, though it dropped to around 16 W with the disks in sleep mode.

QNAP TS-451+ RAID Expansion and Rebuild / Power Consumption
Activity	Duration (HH:MM:SS)	Avg. Power (W)
Single Disk Init	-	20.47 W
JBOD to RAID-1 Migration	08:21:12	33.33 W
RAID-1 (2D) to RAID-5 (3D) Migration	20:52:59	44.11 W
RAID-5 (3D) to RAID-5 (4D) Expansion	22:35:18	53.09 W
RAID-5 (4D) Rebuild	08:35:53	56.16 W

The graphs below show the power consumption and rebuild duration when repairing a RAID-5 volume for the various 4-bay NAS units that have been evaluated before.

The TS-451+ takes very little time to rebuild the NAS volume. However, the power consumption is on the higher end of the spectrum. Overall, the efficiency (energy consumerd in the process) should definitely be in QNAP's favor.

Concluding Remarks

The TS-x51+ series is the latest addition to QNAP's lineup targeting the home users / SOHO market. There is a distinct multimedia focus with the bundled IR remote. Braswell would have further strengthened the multimedia credentials, but QNAP has opted to strangely continue with Bay Trail for this product line. Other than that, the performance of the unit is just slightly better than that of the TS-451 that we evaluated last year.

QNAP's firmware, though, has been improving by leaps and bounds with every release. QTS 4.2 has brought in a host of new features. The NAS is pretty flexible and provides a large number of add-ons as well as third party apps. The TS-451+ is also capable of supporting multiple virtual machines, though, with just two network ports, it is unlikely that consumers will want to run more than one. Earlier this year, QNAP launched Container Station, which adds Docker functionality to QTS. Obviously, QNAP's transcoding functionality that takes advantage of Intel's Quick Sync engine is present in the TS-451+ also. Along with the Hybrid Desk app (Chrome / Kodi etc.) and QvPC technology, the NAS can be used just like a PC for most purposes.

The above advantages aside, there are certain areas where QNAP could improve. One of the main points is the quality of the drive bays in the TS-451+. At the target price point, we need metal drive trays. On a more generic note, QNAP needs to move to Braswell soon while also delivering support for the btrfs file system (in addition to the currently supported EXT4).

The TS-x51+ series has four members - the TS-451+-8G-US coming in at $649, the TS-451+-2G-US coming in at $529, the TS-251+-8G-US coming in at $499 and the TS-251+-2G-US at $399. The price point is definitely an issue, as the TS-453 Pro with 8 GB of memory is currently available in Amazon for $645. For a similar amount, users can get two extra LAN port as well as an extra USB 3.0 port, and there is no power brick to deal with. The only downside is that there is no bundled IR remote in the TS-453 Pro. For users who don't need the IR remote control capabilities (i.e, those who are not going to hook up the NAS to a display and run XBMC / Kodi directly on the NAS), the TS-453 Pro is a much better choice in all respects.

In terms of firmware capabilities and performance for this market segment, there is nothing to complain about in the QNAP TS-451+. The NAS with the included IR remote is attractive for multimedia enthusiasts who want to use the unit also as a media player. At the currently available price points, we would recommend the TS-453 Pro over the TS-451+. For the TS-451+ to become a compelling choice, the pricing aspect needs to be addressed.