Original Link: https://www.anandtech.com/show/8985/synology-ds2015xs-review-an-armbased-10g-nas
Synology DS2015xs Review: An ARM-based 10G NAS
by Ganesh T S on February 27, 2015 8:20 AM EST- Posted in
- Storage
- Arm
- NAS
- 10G Ethernet
- Synology
- Enterprise
Introduction and Testbed Setup
Synology is one of the most popular COTS (commercial off-the-shelf) NAS vendors in the SMB / SOHO market segment. The NAS models introduced by them in 2014 were mostly based on Intel Rangeley (the Atom-based SoCs targeting the storage and communication market). However, in December, they sprang a surprise by launching the DS2015xs, an ARM-based model with dual 10GbE ports. We covered the launch of the Synology DS2015xs in December, and provided some details about the Annapurna Labs AL514 SoC in it.
ARM-based SoCs for SMB / SOHO NAS units typically support up to 4 bays and come with dual GbE links. Intel's offerings have had a virtual monopoly in the other tiers of the market. Synology's DS2015xs, with its native 10G capabilities, brings in another contender into the market.
The DS2015xs is 8-bay NAS unit presented as a step-up from the DS1815+. While the DS1815+ can expand up to a total of 18 bays with two DX513 expansion chassis, the DS2015xs is compatible with the 12-bay DX1215 expander (for a total of 20 bays). The main step-up from the DS1815+ is the presence of two built-in 10G SFP+ links (supporting direct-attach copper cables). The gallery below takes us around the unit's chassis.
The specifications of the Synology DS2015xs are provided in the table below
Synology DS2015xs Specifications | |
Processor | Annapurna Labs AL514 SoC (Quad-Core Cortex-A15 @ 1.7 GHz) |
RAM | 4 GB |
Drive Bays | 8x 3.5"/2.5" SATA II / III HDD / SSD (Hot-Swappable) |
Network Links | 2x 1 GbE RJ-45 + 2x 10GbE SFP+ |
External I/O Peripherals | 2x USB 3.0, 1x Infiniband for Expansion Bay |
Expansion Slots | N/A |
VGA / Display Out | N/A |
Full Specifications Link | Synology DS2015xs Specifications |
Price | USD 1400 |
The Synology DS2015xs runs the latest DiskStation Manager OS, which, subjectively speaking, is one of the best COTS NAS operating systems in the market. Geared towards both novice and power users, it also provides SSH access. Some additional aspects can be gleaned through SSH. For example, the unit runs on Linux kernel version 3.2.40. The AL514 SoC has hardware acceleration for cryptography and two in-built USB 3.0 ports. There are also four network links (we know from external inspection that two are 10GbE, while the others are 1GbE) with unified drivers for both types of interfaces.
In the rest of the review, we will first take a look at the performance of the unit as a direct-attached storage device. This is followed by benchmark numbers for both single and multi-client scenarios across a number of different client platforms as well as access protocols. We have a separate section devoted to the performance of the NAS with encrypted shared folders. Prior to all that, we will take a look at our testbed setup and testing methodology.
Testbed Setup and Testing Methodology
The Synology DS2015xs can take up to 8 drives. Users can opt for different RAID types depnding on their requirements. We expect typical usage to be with multiple volumes in a RAID-5 or RAID-6 disk group. However, to keep things consistent across different NAS units, we benchmarked a SHR volume with single disk redundancy (RAID-5). Tower / desktop form factor NAS units are usually tested with Western Digital RE drives (WD4000FYYZ). However, the presence of 10-GbE on the DS2015xs meant that SSDs had to be used to bring out the maximum possible performance. Therefore, evaluation of the unit was done by setting up a RAID-5 volume with eight OCZ Vector 4 120 GB SSDs. Our testbed configuration is outlined below.
AnandTech NAS Testbed Configuration | |
Motherboard | Asus Z9PE-D8 WS Dual LGA2011 SSI-EEB |
CPU | 2 x Intel Xeon E5-2630L |
Coolers | 2 x Dynatron R17 |
Memory | G.Skill RipjawsZ F3-12800CL10Q2-64GBZL (8x8GB) CAS 10-10-10-30 |
OS Drive | OCZ Technology Vertex 4 128GB |
Secondary Drive | OCZ Technology Vertex 4 128GB |
Tertiary Drive | OCZ Z-Drive R4 CM88 (1.6TB PCIe SSD) |
Other Drives | 12 x OCZ Technology Vertex 4 64GB (Offline in the Host OS) |
Network Cards | 6 x Intel ESA I-340 Quad-GbE Port Network Adapter |
Chassis | SilverStoneTek Raven RV03 |
PSU | SilverStoneTek Strider Plus Gold Evolution 850W |
OS | Windows Server 2008 R2 |
Network Switch | Netgear ProSafe GSM7352S-200 |
The above testbed runs 25 Windows 7 VMs simultaneously, each with a dedicated 1 Gbps network interface. This simulates a real-life workload of up to 25 clients for the NAS being evaluated. All the VMs connect to the network switch to which the NAS is also connected (with link aggregation, as applicable). The VMs generate the NAS traffic for performance evaluation.
Thank You!
We thank the following companies for helping us out with our NAS testbed:
- Thanks to Intel for the Xeon E5-2630L CPUs and the ESA I-340 quad port network adapters
- Thanks to Asus for the Z9PE-D8 WS dual LGA 2011 workstation motherboard
- Thanks to Dynatron for the R17 coolers
- Thanks to G.Skill for the RipjawsZ 64GB DDR3 DRAM kit
- Thanks to OCZ Technology for the two 128GB Vertex 4 SSDs, twelve 64GB Vertex 4 SSDs and the OCZ Z-Drive R4 CM88
- Thanks to SilverStone for the Raven RV03 chassis and the 850W Strider Gold Evolution PSU
- Thanks to Netgear for the ProSafe GSM7352S-200 L3 48-port Gigabit Switch with 10 GbE capabilities.
DAS Evaluation Setup and Methodology
In addition to our standard NAS evaluation suite, the Synology DS2015xs also warrants investigation under ideal network conditions as a direct-attached storage unit. The presence of 10G network links in the unit has prompted Synology
The Emulex PCIe NIC doesn't support teaming under Windows 8.1. Therefore, we had to install Windows Server 2012 R2 on the additional SSD to make our DAS testbed dual-boot for evaluating NAS units. The DHCP Server feature was also activated on the teamed port to which the NAS's 10G ports were connected. On the NAS side, the ports were set up for teaming too and configured to receive an IP address from a DHCP server. The MTU for the interface was configured to be 9000 bytes. The details of the tests that were run in this mode will be presented along with the performance numbers in the next section.
Direct-Attached Storage Performance
The presence of 10G ports on the Synology DS2015xs presents some interesting use-cases. As an example, video production houses have a need for high-speed storage. Usually, direct-attached storage units suffice. Thunderbolt is popular for this purpose - for both single-user modes as well as part of a SAN. However, as 10G network interfaces become common and affordable, there is scope for NAS units to act as a direct-attached storage units also. In order to evaluate the DAS performance of the Synology DS2015xs, we utilized the DAS testbed augmented with an appropriate CNA (converged network adapter), as described in the previous section. To get an idea of the available performance for different workloads, we ran a couple of quick artificial benchmarks along with a subset of our DAS test suite.
CIFS
In the first case, we evaluate the performance of a CIFS share created in a RAID 5 volume. One of the aspects to note is that the direct link between the NAS and the testbed is configured with a MTU of 9000 (compared to the default of 1500 used for the NAS benchmarks).
Our first artificial benchmark is CrystalDiskMark, which tackles sequential accesses as well as 512 KB and 4KB random accesses. For 4K accesses, we have a repetition of the benchmark at a queue depth of 32. As the screenshot above shows, Synology DS2015xs manages around 675 MBps reads and 703 MBps writes. The write benchmark number corresponds correctly to the claims made by Synology in their marketing material, but the 675 MBps read speeds are a far cry from the promised 1930 MBps. We moved on to ATTO, another artificial benchmark, to check if the results were any different.
ATTO Disk Benchmark tackles sequential accesses with different block sizes. We configured a queue depth of 10 and a master file size of 4 GB for accesses with block sizes ranging from 512 bytes to 8 MB and the results are presented above. In this benchmark, we do see 1 MB block sizes giving read speeds of around 1214 MBps.
Synology DS2015xs - 2x10 Gbps LACP - RAID-5 CIFS DAS Performance (MBps) | ||
Read | Write | |
Photos | 594.69 | 363.47 |
Videos | 915.95 | 500.09 |
Blu-ray Folder | 949.32 | 543.93 |
For real-world performance evaluation, we wrote and read back multi-gigabyte folders of photos, videos and Blu-ray files. The results are presented in the table below. These numbers show that it is possible to achieve speeds close to 1 GBps for real-life workloads. The advantage of a unit like the DS2015xs is that the 10G interfaces can be used as a DAS interface, while the other two 1G ports can connect the unit to the rest of the network for sharing the contents seamlessly with other computers.
iSCSI
We configured a block-level (Single LUN on RAID) iSCSI LUN in RAID-5 using all available disks. Network settings were retained from the previous evaluation environment. The same benchmarks were repeated in this configuration also.
The iSCSI performance seems to be a bit off compared to what we got with CIFS. Results from the real-world performance evaluation suite are presented in the table below. These numbers track what we observed in the artificial benchmarks too.
Synology DS2015xs - 2x10 Gbps LACP - RAID-5 iSCSI DAS Performance (MBps) | ||
Read | Write | |
Photos | 535.28 | 532.54 |
Videos | 770.41 | 483.97 |
Blu-ray Folder | 734.51 | 505.3 |
Performance Analysis
The performance numbers that we obtained with teamed ports (20 Gbps) were frankly underwhelming. The more worrisome aspect was that we couldn't replicate Synology's claims of upwards of 1900 MBps throughput for reads. In order to determine if there were any issues with our particular setup, we wanted to isolate the problem to either the disk subsystem on the NAS side or the network configuration. Unfortunately, Synology doesn't provide any tools to evaluate them separately. For optimal functioning, 10G links require careful configuration on either side.
iPerf is the tool of choice for many when it comes to ensuring that the network segment is operating optimally. Unfortunately, iPerf for DSM requires an optware package that is not yet available for the Alpine platform. On the positive side, Synology had uploaded the tool chain for Alpine on SourceForge - this helped us to cross-compile iPerf from source for the DS2015xs. Armed with iPerf on both the NAS and the testbed side, we proceeded to evaluate the links operating simultaneously without the teaming overhead.
The screenshot above shows that the two links together saturated at around 5 Gbps (out of the theoretically possible 20 Gbps), but the culprit was our cross-compiled iPerf executable (with each instance completely saturating one core - 25% of the CPU).
In the CIFS case, the smbd process is not multi-threaded, and this severely affects the utilization of the 10G links fully.
In the iSCSI case, the iscsi_trx process also seems to saturate one CPU core, leading to similar results for 10G link utilization.
On the whole, the 10G links are getting utilized, but not to the full possible extent. The utilization is definitely more than, say, four single GbE links teamed together, but the presence of two 10G links had us expecting more from the unit as a DAS.
Single Client Performance - CIFS & iSCSI on Windows
The single client CIFS and iSCSI performance of the Synology DS2015xs was evaluated on the Windows platforms using Intel NASPT and our standard robocopy benchmark. This was run from one of the virtual machines in our NAS testbed. All data for the robocopy benchmark on the client side was put in a RAM disk (created using OSFMount) to ensure that the client's storage system shortcomings wouldn't affect the benchmark results. It must be noted that all the shares / iSCSI LUNs are created in a RAID-5 volume. One important aspect to note here is that only the Synology DS2015xs uses SSDs. The other NAS units in each of the graphs below were benchmarked with hard drives.
We created a 250 GB iSCSI LUN / target and mapped it on to a Windows VM in our testbed. The same NASPT benchmarks were run and the results are presented below. The observations we had in the CIFS subsection above hold true here too.
The DS2015xs loses out to the Asustor AS7008T and sometimes, even the DS1815+, when it comes to certain benchmarks. This shows that when it comes to access from a single GbE-equipped client, x86-based units can perform better despite being handicapped by hard drives.
Single Client Performance - CIFS and NFS on Linux
A CentOS 6.2 virtual machine was used to evaluate NFS and CIFS performance of the NAS when accessed from a Linux client. We chose IOZone as the benchmark for this case. In order to standardize the testing across multiple NAS units, we mount the CIFS and NFS shares during startup with the following /etc/fstab entries.
//<NAS_IP>/PATH_TO_SMB_SHARE /PATH_TO_LOCAL_MOUNT_FOLDER cifs rw,username=guest,password= 0 0
<NAS_IP>:/PATH_TO_NFS_SHARE /PATH_TO_LOCAL_MOUNT_FOLDER nfs rw,relatime,vers=3,rsize=32768,wsize=32768,namlen=255,hard,proto=tcp,timeo=600,retrans=2, sec=sys,mountaddr <NAS_IP>,mountvers=3,mountproto=udp,local_lock=none,addr=<NAS_IP> 0 0
The following IOZone command was used to benchmark the CIFS share:
IOZone -aczR -g 2097152 -U /PATH_TO_LOCAL_CIFS_MOUNT -f /PATH_TO_LOCAL_CIFS_MOUNT/testfile -b <NAS_NAME>_CIFS_EXCEL_BIN.xls > <NAS_NAME>_CIFS_CSV.csv
IOZone provides benchmark numbers for a multitude of access scenarios with varying file sizes and record lengths. Some of these are very susceptible to caching effects on the client side. This is evident in some of the graphs in the gallery below.
Readers interested in the hard numbers can refer to the CSV program output here.
The NFS share was also benchmarked in a similar manner with the following command:
IOZone -aczR -g 2097152 -U /nfs_test_mount/ -f /nfs_test_mount/testfile -b <NAS_NAME>_NFS_EXCEL_BIN.xls > <NAS_NAME>_NFS_CSV.csv
The IOZone CSV output can be found here for those interested in the exact numbers.
A summary of the bandwidth numbers for various tests averaged across all file and record sizes is provided in the table below. As noted previously, some of these numbers are skewed by caching effects. A reference to the actual CSV outputs linked above make the entries affected by this effect obvious.
Synology DS2015xs - Linux Client Performance (MBps) | ||
IOZone Test | CIFS | NFS |
Init Write | 86 | 82 |
Re-Write | 85 | 82 |
Read | 50 | 120 |
Re-Read | 50 | 122 |
Random Read | 33 | 70 |
Random Write | 80 | 82 |
Backward Read | 32 | 58 |
Record Re-Write | 56 | 1719* |
Stride Read | 45 | 117 |
File Write | 85 | 83 |
File Re-Write | 85 | 82 |
File Read | 35 | 95 |
File Re-Read | 36 | 97 |
*: Benchmark number skewed due to caching effect |
Multi-Client Performance - CIFS on Windows
We put the Synology DS2015xs through some IOMeter tests with a CIFS share being accessed from up to 25 VMs simultaneously. The following four graphs show the total available bandwidth and the average response time while being subject to different types of workloads through IOMeter. The tool also reports various other metrics of interest such as maximum response time, read and write IOPS, separate read and write bandwidth figures etc. Detailed listings of the IOMeter benchmark numbers (including IOPS and maximum response times) for each configuration are linked below:
We see that the sequential accesses get saturated around 700 MBps, similar to what we found in our evaluation of the unit as a DAS. In the Random 8K 70% Reads case, we see a sudden drop after more than five clients come into the mix - we believe it has to do with the smbd processes saturating the CPU cores completely. On the positive side, we find that the bandwidth numbers and response times are excellent across the board, better than all the other NAS units that we are comparing against.
Multi-Client iSCSI Evaluation
As virtualization becomes more and more popular even in home / power user settings, the importance of the iSCSI feature set of any COTS NAS can't be overstated. Starting with our ioSafe 1513+ review, we have started devoting a separate section (in the reviews of NAS units targeting SMBs and SMEs) to the evaluation of iSCSI performance. Since we have already looked at the way iSCSI LUNs are implemented in DSM in the ioSafe 1513+ review, it won't be discussed in detail.
We evaluated the performance of the Synology DS2015xs with file-based LUNs as well as configuring a RAID-5 disk group with single as well as multiple LUNs. The standard IOMeter benchmarks that we used for multi-client CIFS evaluation were utilized for iSCSI evaluation also. The main difference to note is that the CIFS evaluation was performed on a mounted network share, while the iSCSI evaluation was done on a 'clean physical disk' (from the viewpoint of the virtual machine).
Performance Numbers
The four IOMeter traces were run on the physical disk manifested by mapping the iSCSI target on each VM. The benchmarking started with one VM accessing the NAS. The number of VMs simultaneously playing out the trace was incremented one by one till we had all 25 VMs in the fray. Detailed listings of the IOMeter benchmark numbers (including IOPS and maximum response times) for each configuration are linked below:
- Synology DS2015xs - 8 SSDs / RAID-5 / 2x 10G - LUNs (Regular Files)
- Synology DS2015xs - 8 SSDs / RAID-5 / 2x 10G - Single LUN on RAID
- Synology DS2015xs - 8 SSDs / RAID-5 / 2x 10G - Multiple LUNs on RAID
The observations we had in the previous section hold true here also. In general, the performance of the DS2015xs iSCSI volumes is better than the others that are presented as comparison points, even considering the SSD vs. HDD aspect.
Encryption Support Evaluation
Consumers looking for encryption capabilities can opt to encrypt a iSCSI share with TrueCrypt or some in-built encryption mechanism in the client OS. However, if requirements dictate that the data must be shared across multiple users / computers, relying on encryption in the NAS is the best way to move forward. Most NAS vendors use the industry-standard 256-bit AES encryption algorithm. One approach is to encrypt only a particular shared folder while the other approach is to encrypt the full volume. Synology supports only folder-level encryption for now in DSM.
On the hardware side, encryption support can be in the form of specialized hardware blocks in the SoC (common in ARM / PowerPC based NAS units). In x86-based systems, accelerated encryption support is dependent on whether the AES-NI instruction is available on the host CPU. On the Annapurna Labs Alpine platform, we have a dedicated crypto block.
The graphs above show that there is a performance impact only for writes to encrypted volumes. The impact is not as bad as that of software encryption, but the slowdown is definitely noticeable. This indicates an imbalance between the cryptography engine's encryption rate and the the rate at which data can be written to the storage media.
Miscellaneous Aspects and Final Words
In order to keep testing consistent across all 8-bay units, we performed all our expansion / rebuild testing as well as power consumption evaluation with the unit configured in RAID-5. The disks used for benchmarking (OCZ Vector 4 SSDs) were also used in this section. The table below presents the average power consumption of the unit as well as time taken for various RAID-related activities.
Synology DS2015xs RAID Expansion and Rebuild / Power Consumption | ||
Activity | Duration (HH:MM:SS) | Avg. Power (W) |
Single Disk Init | 00:01:00 | 18.64 W |
JBOD to RAID-1 Migration | 00:19:14 | 20.91 W |
RAID-1 (2D) to RAID-5 (3D) Migration | 00:25:07 | 23.95 W |
RAID-5 (3D) to RAID-5 (4D) Expansion | 00:23:48 | 26.92 W |
RAID-5 (4D) to RAID-5 (5D) Expansion | 00:24:19 | 29.86 W |
RAID-5 (5D) to RAID-5 (6D) Expansion | 00:24:20 | 32.82 W |
RAID-5 (6D) to RAID-5 (7D) Expansion | 00:22:49 | 35.8 W |
RAID-5 (7D) to RAID-5 (8D) Expansion | 00:20:03 | 38.79 W |
RAID-5 (8D) Rebuild | 00:13:12 | 32.77 W |
We don't have any comparison graphs for RAID rebuild duration / power consumption since other 8-bay units were subject to RAID operations with 4 TB hard drives. The usage of 120 GB SSDs prevents us from making any concrete conclusions about the efficiency of Synology's expansion, migration and rebuild operations.
Concluding Remarks
Coming to the business end of the review, it is clear that the presence of two 10G ports enables the DS2015xs to achieve high transfer rates. Compared to the last 10G desktop NAS we evaluated (based on an Intel Xeon platform), the power consumption is much lower At $1400, the DS2015xs is affordably priced, considering the 10G features built into it. DSM's business-oriented features (very important for the DS2015xs's target market) are stable and industry-leading.
The above advantages aside, there are certain areas of concern / scope for improvement: Synology needs to supply a way to test the network performance (provide an optimized iPerf package) isolated from the storage subsystem. The SoC vendor, Annapurna Labs is in the process of being purchased by Amazon. It is not clear what effect that would have on the DS2015xs's successors. It is likely that other ARM server vendors might want to make a play for future products in the lineup.
It is impossible not to conclude this review without a car analogy for the DS2015xs. The presence of dual 10G ports and eight native SATA III ports might delude consumers into thinking that they are purchasing a Ferrari. Unfortunately, consumers only get a souped up Volkswagen engine inside. That said, with tempered expectations, the $1400 Synology DS2015xs can fulfill the role of a cost-effective power-efficient high performance NAS.