For a number of months I have been wondering when 10GBase-T would be getting some prime time in the consumer market. Aside from add-in cards, there was no onboard solution, until ASRock announced the X99 WS-E/10G. We were lucky enough to get one in for review.

10GBase-T is somewhat of an odd standard. Based on upgraded RJ-45 connections, it pushes the standard of regular wired networking in terms of performance and capability. The controllers required for it are expensive, as the situations that normally require this bandwidth tend to use different standards that afford other benefits such as lower power, lower heat generation and more efficient signaling standards. Put bluntly, 10GBase-T is hot, power hungry, expensive, but ultimately the easiest to integrate into a home, small office or prosumer environment. Users looking into 10GBase-T calculate cost in hundreds of monies per port, rather than pennies, as the cheapest unmanaged switches cost $800 or so. A standard two port X540-T2 PCIe 2.0 x8 card can cost $400-800 depending on your location, meaning a minimum $2000 for a 3 system setup.

The benefits of 10GBase-T outside the data center sound somewhat limited. It doesn't increase your internet performance, as that is determined by the line outside the building. For a home network, its best use is in computer to computer data transfer. Normally a prosumer environment might have a server or workstation farm for large dataset analysis and GBit just isn't enough. Or the most likely home scenario is streaming lossless 4K content to several devices at once. For most users this sounds almost a myth, but for a select few it is a reality, or at least something near it. Some users are teaming individual GBit ports for similar connectivity as well.

Moving the 10GBase-T controller and ports ultimately frees up PCIe slots for other devices, and makes integration easier, although you lose the ability to transfer the card to another machine if needed. The X540-BT2 used in the X99 WS-E/10G has eight PCIe 3.0 lanes on a 40 PCIe lane CPU, but can also work with four lanes via the 28-lane i7-5820K CPU if required. Using the controller on the motherboard also helps with pricing, providing an integrated system and hopefully shaving $100 or so from the ultimate cost. That being said, as it ends up in the high end model, it is aimed at those where hardware cost is a minimal part of their prosumer activities, where an overclocked i7-5960X system with 4+ PCIe devices is par for the course.

ASRock X99 WS-E/10G Overview

In an ideal testing scenario, we would test motherboards the same way we do medicine – with a double blind randomized test. In this circumstance, there would be no markings to give away who made the device, and during testing there would be no indication of the device either. With CPUs this is relatively easy if someone else sets the system up. With motherboards, it is almost impossible due to the ecosystem of motherboard design that directly impacts expectation and use model. Part of the benefit of a system is in the way it is presented as well as the ease of use of software, to the point where manufacturers will spend time and resources developing the extra tools. Providing the tools is easy enough, but developing it into an experience is an important aspect. So when ASRock presents a motherboard with 10GBase-T, the main key points here are ‘10GBase-T functionality’ coming from ‘ASRock’.

Due to the cost of the 10GBase-T controller, the Intel X540-BT2, ASRock understandably went high-end in their first implementation. This means a full PCIe 3.0 x16/x16/x16/x16 layout due to the use of two PLX 8747 chips that act as FIFO buffer/muxes to increase the lane count. For those new to PLX 8747 chips, we went in-depth on their function when they were first released which you can read here. These PLX chips also are quite expensive, at least adding $40 each to the cost of the board for the consumer, but allow ASRock to implement top inter-GPU bandwidth.  This means that from the 40 PCIe lanes of an LGA2011-3 CPU, 8 go to the X540-BT2 and 16 each go to the PLX chips which output 32 each. For users wanting to go all out with single slot PCIe co-processors, the X99 WS-E/10G will allow an x16/x8/x8/x8/x8/x8/x8 arrangement.

If the WS in the name was not a giveaway, with the cost of these extra controllers, ASRock is aiming at the 1P workstation market. As a result the motherboard has shorter screws to allow 1U implementation and full Xeon support with ECC/RDIMM up to 128GB. The power delivery package is ASRock’s 12-phase solution along with the Super Alloy branding indicating XXL heatsinks as well as server grade components. The two PLX chips are cooled by a large heatsink with a small fan, although this can be disabled if the users cooling is sufficient. Another couple of nods to the WS market is also the two Intel I210 network interfaces with the dual 10GBase-T, affording a potential teaming rate of 22 Gbps all in. There is also a USB Type-A port sticking out for license dongles as well as a SATA DOM port. TPM, COM and two BIOS chips are also supported.

On the consumer side of the equation, the chipset IO is split into four lanes for an M.2 x4 port, the two Intel I210 NICs mentioned before and a SATA Express implementation. The M.2 slot has some PCIe sharing duties with a Marvell 9172 SATA controller as well, meaning that using the Marvell SATA ports puts the M.2 into x2 mode. The board has 12 total SATA ports, with 6 PCH RAID capable, four PCH non-RAID capable and two from the Marvell. Alongside this is eight USB 3.0 ports, four from two onboard headers and four ports on the rear panel from an ASMedia ASM1074 hub. An eSATA port is on the rear panel as well, sharing bandwidth with a non-RAID SATA ports. Finally the audio solution is ASRock’s upgraded ALC1150 package under the Purity Sound 2 branding.

Performance wise, ASRock uses an aggressive form of MultiCore Turbo to score highly in our CPU tests. Due to the 10G controller, the power consumption is higher than other motherboards we have tested, and it also impacts the DPC Latency. USB 2.0 speed was a little slow, and the audio had a low THD+N result, but POST times were ballpark for X99. The software and BIOS from ASRock followed similarly from our previous ASRock X99 WS review.

The 10GBase-T element of the equation was interesting, given that for PC-to-PC individual transfers from RAMDisk to RAMDisk peaked at 2.5 Gbps. To get the most from the protocol the data transfer requires several streams (more than one transfer function to allow for interleaving), at least four for 6 Gbps+ or eight for 8 Gbps+. One bottleneck in the transfer is the CPU, showing 50% load on an eight-thread VM during transfer using five streams, perhaps indicating that an overclocked CPU (or something like the i7-4790K with a higher threaded speed) might be preferable.

Whenever a motherboard company asks what a user looks for in a motherboard, I always mention that if they have a particular need, they will only look at motherboards that have the functionality. Following this, users would look choosing the right socket, then filter by price, brand, looks and reviews (one would hope in that vague order). The key point here being that the X99 WS-E/10G caters to that specific crowd that need a 10GBase-T motherboard. If you do not need it, the motherboard is overly expensive.

Visual Inspection

Motherboards with lots of additions tend to be bigger than usual, and the WS-E/10G sits in the E-ATX form factor. This allows the addition of the X540-BT2 controller and the two PLX 8747 switches with more PCB room for routing. As the 10G controller is rated at 14W at full tilt it comes covered with a large heatsink which is connected via a heatpipe to the heatsink covering the power delivery. The smaller heatsink covering the chipset and two PLX chips is not connected to the others, however it does have a small fan (which can be disconnected) to improve cooling potential.

As this motherboard is oriented towards the workstation market we get features such as COM and TPM headers, with a total of five fan headers around the motherboard. The two CPU fan headers, one four-pin and one three-pin, are at the top right of the board, with a 3-pin CHA header just above the SATA ports and another just below. The final header is on the bottom panel, this time four-pin. The ‘white thing that looks like a fan header’ at the bottom of the board is actually used for SATA DOM power. Note that HDD Saver does not feature on this motherboard.

The DRAM slots are single-sided with the latches due to the close proximity of the first PCIe slot, which means that users should ensure that their DRAM is fully pushed in at both ends. Next to the DRAM is one of the PCIe power connectors, a horrible looking 4-pin molex connector right in the middle of the board. I asked ASRock about these connectors (because I continually request they be replaced) and ASRock’s response was that they would prefer a single connector at the bottom but some users complain that their cases will not allow another connector angled down in that location, so they put one here as well. Users should also note that only one needs to be connected when 3+ PCIe devices are used to help boost power. I quizzed them on SATA power connectors instead, or a 6-pin PCIe, however the response was not enthusiastic.

Next to this power connector is a USB 2.0 type-A port on the motherboard itself, which we normally see on server/workstation motherboards for USB license keys or other forms of not-to-be-removed devices.

On the right hand side of the motherboard is our TPM header followed by the 24-pin ATX power connector and two USB 3.0 headers, where both of these come from the PCH. With the SATA ports there are twelve in total in this segment with the first two being powered by a Marvell controller. The next ten are from the PCH with the first six RAID capable, then the next four are not. As part of this final four there is also a SATA Express port coming from the chipset. For more connectivity we have a black SATA DOM port at the bottom of the board and a PCIe 2.0 x4 M.2 slot from the chipset supporting 2230 to 22110 sized devices. If a device is plugged into the final four SATA ports, the M.2 bandwidth drops to M.2 x2. This suggests that ASRock can partition some of the bandwidth from the second non-RAID AHCI controller in the chipset for M.2 usage, and that the second AHCI controller is in-part based on PCIe. This further implicates my prediction that the chipset is just turning into a mass of PCIe lanes / FPGA as required by the motherboard manufacturer.

At the bottom of the motherboard are our power/reset buttons alongside the two-digit debug. The two BIOS chips are also here with a BIOS select switch, two SATA-SGPIO headers, two USB 2.0 headers, a COM header, a Thunderbolt header, two of the fan headers and that ugly molex power connector. As usual the front panel audio and control headers are here too, as well as two other headers designated FRONT_LAN, presumably to allow server builders to route the signals from the network ports to LEDs on the front of the case.

The audio subsystem uses an upgraded Realtek ALC1150 package, meaning an EMI shield, PCB separation and enhanced filter caps. The PCIe layout is relatively easy to follow:

From the 40 PCIe lanes from the CPU, these are split into x16/x16/x8. The final x8 goes to the 10GBase-T controller, whereas the other lanes get filtered into one PLX controller each. This gives the effect of muxing 16 lanes into 32 (with an extra buffer), allowing each PLX controller to feed two x16 slots for a total of four PCIe 3.0 x16 (hence x16/x16/x16/x16 support). Three of these x16 slots are quick switched to x8 slots, creating x8/x8 from three of the x16 ports.

This means:

Four PCIe devices or less: x16/-/x16/-/x16/-/x16
Four to Seven PCIe devices: x8/x8/x8/x8/x8/x8/x16

So for anyone that wants to strap on some serious PCIe storage, RAID cards or single slot PCIe co-processors, everyone gets at least PCIe 3.0 x8 bandwidth.

For users on the i7-5820K, things are a little different but not so much. Due to only having 28 PCIe lanes, the outputs are split x16/x8/x4, with x4 going to the X540. This leaves x16 and x8 going to the PLX controllers, but in both cases each PLX chip will configure to 32 PCIe lanes, still giving an x16/x16/x16/x16 or x8/x8/x8/x8/x8/x8/x16 arrangement. With only four lanes, the two 10GBase-T ports are still designated to work with PCIe 3.0 x4 (given the original requirement of PCIe 2.0 x8 for the controller), but full bandwidth might not be possible according to Intel’s FAQ on the X540 range – check point 2.27 here.

The rear panel removes any PS/2 ports and gives four USB 2.0 alongside four USB 3.0, with the latter coming from an ASMedia hub. The two network ports on the left are from Intel I210 controllers, whereas the two on the right are the 10GBase-T ports from the Intel X540-BT2 controller. There is a Clear_CMOS button, an eSATA port and the audio jacks to round off the set.

Board Features

ASRock X99 WS-E/10G
Price	US (Newegg)
Size	E-ATX
CPU Interface	LGA2011-3
Chipset	Intel X99
Memory Slots	Eight DDR4 DIMM slots Supporting up to 64 GB UDIMM, 128 GB RDIMM Up to Quad Channel, 1066-3200 MHz
Video Outputs	None
Network Connectivity	2 x Intel I210 (1Gbit) 2 x Intel X540-BT2 (10GBase-T)
Onboard Audio	Realtek ALC1150
Expansion Slots	4 x PCIe 3.0 x16 3 x PCIe 3.0 x8
Onboard Storage	6 x SATA 6 Gbps, RAID 0/1/5/10 4 x S_SATA 6 Gbps, no RAID 2 x SATA 6 Gbps, Marvell 9172 1 x SATA Express 1 x M.2 PCIe 2.0 x4 / x2
USB 3.0	4 x USB 3.0 on Rear Panel (ASMedia ASM1042 Hub) 2 x USB 3.0 Headers onboard (PCH)
Onboard	12 x SATA 6 Gbps 1 x SATA DOM 1 x M.2 x4 2 x USB 2.0 Headers 2 x USB 3.0 Headers 5 x Fan Headers 1 x USB 2.0 Type-A TPM Header COM Header Thunderbolt Header 2 x FRONT_LAN Headers 2 x SATA_SPGIO Headers Power/Reset Switches Two Digit Debug BIOS Switch SATA DOM Power Front Panel Header Front Audio Header
Power Connectors	1 x 24-pin ATX 1 x 8-pin CPU 2 x VGA Molex
Fan Headers	2 x CPU (4-pin, 3-pin) 3 x CHA (4-pin, 2 x 3-pin)
IO Panel	2 x USB 2.0 2 x USB 3.0 (ASMedia Hub) 2 x Intel I210 Gbit Network 2 x Intel X540-BT2 10GBase-T Network eSATA Clear_CMOS Button Audio Jacks
Warranty Period	3 Years
Product Page	Link

Testing the 10GBase-T

Many thanks to Brett Howse for his help!

As an indication of how far away from mainstream adoption of 10GBase-T we are, our testing setup was not ready to receive 10GBase-T – we have no switches or other PCIe cards in house to test the ports.  Apart from that, one of the downsides of testing network ports as a whole is that any test out one machine and into another, meaning that and any result will always be at the whims of the lowest common denominator between the two systems. The easiest way to test is therefore to essentially loop back on itself. As the X540-BT2 is a dual port solution, this was ideal for the testing scenario.

We set up the system with ESXi and two Windows Server 2012 VMs, each allocated with 8 threads, 16 GB of DRAM and one of the 10GBase-T ports with custom IPs. We then used LAN Speed Test to set up a server on one VM and a client on the other. LAN Speed Test allows us to simulate multiple clients through the same cable, effectively probing our ports similar to a SOHO/SMB environment.

We organized a series of send/receive commands to go through the connections with differing numbers of streams and differing amounts of data per stream. The system was set to repeat for 10 minutes, and the peak transfer rate was recorded. Results shown are in Gbps.

There are several key points to note with these results.

• Firstly, single client speed never broke 2.2 Gbps. This puts an absolute limit on individual point-to-point communication speed in our system setup.
• Next, with between 4-9 clients the speed is fairly consistent between 6.7 and 8 Gbps, no matter what the size of the transfer is.
• Thus in order to get peak throughput, at least 10 concurrent connections need to be made. This is when 8+ Gbps is observed.
• With a small number of clients, a longer transfer results in higher overall speed. However with a larger amount of clients, faster transfers results in higher peak speed.

As part of the test, we also examined CPU usage when each stream was set for 1GB transfers. Normally CPU usage for a standard 1Gbit Ethernet port is minimal although one of the factors that some companies like to point to for increased gaming rates. Normally this rarely goes above a few percent on a quad core system, but with the X540-BT2 that changed, especially when the system was being hammered.

As the main VM alternated between reading and writing from the server VM, reading CPU usage peaked after 10 concurrent clients but write CPU usage shot up very quickly to 50% and stayed there. This might explain the slow increases in peak performance we observed, if the software simply ran out of threads and was only geared for four threads.

With a 4 core VM we saw 100% usage during writes, and above was the result in terms of CPU performance monitoring.

This marks an interesting juncture, suggesting that a faster single threaded CPU could deliver better performance and that the X540T-BT2 would be better attached to a Haswell/Broadwell platform – at least in our testing scenario.  The truth of the matter is that fast connectivity technology runs on optimized FPGAs because general purpose CPUs cannot keep up with what is needed.

So where does leave the X99 WS-E/10G? The best example I could propose is in that SOHO/SMB environment where the system is connected to a 10Gbit/1Gbit mixed switch that has 24+ clients that all need to access the system, either as a virtualized workspace, some form of storage or a joint streaming venture. As the former, it allows the rest of the office to use very basic machines and rely on the grunt of the virtualized environment to perform tasks.

Additional: As mentioned in the comments, this is almost the complete out-of-the-box scenario where the only thing configured is the RAMDisk transfers and multiple stream application, whereas most users might be limited by the SSD speed. Jammrock in the comments has posted a list in helping to optimize the connection solely for individual point-to-point transfers and is worth a read.

The ASRock BIOS and software has changed little since our review of the ASRock X99 WS, and hence is updated here with the WS-E/10G specific information.

ASRock X99 WS-E/10G BIOS

The ASRock BIOS shifted slightly between Z87 and Z97, and it is the Z97 implementation we get on X99. This means the starry background is gone, replaced with a dark and blue-crystalline image. The icons are also reduced to mere text with a small image and the menu sections are of equal size which I prefer. There is also an adjustment to the OC Tweaker layout, which we will come to.

There is no ‘Easy Mode’ for the ASRock BIOS, with entry direct into this first Main screen. The right hand side of the BIOS is for description of the option selected, and the main segment shows information such as the BIOS version, CPU installed, CPU speed and memory installed. As mentioned in previous reviews, having some set of information relating to CPU voltages, temperatures and fan speeds might be useful here. If ASRock was to do an Easy Mode, it would make sense to have a one-button overclock option as well.

OC Tweaker takes heed of one of my previous complains about the ASRock overclock options. Previously there used to be a long list of options that continuously grew, depending on what options were left open to the user. For X99, ASRock has condensed these options into separate submenus similar in design to those we see on a GIGABYTE motherboard. Note that ASUS and MSI BIOSes currently go half-way between these extremes, offering some basic options with the more advanced ones in separate menus.

ASRock cuts straight to the chase – the first two options deal with overclocking. We have touched on features like Multi Core Enhancement many times previously, but it is interesting to have it so high up on the list of options. The Optimized CPU OC Setting gives some easy selections for overclocks based on ASRock’s internal testing:

In line with our overclock testing, 4.0 GHz and 4.2 GHz should be achievable by all CPUs, with 4.4 GHz by most and 4.5 GHz by around half, given sufficient cooling methods.

The OC Tweaker menu is split into four sections. CPU Configuration gives options for CPU Ratio, multiplier, frequency, turbo boost and power limits:

DRAM Configuration allows for XMP application, DRAM strap adjustment, DRAM voltage and sub-timing adjustment, all the way through primary timings, secondary timings, tertiaries and others.

ASRock still uses the DRAM Tweaker function to fine tune the JEDEC and XMP responses to each memory kit. The ASRock BIOS also offers a DRAM Frequency overclocking preset list which will adjust the CPU strap and BCLK frequency needed to achieve various speeds.

The FIVR Configuration menu from the OC Tweaker page deals primarily with the CPU and CPU Cache/uncore voltages.

The Voltage Configuration menu has the other voltages on the system, including VCIN to the FIVR and load line calibration. The Load Line Calibration option also comes with a handy graph showing what each setting means:

Unfortunately this arrangement of voltages is mildly confusing. Typically an overclocker will adjust the CPU voltage and the Load Line Calibration settings enough to get the right combination, and ASRock has placed these options in two different menus. If the CPU VCore option could be duplicated in the Voltage Configuration menu, this would be preferable.

The Advanced tab deals with other elements of the system configuration, such as the chipset, storage and super IO calibrations. This includes fixing power modes, PCIe linking speeds, disabling or enabling onboard controllers and adjusting RAID configurations.

An adjustment to the X99 WS BIOS is that the Active Page on Entry setting has been moved here.

By default the ‘Above 4G Decoding’ is disabled, and I am surprised not to see an option here to enable the Thunderbolt header as is typically present in TB capable products.

The Tools menu is typically where ASRock tries out its more unusual ideas, although we do not get anything new for X99 that we have not seen on Z97 already. The System Browser shows the different elements of the motherboard and what is detected:

The Online Management Guard (OMG) allows the user to enable or disable the network ports at various times of the day, as determined by the system time:

(It is amusing that this feature is aimed more at controlling when a child is allowed online, but X99 is not exactly a system configuration used as secondary machines.)

The Dehumidifier function now gets its own sub menu, allowing fans to spin for a fixed time after the machine is in S5:

The H/W Monitor tab is where the temperatures, fan speeds and voltage monitoring tools are all kept, and we have seen an effort by the other manufacturers to improve the fan control situation. ASRock now joins them with the FAN-Tastic tuning option, moving into a more interactive type of scenario. This extends the multi-point gradient options into something more clickable, similar to the ASUS implementation. For example, a user can put a fan into one of five modes or customize the profile:

Rather than using the interface, users can do it the older way:

There is also an option to select which temperature sensor is being read from to make these adjustments.

The rest of the BIOS deals with security, boot order and fast boot options. In the Save and Exit menu we do get options for Boot Override:

ASRock X99 WS-E/10G Software

The software package from ASRock has gone through a slow evolution since Z77 into a single interface for all of ASRock’s functionality called A-Tuning. With the overclocking and gaming models the interface is slightly adjusted, but the green tint follows the majority of ASRock’s motherboard range. However the newest element to ASRock’s like is the APP Shop. This is essentially ASRock’s curated version of the Play Store/Microsoft Store, with only software ASRock feels is suited to their motherboard range.

Currently the software is fairly limited to Chrome, ASRock’s own software programs and a few Asian free-to-play titles. While offering this method to obtain software is interesting, it does open up a lot of questions. Will there be to-pay titles? What happens if one element of the store is filled with malware?

The APP Shop also offers a BIOS and Drivers section to update the system, but as of yet we have not had it working properly in our testing.

One suggestion has been that this software will only update the packages it downloads. There is another update tool in A-Tuning.

A-Tuning

The initial interface revolves around overclocking, giving the following three modes for automatic frequency adjustments:

Power Saving puts the CPU into a low power mode (1.2 GHz on the 5960X) and requires the system have full CPU load before slowly ramping up the speed over the next 6-10 seconds. This keeps power consumption down, but perhaps increases the responsiveness of the system by not having that initial high single core frequency. Standard mode is the default setting, and Performance mode puts the CPU into high frequency mode for any loading. Performance Mode also opens up the Advanced menu:

Here we have a list of Optimized CPU OC Settings similar to the BIOS and an auto tuning section. There is no list of auto tuning options similar to ASUS, for adjusting the stress tests or the optimum CPU temperature, although I would imagine that all the manufacturers might move in that direction at some point in the future.

The tools menu has a lot of space for ASRock to add in future features, but currently houses the ones they do have. XFast RAM allows the system to partition some of the RAM into a RAMDisk while also providing some RAMCache options:

XFast LAN is a customized interface for cFos, allowing users to prioritize certain applications over others:

Personally I find this interface extremely cumbersome, especially if there are a lot of applications to deal with. ASRock could design something with less white space and a more efficient amalgamation of the A-Tuning visual dimensions to make the process a lot easier. There is access to cFos directly with the Advance Setting button:

The software works with all the network ports on board, including the 10GBase-T ones.

Fast Boot enables options relating to UEFI quick booting by disabling certain options until the OS is loaded:

The Online Management Guard (OMG [!]) has been around for several generations of ASRock motherboards now, and offers the user the ability to disable the networking ports during certain times of the day.

ASRock’s Fan software in the software now mirrors that in the BIOS, giving a better sense for the user in what to adjust:

The FAN Test will detect the RPM for a given fan power, although the graph on the left is misnamed – what ASRock calls ‘FAN Speed (%)’ is actually deceptive because it means ‘Fan Power’ and the user has to do the mathematics in their head to adjust based on the table in the middle. If ASRock was on the ball, they would do the conversion in software and adjust the graph to read ‘Fan Speed (RPM)’ and adjust the axis appropriately from lowest Fan Speed to highest Fan Speed. Note that the high fan speeds above are actually the speeds from my liquid cooling pump.

The Dehumidifier tool in the software is identical to that in the BIOS, allowing the system to enable the fans after the system has been shut off in order to equalize the air temperature inside and outside the case. This has benefits in humid environments where the moisture in the air may condense inside the case during a cool night after a hot day.

The USB Key tool allows users to assign a particular USB drive with login data for particular Windows users. This means that users need not remember a particular long password to log in, and only specified users are allowed to log in. Though lose the USB drive and lose the ability to log in.

One of the newer tools in ASRock’s list is the DISK Health Report. This gives the critical information on the drives attached to the system, allowing SSD users to see the life of their drive. This drive has been at the heart of my motherboard test beds now for almost three years and is still going forward.

The next tab along the top is the OC Tweaker, featuring the more critical options from the BIOS for manual overclocking along with some options to save overclock profiles. The way this is shown in ASRock’s software is quite user-unfriendly, and I would suggest that the next iteration of the software gives an experienced user an easier way to adjust frequencies and voltages without dealing with sliding bars and scrolling.

The System Info tab gives the hardware monitor page by default, giving information about the frequencies, fan speeds and voltages in the system. Most other manufacturers have a way of recording this data, or seeing it plotted on a graph while running a stress test, but ASRock is behind on this front at this time.

The Hardware Monitor section of System Info is identical to that in the BIOS, showing where hardware is installed with a mouse over giving basic details. This is handy for investigating which memory stick, USB or PCIe device is not being detected.

The Live Update tab is, by comparison to MSI, limited. Although I knew there were updates to the platform when I run this software, it failed to find the updated drivers. Here it also does not say how big each download is. If a user is on a limited or slow bandwidth package having to download 300MB of audio or graphics drivers can be detrimental.

While ASRock’s software package is presented in a good way, and there are a number of helpful tools, there are various aspects here that miss the mark in terms of user experience.

ASRock X99 WS-E/10G In The Box

High end motherboards that cater to a general market tend to feature enough extras in the box especially if they are aligned to a gaming brand or the overclocking crowd. With motherboards that focus on one particular feature, the trend tends to be the opposite, as these motherboards are purchased exclusively for that feature. With the X99 WS-E/10G aiming at the higher end of the market on multiple fronts, there offered much more possibility from ASRock, and as such multi-SLI bridges are included.

In the box we get the following:

Rear IO Shield
Driver DVD
Manuals
Six SATA Cables
Flexi 2-way SLI Cable
Rigid 3-way SLI Cable
Rigid 4-way SLI Cable

Many thanks to...

We must thank the following companies for kindly providing hardware for our test bed:

Thank you to OCZ for providing us with PSUs and SSDs.
Thank you to G.Skill for providing us with memory.
Thank you to Corsair for providing us with an AX1200i PSU.
Thank you to MSI for providing us with the NVIDIA GTX 770 Lightning GPUs.
Thank you to Rosewill for providing us with PSUs and RK-9100 keyboards.
Thank you to ASRock for providing us with some IO testing kit.
Thank you to Cooler Master for providing us with Nepton 140XL CLCs.

Test Setup

Test Setup
Processor	Intel Core i7-5960X ES 8 Cores, 16 Threads, 3.0 GHz (3.5 GHz Turbo)
Motherboards	ASRock X99 WS-E/10G
Cooling	Cooler Master Nepton 140XL
Power Supply	OCZ 1250W Gold ZX Series Corsair AX1200i Platinum PSU
Memory	Corsair DDR4-2133 C15 4x8 GB 1.2V G.Skill Ripjaws 4 DDR4-2133 C15 4x8 GB 1.2V
Memory Settings	JEDEC @ 2133
Video Cards	MSI GTX 770 Lightning 2GB (1150/1202 Boost)
Video Drivers	NVIDIA Drivers 332.21
Hard Drive	OCZ Vertex 3 256GB
Optical Drive	LG GH22NS50
Case	Open Test Bed
Operating System	Windows 7 64-bit SP1
USB 2/3 Testing	OCZ Vertex 3 240GB with SATA->USB Adaptor
WiFi Testing	D-Link DIR-865L 802.11ac Dual Band Router

ASRock X99 WS-E/10G Overclocking

Experience with ASRock X99 WS-E/10G

Nothing out of the ordinary for overclocking with the WS-E/10G - we get several automatic overclocking options in the software which are mirrored in the BIOS under 'Optimized CPU Setting'. This offers 4.0 GHz, 4.2 GHz, 4.4 GHz and 4.5 GHz options with varying levels of load line calibration and voltage.

While manual overclocking is not usually the focus of workstation motherboards, ASRock puts their usual array of options in to allow control of voltages, the FIVR, and the DRAM. We didn't achieve anything special in terms of overclocking due our mid-performance CPU.

Methodology

Our standard overclocking methodology is as follows. We select the automatic overclock options and test for stability with PovRay and OCCT to simulate high-end workloads. These stability tests aim to catch any immediate causes for memory or CPU errors.

For manual overclocks, based on the information gathered from previous testing, starts off at a nominal voltage and CPU multiplier, and the multiplier is increased until the stability tests are failed. The CPU voltage is increased gradually until the stability tests are passed, and the process repeated until the motherboard reduces the multiplier automatically (due to safety protocol) or the CPU temperature reaches a stupidly high level (100ºC+). Our test bed is not in a case, which should push overclocks higher with fresher (cooler) air.

Overclock Results

System Performance

Power Consumption

Power consumption was tested on the system while in a single MSI GTX 770 Lightning GPU configuration with a wall meter connected to the OCZ 1250W power supply. This power supply is Gold rated, and as I am in the UK on a 230-240 V supply, leads to ~75% efficiency > 50W, and 90%+ efficiency at 250W, suitable for both idle and multi-GPU loading. This method of power reading allows us to compare the power management of the UEFI and the board to supply components with power under load, and includes typical PSU losses due to efficiency. These are the real world values that consumers may expect from a typical system (minus the monitor) using this motherboard.

While this method for power measurement may not be ideal, and you feel these numbers are not representative due to the high wattage power supply being used (we use the same PSU to remain consistent over a series of reviews, and the fact that some boards on our test bed get tested with three or four high powered GPUs), the important point to take away is the relationship between the numbers. These boards are all under the same conditions, and thus the differences between them should be easy to spot.

Power consumption all around was very high for the X99 WS-E/10G, with a long idle to load delta of 157W for the CPU; most other motherboards hover between 130 and 140W, so this compounds the peak power issue.

Windows 7 POST Time

Different motherboards have different POST sequences before an operating system is initialized. A lot of this is dependent on the board itself, and POST boot time is determined by the controllers on board (and the sequence of how those extras are organized). As part of our testing, we look at the POST Boot Time using a stopwatch. This is the time from pressing the ON button on the computer to when Windows 7 starts loading. (We discount Windows loading as it is highly variable given Windows specific features.) 

Despite having the extra X540-BT2 controller, POST time on the board was within the expected range.

Rightmark Audio Analyzer 6.2.5

Rightmark:AA indicates how well the sound system is built and isolated from electrical interference (either internally or externally). For this test we connect the Line Out to the Line In using a short six inch 3.5mm to 3.5mm high-quality jack, turn the OS speaker volume to 100%, and run the Rightmark default test suite at 192 kHz, 24-bit. The OS is tuned to 192 kHz/24-bit input and output, and the Line-In volume is adjusted until we have the best RMAA value in the mini-pretest. We look specifically at the Dynamic Range of the audio codec used on board, as well as the Total Harmonic Distortion + Noise.

Dynamic Range of X99 WS-E/10G

While our Dynamic Range results were reasonable, the THD+N results were a lot lower than expected.

USB Backup

For this benchmark, we transfer a set size of files from the SSD to the USB drive using DiskBench, which monitors the time taken to transfer. The files transferred are a 1.52 GB set of 2867 files across 320 folders – 95% of these files are small typical website files, and the rest (90% of the size) are small 30 second HD videos. In an update to pre-Z87 testing, we also run MaxCPU to load up one of the threads during the test which improves general performance up to 15% by causing all the internal pathways to run at full speed.

Both USB 2.0 and USB 3.0 speeds were on the high end in comparison to the other X99 motherboards tested.

DPC Latency

Deferred Procedure Call latency is a way in which Windows handles interrupt servicing. In order to wait for a processor to acknowledge the request, the system will queue all interrupt requests by priority. Critical interrupts will be handled as soon as possible, whereas lesser priority requests such as audio will be further down the line. If the audio device requires data, it will have to wait until the request is processed before the buffer is filled.

If the device drivers of higher priority components in a system are poorly implemented, this can cause delays in request scheduling and process time.  This can lead to an empty audio buffer and characteristic audible pauses, pops and clicks. The DPC latency checker measures how much time is taken processing DPCs from driver invocation. The lower the value will result in better audio transfer at smaller buffer sizes. Results are measured in microseconds.

DPC results were rather high with the X540-BT2 enabled in the BIOS, showing 772 microseconds. Initially it was thought this was due to the GPU attached, but on disabling the 10G ports it reduced down to 86 microseconds which is more manageable. Unfortunately the main reason for getting this motherboard would be the 10G ports, so it would seem some optimization is needed.

CPU Performance

Readers of our motherboard review section will have noted the trend in modern motherboards to implement a form of MultiCore Enhancement / Acceleration / Turbo (read our report here) on their motherboards. This does several things, including better benchmark results at stock settings (not entirely needed if overclocking is an end-user goal) at the expense of heat and temperature. It also gives in essence an automatic overclock which may be against what the user wants. Our testing methodology is ‘out-of-the-box’, with the latest public BIOS installed and XMP enabled, and thus subject to the whims of this feature. It is ultimately up to the motherboard manufacturer to take this risk – and manufacturers taking risks in the setup is something they do on every product (think C-state settings, USB priority, DPC Latency / monitoring priority, memory subtimings at JEDEC). Processor speed change is part of that risk, and ultimately if no overclocking is planned, some motherboards will affect how fast that shiny new processor goes and can be an important factor in the system build.

For reference, the ASRock X99 WS-E/10G has MultiCore Turbo enabled by default on BIOS 1.01A.

Point Calculations – 3D Movement Algorithm Test: link

3DPM is a self-penned benchmark, taking basic 3D movement algorithms used in Brownian Motion simulations and testing them for speed. High floating point performance, MHz and IPC wins in the single thread version, whereas the multithread version has to handle the threads and loves more cores.

Compression – WinRAR 5.0.1: link

Our WinRAR test from 2013 is updated to the latest version of WinRAR at the start of 2014. We compress a set of 2867 files across 320 folders totaling 1.52 GB in size – 95% of these files are small typical website files, and the rest (90% of the size) are small 30 second 720p videos.

Image Manipulation – FastStone Image Viewer 4.9: link

Similarly to WinRAR, the FastStone test us updated for 2014 to the latest version. FastStone is the program I use to perform quick or bulk actions on images, such as resizing, adjusting for color and cropping. In our test we take a series of 170 images in various sizes and formats and convert them all into 640x480 .gif files, maintaining the aspect ratio. FastStone does not use multithreading for this test, and thus single threaded performance is often the winner.

Video Conversion – Handbrake v0.9.9: link

Handbrake is a media conversion tool that was initially designed to help DVD ISOs and Video CDs into more common video formats. The principle today is still the same, primarily as an output for H.264 + AAC/MP3 audio within an MKV container. In our test we use the same videos as in the Xilisoft test, and results are given in frames per second.

Rendering – PovRay 3.7: link

The Persistence of Vision RayTracer, or PovRay, is a freeware package for as the name suggests, ray tracing. It is a pure renderer, rather than modeling software, but the latest beta version contains a handy benchmark for stressing all processing threads on a platform. We have been using this test in motherboard reviews to test memory stability at various CPU speeds to good effect – if it passes the test, the IMC in the CPU is stable for a given CPU speed. As a CPU test, it runs for approximately 2-3 minutes on high end platforms.

Synthetic – 7-Zip 9.2: link

As an open source compression tool, 7-Zip is a popular tool for making sets of files easier to handle and transfer. The software offers up its own benchmark, to which we report the result.

Gaming Performance

An interesting point to consider is the addition of the PLX8747 chips to allow for x16/x16/x16/x16 PCIe layouts. In previous reviews we have noted that these chips barely reduce the frame rate if at all, however one could suggest that as the X99 WS-E/10G uses two of them (and we split our SLI testing cards between them), there might be more room for additional delays.

Ultimately however our results were within the same range as the other X99 motherboards, suggesting that PCIe bandwidth on this scale (1080p, max settings) is not affected. It would be interesting to see a three or four-way SLI setup powering several 4K monitors.

F1 2013

First up is F1 2013 by Codemasters. I am a big Formula 1 fan in my spare time, and nothing makes me happier than carving up the field in a Caterham, waving to the Red Bulls as I drive by (because I play on easy and take shortcuts). F1 2013 uses the EGO Engine, and like other Codemasters games ends up being very playable on old hardware quite easily. In order to beef up the benchmark a bit, we devised the following scenario for the benchmark mode: one lap of Spa-Francorchamps in the heavy wet, the benchmark follows Jenson Button in the McLaren who starts on the grid in 22nd place, with the field made up of 11 Williams cars, 5 Marussia and 5 Caterham in that order. This puts emphasis on the CPU to handle the AI in the wet, and allows for a good amount of overtaking during the automated benchmark. We test at 1920x1080 on Ultra graphical settings.

Bioshock Infinite

Bioshock Infinite was Zero Punctuation’s Game of the Year for 2013, uses the Unreal Engine 3, and is designed to scale with both cores and graphical prowess. We test the benchmark using the Adrenaline benchmark tool and the Xtreme (1920x1080, Maximum) performance setting, noting down the average frame rates and the minimum frame rates.

Tomb Raider

The next benchmark in our test is Tomb Raider. Tomb Raider is an AMD optimized game, lauded for its use of TressFX creating dynamic hair to increase the immersion in game. Tomb Raider uses a modified version of the Crystal Engine, and enjoys raw horsepower. We test the benchmark using the Adrenaline benchmark tool and the Xtreme (1920x1080, Maximum) performance setting, noting down the average frame rates and the minimum frame rates.

Sleeping Dogs

Sleeping Dogs is a benchmarking wet dream – a highly complex benchmark that can bring the toughest setup and high resolutions down into single figures. Having an extreme SSAO setting can do that, but at the right settings Sleeping Dogs is highly playable and enjoyable. We run the basic benchmark program laid out in the Adrenaline benchmark tool, and the Xtreme (1920x1080, Maximum) performance setting, noting down the average frame rates and the minimum frame rates.

Battlefield 4

The EA/DICE series that has taken countless hours of my life away is back for another iteration, using the Frostbite 3 engine. AMD is also piling its resources into BF4 with the new Mantle API for developers, designed to cut the time required for the CPU to dispatch commands to the graphical sub-system. For our test we use the in-game benchmarking tools and record the frame time for the first ~70 seconds of the Tashgar single player mission, which is an on-rails generation of and rendering of objects and textures. We test at 1920x1080 at Ultra settings.

ASRock X99 WS-E/10G Conclusion

One could invite a saying 'the mind wants what the body can't have'. For a number of months and launches, I have wondered why there was a lack of 10GBase-T on consumer motherboards. The simple answer is that the X540 chips, or the Broadcom variants, are not only expensive but also power hungry enough require their own cooling but also require PCIe 2.0 x8 as a recommended minimum. It still baffles me why, despite these issues, it took so long to get on a product.

Our testing however shows the reality of the situation. In a single user point to point transfer, we yielded just over 2.0 Gbps, only 20% of the supposed rating. This was with a 1 GB transfer, with higher sizes increasing the speed up to a point. In order to get more than 2.0 Gbps, we needed to instigate multiple access streams to simulate more than one transfer request. The results pushed us into the 6-8 Gbps range from 4-10 streams and above 8 Gbps for 10+.

This puts a limit on the usefulness for a single individual. It means that the 10GBase-T network interface benefits from individuals that can emulate multiple access streams through software development (bulk information transfer such as MPI or rendering data) or in an SOHO/SMB environment where many users might want to be situated on VMs located on the machine. The motherboard is aimed at the workstation market with support for Xeons and RDIMMs, as well as 1U height clearance for servers too. I can imagine a server or workstation environment using this with several PCIe co-processors attached, each one assigned to a VM and being connected to via the 10G ports.

Due to the price of the controller, ASRock put the 10GBase-T on their highest end motherboard model under the premise that only extreme users will need it. As a result the board is equipped with two PLX8747 chips to allow for x16/x16/x16/x16 operation in a four-way GPU arrangement or x8/x8/x8/x8/x8/x8/x16 when single slot cards are in play. This opens up the market to PCIe coprocessors, RAID card arrangements and FPGA implementation workstations aplenty.  Elsewhere on the board are twelve total SATA ports, eight USB 3.0 ports, two Intel I210 GBit ports alongside the two Intel X540-BT2 10Gbit ports, TPM, SATA DOM, M.2 x4 and an enhanced Realtek ALC1150 audio codec solution.

Benchmark results were pretty much ballpark for X99 at stock levels, with multicore turbo putting the CPU results up nearer the top. One disappointing note was the DPC Latency which was reasonable only when the X540 10GBit ports were disabled, suggesting that the combination of 10G drivers and BIOS are not yet optimized for this sort of scenario. However on the plus side POST times were not affected by the X540 controller.

In the end, despite not yet knowing the price of the ASRock X99 WS-E/10G, we can say that it will be expensive. This means possibly in the $700-900 range, due to all the higher end connectivity in play. For that reason alone, the only way this board will be sold is to those that need 10G but also multi-GPU bandwidth. With all that said, I'm still glad ASRock has shown that 10G is possible in the consumer space.