If you're running a large dataset with high performance requirements, that $50,000 could pay for itself if you can do more business with it.
Take Amazon - if they can get a 25% increase in usable, full load storage from these devices than they can with a device half the price, it'll still pay for itself pretty damned quickly. Especially if it uses less power, requiring less CRAC, more storage per rack, etc.
Once you get beyond a certain scale (in terms of customer base and performance requirements) this stuff really, really matters - $150,000 of storage is cheaper than having to blow millions on a new DC.
My thoughts exactly. A cool $406,640 for the 136TB version! That's one crazy expensive 1U server!
When you are buying this many TBs, you'd think you should at least be getting < $1/GB. On the consumer end, I can't wait until they can get something like this down to the $0.1/GB range. Not sure if that'll even happen in 10 years though... I can dream.
Big enterprise storage is far more expensive than consumer storage. You're primarily paying for all the custom software/hardware needed to build SAN systems; and the small number of customers buying that sort of hardware (relative to the consumer or SMB markets) means each one ends up paying for a significant fraction of the total custom engineering/integration labor needed to make it work.
I know of distributed file systems that are disk-based that are more than $1/GB for I can say, from an HPC/Big Data/Enterprise standpoint, these are worth the money if they provide some lowered TCO/increased performance based on density and reliability. Cost of hardware is NOTHING compared to total TCO that large enterprise faces when it comes to Infrastructure.
Why aren't you impressed? That's very cheap for Enterprise flash storage. Fusion-IO charges 60% more than that for just the raw flash PCIe cards themselves (let alone the appliance they go in), and Intel charges about 30% more for the raw cards (again, before you even get into the appliance cost).
What impresses me is the sheer amount of silicon in the 136 TB box. Using Micron's 16 nm, 128 Gbit NAND, that's 8,500 dies in a 1U enclosure. With a die size of ~173 mm^2, that's almost 1.5 m^2 of silicon, or more than 24 wafers worth! It should be called sandBox instead of skyHawk, but I guess that name wouldn't be sufficiently "cloudy".
It would be interesting to see more information about redundancy and backups. Do they have tooling like Netapp for mirroring volumes within the chassis or to a volume on another chassis? How about redundancy within the volume? What's the primary market they are going for? Enterprise class storage with carrier grade safety built in or high performance, fault tolerant storage?
Skyera, I would like to offer my services for an extended test of the 136TB version at my house. I'm happy to test one for 10 years to prove reliability, and I'll even do it for free!
Kristian, I personally like all of the enterprise coverage. Besides just being generally interesting, there is often some (slow) trickle down, at least to the still-pricey enthusiast level.
I'm particularly interested in something that demonstrates why SAN systems are able to be sold for so much more than the incorrect "it's just a giant box full of drives" assumption would suggest.
Mostly support and the much smaller economies of scale to support the large engineering costs. While they may look like a box of drives, they are run by highly customized software and hardware components. In addition, most parts are stocked for a decade and available almost anywhere in hours, with paid engineering for onsite repair and install.
Compression and de-dupe (a la Sandforce???) may not be such a good fit in Enterprise Storage, given the widely used encryption which makes these things irrelevant.
Great write up, I have never heard of Cloudera, but this is a very interesting proposition.
My company offers Nimble storage almost exclusively to our customers. While I originally thought Nimble was all about SSD and flash, their message seems to have a lot to do with compression, sequential writes, and ultimately saving space in datacenter racks. Also since Nimble claim that their performance is not l limited by storage technologies but instead limited by CPU, I feel like more coverage here at AnandTech is highly appropriate.
I met with the CEO of Nimble at FMS, so that's definitely a company that I want and will be covering in the future. Their approach is similar to Skyera's (i.e. they buy NAND and build their own blades), but I'll have to check in with them for more details.
Wait for SkyEagle, website currently says that they will have 300TB in 1U and add on top of that compression and deduplication that would push it up above the 1PB effective capacity mark. That is in 1U.
While other flash array vendors boast about multi-terabyte capacities in a full rack, Skyera puts over 300TB of raw flash capacity in a single 1U skyEagle array, or over 10PB in a standard size rack. skyEagle’s on-board hardware-accelerated data compression and data de-duplication dramatically increase usable capacity well beyond the physical capacity."
They really should have gone with some PCIExpress interface. The latency with iSCSI would be too large for databases and other IOPS intensive applications, which SSD arrays are good for.
It is an enterprise storage array so won't be sitting on the PCIExpress interface. iSCSI is what it is and the Skyera SkyHawk does very well here delivering sub millisecond response with SQL databases during our testing which pushed it to 100k IOPS (and it more that it could give easily).
As far as enterprise storage goes, I am more interested in the following things: - RAS features: can you replace/hotplug a flash module in this box when it fails? - OS/Hypervisor support: At the moment, only the Skyhawk is listed in VMware's HCL. Not the SkyhawkFS - Management: Can you cluster/scale-out one box to multiple boxes once 136 TB is not enough, all with single instance management and single namespace (e.g. for NFS) - Protocol support: Right now only iSCSI and NFSv3 are supported. What about FCoE, SMBv3 and pNFS (NFSv4.1)?
There is an error in the article "For comparison, Pure Storage, which is one of the leading all-flash array suppliers, only offers up to 11TB in 2U form factor, so Skyera has an enormous advantage in terms of density. "
The Pure is 2U for just the controllers, to get 11TB RAW from Pure you need both controllers and storage shelves so you are looking at a minimum of 4U or 6U (depending on which shelf option you choose). That is with the lowerst end controllers. If you step up to the beefier controllers then they are 4U for just the controllers then you need to add the storage shelves.
I've tested the Skyera SkyHawk and it is an awesome device.
We purchased 3 skyHawk units and overall we like them very much however there are many bugs and limitations with the management piece of their solution. I'm excited for future improvements in skyEagle such as replication, dual-controllers, FC, etc.
Oh btw, I believe they will have a direct PCI express connection option at some point too. I'm not quite sure how they will connect that way though.
We currently own 4 Skyhawks. While it is true you need to build some "stuff" around it for full redundancy etc. they certainly are some incredible devices. With the right tools and setup they are rock solid and freaking fast. Of course we also had some issues but one should count this in when dealing with a startup. We call it "fun" :) Currently we are testing dedupe appliances in front of them which ceertainly will get us far below 0.50$ per usable GB...
BTW. it is not possible to replace blades in skyhawk (this feature will come in skyeagle where every part can be hot-swapped, even blades), but if cells (and/or full blades) fail you will simply lose some usable flash (after over provisioning flash runs out...) but raid-se will still be in-place. If you really wanted to replace a blade this would need some downtime
We’ve updated our terms. By continuing to use the site and/or by logging into your account, you agree to the Site’s updated Terms of Use and Privacy Policy.
Facebook
Twitter
RSS
34 Comments
Back to Article
mevans336 - Wednesday, October 29, 2014 - link
So their base model costs $50,000 for 16TB? All flash or not, I'm not impressed.Beany2013 - Wednesday, October 29, 2014 - link
If you're running a large dataset with high performance requirements, that $50,000 could pay for itself if you can do more business with it.Take Amazon - if they can get a 25% increase in usable, full load storage from these devices than they can with a device half the price, it'll still pay for itself pretty damned quickly. Especially if it uses less power, requiring less CRAC, more storage per rack, etc.
Once you get beyond a certain scale (in terms of customer base and performance requirements) this stuff really, really matters - $150,000 of storage is cheaper than having to blow millions on a new DC.
3DoubleD - Wednesday, October 29, 2014 - link
My thoughts exactly. A cool $406,640 for the 136TB version! That's one crazy expensive 1U server!When you are buying this many TBs, you'd think you should at least be getting < $1/GB. On the consumer end, I can't wait until they can get something like this down to the $0.1/GB range. Not sure if that'll even happen in 10 years though... I can dream.
DanNeely - Wednesday, October 29, 2014 - link
Big enterprise storage is far more expensive than consumer storage. You're primarily paying for all the custom software/hardware needed to build SAN systems; and the small number of customers buying that sort of hardware (relative to the consumer or SMB markets) means each one ends up paying for a significant fraction of the total custom engineering/integration labor needed to make it work.Ushio01 - Wednesday, October 29, 2014 - link
HEH, in the rackmount SSD space Skyera are a budget brand. If you want expensive go look at Kove and Kaminario rackmount SSD's.PCTC2 - Wednesday, October 29, 2014 - link
I know of distributed file systems that are disk-based that are more than $1/GB for I can say, from an HPC/Big Data/Enterprise standpoint, these are worth the money if they provide some lowered TCO/increased performance based on density and reliability. Cost of hardware is NOTHING compared to total TCO that large enterprise faces when it comes to Infrastructure.Guspaz - Wednesday, October 29, 2014 - link
Why aren't you impressed? That's very cheap for Enterprise flash storage. Fusion-IO charges 60% more than that for just the raw flash PCIe cards themselves (let alone the appliance they go in), and Intel charges about 30% more for the raw cards (again, before you even get into the appliance cost).UltraTech79 - Saturday, November 1, 2014 - link
Why isnt he? Because he is like a monkey looking at a CPU tossed in his cage. He doesn't know what he's looking at.repoman27 - Thursday, October 30, 2014 - link
What impresses me is the sheer amount of silicon in the 136 TB box. Using Micron's 16 nm, 128 Gbit NAND, that's 8,500 dies in a 1U enclosure. With a die size of ~173 mm^2, that's almost 1.5 m^2 of silicon, or more than 24 wafers worth! It should be called sandBox instead of skyHawk, but I guess that name wouldn't be sufficiently "cloudy".nekoken - Wednesday, October 29, 2014 - link
It would be interesting to see more information about redundancy and backups. Do they have tooling like Netapp for mirroring volumes within the chassis or to a volume on another chassis? How about redundancy within the volume? What's the primary market they are going for? Enterprise class storage with carrier grade safety built in or high performance, fault tolerant storage?Bob Todd - Wednesday, October 29, 2014 - link
Skyera, I would like to offer my services for an extended test of the 136TB version at my house. I'm happy to test one for 10 years to prove reliability, and I'll even do it for free!Kristian, I personally like all of the enterprise coverage. Besides just being generally interesting, there is often some (slow) trickle down, at least to the still-pricey enthusiast level.
Mikemk - Thursday, October 30, 2014 - link
I'd like to test one too.DanNeely - Wednesday, October 29, 2014 - link
I'm particularly interested in something that demonstrates why SAN systems are able to be sold for so much more than the incorrect "it's just a giant box full of drives" assumption would suggest.irwincur - Wednesday, October 29, 2014 - link
Mostly support and the much smaller economies of scale to support the large engineering costs. While they may look like a box of drives, they are run by highly customized software and hardware components. In addition, most parts are stocked for a decade and available almost anywhere in hours, with paid engineering for onsite repair and install.FunBunny2 - Wednesday, October 29, 2014 - link
Compression and de-dupe (a la Sandforce???) may not be such a good fit in Enterprise Storage, given the widely used encryption which makes these things irrelevant.Bob Todd - Wednesday, October 29, 2014 - link
These aren't meant to just be a corp file server that *might* have some encrypted data on it.Kristian Vättö - Thursday, October 30, 2014 - link
Skyera doesn't actually do de-duplication. That was a misunderstanding on my part, sorry about that. I've updated the story.hojnikb - Wednesday, October 29, 2014 - link
What nand controller does this use ?Kristian Vättö - Thursday, October 30, 2014 - link
It's an FBGA based like most enterprise controllers are. Skyera does have an ASIC in the development, though.extide - Friday, October 31, 2014 - link
FPGA, you mean :)LarsBars - Wednesday, October 29, 2014 - link
Great write up, I have never heard of Cloudera, but this is a very interesting proposition.My company offers Nimble storage almost exclusively to our customers. While I originally thought Nimble was all about SSD and flash, their message seems to have a lot to do with compression, sequential writes, and ultimately saving space in datacenter racks. Also since Nimble claim that their performance is not l limited by storage technologies but instead limited by CPU, I feel like more coverage here at AnandTech is highly appropriate.
LarsBars - Wednesday, October 29, 2014 - link
*Skyera, whoops!Kristian Vättö - Thursday, October 30, 2014 - link
I met with the CEO of Nimble at FMS, so that's definitely a company that I want and will be covering in the future. Their approach is similar to Skyera's (i.e. they buy NAND and build their own blades), but I'll have to check in with them for more details.Mikemk - Thursday, October 30, 2014 - link
Forget 136TB. I want a pettabyterriiicchh - Thursday, October 30, 2014 - link
Wait for SkyEagle, website currently says that they will have 300TB in 1U and add on top of that compression and deduplication that would push it up above the 1PB effective capacity mark. That is in 1U.dabotsonline - Thursday, October 30, 2014 - link
"300TB IN 1U FORM FACTORWhile other flash array vendors boast about multi-terabyte capacities in a full rack, Skyera puts over 300TB of raw flash capacity in a single 1U skyEagle array, or over 10PB in a standard size rack. skyEagle’s on-board hardware-accelerated data compression and data de-duplication dramatically increase usable capacity well beyond the physical capacity."
http://www.skyera.com/products/skyeagle/1u-form-fa...
That'll be over $1m for 1U, won't it?!
vFunct - Thursday, October 30, 2014 - link
They really should have gone with some PCIExpress interface. The latency with iSCSI would be too large for databases and other IOPS intensive applications, which SSD arrays are good for.vFunct - Thursday, October 30, 2014 - link
And yes, I'd like to hear more about large database servers in future articles.Actually, a thorough review of an 8-socket IBM x3950 X6 would be useful to me..
rriiicchh - Thursday, October 30, 2014 - link
It is an enterprise storage array so won't be sitting on the PCIExpress interface. iSCSI is what it is and the Skyera SkyHawk does very well here delivering sub millisecond response with SQL databases during our testing which pushed it to 100k IOPS (and it more that it could give easily).The SkyEagle will be coming with a FC option.
avbohemen - Thursday, October 30, 2014 - link
As far as enterprise storage goes, I am more interested in the following things:- RAS features: can you replace/hotplug a flash module in this box when it fails?
- OS/Hypervisor support: At the moment, only the Skyhawk is listed in VMware's HCL. Not the SkyhawkFS
- Management: Can you cluster/scale-out one box to multiple boxes once 136 TB is not enough, all with single instance management and single namespace (e.g. for NFS)
- Protocol support: Right now only iSCSI and NFSv3 are supported. What about FCoE, SMBv3 and pNFS (NFSv4.1)?
rriiicchh - Thursday, October 30, 2014 - link
There is an error in the article "For comparison, Pure Storage, which is one of the leading all-flash array suppliers, only offers up to 11TB in 2U form factor, so Skyera has an enormous advantage in terms of density. "The Pure is 2U for just the controllers, to get 11TB RAW from Pure you need both controllers and storage shelves so you are looking at a minimum of 4U or 6U (depending on which shelf option you choose). That is with the lowerst end controllers. If you step up to the beefier controllers then they are 4U for just the controllers then you need to add the storage shelves.
I've tested the Skyera SkyHawk and it is an awesome device.
acrophile - Friday, October 31, 2014 - link
We purchased 3 skyHawk units and overall we like them very much however there are many bugs and limitations with the management piece of their solution. I'm excited for future improvements in skyEagle such as replication, dual-controllers, FC, etc.Oh btw, I believe they will have a direct PCI express connection option at some point too. I'm not quite sure how they will connect that way though.
Delo123 - Monday, November 17, 2014 - link
We currently own 4 Skyhawks. While it is true you need to build some "stuff" around it for full redundancy etc. they certainly are some incredible devices. With the right tools and setup they are rock solid and freaking fast. Of course we also had some issues but one should count this in when dealing with a startup. We call it "fun" :) Currently we are testing dedupe appliances in front of them which ceertainly will get us far below 0.50$ per usable GB...BTW. it is not possible to replace blades in skyhawk (this feature will come in skyeagle where every part can be hot-swapped, even blades), but if cells (and/or full blades) fail you will simply lose some usable flash (after over provisioning flash runs out...) but raid-se will still be in-place. If you really wanted to replace a blade this would need some downtime