Ever since you guys reported here on HBM being used in the consumer space on that famous small AMD GPU, I felt HBM was going to be mainstream. Going wide, stacked and close seems obvious when you need more bandwidth and lesser latencies.
Of course wide, stacked and close were a bit novel then, but pretty ubiquitous today, where I believe the vast majority of SoCs actually carry their DRAM burden on their back?[top? It’s so hard to tell with all this flip-chippery going on…] and that doesn’t seem to put them into HPC price ranges, where HBM thrives and gets significant scale.
So what gives? HBM from what I gather is still 10x on price vs DRAM and GDDR something tends to be 2-3x. And GDDR actually seems to require quite a bit of special signal and logic engineering to fine tune timing along much longer and off-die carrier traces. By comparison, HBM seemed almost less of a challenge for transmission logic.
That wide and close (stacked, too?) isn’t an entirely bad idea has been demonstrated by the rotten fruit cult, which seems to use commodity RAM-parts if not packaging.
Naïve as I am, I keep thinking that the major obstacle to HBM commodity pricing has been its niche position and therefore scale, but that should flip naturally given the rather general memory wall pressure.
So where are the real technological/economical barriers to making HBM if not the exclusive RAM on mobile and desktop, then at least a more normal part of any workstation?
Chiplet designs are several generations established even in the consumer space, and with 192GB max capacity on an Apple chip, which would be interesting if it wasn’t tied into their indenture, even my RAM expansion anxiety is receding, perhaps CXL to the rescue in corner cases.
Stacking is extremely normal these days, from flash to just high-density DRAM, so that wouldn’t really explain the price gap. Wide is never cheap, but there is such a variety of materials and packaging technologies addressing this and then where you can put CCDs, you can also put HBM, especially with Infinity Fabric, I believe.
Having to tie down RAM capacity at SoC production is a bother, but even today there is little difference to soldered RAM, which is never near the capacity nor the price that I prefer and believe adequate. So there is a bit of a risk of winding up with inventory that won’t sell at optimal prices.
What am I missing?
TSVs I guess. But as the AMD V-cache chips prove, even TSVs don’t convert commodity parts in unobtainium any more, they have become a manageable part of the technology stack.
Of course, when you need to stack DRAM chips not just in two layers but some nice binary multiple, the risk of something going wrong in the middle killing the entire stack needs to be paid somewhere. But unless you’re in HPC territory, your HBM stacks don’t need to be that high, TSVs don’t have to run the entire building, just enough floors until the bandwidth is right. And then you can stack these bundles again, using wires at the edge or even lower density interposer chiplets creating risk breakers or intermediate level lobbies, without having to expand in terms of ground floor space.
One of the greatest benefits of the PC space during the last four decades has been its modularity, the ability for an owner to compose what he wanted. The point of composition has now firmly moved to the SoC level, while traditional players still try to claw out ever bigger parts of the cake for their own.
That obviously leaves a lot of potential for creative solutions on the table, you wind up paying excess capacity at every component, because a more synergic solution only works with the kind of scale that clouds and Apple can muster.
Alas the PC started with a processor ASIC and lots of TTL to build everything else. Chipsets ingested TTL chips and then other ASICs and today we see not just single chip PCs, but pretty much rack-on-chip coming and only the big boys still get to play personal computing.
Well, more pins means more money. There's very little competition, from what I've read, on the packaging front. I've heard that HBM has one or more patents on it and you have to license them all in order to produce HBM. If you only have 1 or 2 companies producing HBM then the price will be close to whatever they make it. Sort of like AMD and Nvidia driving up recent GPU pricing. You also need an interposer as normal PCB routing doesn't work for such high-density interconnects. This means a whole slab of silicon for the chips to sit on.
> So what gives? HBM from what I gather is still 10x on price vs DRAM...
If demand outstrips supply, the price won't go down, and instead will continue to stay high. Until Datacenter/AI GPU compute doesn't demand every last chip of HBM it can get it's hands on, the chances of price dropping and the tech being implemented in the consumer space is very low.
We’ve updated our terms. By continuing to use the site and/or by logging into your account, you agree to the Site’s updated Terms of Use and Privacy Policy.
Facebook
Twitter
RSS
4 Comments
Back to Article
abufrejoval - Thursday, July 27, 2023 - link
Ever since you guys reported here on HBM being used in the consumer space on that famous small AMD GPU, I felt HBM was going to be mainstream. Going wide, stacked and close seems obvious when you need more bandwidth and lesser latencies.Of course wide, stacked and close were a bit novel then, but pretty ubiquitous today, where I believe the vast majority of SoCs actually carry their DRAM burden on their back?[top? It’s so hard to tell with all this flip-chippery going on…] and that doesn’t seem to put them into HPC price ranges, where HBM thrives and gets significant scale.
So what gives? HBM from what I gather is still 10x on price vs DRAM and GDDR something tends to be 2-3x. And GDDR actually seems to require quite a bit of special signal and logic engineering to fine tune timing along much longer and off-die carrier traces. By comparison, HBM seemed almost less of a challenge for transmission logic.
That wide and close (stacked, too?) isn’t an entirely bad idea has been demonstrated by the rotten fruit cult, which seems to use commodity RAM-parts if not packaging.
Naïve as I am, I keep thinking that the major obstacle to HBM commodity pricing has been its niche position and therefore scale, but that should flip naturally given the rather general memory wall pressure.
So where are the real technological/economical barriers to making HBM if not the exclusive RAM on mobile and desktop, then at least a more normal part of any workstation?
Chiplet designs are several generations established even in the consumer space, and with 192GB max capacity on an Apple chip, which would be interesting if it wasn’t tied into their indenture, even my RAM expansion anxiety is receding, perhaps CXL to the rescue in corner cases.
Stacking is extremely normal these days, from flash to just high-density DRAM, so that wouldn’t really explain the price gap. Wide is never cheap, but there is such a variety of materials and packaging technologies addressing this and then where you can put CCDs, you can also put HBM, especially with Infinity Fabric, I believe.
Having to tie down RAM capacity at SoC production is a bother, but even today there is little difference to soldered RAM, which is never near the capacity nor the price that I prefer and believe adequate. So there is a bit of a risk of winding up with inventory that won’t sell at optimal prices.
What am I missing?
TSVs I guess. But as the AMD V-cache chips prove, even TSVs don’t convert commodity parts in unobtainium any more, they have become a manageable part of the technology stack.
Of course, when you need to stack DRAM chips not just in two layers but some nice binary multiple, the risk of something going wrong in the middle killing the entire stack needs to be paid somewhere.
But unless you’re in HPC territory, your HBM stacks don’t need to be that high, TSVs don’t have to run the entire building, just enough floors until the bandwidth is right. And then you can stack these bundles again, using wires at the edge or even lower density interposer chiplets creating risk breakers or intermediate level lobbies, without having to expand in terms of ground floor space.
One of the greatest benefits of the PC space during the last four decades has been its modularity, the ability for an owner to compose what he wanted. The point of composition has now firmly moved to the SoC level, while traditional players still try to claw out ever bigger parts of the cake for their own.
That obviously leaves a lot of potential for creative solutions on the table, you wind up paying excess capacity at every component, because a more synergic solution only works with the kind of scale that clouds and Apple can muster.
Alas the PC started with a processor ASIC and lots of TTL to build everything else. Chipsets ingested TTL chips and then other ASICs and today we see not just single chip PCs, but pretty much rack-on-chip coming and only the big boys still get to play personal computing.
ballsystemlord - Thursday, July 27, 2023 - link
(Holy wall of text, Batman!)> What am I missing?
Well, more pins means more money. There's very little competition, from what I've read, on the packaging front.
I've heard that HBM has one or more patents on it and you have to license them all in order to produce HBM.
If you only have 1 or 2 companies producing HBM then the price will be close to whatever they make it. Sort of like AMD and Nvidia driving up recent GPU pricing.
You also need an interposer as normal PCB routing doesn't work for such high-density interconnects. This means a whole slab of silicon for the chips to sit on.
I think that's everything.
James5mith - Saturday, July 29, 2023 - link
> So what gives? HBM from what I gather is still 10x on price vs DRAM...If demand outstrips supply, the price won't go down, and instead will continue to stay high. Until Datacenter/AI GPU compute doesn't demand every last chip of HBM it can get it's hands on, the chances of price dropping and the tech being implemented in the consumer space is very low.
Calle2003 - Thursday, July 27, 2023 - link
32 Gigabit only?