Probably nothing in the next couple of generations of GPUs will benefit much from the additional bandwidth, but if it can be done without a drastic cost increase, why not as it may become relevant in the future. Devices that are currently using say 4x could get by just fine on 2x and free up more PCIe lanes for other components or reduce the overall need and maybe save cost by reducing PCB wiring complexity.
You're missing the point. This is not about your consumer GPUs.
It's primarily about networking, storage, HPC, and multi-GPU deep learning.
For consumers, maybe M.2 drives will get replaced by something with only half the lanes. Perhaps you'll also start seeing more CPUs and GPUs with only x8 interfaces (and corresponding cost savings).
The question is fair. We've had 16xgen3 for a while now but only recently with the 1080ti and 2080ti was there any real need to use anything above 8xgen3. And this cause a serious issue for consumer CPUs since with NVME you start needing more lanes. Basically you could reach a bottleneck with a gpu NVME combo. You always think you're not gonna need it, but right now there are more than enough use-cases that show that gen4 for consumers should have been last year's news. By the time PC parts come out we're going to be well past gen3 being an acceptable compromise.
With NVMe getting more wide spread, I would rather see an increase in CPU lanes to cover at least one 4x device in addition to the 16x GPU on mainstream platforms. You know, like AMDs Ryzen. Thats one short-coming I hope Intel really fixes.
That way you can run a full x16 device, and one SSD. And if you need more, you can split 8 off of the GPU an run 2 more full bandwidth SSDs.
This would overall be more future proof, even if switching to PCIe4 or 5 would hold us off for a while.
Why are people under the impression NMVE is shared witgh the PCI-e16x slot ? this is simply not true. AM4 for example has: - 16 dedicated lanes from the cpu to pci-e16x - 4 dedicated lanes from the cpu to the NMVE slot - 4 dedicated lanes from the cpu to the chipset...
^this. I really fail to understand why people think GPU would need more bandwidth even though there have been multiple benchmarks done that this is not the case. As long as CPU/motherboard keeps feeding data at the rate GPU requires it won't be affecting GPU performance and that point is relatively low... like mining crypto was perfectly fine with 1x lane because there is not that much traffic between GPU and rest of the system.
I would correct "maybe M.2 drives will get replaced" assumption since m.2 is essentially using (depending on solution) either 2x or 4x pcie lanes, so having them using standard that doubles transfer speed doubles theoretical m.2 speed as well, so there is no need to replace m.2, just use newer PCIE standard and we get benefits from having same standard on laptops, desktops and servers. If you want something beefier you get PCIe card like one of those optain 905p or pcie add-on card where you can stack m.2 drives and set them up on raid 0 mode. Some youtuber already made a video about this, and I think this is the point AMD should be saying on their marketing videos and a thing that deserves more investigation... like why games don't really load faster if you have 10x higher bandwidth to your data and extremely good latencies. There is some hidden bottleneck that should be addressed ASAP.
Why do you need 16 GB/sec storage, in a desktop or laptop? That's what doesn't make sense to me about transitioning M.2 to PCIe 5.0. I think the mainstream PC market will prefer the cost savings of reducing lane count, and be happy with a x2 (or even x1) interface.
because games aren't raw data...! shaders need compilation, gpu usage goes up, cpu usage goes up, assets get uncompressed from archives, in rare cases vsync/fps cap limits loading speed, in other cases the engine is too single threaded
did you not notice nvidia, amd, AND steam added cached shaders features that explicitly state they improve loading time?
there's no hidden bottleneck, it already is being addressed by engineers on multiple sides
I think PCIe 4.0 could reduce latency for VR use cases. That's the main consumer GPU use case I see for it... for those consumers using VR, anyhow. I don't know how much that would really carry over to PCIe 5, though.
I'm not saying it won't affect frame rates (esp. 99th percentile) at the margins, but we're not talking about a big impact at x16 connectivity.
Since we know Intel is using CXL for datacenter GPUs and that CXL can work atop PCIe 5, it will be interesting to see if Intel provides this to consumers, in some premium Titan-like product.
There is definitely some hidden bottleneck. My rig has 32 GB RAM and back when games were less than 24 GB, I created a RAM 24 GB RAM drive and loaded the entire game to the RAM drive. This still left 8 GB for system RAM at a time when 2GB was more than enough. Anyway even loading from the RAM drive the game didn't seem to load any faster. Back then I blamed it on my internet connect / ping / latency. Now days I have a fiber optic link to the internet, but I'm not convinced the network speed is the bottleneck. Could be protocol overhead and how Windows transfers data?
This is basically my point, except I wouldn't say that RAID controllers will get much narrower, depending on whether it's a SSD RAID or not. If SSD, then you'll probably need the additional bandwidth.
However, for HDD RAIDs, practical limits on RAID sizes would mean that controllers' interfaces will probably be cut in half.
no, but ssd that could saturate that slot would be fun. We ALREADY have 90% used on 5GB/s Corsair SSD and its YET unreleased to masses. 10GB/s SSD from PCIE5.0 would be insane.
It seems a bit odd, but that graph is actually putting the bandwidth gains in a very positive light. If you plot those points in a log scale, PCIe bandwidth has fallen short of the historical trend from 2014 until this announcement. This announcement puts everything back on track, but if you go by the "doubling every 3 years" like in the slides, PCIe 6.0 should have come out last year with a max bandwidth of ~128 GB/sec.
There were several articles in the recent past claiming that PCIe 4.0 would be too expensive for the mainstream. So it’ll be interesting to see the price difference for motherboards and cards.
"too expensive" is an odd statement. If it were too expensive for AMD produce then they wouldn't produce it, because they would have performed the necessary marginal analysis. If it were going to be too expensive for consumers, then again AMD wouldn't include it.
So reality is contradicting blogger speculation, I guess? *shrug*
Also, you don't know what guidance AMD could've given board designers, with the knowledge that they'd want to enable PCIe 4.0, in the future. Remember, AMD *planned* for AM4 to last a while, so they probably had some idea PCIe 4.0 would be coming.
"Works", sort of. You likely get one slot with a direct connection to the CPU doing it. For mainboards right now, the big advantage in PCIe4 is the increased bandwidth for the chipset to communicate with the CPU - and thats of course only available on new boards where the chipset speaks PCIe 4.0
What an odd thing to say or believe if heard. Its like claiming its been stated that DDR5 will be too expensive for the mainstream.. At what point in time are they referring?? Before its mass release to the public world then YES. PCIE 6, 7, 8, DDR 5,6,7 etc will all probably exist. And the mainstream will have access. Given time. Its how the PC business moves forward - with upgrades.
Well... I could point to 10 Gigabit, as a counterexample.
Now, you might say there was no consumer demand for it, but I'd say that's even more true of PCIe 4.0. The only reason we're getting it now is that AMD decided to give it to us.
Until someone cites the original claim that it'd be too expensive, we cannot pick apart the reasoning behind it.
The Zen 2 chips are PCIe 4.0 complaint anyway due to also being used in EPYC chips for servers so there is no extra cost at that level. As some Ryzen 2 series boards are also partially PCIe 4.0 complaint that suggests that the cost is not significant which is why it has been rolled out now even further. In the short term it may only be SSDs that benefit although that seems to be very much diminishing returns for consumers. As for 10Gb Ethernet, I hear people complaining that the hardware is still expensive; switches etc. Add that to low consumer interest for 10Gb Ethernet and no wonder it hasn't taken off yet. Technology that is cheap and easy to pitch to consumers tends to get implemented first.
> The Zen 2 chips are PCIe 4.0 complaint anyway due to also being used in EPYC chips for servers
I doubt it. The I/O chiplet in AM4 Ryzens is certainly different than what EPYC is using. That said, they did probably 90% of the work for EPYC, so maybe it wasn't a big deal to use it in the desktop chips.
Still, that doesn't change my point, which is that we're not getting PCIe 4.0 by popular demand.
You appear to be arguing the same side as I am, which is that we got PCIe 4.0 because it was apparently fairly cheap/easy. On the flip side, 10 Gigabit Ethernet is taking forever, because even though the demand has been stronger, longer than PCIe 4.0, it's still not been enough to drive down costs to the point where even typical enthusiasts would bite.
Thanks, I keep forgetting that the I/O die hosts the controller. With laptops, tablets and smart phones being used more than desktops seemingly, WiFi tends to get more traction than ethernet. It took ages for many ISPs to add gigabit ethernet to their routers which says something about the inertia.
PCIe 4.0 presents challenges but will obviously see widespread adoption. AMD is there now, and Intel will follow suit as soon as they can figure out how to manufacture some new lake that isn't Skylake. Furthermore, Thunderbolt 3 already exists in the consumer space with a significantly higher signaling rate, albeit at a premium.
I have a seriously hard time believing PCIe 5.0 will ever grace a consumer platform. NRZ with a Nyquist rate of 16 GHz and no FEC? I just do not see it happening. Here's a list of NRZ PHYs that operate at or above 8 GT/s for comparison:
A lot of these protocols, even DisplayPort and HDMI, have provisions for FEC. Consumers never saw SATA 12Gb/s because it was cheaper to converge with PCIe. 10 GbE was a hit in the datacenter but rarely seen on the desktop. The price of NICs was initially too high, that of switches remains exorbitant, and power requirements all but preclude it from mobile. Yet we're to believe that there will be a consumer interconnect that outstrips everything on the market except for the fastest FPGA transceiver available? What new magic will make this possible?
Do I see this correctly? PCIe-5 x16 is faster than any current dual-channel memory, and basically as fast as quad-channel memory? Does that mean PCIe 5 is really fast, current memory bus speeds really suck, or both?
Bandwidth != latency. For DDR4, "I want that byte!" has single-digit nanoseconds of latency (usually just below 10ns). For PCIe, even with DMA, you're looking at several hundred to over a thousand nanoseconds (or in worst-case contention, maybe several microseconds). If all you're doing is shoving a big block of data across a bus then raw bandwidth is great. If you want to put a memory interface over that bus, then you would very quickly notice the difference between DDR and PCIe.
Power dissipation may be a problem - the faster that transistors switch the more power is needed. As PCIe 5.0 has a data rate around 32Gbits/sec per lane this implies that the chips are going to get hot. (The X570 chipset which uses PCIe 4.0 needs active cooling - this problem will be worse for PCIe 5.0). It is possible that CPUs may need a separate chip in their package to do a mux/demux job to reduce the data rates to the level that the rest of the silicon can handle (transistors built to operate at 32GHz are unlikely to be able to use the same process as the rest of the CPU running at 5GHz or less). There is also going to need to be some serious microwave engineering on motherboards, add in cards and connectors to deal with the 32GHz signals. (At 32GHz a full wavelength is under 1mm!!) Screening of PC cases may well need to be much improved to stop PCs interfering with other users of the 32GHz frequency range.
So "actual bandwidth" in graph is duplex (bidirectional), whereas I/O bandwidth is bandwidth for one direction? It's in my question below. In this case you're right, there is a mistake in Anand's table ("Full Duplex").
unidirectional = 64GB/s x16 PCIe 5.0 Full Duplex = 64GB/s x16 PCIe 5.0 Aggregate (The crap that marketing teams do that adds the bandwidth of both directions of a full duplex link) = 128GB/s x16 PCIe 5.0
1GbE = 1Gbps full duplex. I.e. 1Gbps in/out simultaneously. It seems like nobody thinks about the fact that speeds in PCIe USED to be reported the same way as network gear, I.e. full duplex.
Kudos to the writer of the article reporting it correctly. I.e. 64GB/s FULL DUPLEX.
Quote from PCI-SIG's press release: "PCIe 5.0 Specification Highlights - Delivers 32 GT/s raw bit rate and up to 128(!!!) GB/s via x16 configuration". Is it a misprint?
PCE-E 4.0 is quite a bit more expensive than 3.0, but you make up the saving by having less lane to reach those bandwidth. I am wondering how much more expensive is it for PCI-E 5.0?
Now the SSD maker have to figure out a way to make 16GB/s PCI-E 5.0 SSD. :D
This could depend on it implemention of PCIe, Just because PCIe 4.0 is more expensive currently, does not mean that PCIe 5.0 will be by another vendor. It just send really fishy that AMD rushes out 4.0 support at very close to time 5.0 spec is out. There must be a reason why Intel skip 4.0.
Because 10nm is delayed. Just because Intel hasn't done it (yet) doesn't mean it's problematic.
HStewart even with the 2 brain cells you have you are aware that PCIe4 support is on Intel's roadmap? Or do I need to deduct another brain cell from your tally?
Intel did NOT skip PCIe 4.0. What you are referring to is their server chips using PCIe 5.0 next year. Moreover, what you are ignoring is CONSUMERS WILL NOT GET PCIe 5.0 ANYTIME SOON!
In fact, Intel is likely to bring PCIe 4.0 to consumers next year. Then you show your lack of knowledge that implementing it required certain signal integrity parameters in order to implement it. That meant more power on the chipset, that meant higher quality PCBs for motherboards, and for beyond 5-7 inch trace paths, you need signal conditioners/boosters and repeaters to drive beyond that distance. All of this is significant engineering challenges, all of which AMD did with their motherboard partners, whose engineers should be lauded for such achievements to so quickly bring it to mainstream implementations.
Instead, your trying to crap on AMD is also crapping on those motherboard engineers that worked out the implementation on their end. Good job! /s
We’ve updated our terms. By continuing to use the site and/or by logging into your account, you agree to the Site’s updated Terms of Use and Privacy Policy.
Facebook
Twitter
RSS
55 Comments
Back to Article
Hxx - Wednesday, May 29, 2019 - link
will 10 duct taped 2018TIs bottleneck the PCIE 5.0 bandwidth limit?PeachNCream - Wednesday, May 29, 2019 - link
Probably nothing in the next couple of generations of GPUs will benefit much from the additional bandwidth, but if it can be done without a drastic cost increase, why not as it may become relevant in the future. Devices that are currently using say 4x could get by just fine on 2x and free up more PCIe lanes for other components or reduce the overall need and maybe save cost by reducing PCB wiring complexity.mode_13h - Wednesday, May 29, 2019 - link
You're missing the point. This is not about your consumer GPUs.It's primarily about networking, storage, HPC, and multi-GPU deep learning.
For consumers, maybe M.2 drives will get replaced by something with only half the lanes. Perhaps you'll also start seeing more CPUs and GPUs with only x8 interfaces (and corresponding cost savings).
StevenD - Thursday, May 30, 2019 - link
The question is fair. We've had 16xgen3 for a while now but only recently with the 1080ti and 2080ti was there any real need to use anything above 8xgen3. And this cause a serious issue for consumer CPUs since with NVME you start needing more lanes. Basically you could reach a bottleneck with a gpu NVME combo.You always think you're not gonna need it, but right now there are more than enough use-cases that show that gen4 for consumers should have been last year's news. By the time PC parts come out we're going to be well past gen3 being an acceptable compromise.
nevcairiel - Thursday, May 30, 2019 - link
With NVMe getting more wide spread, I would rather see an increase in CPU lanes to cover at least one 4x device in addition to the 16x GPU on mainstream platforms. You know, like AMDs Ryzen. Thats one short-coming I hope Intel really fixes.That way you can run a full x16 device, and one SSD. And if you need more, you can split 8 off of the GPU an run 2 more full bandwidth SSDs.
This would overall be more future proof, even if switching to PCIe4 or 5 would hold us off for a while.
mode_13h - Thursday, May 30, 2019 - link
Um, what on earth are you doing that you need so much storage bandwidth in a _desktop_?cyberbug - Sunday, June 2, 2019 - link
Why are people under the impression NMVE is shared witgh the PCI-e16x slot ? this is simply not true.AM4 for example has:
- 16 dedicated lanes from the cpu to pci-e16x
- 4 dedicated lanes from the cpu to the NMVE slot
- 4 dedicated lanes from the cpu to the chipset...
mode_13h - Thursday, May 30, 2019 - link
In case you didn't notice, neither this article nor that comment were about Gen 4.compvter - Thursday, May 30, 2019 - link
^this. I really fail to understand why people think GPU would need more bandwidth even though there have been multiple benchmarks done that this is not the case. As long as CPU/motherboard keeps feeding data at the rate GPU requires it won't be affecting GPU performance and that point is relatively low... like mining crypto was perfectly fine with 1x lane because there is not that much traffic between GPU and rest of the system.I would correct "maybe M.2 drives will get replaced" assumption since m.2 is essentially using (depending on solution) either 2x or 4x pcie lanes, so having them using standard that doubles transfer speed doubles theoretical m.2 speed as well, so there is no need to replace m.2, just use newer PCIE standard and we get benefits from having same standard on laptops, desktops and servers. If you want something beefier you get PCIe card like one of those optain 905p or pcie add-on card where you can stack m.2 drives and set them up on raid 0 mode. Some youtuber already made a video about this, and I think this is the point AMD should be saying on their marketing videos and a thing that deserves more investigation... like why games don't really load faster if you have 10x higher bandwidth to your data and extremely good latencies. There is some hidden bottleneck that should be addressed ASAP.
mode_13h - Thursday, May 30, 2019 - link
Why do you need 16 GB/sec storage, in a desktop or laptop? That's what doesn't make sense to me about transitioning M.2 to PCIe 5.0. I think the mainstream PC market will prefer the cost savings of reducing lane count, and be happy with a x2 (or even x1) interface.kn00tcn - Friday, May 31, 2019 - link
because games aren't raw data...! shaders need compilation, gpu usage goes up, cpu usage goes up, assets get uncompressed from archives, in rare cases vsync/fps cap limits loading speed, in other cases the engine is too single threadeddid you not notice nvidia, amd, AND steam added cached shaders features that explicitly state they improve loading time?
there's no hidden bottleneck, it already is being addressed by engineers on multiple sides
mode_13h - Friday, May 31, 2019 - link
I think PCIe 4.0 could reduce latency for VR use cases. That's the main consumer GPU use case I see for it... for those consumers using VR, anyhow. I don't know how much that would really carry over to PCIe 5, though.I'm not saying it won't affect frame rates (esp. 99th percentile) at the margins, but we're not talking about a big impact at x16 connectivity.
Since we know Intel is using CXL for datacenter GPUs and that CXL can work atop PCIe 5, it will be interesting to see if Intel provides this to consumers, in some premium Titan-like product.
stancilmor - Monday, June 3, 2019 - link
There is definitely some hidden bottleneck. My rig has 32 GB RAM and back when games were less than 24 GB, I created a RAM 24 GB RAM drive and loaded the entire game to the RAM drive. This still left 8 GB for system RAM at a time when 2GB was more than enough. Anyway even loading from the RAM drive the game didn't seem to load any faster. Back then I blamed it on my internet connect / ping / latency. Now days I have a fiber optic link to the internet, but I'm not convinced the network speed is the bottleneck. Could be protocol overhead and how Windows transfers data?rtho782 - Thursday, May 30, 2019 - link
No, but given that such GPUs can cope with a PCIe 3.0 x8 link easily, they will be able to cope with a PCIe 5.0 x2 link.Other devices like hard drive controllers that previously needed to be x4 or x8 can now be x1 or x2.
Suddenly the 20 lanes from your CPU can go a lot further.
mode_13h - Thursday, May 30, 2019 - link
This is basically my point, except I wouldn't say that RAID controllers will get much narrower, depending on whether it's a SSD RAID or not. If SSD, then you'll probably need the additional bandwidth.However, for HDD RAIDs, practical limits on RAID sizes would mean that controllers' interfaces will probably be cut in half.
deil - Thursday, May 30, 2019 - link
no, but ssd that could saturate that slot would be fun.We ALREADY have 90% used on 5GB/s Corsair SSD and its YET unreleased to masses.
10GB/s SSD from PCIE5.0 would be insane.
Someguyperson - Wednesday, May 29, 2019 - link
It seems a bit odd, but that graph is actually putting the bandwidth gains in a very positive light. If you plot those points in a log scale, PCIe bandwidth has fallen short of the historical trend from 2014 until this announcement. This announcement puts everything back on track, but if you go by the "doubling every 3 years" like in the slides, PCIe 6.0 should have come out last year with a max bandwidth of ~128 GB/sec.The Hardcard - Wednesday, May 29, 2019 - link
There were several articles in the recent past claiming that PCIe 4.0 would be too expensive for the mainstream. So it’ll be interesting to see the price difference for motherboards and cards.sorten - Wednesday, May 29, 2019 - link
That's odd. It'll be a feature of every AMD motherboard moving forward until it's replaced with 5.0.imaheadcase - Wednesday, May 29, 2019 - link
What is odd? It is expensive on the AMD motherboards. Just how it is.sorten - Wednesday, May 29, 2019 - link
"too expensive" is an odd statement. If it were too expensive for AMD produce then they wouldn't produce it, because they would have performed the necessary marginal analysis. If it were going to be too expensive for consumers, then again AMD wouldn't include it.So reality is contradicting blogger speculation, I guess? *shrug*
alchemist83 - Wednesday, May 29, 2019 - link
LOL. Them bloggers be so clever. Cos they got plenty mugs to listen and take every BS sentence as gospel.alchemist83 - Wednesday, May 29, 2019 - link
Wot? Just babble.. Expensive ON AMD mobos? How so. What does that even mean? English, grammar, words, semantics. Get sum.PixyMisa - Wednesday, May 29, 2019 - link
Except that it even works on existing motherboards originally designed for PCIe 3.0.mode_13h - Wednesday, May 29, 2019 - link
Only on slots near the CPU.Also, you don't know what guidance AMD could've given board designers, with the knowledge that they'd want to enable PCIe 4.0, in the future. Remember, AMD *planned* for AM4 to last a while, so they probably had some idea PCIe 4.0 would be coming.
nevcairiel - Thursday, May 30, 2019 - link
"Works", sort of. You likely get one slot with a direct connection to the CPU doing it. For mainboards right now, the big advantage in PCIe4 is the increased bandwidth for the chipset to communicate with the CPU - and thats of course only available on new boards where the chipset speaks PCIe 4.0mode_13h - Thursday, May 30, 2019 - link
Yes, exactly.alchemist83 - Wednesday, May 29, 2019 - link
Exactly!alchemist83 - Wednesday, May 29, 2019 - link
What an odd thing to say or believe if heard. Its like claiming its been stated that DDR5 will be too expensive for the mainstream.. At what point in time are they referring?? Before its mass release to the public world then YES. PCIE 6, 7, 8, DDR 5,6,7 etc will all probably exist. And the mainstream will have access. Given time. Its how the PC business moves forward - with upgrades.Deses - Wednesday, May 29, 2019 - link
It's just basic demand and offer. When everyone moves from the old technology to the newer one and everyone mass produces it, costs go down.It's been always like that...
mode_13h - Wednesday, May 29, 2019 - link
Well... I could point to 10 Gigabit, as a counterexample.Now, you might say there was no consumer demand for it, but I'd say that's even more true of PCIe 4.0. The only reason we're getting it now is that AMD decided to give it to us.
Until someone cites the original claim that it'd be too expensive, we cannot pick apart the reasoning behind it.
smilingcrow - Wednesday, May 29, 2019 - link
The Zen 2 chips are PCIe 4.0 complaint anyway due to also being used in EPYC chips for servers so there is no extra cost at that level.As some Ryzen 2 series boards are also partially PCIe 4.0 complaint that suggests that the cost is not significant which is why it has been rolled out now even further.
In the short term it may only be SSDs that benefit although that seems to be very much diminishing returns for consumers.
As for 10Gb Ethernet, I hear people complaining that the hardware is still expensive; switches etc.
Add that to low consumer interest for 10Gb Ethernet and no wonder it hasn't taken off yet.
Technology that is cheap and easy to pitch to consumers tends to get implemented first.
mode_13h - Wednesday, May 29, 2019 - link
> The Zen 2 chips are PCIe 4.0 complaint anyway due to also being used in EPYC chips for serversI doubt it. The I/O chiplet in AM4 Ryzens is certainly different than what EPYC is using. That said, they did probably 90% of the work for EPYC, so maybe it wasn't a big deal to use it in the desktop chips.
Still, that doesn't change my point, which is that we're not getting PCIe 4.0 by popular demand.
You appear to be arguing the same side as I am, which is that we got PCIe 4.0 because it was apparently fairly cheap/easy. On the flip side, 10 Gigabit Ethernet is taking forever, because even though the demand has been stronger, longer than PCIe 4.0, it's still not been enough to drive down costs to the point where even typical enthusiasts would bite.
smilingcrow - Thursday, May 30, 2019 - link
Thanks, I keep forgetting that the I/O die hosts the controller.With laptops, tablets and smart phones being used more than desktops seemingly, WiFi tends to get more traction than ethernet.
It took ages for many ISPs to add gigabit ethernet to their routers which says something about the inertia.
repoman27 - Thursday, May 30, 2019 - link
PCIe 4.0 presents challenges but will obviously see widespread adoption. AMD is there now, and Intel will follow suit as soon as they can figure out how to manufacture some new lake that isn't Skylake. Furthermore, Thunderbolt 3 already exists in the consumer space with a significantly higher signaling rate, albeit at a premium.I have a seriously hard time believing PCIe 5.0 will ever grace a consumer platform. NRZ with a Nyquist rate of 16 GHz and no FEC? I just do not see it happening. Here's a list of NRZ PHYs that operate at or above 8 GT/s for comparison:
PCIe Gen 3 - 8.0 GT/s, 128b/130b encoding
DisplayPort HBR3 - 8.1 GT/s, 8b/10b encoding
USB 3 Gen 2 - 10 GT/s, 128b/132b encoding
Thunderbolt / Thunderbolt 2 / InifiniBand FDR10 - 10.3125 GT/s, 64b/66b encoding
Intel Ultra Path Interconnect (UPI) - 10.4 GT/s
SAS-3 12Gb/s - 12 GT/s, 8b/10b encoding
HDMI 2.1 FRL - 12 GT/s, 16b/18b encoding
AMD Infinity Fabric InterSocket (IFIS) - 12.8 GT/s (@DDR4-3200)
Fibre Channel 16GFC - 14.025 GT/s, 64b/66b encoding
InfiniBand FDR - 14.0625 GT/s, 64b/66b encoding
PCIe Gen 4 - 16 GT/s, 128b/130b encoding
NVLink 1.0 - 20 GT/s
Thunderbolt 3 - 20.625 GT/s, 64b/66b encoding
SAS-4 24G - 22.5 GT/s 128b/150b encoding (128b/130b + 20b RS FEC)
NVLink 2.0 - 25 GT/s
InfiniBand EDR / Intel Omni-Path Architecture (OPA) - 25.78125 GT/s, 64b/66b encoding
Fibre Channel 32GFC - 28.05 GT/s, 256b/257b encoding
Intel Stratix 10 GXE Transceiver - 28.9 GT/s
PCIe Gen 5 - 32 GT/s, 128b/130b encoding
Xilinx UltraScale+ GTY Transceiver - 32.75 GT/s
A lot of these protocols, even DisplayPort and HDMI, have provisions for FEC. Consumers never saw SATA 12Gb/s because it was cheaper to converge with PCIe. 10 GbE was a hit in the datacenter but rarely seen on the desktop. The price of NICs was initially too high, that of switches remains exorbitant, and power requirements all but preclude it from mobile. Yet we're to believe that there will be a consumer interconnect that outstrips everything on the market except for the fastest FPGA transceiver available? What new magic will make this possible?
Gradius2 - Wednesday, May 29, 2019 - link
AFAIK its 128GB/s. NOT 64mode_13h - Wednesday, May 29, 2019 - link
PCIe 5.0 x16 is 64 GB/sec, unidirectional.eastcoast_pete - Wednesday, May 29, 2019 - link
Do I see this correctly? PCIe-5 x16 is faster than any current dual-channel memory, and basically as fast as quad-channel memory? Does that mean PCIe 5 is really fast, current memory bus speeds really suck, or both?mode_13h - Wednesday, May 29, 2019 - link
Since Sandybridge, Intel (and I assume recent AMD) CPUs can snoop L3 cache for PCIe transactions. So, you don't always have to go through RAM.Also, Intel server CPUs have 6-channel memory and are moving to 8, while EPYC already has 8. And DDR5 is on the horizon.
mode_13h - Wednesday, May 29, 2019 - link
Here you go: https://www.intel.com/content/www/us/en/io/data-di...Lord of the Bored - Wednesday, May 29, 2019 - link
Probably both.edzieba - Thursday, May 30, 2019 - link
Bandwidth != latency. For DDR4, "I want that byte!" has single-digit nanoseconds of latency (usually just below 10ns). For PCIe, even with DMA, you're looking at several hundred to over a thousand nanoseconds (or in worst-case contention, maybe several microseconds). If all you're doing is shoving a big block of data across a bus then raw bandwidth is great. If you want to put a memory interface over that bus, then you would very quickly notice the difference between DDR and PCIe.xception - Thursday, May 30, 2019 - link
With the sizes of video cards lately makes you wonder why they aren't extending the slots to x32 as well.Duncan Macdonald - Thursday, May 30, 2019 - link
Power dissipation may be a problem - the faster that transistors switch the more power is needed. As PCIe 5.0 has a data rate around 32Gbits/sec per lane this implies that the chips are going to get hot. (The X570 chipset which uses PCIe 4.0 needs active cooling - this problem will be worse for PCIe 5.0).It is possible that CPUs may need a separate chip in their package to do a mux/demux job to reduce the data rates to the level that the rest of the silicon can handle (transistors built to operate at 32GHz are unlikely to be able to use the same process as the rest of the CPU running at 5GHz or less).
There is also going to need to be some serious microwave engineering on motherboards, add in cards and connectors to deal with the 32GHz signals. (At 32GHz a full wavelength is under 1mm!!) Screening of PC cases may well need to be much improved to stop PCs interfering with other users of the 32GHz frequency range.
brakdoo - Thursday, May 30, 2019 - link
Your numbers are one direction, not duplex...Tigran - Thursday, May 30, 2019 - link
So "actual bandwidth" in graph is duplex (bidirectional), whereas I/O bandwidth is bandwidth for one direction? It's in my question below. In this case you're right, there is a mistake in Anand's table ("Full Duplex").James5mith - Thursday, May 30, 2019 - link
They aren't.unidirectional = 64GB/s x16 PCIe 5.0
Full Duplex = 64GB/s x16 PCIe 5.0
Aggregate (The crap that marketing teams do that adds the bandwidth of both directions of a full duplex link) = 128GB/s x16 PCIe 5.0
1GbE = 1Gbps full duplex. I.e. 1Gbps in/out simultaneously. It seems like nobody thinks about the fact that speeds in PCIe USED to be reported the same way as network gear, I.e. full duplex.
Kudos to the writer of the article reporting it correctly. I.e. 64GB/s FULL DUPLEX.
Tigran - Thursday, May 30, 2019 - link
Quote from PCI-SIG's press release: "PCIe 5.0 Specification Highlights - Delivers 32 GT/s raw bit rate and up to 128(!!!) GB/s via x16 configuration". Is it a misprint?Tigran - Thursday, May 30, 2019 - link
I can see now it's not a misprint - the same in graph. What's the difference between actual and I/O bandwidth?arashi - Thursday, May 30, 2019 - link
GT measures raw data Transfer. Bytes or bits measure effective data transferred.Due to checksums GT/s > Gb/s.
8GT/s ~= 1GB/s
32GT/s * ~1 b per T * 16 lanes * 1/8 b per B * 2 directions = ~128GB/s
ksec - Thursday, May 30, 2019 - link
PCE-E 4.0 is quite a bit more expensive than 3.0, but you make up the saving by having less lane to reach those bandwidth. I am wondering how much more expensive is it for PCI-E 5.0?Now the SSD maker have to figure out a way to make 16GB/s PCI-E 5.0 SSD. :D
HStewart - Thursday, May 30, 2019 - link
This could depend on it implemention of PCIe, Just because PCIe 4.0 is more expensive currently, does not mean that PCIe 5.0 will be by another vendor. It just send really fishy that AMD rushes out 4.0 support at very close to time 5.0 spec is out. There must be a reason why Intel skip 4.0.arashi - Thursday, May 30, 2019 - link
Because 10nm is delayed. Just because Intel hasn't done it (yet) doesn't mean it's problematic.HStewart even with the 2 brain cells you have you are aware that PCIe4 support is on Intel's roadmap? Or do I need to deduct another brain cell from your tally?
ajc9988 - Friday, May 31, 2019 - link
Intel did NOT skip PCIe 4.0. What you are referring to is their server chips using PCIe 5.0 next year. Moreover, what you are ignoring is CONSUMERS WILL NOT GET PCIe 5.0 ANYTIME SOON!In fact, Intel is likely to bring PCIe 4.0 to consumers next year. Then you show your lack of knowledge that implementing it required certain signal integrity parameters in order to implement it. That meant more power on the chipset, that meant higher quality PCBs for motherboards, and for beyond 5-7 inch trace paths, you need signal conditioners/boosters and repeaters to drive beyond that distance. All of this is significant engineering challenges, all of which AMD did with their motherboard partners, whose engineers should be lauded for such achievements to so quickly bring it to mainstream implementations.
Instead, your trying to crap on AMD is also crapping on those motherboard engineers that worked out the implementation on their end. Good job! /s
maisonier - Thursday, May 30, 2019 - link
When are we going to get HBM memory in desktop computers? ... we are still using the crap of DDR and DUAL CHANNEL !!!