It would be interesting to see an AM4 board that was outfitted with a PAX 52xG4 and an X570 chipset. IT would be interesting to see how well a 3950 could handle a multi-GPU professional workload with the x16 link connected to the switch and 4 GPUs attached via 4 X8 links from the PAX. You'd still have good IO numbers through the X570, and the CPU connected x4 link for a boot and scratch drive.
I dont think the small cost savings (and performance penalty) over HEDT would be worth it.
More extreme scenarios are interesting. One could, for example, hang a boatload of GPUs off a single EPYC or Ice Lake board, which is way cheaper (and possibly faster?) than wiring a bunch of 4-8 GPU nodes together.
The current NVMe 4.0 switch cards are like $400 just for the card, a little more than 3.0 switch cards.
Buying an X299 HEDT platform gives you 50% more lanes and bandwidth, for only ~$400 over a consumer platform. TR4 starts a little higher at ~$800 but gives you 100% more lanes and 200% more bandwidth. HEDT also has the memory bandwidth to actually saturate all the links at once.
Yeah, maybe it's a niche of a niche, but currently it's kinda weird trying to build a small-ish storage server: Threadripper 3 would be neat with it's 64 PCIe lanes and ECC support, but $1400 for the "lowest-end" 3960X is a tough sale. Ryzen has ECC support (though not validated by AMD), but only 24 PCIe lanes. Threadripper 1 is great since it's dirt cheap now, but you're dealing with a dead-end platform. EPYC seems to be the go-to-choice, but even that is expensive.
Of course, an NVMe card with a PCIe Switch on it is the probable solution here, but having a "storage server" board/cpu combination for hundreds instead of thousands of dollars would be nice.
What's the need for so many lanes in a storage server?
An 8x slot is plenty for basic storage.
SAS card + expander and away you go.if you are talking SSD then you want more actual proper lanes and not a switch ideally so Threadripper/Epyc would be the requirement.
Look into the Asus Pro WS X570-Ace. It has the two CPU connected slots (x16/0, x8/x8) in addition to a unique third x8 chipset connected slot. Install a 16, 24 or 32-port SATA HBA (eg, Adaptec ASR-72405 or an LSI 8/16 port + Intel Expander) and you're off to the races. This still leaves you with available onboard NVMe M.2 slots and PCH sourced SATA ports, too, in addition to the two primary PCI Express slots. You could just as easily drop in another HBA and run the GPU at x8, a minimal performance hit at this time.
I've got my system set up to do a bit of everything, but part of that is VMs and file/Plex server duty. I have an 3950X + X570 Taichi which is populated with a GPU, LSI 9211-8i SAS HBA, discrete audio card, three NVMe M.2 drives and 12 SATA drives - four SSDs configured in RAID from the motherboard's ports, and the remaining eight are HDDs attached to the HBA. Works great.
I was just thinking about this the other day. AMD's approach isn't the best when it comes to chipsets. IMO costumers would be better served if they pushed 64 lanes (in a future socket) directly to a switch and allowed motherboard manufacturers to decide how to delegate those 64 lanes. Switches add a small amount of latency, but the tradeoff is worth it IMO. An example would be providing a PCIE 3.0 x16 slot from 8 PCIE 4.0 lanes. Another is a motherboard that is more storage oriented, with quadruple PCIE 4.0 m.2 slots.
I sincerely hope you're talking about a HEDT or server platform. Otherwise, you would be increasing base motherboard costs by at least a hundred dollars, though more likely 2-3x that. Traces for 64 PCIe lanes will require more pcb layers, the switch would be expensive, etc.
The consumer market for PCIe switches is dead and these Micron Switches are not for consumers (making redundant the lengthy introduction to this article).
Multi-GPU systems for consumers are a thing of the past and very few need more than 1 NVME drive.
PCIe 4.0 switches currently would only make sense for a AMD B550 system wanting to have more than 1 PCIe 4.0 M.2 slot and for that niche using the X570 chipset makes more sense. Future platforms won't even allow for this marginal case as all PCIe lanes will be upgraded to 4.0.
I very much want more than 1 nvme drive to be standard on all mobos. SSD SATA is a dying technology, and even cheap SSDs max it out (for bulk transfer at least, still a way to go on random).
I've run the same daily use computer on fast SATA SSD and medium speed nvme (2GB/sec max) and there was a significant performance uplift when moving to nvme. Everything 'felt' far faster, not just app opening.
Also, nvme is easier to install for newbies - slot it in, do up one screw, and that's it - compared to SATA - fiddle around with up to 4 screws, and 2 different cables per SSD, and one of these cables often goes to other drives as well, or might not have any spare connectors available.
There is no "significant performance uplift" moving from decent SATA SSD to nvme.
You are talking complete rubbish.
Stop lying.
There is a significant performance uplift moving from HDD to any SSD. Going from Sara to nvme there is only small benefits which you wouldn't notice for daily use.
High end use. Sure if you are moving large amounts of data around or need high speed reads.
You've never seen new cases have you? I don't use screws for HDD/SSD, haven't use one for years. Also there's no discernable difference between SATA and NVMe for daily use. Unless all you do is copy huge files.
It’s a thing that is possible and likely in the future. AMD’s multi-die approach has large implications for GPUs, which have the largest monolithic consumer dies in history. Larger dies have much lower yield and cost much more. If you split your GPU across dies then you might also look at splitting it across cards too. On paper there is no reason SIMD can’t perfectly scale across cards. Hell, it does in HPC.
Mistake in this line: "Those interested in Microchip are advised to content their local representative." I think it was supposed to be: "Those interested in Microchip are advised to contact their local representative."
Having grown up with Lego, these things hit home in a territory where fact checking remains de-activated: I immediately imagine a switched backplane, that could be filled with the type of AMD/Subor FireFlight SoCs in 1-8 increments to create a stackable performance console.
Thinking about the BIOS initialization code and teaching the OS and game engine schedulers how to operate the topology turns it into more of a nightmare.
And then I can't help to think that I'd really prefer these being Infinity Fabric switch chips, not just PCIe 4: Not that I really know, but I'd imagine that remote memory access over PCIe could cost quite a bit of latency vs. IF configured for memory (when dealing with memory) and PCIe (when dealing with I/O).
But with 12 or even 16 cores in an AM4 socket at current prices, stacking has prettey much happened for any CPU workload I might personally require, while GPUs don't take kindly even to the memory latency of HBM or GDDR6. PCIe? Might as well be using punch cards!
With the adult slowly creaking back in, I can't help but think that these are rather late, even in the worlds of all-flash storage arrays and 400Gbit/s Ethernet.
Somehow, I still think there used to be times when a PLX or IDT switch on a standard ATX mainboard was just €50 extra for "Pro"... Could be some misfiring neurons, but I sure never appreciated how Avago killed a lot of consumer accessible innovation with these crazy price hikes.
We’ve updated our terms. By continuing to use the site and/or by logging into your account, you agree to the Site’s updated Terms of Use and Privacy Policy.
Facebook
Twitter
RSS
36 Comments
Back to Article
jeremyshaw - Monday, June 1, 2020 - link
So these have been on their site for a while now (maybe not spec sheets, but most of the other material, iirc). Is there any change? Availability?Ian Cutress - Monday, June 1, 2020 - link
It looks like they'll now be available. Microchip put up the press release last week.lightningz71 - Monday, June 1, 2020 - link
It would be interesting to see an AM4 board that was outfitted with a PAX 52xG4 and an X570 chipset. IT would be interesting to see how well a 3950 could handle a multi-GPU professional workload with the x16 link connected to the switch and 4 GPUs attached via 4 X8 links from the PAX. You'd still have good IO numbers through the X570, and the CPU connected x4 link for a boot and scratch drive.brucethemoose - Monday, June 1, 2020 - link
I dont think the small cost savings (and performance penalty) over HEDT would be worth it.More extreme scenarios are interesting. One could, for example, hang a boatload of GPUs off a single EPYC or Ice Lake board, which is way cheaper (and possibly faster?) than wiring a bunch of 4-8 GPU nodes together.
Jorgp2 - Tuesday, June 2, 2020 - link
HEDT is cheaper than using switches.The current NVMe 4.0 switch cards are like $400 just for the card, a little more than 3.0 switch cards.
Buying an X299 HEDT platform gives you 50% more lanes and bandwidth, for only ~$400 over a consumer platform. TR4 starts a little higher at ~$800 but gives you 100% more lanes and 200% more bandwidth.
HEDT also has the memory bandwidth to actually saturate all the links at once.
MenhirMike - Monday, June 1, 2020 - link
Yeah, maybe it's a niche of a niche, but currently it's kinda weird trying to build a small-ish storage server: Threadripper 3 would be neat with it's 64 PCIe lanes and ECC support, but $1400 for the "lowest-end" 3960X is a tough sale. Ryzen has ECC support (though not validated by AMD), but only 24 PCIe lanes. Threadripper 1 is great since it's dirt cheap now, but you're dealing with a dead-end platform. EPYC seems to be the go-to-choice, but even that is expensive.Of course, an NVMe card with a PCIe Switch on it is the probable solution here, but having a "storage server" board/cpu combination for hundreds instead of thousands of dollars would be nice.
supdawgwtfd - Monday, June 1, 2020 - link
What's the need for so many lanes in a storage server?An 8x slot is plenty for basic storage.
SAS card + expander and away you go.if you are talking SSD then you want more actual proper lanes and not a switch ideally so Threadripper/Epyc would be the requirement.
Jorgp2 - Tuesday, June 2, 2020 - link
Or the need for high performance CPUs.Atom C3000, Xeon D, or Ryzen embedded have enough performance and IO while using much less power.
Slash3 - Monday, June 1, 2020 - link
Look into the Asus Pro WS X570-Ace. It has the two CPU connected slots (x16/0, x8/x8) in addition to a unique third x8 chipset connected slot. Install a 16, 24 or 32-port SATA HBA (eg, Adaptec ASR-72405 or an LSI 8/16 port + Intel Expander) and you're off to the races. This still leaves you with available onboard NVMe M.2 slots and PCH sourced SATA ports, too, in addition to the two primary PCI Express slots. You could just as easily drop in another HBA and run the GPU at x8, a minimal performance hit at this time.I've got my system set up to do a bit of everything, but part of that is VMs and file/Plex server duty. I have an 3950X + X570 Taichi which is populated with a GPU, LSI 9211-8i SAS HBA, discrete audio card, three NVMe M.2 drives and 12 SATA drives - four SSDs configured in RAID from the motherboard's ports, and the remaining eight are HDDs attached to the HBA. Works great.
Jorgp2 - Tuesday, June 2, 2020 - link
According to AMD fanboys, every AM4 motherboard already has one of these. To magically convert the 16 lanes from the CPU into 32.As for your suggestion, I think using 4x4 bifurcation you could get most of the performance without the cost of a switch.
Spunjji - Wednesday, June 3, 2020 - link
Weird how certain posters have more to say about what AMD fanboys are saying than any AMD fanboys can be seen to say for themselves.DigitalFreak - Wednesday, June 3, 2020 - link
What are you babbling on about?ballsystemlord - Monday, June 1, 2020 - link
Now if only some Ryzen MBs had these.supdawgwtfd - Monday, June 1, 2020 - link
For what purpose?Slash3 - Monday, June 1, 2020 - link
Acting as a PCI Express 4.0 switch, I'd venture to guess.dotjaz - Tuesday, June 2, 2020 - link
Again, for what purpose? I assume you want more than 3 x16 connectors.bananaforscale - Tuesday, June 2, 2020 - link
3 x16 connectors isn't 3 *electrically* x16 connectors in Ryzen. Multiplexed x16 is still faster than unmultiplexed anything less.Jorgp2 - Tuesday, June 2, 2020 - link
Wasting moneyYB1064 - Monday, June 1, 2020 - link
This is a winner in PXIe mainframes.eek2121 - Monday, June 1, 2020 - link
I was just thinking about this the other day. AMD's approach isn't the best when it comes to chipsets. IMO costumers would be better served if they pushed 64 lanes (in a future socket) directly to a switch and allowed motherboard manufacturers to decide how to delegate those 64 lanes. Switches add a small amount of latency, but the tradeoff is worth it IMO. An example would be providing a PCIE 3.0 x16 slot from 8 PCIE 4.0 lanes. Another is a motherboard that is more storage oriented, with quadruple PCIE 4.0 m.2 slots.blaktron - Monday, June 1, 2020 - link
And run your GPU through your chipset? Why? Its a more complex architecture with only downsides for like 99% of users.sftech - Monday, June 1, 2020 - link
My R9 295x2 shipped with a PAX switch and was quick. That said a PCIe switch is a high-end option, not a low cost venturebananaforscale - Tuesday, June 2, 2020 - link
The 99% already have options. Nobody's saying all boards should have a PCIe switch.Pewzor - Monday, June 1, 2020 - link
You don't want to run pcie4 thru chipsets.What does Intel have to offer currently anyways.
dotjaz - Tuesday, June 2, 2020 - link
That's not what switches do, it's a SWITCH, not a converter. PCIE 4.0 x8 would be provided as is.eek2121 - Tuesday, June 2, 2020 - link
I don’t think you understand what a switch is.Valantar - Tuesday, June 2, 2020 - link
I sincerely hope you're talking about a HEDT or server platform. Otherwise, you would be increasing base motherboard costs by at least a hundred dollars, though more likely 2-3x that. Traces for 64 PCIe lanes will require more pcb layers, the switch would be expensive, etc.Arsenica - Monday, June 1, 2020 - link
The consumer market for PCIe switches is dead and these Micron Switches are not for consumers (making redundant the lengthy introduction to this article).Multi-GPU systems for consumers are a thing of the past and very few need more than 1 NVME drive.
PCIe 4.0 switches currently would only make sense for a AMD B550 system wanting to have more than 1 PCIe 4.0 M.2 slot and for that niche using the X570 chipset makes more sense. Future platforms won't even allow for this marginal case as all PCIe lanes will be upgraded to 4.0.
People needing more than
Tomatotech - Monday, June 1, 2020 - link
I very much want more than 1 nvme drive to be standard on all mobos. SSD SATA is a dying technology, and even cheap SSDs max it out (for bulk transfer at least, still a way to go on random).I've run the same daily use computer on fast SATA SSD and medium speed nvme (2GB/sec max) and there was a significant performance uplift when moving to nvme. Everything 'felt' far faster, not just app opening.
Also, nvme is easier to install for newbies - slot it in, do up one screw, and that's it - compared to SATA - fiddle around with up to 4 screws, and 2 different cables per SSD, and one of these cables often goes to other drives as well, or might not have any spare connectors available.
supdawgwtfd - Monday, June 1, 2020 - link
I call bullshit.There is no "significant performance uplift" moving from decent SATA SSD to nvme.
You are talking complete rubbish.
Stop lying.
There is a significant performance uplift moving from HDD to any SSD. Going from Sara to nvme there is only small benefits which you wouldn't notice for daily use.
High end use. Sure if you are moving large amounts of data around or need high speed reads.
Daily usage is neither of these.
brucethemoose - Monday, June 1, 2020 - link
I not so sure about that right now.Either way, PS5/XBX ports will absolutely change that in the near future.
dotjaz - Tuesday, June 2, 2020 - link
You've never seen new cases have you? I don't use screws for HDD/SSD, haven't use one for years. Also there's no discernable difference between SATA and NVMe for daily use. Unless all you do is copy huge files.willis936 - Monday, June 1, 2020 - link
It’s a thing of the past... for now.It’s a thing that is possible and likely in the future. AMD’s multi-die approach has large implications for GPUs, which have the largest monolithic consumer dies in history. Larger dies have much lower yield and cost much more. If you split your GPU across dies then you might also look at splitting it across cards too. On paper there is no reason SIMD can’t perfectly scale across cards. Hell, it does in HPC.
sftech - Monday, June 1, 2020 - link
Now I can design a 16x pcie4 to #12 m.2 slot board, yee-hawdrzzz - Tuesday, June 2, 2020 - link
Mistake in this line:"Those interested in Microchip are advised to content their local representative."
I think it was supposed to be:
"Those interested in Microchip are advised to contact their local representative."
abufrejoval - Tuesday, June 2, 2020 - link
Having grown up with Lego, these things hit home in a territory where fact checking remains de-activated: I immediately imagine a switched backplane, that could be filled with the type of AMD/Subor FireFlight SoCs in 1-8 increments to create a stackable performance console.Thinking about the BIOS initialization code and teaching the OS and game engine schedulers how to operate the topology turns it into more of a nightmare.
And then I can't help to think that I'd really prefer these being Infinity Fabric switch chips, not just PCIe 4: Not that I really know, but I'd imagine that remote memory access over PCIe could cost quite a bit of latency vs. IF configured for memory (when dealing with memory) and PCIe (when dealing with I/O).
But with 12 or even 16 cores in an AM4 socket at current prices, stacking has prettey much happened for any CPU workload I might personally require, while GPUs don't take kindly even to the memory latency of HBM or GDDR6. PCIe? Might as well be using punch cards!
With the adult slowly creaking back in, I can't help but think that these are rather late, even in the worlds of all-flash storage arrays and 400Gbit/s Ethernet.
Somehow, I still think there used to be times when a PLX or IDT switch on a standard ATX mainboard was just €50 extra for "Pro"... Could be some misfiring neurons, but I sure never appreciated how Avago killed a lot of consumer accessible innovation with these crazy price hikes.