Wow, if you'd told me Titan would be usurped in the next top500 list I would not have been terribly surprised, technology advances. If you'd told me China would take it's place with a system nearly twice as powerful, built on Xeon Phi cards, I would have laughed. I have to wonder just how much this cost, IIRC Titan cost hundreds of millions just to upgrade an underlying structure, this is all new and built on non-commercially available parts.
It's not a trivial engineering feat to maintain almost the same level of efficiency while almost doubling performance. Pretty impressive. Gotta say, Xeon Phi looks pretty promising.
Meh. The difference between a lot of these systems is just number of cabinets, essentially. Doubling performance with the same number of chips, yeah, probably more power. But when you're just talking about a whole mess of cabinets tied together with QDR Infiniband, meh. Doubling performance means double power, same efficiency.
The inter-cabinet interconnects are the secret to keeping up the scaling of the machine in terms of total number of nodes and total throughput. There are many solutions to interconnects although the driver code to tie them together are really the most difficult thing to do. As far as Xeon Phi chips, since the West were lackluster in its adoption, the Chinese are willing to sweep them up to built this beast. And there will be a few more similar ones built for Universities in China as well. it shows their technological progress to date and how come the Japanese are not competing in this space ?. Fujitsu is still a supercomputer builder. They probably contributed to the design of this machine and remain low keyed just to be safe ... Well done China, you have proven one point here.
It does, but compared to the two twelve core Xeons and 3 Phis, the interconnect power consumption is going to be miniscule.
The BlueGene/Q systems are still my favorite. They've actually hit 14 PFlops in a benchmark run of a real application (not just Linpack,) and have actually used a million cores at once. The Sequoia system has as much memory as all of Tianhe-2. Until we get Graph500 results for Titan or Tianhe-2, I'm not going to be impressed. The entire architecture of BlueGene - the chips, the boards, the network topology, etc., are all designed to be scalable. Titan, Tianhe-2, etc., just hook everything up to a big router. Give me a torus over a fat tree any day.
I wonder how much of the performance improvement is due to the Xeon CPUs vs. the AMD CPUs in the Titan. I've heard AMD does better in terms of floating-point performance per dollar, and perhaps even FP performance per watt, but I would imagine Intel could do better in terms of absolute FP performance. In other words, perhaps the increase in performance is not really due using Xeon Phi vs. NVIDIA Tesla, and maybe the decreased efficiency is due (at least partially) to Xeon CPUs? In any case, I don't think Cray is allowed to sell their computers to China.
Here's a nice counter-point: In 2004 I helped build one of the bigger TOP500 computers. We saw about 20 MFLOPS/watt per node, 16 MFLOPS/watt for the entire system (not counting power used for facility cooling).
Ryan, it might be worth mentioning that the Tianhe-2 represents the first introduction of multi-socket Xeon Ivy Bridge Xeon E5-2692.Xeon Ivy Bridge E5s also powered the 54th and 330th fastest supercomputers on this list. No Xeon Ivy Bridge E5 models are listed on the latest Intel Price List.
Tianhe-2 is the first ever top 10 supercomputer that consists entirely of processors fabricated at 22 nm.
To my surprise there was no Oracle Sparc T5 supercomputer in the 500 list.
Some other interesting observations: 1) only 26 petaflop supercomputers on this list, up from 23 petaflop supercomputers in the last list. (would have thought that more than 3 petaflop supercomputers were purchased in 7 months) 2) the 500th most powerful supercomputer only had 96.6 teraflops, up a measly 26% from 76.5 teraflops in the last list. This is the second straight list with modest growth in teraflops for the 500th most powerful system. What far will this trend go? 3) The sum of computational power for all 500 supercomputer systems was 223 petaflops, up 38% from 162 petaflops in the November, 2012. (the November 2012 list showed in increase of 31% from 123 petaflops in the June, 2012 list) 4) By performance, the most powerful supercomputer had 15.2% of the combined petaflops of all 500 supercomputers. This is almost certainly a record high percentage since the 500 list was first introduced in 1993.
We’ve updated our terms. By continuing to use the site and/or by logging into your account, you agree to the Site’s updated Terms of Use and Privacy Policy.
Facebook
Twitter
RSS
16 Comments
Back to Article
yougotkicked - Monday, June 17, 2013 - link
Wow, if you'd told me Titan would be usurped in the next top500 list I would not have been terribly surprised, technology advances. If you'd told me China would take it's place with a system nearly twice as powerful, built on Xeon Phi cards, I would have laughed. I have to wonder just how much this cost, IIRC Titan cost hundreds of millions just to upgrade an underlying structure, this is all new and built on non-commercially available parts.testbug00 - Monday, June 17, 2013 - link
How much did it cost China? Probably not to much.How much did it cost Intel? Probably quite a bit.
karasaj - Monday, June 17, 2013 - link
I'm not sure why it would cost Intel anything... I'm pretty sure they're lining their pockets right now :)chizow - Monday, June 17, 2013 - link
Titan cost about $60M originally and about $100M to upgrade based on the various news articles you'll find quoting the ORNL director.Looks like the Chinese used brute force with Tianhe-2 though, less than 2x the throughput at over 2x the power consumption.
hammer256 - Monday, June 17, 2013 - link
It's not a trivial engineering feat to maintain almost the same level of efficiency while almost doubling performance. Pretty impressive.Gotta say, Xeon Phi looks pretty promising.
testbug00 - Monday, June 17, 2013 - link
Having talked with someone who does HPC and has worked with GK110 silicon + other video cards i can say it is not :/It has some impressive parts, but to many issues (in this generation) i think in 3-4 generations it will be promising.
hammer256 - Monday, June 17, 2013 - link
Oh? If possible, can you elaborate on the issues with Phi? I'm super curious. Thanks :)marimvibe - Monday, June 17, 2013 - link
Meh. The difference between a lot of these systems is just number of cabinets, essentially. Doubling performance with the same number of chips, yeah, probably more power. But when you're just talking about a whole mess of cabinets tied together with QDR Infiniband, meh. Doubling performance means double power, same efficiency.TeXWiller - Monday, June 17, 2013 - link
This system does have a custom interconnect, though.fteoath64 - Tuesday, June 18, 2013 - link
The inter-cabinet interconnects are the secret to keeping up the scaling of the machine in terms of total number of nodes and total throughput. There are many solutions to interconnects although the driver code to tie them together are really the most difficult thing to do. As far as Xeon Phi chips, since the West were lackluster in its adoption, the Chinese are willing to sweep them up to built this beast. And there will be a few more similar ones built for Universities in China as well. it shows their technological progress to date and how come the Japanese are not competing in this space ?. Fujitsu is still a supercomputer builder. They probably contributed to the design of this machine and remain low keyed just to be safe ... Well done China, you have proven one point here.marimvibe - Tuesday, June 18, 2013 - link
It does, but compared to the two twelve core Xeons and 3 Phis, the interconnect power consumption is going to be miniscule.The BlueGene/Q systems are still my favorite. They've actually hit 14 PFlops in a benchmark run of a real application (not just Linpack,) and have actually used a million cores at once. The Sequoia system has as much memory as all of Tianhe-2. Until we get Graph500 results for Titan or Tianhe-2, I'm not going to be impressed. The entire architecture of BlueGene - the chips, the boards, the network topology, etc., are all designed to be scalable. Titan, Tianhe-2, etc., just hook everything up to a big router. Give me a torus over a fat tree any day.
Ktracho - Monday, June 17, 2013 - link
I wonder how much of the performance improvement is due to the Xeon CPUs vs. the AMD CPUs in the Titan. I've heard AMD does better in terms of floating-point performance per dollar, and perhaps even FP performance per watt, but I would imagine Intel could do better in terms of absolute FP performance. In other words, perhaps the increase in performance is not really due using Xeon Phi vs. NVIDIA Tesla, and maybe the decreased efficiency is due (at least partially) to Xeon CPUs? In any case, I don't think Cray is allowed to sell their computers to China.protomech - Monday, June 17, 2013 - link
Here's a nice counter-point: In 2004 I helped build one of the bigger TOP500 computers. We saw about 20 MFLOPS/watt per node, 16 MFLOPS/watt for the entire system (not counting power used for facility cooling).hammer256 - Monday, June 17, 2013 - link
So 2 orders of magnitude improvement in efficiency in 9 years. That's pretty sweet.marimvibe - Monday, June 17, 2013 - link
Graph500 or GTFO.1008anan - Tuesday, June 18, 2013 - link
Ryan, it might be worth mentioning that the Tianhe-2 represents the first introduction of multi-socket Xeon Ivy Bridge Xeon E5-2692.Xeon Ivy Bridge E5s also powered the 54th and 330th fastest supercomputers on this list. No Xeon Ivy Bridge E5 models are listed on the latest Intel Price List.Tianhe-2 is the first ever top 10 supercomputer that consists entirely of processors fabricated at 22 nm.
To my surprise there was no Oracle Sparc T5 supercomputer in the 500 list.
Some other interesting observations:
1) only 26 petaflop supercomputers on this list, up from 23 petaflop supercomputers in the last list. (would have thought that more than 3 petaflop supercomputers were purchased in 7 months)
2) the 500th most powerful supercomputer only had 96.6 teraflops, up a measly 26% from 76.5 teraflops in the last list. This is the second straight list with modest growth in teraflops for the 500th most powerful system. What far will this trend go?
3) The sum of computational power for all 500 supercomputer systems was 223 petaflops, up 38% from 162 petaflops in the November, 2012. (the November 2012 list showed in increase of 31% from 123 petaflops in the June, 2012 list)
4) By performance, the most powerful supercomputer had 15.2% of the combined petaflops of all 500 supercomputers. This is almost certainly a record high percentage since the 500 list was first introduced in 1993.