Eight years may be an age (or two) in the computer hardware industry, but Nvidia is nevertheless presenting a product that will be available in 2017.
Inpai have leaked a slide presented by Bill Dally, chief scientist, Nvidia that talks of a GPU + CPU combination, from Nvidia.
The slide talks of a 2017 GPU Node that features 2400 "throughput cores" and 16 CPUs on a single chip. The TDP stays at 300W, which is the maximum TDP for commercial GPUs today. This combination boats of 40 TFlops Single Precision and 13 TFlops. This seems in stark contrast to Nvidia's earlier claims of a 570x increase in performance by 2015. We are looking at more like a 16x increase according to the slide, which is right on with the Moore's law
Some more features - 128GB DRAM at a cool 2TB/s bandwidth, 512GB of phase-change memory.
Nvidia talks of Cabinets and Systems, clear indication that this system is aimed at HPC. A "system" will contain 128 cabinets of 384 nodes, or 49152 GPU Nodes. The performance? 2 Exaflops. That is 2 million TFlops, or 2000 PFlops. That is an incredible 1400 times more powerful than the current fastest supercomputer in the world - IBM Roadrunner. In fact, even a Cabinet offers 10x the performance of Roadrunner, at 4% the power consumption.
It is not clear what the CPU part of the 2017 Node is all about - or if it is even x86, or a native Nvidia design. Regardless, Nvidia seems to be going the Fusion way as well. We will have to wait and observe the ExaScale Machine develop over the next 8 years.
