Re: Is Intel exceptionally unsuccessful as an architecture designer?

On 18/09/2024 20:48, MitchAlsup1 wrote:

On Wed, 18 Sep 2024 17:01:48 +0000, David Brown wrote:
 

On 18/09/2024 18:00, Michael S wrote:

On Wed, 18 Sep 2024 15:50:09 GMT
>
It's not easy to get 256GB via HBM.
To give one example, Fujitsu A64Fx got only 32GB.
It was 5 years ago and some progress was made since then, but density
improvements nowadays are sloooooooow.
>

>
<https://www.servethehome.com/micron-hbm3e-12-high-36gb-higher-capacity-ai-accelerators-shipping/>
<https://www.servethehome.com/a-quick-introduction-to-the-nvidia-gh200-aka-grace-hopper-arm/>
>
It's certainly not /cheap/ to have 256GB (or more) with HBM, but it is
not unrealistic.

 Consider the cost of the power it takes to feed a rack that consumes
100KW
continuously for a year, and don't forget to add in the cooling costs to
remove that 100KW from that rack while computing the cost of the power.
Using $0.15 /KWh = $170,000 per year per rack (including cooling).
 The cost of 256GB of memory fades into insignificance.

Sure - when other costs are big enough, some costs are insignificant.
But to fill your rack with 100 KW worth of equipment, you will have a lot of processors or accelerators that have 256 GB of HBM - it's not just a single device taking 100 KW !