Re: ongoing infrastructure changes with AI in the USA

On Mon, 18 Nov 2024 14:12:34 GMT, scott@slp53.sl.home (Scott Lurndal)
wrote:

Lynn McGuire <lynnmcguire5@gmail.com> writes:

On 11/14/2024 9:00 AM, Scott Lurndal wrote:

Lynn McGuire <lynnmcguire5@gmail.com> writes:

I am on the periphery of the ongoing blanketing of the USA with AI
servers.  I have a few facts that might just blow you away.
>
The expected number of AI servers in the USA alone is presently a
million (SWAG).  The current cost for a single AI server is $500,000 US.
    1,000,000 x $500,000 = $500 billion US of capital.

 
First, What is your source for this data?  Be specific.
 
Second, define precisely what an "AI server" is.

>
BTW, I should have mentioned that the AI Servers are not uniform in any
of their aspects.  I just gave the specs for one of the high end
machines that the manufacturer has a three month waiting list for.

>
Some of us actually produce ML hardware.  The term AI is an marketing
gimic, not reality.
>
ML hardware ranges from custom logic in a desktop CPU to massively parallel
specialized hardware (e.g. plug-in GPUs).   An ML-enabled server can
range from a simple ML accelerator block in the CPU itself (Apple,
Google, Amazon, Marvell) to a large, expensive, power-hungry GPU from Nvidia.
>
Yes, there are large racks of ML-enabled servers, particularly for
training.  No, they're not really anything special other than using
high-end, high performance CPUs, GPUs and high-speed interconnects
(Infiniband, 100 and 400Gb ethernet).      Basically no different
than any supercomputer in scale and power requirements.

1 million of them will rather increase the power requirements,
possibly causing shortages.

I would say "perhaps Congress should pass a law allowing local power
systems to refuse to connect to these things" but, judging from the
last two years, Congress will be doing well to be sufficiently
organized by Jan 6 to do its duty to the country, never mind passing
any laws any time soon. Well, except a massive Tax Cut for 1%-ers, of
course.
--
"Here lies the Tuscan poet Aretino,
Who evil spoke of everyone but God,
Giving as his excuse, 'I never knew him.'"