Re: ongoing infrastructure changes with AI in the USA

On Fri, 15 Nov 2024, Lynn McGuire wrote:

On 11/15/2024 3:23 AM, D wrote:

  On Thu, 14 Nov 2024, Lynn McGuire wrote:
 

On 11/14/2024 2:18 PM, D wrote:

  On Thu, 14 Nov 2024, Scott Lurndal wrote:
 

Lynn McGuire <lynnmcguire5@gmail.com> writes:

I am on the periphery of the ongoing blanketing of the USA with AI
servers.  I have a few facts that might just blow you away.
 The expected number of AI servers in the USA alone is presently a
million (SWAG).  The current cost for a single AI server is $500,000 US.
   1,000,000 x $500,000 = $500 billion US of capital.

 First, What is your source for this data?  Be specific.
 Second, define precisely what an "AI server" is.

 My guess would be a server stuffed with GPU:s.

 1. A huge rackmount frame the size of a refrigerator
2. Two Intel Zeons with 64 or 128 cpus each
3. Many terabytes of ram in 20 to 40 memory slots
4. 10 to 40 SSD hard drives of 1 TB to 8 TB
5. four to twelve Nvidea H100 or H200 boards
6. multiple power supplies
7. etc, etc, etc, etc, etc, etc
 In other words, it looks like a 1980s / 1990s mini computer in a rack configuration with many swapable / upgradable modules.
 Here is just one variant:
https://www.dell.com/en-us/dt/servers/specialty-servers/poweredge-xe- servers.htm#scroll=off&tab0=0&accordion0
 Lynn

 Just what I would have expected! It will be fun if there's another AI- winter. Then we'll be eating GPU:s for breakfast given all the oversupply that will exist! When do you think disillusionment will set in (if at all)?

>
The AI disillusionment will start when the companies try to layoff their customer support departments and find out that they cannot do so.
>
Lynn

Let us study the example of Klarna with great interest! I think they did exactly that. Instead of having people answering questions about loans and credit, they will let the LLM:s do it and I then assume have a vastly smaller teams of humans catching the hallucinations and edge cases.