Re: Windows-on-ARM Laptop Is A “Frequently-Returned Item” On Amazon

rbowman <bowman@montana.com> wrote:

On Mon, 24 Mar 2025 17:48:25 -0000 (UTC), Chris wrote:
 

Not quite sure what exactly you mean by "inference", the latest Mac
Studio that can go up to 512GB RAM is certainly heading in the right
direction for local LLM training.

 
Inference is when new data is fed into an existing model to reach
conclusions.

Yes, I know. Paul seems to have a different definition, however, as
inference does not need a 4MW supercomputer as he seems to be implying. I
have Ollama running completely locally on mac laptop with 32GB RAM.

This is a decent overview of creating and training a model.
 
https://www.nitorinfotech.com/blog/training-large-language-models-llms-
techniques-and-best-practices/
 
I haven't looked at this series but there a many PyTorch or TensorFlow
tutorials online.
 
https://pytorch.org/tutorials/beginner/basics/intro.html
 
The original MNIST database was 60,000 pre-scaled images of hand written
digits. Fashion MNIST has 60,000 grayscale  clothing images and 10,000
test images. This isn't NLP and the datasets are laughingly small. If
you're lucky you have a Nvidia GPU. So far most packages use CUDA which is
a Nvidia thing. Some can be set to use the CPU. In either case you might
want to go for a cup of coffee while the epochs spool by minimizing the
loss function.

Yes, CUDA is the dominant interface, but not the only game in town. There
are other NPUs that can give nVidia a run for its money.

You do realize the players in the field are using banks of $40,000 Nvidia
GPUs and are trying to buy nuclear plants to power them? If you want to
talk about ChatGPT here are some numbers.

Sure, but there's a whole spectrum of needs for deep learning methods that
are far more modest and still very useful.

Machine learning has been around since the 1960s and has had real world
uses for a lot of that time.

https://www.moomoo.com/community/feed/109834449715205
https://www.forbes.com/sites/katharinabuchholz/2024/08/23/the-extreme-
cost-of-training-ai-models/
 
Not coming to a desktop near you any time soon.

I created my first model in 2006/7 with no need for a GPU.