Re: Short Vectors Versus Long Vectors

John Savard wrote:

On Tue, 23 Apr 2024 02:14:32 +0000, mitchalsup@aol.com (MitchAlsup1)
wrote:

CRAY machines stayed "in style" as long as memory latency remained smaller
than the length of a vector (64 cycles) and fell out of favor when the cores
got fast enough that memory could no longer keep up.
>
I whish them well, but I expect it will not work out as they desire.....

I know that you've said this about Cray-style vectors.

I had thought the cause was much simpler. As soon as chiips like the
486 DX and then the Pentium II became available, a Cray-style machine
would have had to be implemented from smaller-scale integrated
circuits, so it would have been wildly uneconomic for the performance
it provided; it made much more sense to use off-the-shelf
microprocessors. Despite their shortcomings theoretically in
architectural terms compared to a Cray-style machine, they offered
vastly more FLOPS for the dollar.

CRAY-XMP was done in MECL 10K gate arrays, offering 10K gates per chip.

After all, the reason the Cray I succeeded where the STAR-100 failed
was that it had those big vector registers - so it did calculations on
a register-to-register basis, rather than on a memory-to-memory basis.

The CRAY-a had much shorter setup sequences than the STAR.
Amdahl's law strikes again.

That doesn't make it immune to considerations of memory bandwidth, but
that does mean that it was designed correctly for the circumstance
where memory bandwidth is an issue. So if you have the kind of
calculation to perform that is suited to a vector machine, wouldn't it
still be better to use a vector machine than a whole bunch of scalar
cores with no provision for vectors?

Let us face facts:: en the large; vector machines are DMA devices
that happen to mangle the data on thee way through.

And if memory bandwidth issues make Cray-style vector machines
impractical, then wouldn't it be even worse for GPUs?

a) It is not pure BW but BW at a latency less than K. CRAY-1 was about 16-cycles (DRAM) CRAY-1S was about 10 cycles (SRAM), XMP
was about 22 cycles, and YMP was about 32 cycles. CRAY-1 and -1S
had 1 port to memory, XMP and YMP had 2Rd and 1W to memory.
b) GPUs use threading to absorb the latency to memory (roughly 400
cycles), along with HW rasterizer, interpolator, texture access,
and an HW OS that can clean up a thread and launch a new thread in
about 8 cycles. That is: GPUs absorb latency by waiting in a way
that does not prevent others from making forward progress.

There are ways to increase memory bandwidth. Use HBM. Use static RAM.
Use graphics DRAM. The vector CPU of the last gasp of the Cray-style
architecture, the NEC SX-Aurora TSUBASA, is even packaged like a GPU.

Even HBM has a latency of standard DRAM (with smaller command cycle
overheads) so, a 5-GHz core using 20ns DRAM with infinite BW between
core and DRAM will still have the core see 100 cycles of latency.
Bandwidth alone does not solve latency bound problems, latency alone
does not solve BW bound problems.

Also, the original Cray I did useful work with a memory no larger than
many L3 caches these days. So a vector machine today wouldn't be as
fast as it would be if it could have, say, a 1024-bit wide data bus to
a terabyte of DRAM. That doesn't necessarily mean that such a CPU,
even when throttled by memory bandwidth, isn't an improvement over an
ordinary CPU.

The 128K DW memory was used for number crunching, but CRAY-1 had an
I/O system that could consume as much BW as a core, so one could
write out the last chunk and read in the next chunk while the current chunk was processing. And it was this I/O system that made
a CRAY-1 faster than its equivalent NEC machine (excepting on certain
benchmarks).

Of course, though, the question is, is it an improvement enough? If
most problems anyone would want to use a vector CPU for today do
involve a large amount of memory, used in a random fashion, so as to
fit poorly in cache, then it might well be that memory bandwidth would
mean that even with a vector architecture well suited to doing a lot
of work, the net result would be only a slight improvement over what
an ordinary CPU could do with the same memory bandwidth.

In essence, if you can teach the compiler to block the numeric algorithm
to fit through (through not in) the cache(s) you can use vector style
CPU architecture.

I would think that a chip is still useful if it can only provide an
improvement for some problems, and that there are ways to increase
memory bandwidth from what ordinary CPUs offer, making it seem likely
that Cray-style vectors are worth doing as a way to improve what a CPU
can do.

Everyone has to have hope on something.

John Savard