Re: Computer architects leaving Intel...

On Sun, 22 Sep 2024 20:43:38 +0000, Paul A. Clayton wrote:

On 9/19/24 11:07 AM, EricP wrote:
[snip]

If the multiplier is pipelined with a latency of 5 and throughput
of 1,
then MULL takes 5 cycles and MULL,MULH takes 6.
>
But those two multiplies still are tossing away 50% of their work.

>
I do not remember how multipliers are actually implemented — and
am not motivated to refresh my memory at the moment — but I
thought a multiply low would not need to generate the upper bits,
so I do not understand where your "50% of their work" is coming
from.

     +-----------+   +------------+
     \  mplier  /     \   mcand  /        Big input mux
      +--------+       +--------+
           |                |
           |      +--------------+
           |     /               /
           |    /               /
           +-- /               /
              /     Tree      /
             /               /--+
            /               /   |
           /               /    |
          +---------------+-----------+
                hi             low        Products
two n-bit operands are multiplied into a 2×n-bit result.
{{All the rest is HOW not what}}

The high result needs the low result carry-out but not the rest of
the result. (An approximate multiply high for multiply by
reciprocal might be useful, avoiding the low result work. There
might also be ways that a multiplier could be configured to also
provide bit mixing similar to middle result for generating a
hash?)
>
I seem to recall a PowerPC implementation did semi-pipelined 32-
bit multiplication 16-bits at a time. This presumably saved area
and power

You save 1/2 of the tree area, but ultimately consume more power.

          while also facilitating early out for small
multiplicands,

Dadda showed that doubling the size of the tree only adds one
4-2 compressor delay to the whole calculation.

               at the cost of some latency and substantial
throughput compared to a fully pipelined multiplication.

Throughput that the rest of the engine could not use.

                                                          If I
remember correctly, this produced a result for 16-bit by 32-bit
multiplication, which is different from generating a low or high
result.
>

And if it does fuse them then the internal uArch cost is the same
as if
you had designed it optimally from the start, except now you have
to pay for a fuser.
>
<sound of soap box being dragged out>
This idea that macro-op fusion is some magic solution is bullshit.

The argument is, at best, of Academic Quality, made by a student
at the time as a way to justify RISC-V not having certain easy
for HW to perform calculations.

1) It's not free.

>
Neither is increasing the number of opcodes or providing extender
prefixes. If one wants binary compatibility, non-fusing
implementations would work.

I did neither and avoided both.

(I tend to favor providing a translation layer between software
distribution format and instruction cache format, which reduces
the binary compatibility constraint.)
>

2) It only works where Decode can see *all* the required lookahead
    instructions, which means you have to pay for an N-lane decoder
    but only get 1 lane.

>
Most fusion is for two adjacent instructions, which significantly
limits the complexity.

To quadratic {BigO( instruction-OpCode-bits ** 2)}

                       The fusable patterns are also a subset of
all pairs of two instructions, so complete two-way decoding may
not be needed.
>
There may also be optimization opportunities from looking ahead.
Mitch Alsup proposed such for branch handling in a scalar
implementation.

I use this, to be clear, as a means to eliminate any need of the
branch delay slot in smaller narrow machines.

                Apart from fusion, there might be advantages for
avoiding bank conflicts in a banked register file. I.e., the cost
of lookahead might be shared by multiple techniques/optimizations.
>
I tend to agree that fusion tends to be a workaround for sub-
optimal instruction encoding, but it seems that encoding involves
a lot of tradeoffs.
>

3) It's probabilistic as it depends on how the fetch buffers get
loaded.
    Eg if the fetch buffer contains a valid instruction but does
not have
    a next instruction, do you stall Decode to see if a fuser
might arrive
    or dispatch it anyway.

>
This is also somewhat true for variable length encodings that
cross fetch boundaries.

In My 1-wide machine, the only time this comes up is when a
long instruction crosses into a new cache line (or page) and
the cache (or TLB) takes a miss.

                        In general a boundary-crossing instruction
would probably stall even if such was not strictly necessary
(e.g., if the missing information is opcode refinement — not
related to instruction routing — or an immediate or even a
register source identifier specifying a value that can have
delayed use (e.g., value of a store, addend of a FMADD).

In my case, immediate data for a ST is not needed until the ST
has retired, so a) it is placed last, b) delay can be tolerated
as long as the pipeline depth.

>
This does seem a weakness, but fusion is not entirely negative
factors.
>

4) It gets exponentially expensive if you start doing multiple
instruction
    lanes because decode has to deal with all the permutations of
    fusion possibilities.

Fusion in an already variable length RISC ISA is already exponential.

>
This is also a factor in mere superscalar decode/execute.
Detecting that an instruction is dependent on another would
normally stall the execution of that instruction.
>
(I feel that encoding some of the dependency information could
be useful to avoid some of this work. In theory, common
dependency detection could also be more broadly useful; e.g.,
operand availability detection and execution/operand routing.)

So useful that it is encoded directly in My 66000 ISA.

5) Any fused instructions leave (multiple) bubbles that should be
    compacted out or there wasn't much point to doing the fusion.

>
Even with reduced operations per cycle, fusion could still provide
a net energy benefit.

Here I disagree:: but for a different reason::
In order for RISC-V to use a 64-bit constant as an operand, it has
to execute either::  AUPIC-LD to an area of memory containing the
64-bit constant, or a 6-7 instruction stream to build the constant
inline. While an ISA that directly supports 64-bit constants in ISA
does not execute any of those.
Thus, while it may save power seen at the "its my ISA" level it
may save power, but when seem from the perspective of "it is
directly supported in my ISA" it wastes power.
There is NO less power expensive way to deliver a constant into
execution as from the instruction stream directly to the function
unit performing the calculation.

In my opinion it is better to have an ISA that is optimal by design
rather than being patched up by fusion later.

>
Fusion is mostly presented for "patching up", but there are also
considerations of diverse microarchitectures. With pre-fused
instructions, an implementation might need to crack some of those
instructions. Software optimized for such an implementation might
also prefer more flexible compile-time scheduling of pre-cracked
operations.

Agreed:: there is a cost of implementing a means by which large
constants can be used in the instruction. I argue that this is
a) only apparent in the smallest implementations, b) is smaller
than the cost in cycles and power that fusion requires.

A load-op instruction is perhaps particularly difficult because
one needs frequent stalls, a skewed (or second chance) pipeline to
hide the load latency, out-of-order execution, or some other stall
avoidance mechanism.
>
There are also constraints in encoding granularity.
>

Some of this inefficiency is caused by clinging to now 40 year old
risc design *guidelines* (ie not even rules) that:
- instructions have at most 1 dest and 2 source registers

>
FMADD seems to have mostly killed the 2-source limit. AArch64's
paired load removes the 2 destination limit. (Paired destinations
were common for early double precision implementations.)

FMAD also provides the operand bussing to support the::
     mem     rd,[Rbase+Rindex<<scale+disp]
addressing mode.
But this was already possible since "disp" always comes from the
instruction, and only goes to the AGEN unit.
FMAD just got rid of all the other excuses not to do the right
thing.

>

- register specifier fields are either source or dest, never both

>
This seems mostly a code density consideration. I think using a
single name for both a source and a destination is not so
horrible, but I am not a hardware guy.

All we HW guys want is the where ever the field is specified,
it is specified in exactly 1 field in the instruction. So, if
field<a..b> is used to specify Rd in one instruction, there is
no other field<!a..!b> specifies the Rd register. RISC-V blew
this "requirement.

- instructions should take at most 1 clock (they never did)

>
That was clearly overconstraining.
>

These self imposed design restrictions cause ISA designers to miss
some possible more optimal solutions. The result is things like
RISC-V's memory reference linkage structures taking 6 instructions
to build a 64-bit PC-relative address. And I'm pretty sure we won't
see any 6 instruction fusers for quite some time.

>
I very much doubt a compiler would generate such outside of some
real-time application where the time constancy might justify the
code bloat.
>

<sound of soap box being dragged back to cupboard>

>
I do not mean my response to be heckling. Your points are very
true. However, I think fusion is a technique — like cracking —
that is a natural part of an architect's toolbox.