Re: Computer architects leaving Intel...

On 9/22/2024 3:43 PM, Paul A. Clayton wrote:

On 9/19/24 11:07 AM, EricP wrote:
[snip]

If the multiplier is pipelined with a latency of 5 and throughput of 1,
then MULL takes 5 cycles and MULL,MULH takes 6.
>
But those two multiplies still are tossing away 50% of their work.

 I do not remember how multipliers are actually implemented — and
am not motivated to refresh my memory at the moment — but I
thought a multiply low would not need to generate the upper bits,
so I do not understand where your "50% of their work" is coming
from.
 The high result needs the low result carry-out but not the rest of
the result. (An approximate multiply high for multiply by
reciprocal might be useful, avoiding the low result work. There
might also be ways that a multiplier could be configured to also
provide bit mixing similar to middle result for generating a
hash?)
 

I guess it might be interesting if one made a bigger multiplier out of 4-bit multipliers, in a way similar to a 4-bit shift-add.
Could in theory do a 64-bit multiply in ~ 16 cycles this way...
But, probably couldn't use this for divide.

I seem to recall a PowerPC implementation did semi-pipelined 32-
bit multiplication 16-bits at a time. This presumably saved area
and power while also facilitating early out for small
multiplicands, at the cost of some latency and substantial
throughput compared to a fully pipelined multiplication. If I
remember correctly, this produced a result for 16-bit by 32-bit
multiplication, which is different from generating a low or high
result.
 

On an FPGA, one could almost argue for doing everything with an unsigned 16*16->32 bit multiplier and some helper instructions.
Function calls to do multiply would be kinda lame though, as in cases like this, function call/return overhead can become a significant part of the total cycle cost (but, still not really cheap enough to justify doing it inline).

And if it does fuse them then the internal uArch cost is the same as if
you had designed it optimally from the start, except now you have
to pay for a fuser.
>
<sound of soap box being dragged out>
This idea that macro-op fusion is some magic solution is bullshit.
1) It's not free.

 Neither is increasing the number of opcodes or providing extender
prefixes. If one wants binary compatibility, non-fusing
implementations would work.
 (I tend to favor providing a translation layer between software
distribution format and instruction cache format, which reduces
the binary compatibility constraint.)
 

Yes, it would be preferable.
Sadly, no "good" VM has caught on.
As-in, sanely designed and can run C and C++ and similar effectively without being overly tied to a specific platform.
Among other things, this does likely mean the core C runtime library will need to run inside the VM, and probably static linked (C library implementations being prone to leak details like how their "stdio" implementation is implemented, etc).

2) It only works where Decode can see *all* the required lookahead
    instructions, which means you have to pay for an N-lane decoder
    but only get 1 lane.

 Most fusion is for two adjacent instructions, which significantly
limits the complexity. The fusable patterns are also a subset of
all pairs of two instructions, so complete two-way decoding may
not be needed.
 There may also be optimization opportunities from looking ahead.
Mitch Alsup proposed such for branch handling in a scalar
implementation. Apart from fusion, there might be advantages for
avoiding bank conflicts in a banked register file. I.e., the cost
of lookahead might be shared by multiple techniques/optimizations.
 I tend to agree that fusion tends to be a workaround for sub-
optimal instruction encoding, but it seems that encoding involves
a lot of tradeoffs.
 

Yeah...
However, the cost of doing fusion is higher than having longer-form variable-length instructions via prefixes...
If one wants a cheapish way to do prefixes on a 1-wide machine, they could transpose the instruction words during fetch, and then only need a single decoder.
So:
   WordA
   PrefixA WordB
   PrefixA PrefixB WordC
Is presented to the decoder as:
   WordA
   WordB PrefixA
   WordC PrefixB PrefixA
So, the decoder doesn't move...
Possibly, a similar trick could be used for 2-wide with limited variable-length, but would get more complicated.

3) It's probabilistic as it depends on how the fetch buffers get loaded.
    Eg if the fetch buffer contains a valid instruction but does not have
    a next instruction, do you stall Decode to see if a fuser might arrive
    or dispatch it anyway.

 This is also somewhat true for variable length encodings that cross fetch boundaries. In general a boundary-crossing instruction
would probably stall even if such was not strictly necessary
(e.g., if the missing information is opcode refinement — not
related to instruction routing — or an immediate or even a
register source identifier specifying a value that can have
delayed use (e.g., value of a store, addend of a FMADD).
 This does seem a weakness, but fusion is not entirely negative
factors.
 

4) It gets exponentially expensive if you start doing multiple instruction
    lanes because decode has to deal with all the permutations of
    fusion possibilities.

 This is also a factor in mere superscalar decode/execute.
Detecting that an instruction is dependent on another would
normally stall the execution of that instruction.
 (I feel that encoding some of the dependency information could
be useful to avoid some of this work. In theory, common
dependency detection could also be more broadly useful; e.g.,
operand availability detection and execution/operand routing.)
 

Superscalar is easier, as here one merely needs to categorize the instruction with feature flags (such as which lanes it is allowed in), and check for register dependencies.
This is less of an issue than what is needed for fusion (namely, detecting specific pairs of instructions via pattern matching); unless the fusion is very limited in scope.

5) Any fused instructions leave (multiple) bubbles that should be
    compacted out or there wasn't much point to doing the fusion.

 Even with reduced operations per cycle, fusion could still provide
a net energy benefit.
 

In my opinion it is better to have an ISA that is optimal by design
rather than being patched up by fusion later.

 Fusion is mostly presented for "patching up", but there are also
considerations of diverse microarchitectures. With pre-fused
instructions, an implementation might need to crack some of those
instructions. Software optimized for such an implementation might
also prefer more flexible compile-time scheduling of pre-cracked
operations.
 A load-op instruction is perhaps particularly difficult because
one needs frequent stalls, a skewed (or second chance) pipeline to
hide the load latency, out-of-order execution, or some other stall
avoidance mechanism.
 There are also constraints in encoding granularity.
 

Some of this inefficiency is caused by clinging to now 40 year old
risc design *guidelines* (ie not even rules) that:
- instructions have at most 1 dest and 2 source registers

 FMADD seems to have mostly killed the 2-source limit. AArch64's
paired load removes the 2 destination limit. (Paired destinations
were common for early double precision implementations.)
 

IMHO:
RISC-V not having register-index load/store, while having things like FMADD, is kinda stupid. Having advanced features while taking a big hit on the lack of cheap features is not ideal.
I had recently been working on getting BGBCC to target RISC-V (generated code still not fully working, but the compiler is now able to do the compiler thing at least).
However, with all of the limits that RISC-V imposes, BGBCC is currently generating output that is around 43% bigger in RISC-V mode than BJX2-XG2 mode (or around 56% bigger than baseline mode).
This is kinda terrible...
A lot of this being because there are lots of cases where BJX2 can encode things in a single instruction that RV64 ends up needing multiple instructions to pull off (along with the loss of predicated instructions and some other things).
Granted, it is also losing pretty hard vs GCC in terms of code density for RV64, so there is likely some room for improvement.
Seems to be a lot of relative penalty for things like constants and for loading/storing global variables, ... Currently, BGBCC is always building constants inline vs using memory loads.
So, say, 6 instructions for a 64-bit constant load, or around 4 instructions to load/store a global variable (relative to GP), 4 instructions whenever the 12-bit displacement fails, ...

- register specifier fields are either source or dest, never both

 This seems mostly a code density consideration. I think using a
single name for both a source and a destination is not so
horrible, but I am not a hardware guy.
 

- instructions should take at most 1 clock (they never did)

 That was clearly overconstraining.
 

These self imposed design restrictions cause ISA designers to miss
some possible more optimal solutions. The result is things like
RISC-V's memory reference linkage structures taking 6 instructions
to build a 64-bit PC-relative address. And I'm pretty sure we won't
see any 6 instruction fusers for quite some time.

 I very much doubt a compiler would generate such outside of some
real-time application where the time constancy might justify the
code bloat.
 

<sound of soap box being dragged back to cupboard>

 I do not mean my response to be heckling. Your points are very
true. However, I think fusion is a technique — like cracking —
that is a natural part of an architect's toolbox.