Re: Why VAX Was the Ultimate CISC and Not RISC

On 3/1/2025 5:58 AM, Anton Ertl wrote:

Lawrence D'Oliveiro <ldo@nz.invalid> writes:

Could the VAX have been designed as a
RISC architecture to begin with? Because not doing so meant that, just
over a decade later, RISC architectures took over the “real computer”
market and wiped the floor with DEC’s flagship architecture,
performance-wise.
>
The answer was no, the VAX could not have been done as a RISC
architecture. RISC wasn’t actually price-performance competitive until
the latter 1980s:
>
    RISC didn’t cross over CISC until 1985. This occurred with the
    availability of large SRAMs that could be used for caches.

 Like other USA-based computer architects, Bell ignores ARM, which
outperformed the VAX without using caches and was much easier to
design.
 As for code size, we see significantly smaller code for RISC
instruction sets with 16/32-bit encodings such as ARM T32/A32 and
RV64GC than for all CISCs, including AMD64, i386, and S390x
<2024Jan4.101941@mips.complang.tuwien.ac.at>.  I doubt that VAX fares
so much better in this respect that its code is significantly smaller
than for these CPUs.
 

Possibly counter-intuitively, I am beating RV64GC in terms of code sizes with 32/64 encoding schemes...
Though, a lot of this is addressing cases where RV64 is inefficient:
   Load/Store Index
   Larger Imm/Disp encodings;
   Load/Store Pair;
   LI (Imm17s), SHORI (Imm16u)
   ...
Seemingly, BGBCC can manage to shave more off the size of the binary mostly by addressing larger imm/disp/constant cases than GCC manages to save by having 16 bit instructions. Though, likely depends a lot of the program how effectively they are used. In the programs I have tested, they reduce the binary size by around 20%, implying around 40% of instructions can be turned into 16-bit forms.
For BGBCC RV+Jx output, it is around 24% smaller, and slightly beating GCC's RV64GC output.
XG3 and XG2 both save more.
   XG3: 38% smaller than RV64G
   XG2: 54% smaller than RV64G
Not entirely sure why there is this much of a delta between XG2 and XG3, they are more or less the same ISA feature wise, though there is a difference that BGBCC treats XG3 like it was a RISC-V variant, and XG3 isn't currently using any predication.
Weirdly, I seem to be winning on code-size, design my priority having been mostly on performance.
Can shave another 2% off the code size for RV64 by filling out the SLT instruction with the other cases:
   SLT , SGE , SEQ , SNE
   SLTU, SGEU, STST, SNTST
Not much, but something...
Or:
* 0000000-ttttt-sssss-010-nnnnn-01-10011  SLT Rn, Rs, Rt
* 0100000-ttttt-sssss-010-nnnnn-01-10011  SGE Rn, Rs, Rt
* 1000000-ttttt-sssss-010-nnnnn-01-10011  SEQ Rn, Rs, Rt
* 1100000-ttttt-sssss-010-nnnnn-01-10011  SNE Rn, Rs, Rt
* 0000000-ttttt-sssss-011-nnnnn-01-10011  SLTU Rn, Rs, Rt
* 0100000-ttttt-sssss-011-nnnnn-01-10011  SGEU Rn, Rs, Rt
* 1000000-ttttt-sssss-011-nnnnn-01-10011  STST Rn, Rs, Rt
* 1100000-ttttt-sssss-011-nnnnn-01-10011  SNTST Rn, Rs, Rt
Not sure why the 'B' extension didn't do this, seemingly they were more inclined to add a bunch of random weird/niche stuff than address issues like this...
Like, seemingly, it would have made more sense to address a lot of the "sharp corners" and "non-orthogonality" before adding a bunch of weird/random stuff?...
But, this was mostly by noting that BGBCC was encoding these mostly using Jumbo encoded forms, and they were a fair chunk of the Jumbo-encoded instructions. Adding 32-bit encodings reduced the number of Jumbo-coded forms (though, with jumbo prefixes still being needed to encode immediate-form variants).
Though, the 32-bit encodings open up the possibility of 64-bit:
   SEQI  Rn, Rs, Imm26s
And similar...
A lot of these other compares need around 3 instructions to encode using just RV64G instructions (or, up to around 4 or 5 for an Imm32 compare).
Granted, BGBCC was using them in a lot of cases where GCC would instead use a branch.
Say:
   if((x<10) && (y<10))
     ...
GCC will use two branches, 4 instructions.
   Granted, this follows nominal C semantics, so works.
BGBCC, RV+JX:
   Will use 4 instructions, with a single branch.
BGBCC, XG2 or XG3:
   Can be encoded in 3 instructions (with a single branch).
Can save an instruction here because my ISA's also have Test-and-Branch.
   Doing these in RV64 needs a AND+BEQ or AND+BNE.
Where:
BTST/BNTST is less common than BEQ/BNE or BLT/BGE, but more common than BLTU/BGEU, ...
I haven't yet tried adding BTST/BNTST to my RV variant, but it gets tempting...
As for debugging efforts (in the Verilog core):
In current testing, I seem to have resolved most of the issues with XG3.
My extended RISC-V variant is still buggy.
One of the remaining issues for XG3 (fixed in my working version) was that the STST and SNTST instructions were mixed up, resulting in some buggy behavior in Doom and demos would desync.
On my extended RV variant, there are rendering glitches, memory corruption, and crashes. Most likely something isn't being decoded correctly somewhere, but the cause is being hard to identify.
I did start some very early work on trying to add Verilog support to BGBCC, but I don't expect this will happen quickly, if at all. Had mostly started for now on some parts of the parsing.
While, structurally, it is quite different from C or BS2 or similar, most basic elements are "similar enough".
General thinking is that most reg/wire/etc types will be handled similar to _BitInt:
   1..128 bits:
     Fixed integer type internally, padded to the next power of 2 size.
     For VL, each would be a bitfield within the base value.
   129+: Padded to the next multiple of 128 bits.
     256, 384, 512, ...
     These types are handled in a similar way to struct types.
May consider augmenting the existing "BitInt" type with support for some additional metadata needed for Verilog semantics.
Would likely need some new internal operators to deal with bit-array operations and similar, with bit-ranges allowed as a pseudo-value type (may exist in constant expressions but will not necessarily exist as an actual value type at runtime).
   Say:
     val[63:32]
Has the (63:32) as a BitRange type, which then has special semantics when used as an array index on an integer type, ...
The previous idea for bitfield extract/insert had turned into a composite BITMOV instruction that could potentially do both operations in a single instruction (along with moving a bitfield directly between two instructions).
Idea here is that it may do, essentially a combination of a shift and a masked bit-select, say:
   Low 8 bits of immediate encode a shift in the usual format:
     Signed 8-bit shift amount, negative is right shift.
   High bits give a pair of bit-offsets used to compose a bit-mask.
     These will MUX between the shifted value and another input value.
I am still not sure whether this would make sense in hardware, but is not entirely implausible to implement in the Verilog.
Would likely be a 2 or 3 cycle operation, say:
   EX1: Do a Shift and Mask Generation;
     May reuse the normal SHAD unit for the shift;
     Mask-Gen will be specialized logic;
   EX2:
     Do the MUX.
   EX3:
     Present MUX result as output (passed over from EX2).
Doing it as a 2-cycle op could be possible, but would be worse for cost (as the MUX output would feed directly into the forwarder chains). While the MUX operation itself shouldn't be too bad (in theory, 32 LUT6's, each MUX'ing 2 bits). The forwarder path tends to multiply the cost of anything fed into it. So, maybe still better to leave it as a 3-cycle operation (the forwarders are less prone to multiply the cost of inputs fed directly from FF's, as some of the cost may be absorbed in the previous cycle in routing the value to the FF's in question, which serve mostly to carry the working output to the next stage).
The Mask-Gen would basically have two 7 or 8 bit fields, with a range of 0..64 for a 64-bit BITMOV (8-bit would allow for potentially doing a 128-bit BITMOV with the same basic logic; using both the Lane 1 and 2 SHAD units as in a SHLDX instruction).
For mask generation:
   Lo: Generates 1s for bits >= Lo;
   Hi: Generates 1s for bits <  Hi;
   Hi> Lo: Combine using AND
   Hi<=Lo: Combine using OR
Granted:
   Lo: Generates 1s for bits >= Lo;
   Hi: Generates 1s for bits <= Hi;
   Hi>=Lo: Combine using AND
   Hi< Lo: Combine using OR
Could allow shaving 1 bit off the needed ranges for the bit offsets (allowing 6 or 7 bits; or Imm20/Imm22 vs Imm24).
A more compact encoding (using the same logic) could be adapted for a bitfield extract, likely internally using ZZR (zero) for the 3rd input.
   Rs: Value to be shifted and MUX'ed;
   Rt: Shift/Mask Control (Immed);
   Rp: Third source, Zero, or Dest
   Rn: Dest.
Though, one possibility (besides separate Shift and Mask handling) is to glue the mask generation and MUX'ing logic directly onto the SHAD units (and probably keeping these as 2-cycle operations, doing the MUX'ing at the end of EX1 rather than EX2).
The considered encoding for this instruction would be a little wonky (I don't currently have any large-immediate 4RI encodings, so this would need to deviate from the usual decoder behavior). Or, I use a more typical 3RI Imm33s encoding, but this requires an additional MOV or similar for the Dest register.
This instruction would likely have more limited applicability to C and would be more difficult for the compiler to use (but could be relevant to bitfields); or inferred from expressions like: ((x>>SHL)&MASK).
However, this operation would likely be very relevant if trying to run Verilog on the ISA (otherwise, extreme bit-twiddly is likely to hurt pretty bad).
The debate is more whether it would make sense to also add it to hardware, or leave it as an emulator-only instruction.
There is nothing too absurd about adding it to hardware, and as noted, it could potentially help with things like bitfields.

Bottom line: If you sent, e.g., me and the needed documents back in
time to the start of the VAX project, and gave me a magic wand that
would convince the DEC management and workforce that I know how to
design their next architecture, and how to compiler for it, I would
give the implementation team RV32GC as architecture to implement, and
that they should use pipelining for that, and of course also give that
to the software people.
 As a result, DEC would have had an architecture that would have given
them superior performance, they would not have suffered from the
infighting of VAX9000 vs. PRISM etc. (and not from the wrong decision
to actually build the VAX9000), and might still be going strong to
this day.  They would have been able to extend RV32GC to RV64GC
without problems, and produce superscalar and OoO implementations.
 

While RISC-V is still not perfect, it is arguably a whole lot better than a lot of the other stuff from that era, so probably works...

OTOH, DEC had great success with the VAX for a while, and their demise
may have been unavoidable given their market position: Their customers
(especially the business customers of VAXen) went to them instead of
IBM, because they wanted something less costly, and they continued
onwards to PCs running Linux when they provided something less costly.
So DEC would also have needed to outcompete Intel and the PC market to
succeed (and IBM eventually got out of that market).
 

It almost seems like they could have tried making a PDP-11 based PC.
Say, if they could make a PDP-11 variant that was cost competitive with the 8086 or M68K, and maybe tried underselling IBM (where the early IBM PC's were very expensive, and Compaq later captured that part of the market by being better and cheaper).
DEC could have maybe had a marketing advantage in, say, "Hey, our crap can run UNIX" and "UNIX is better than DOS".
This alongside any VAX based workstations.
...