Re: Constant Stack Canaries

On 3/31/2025 11:04 AM, Stephen Fuld wrote:

On 3/30/2025 1:14 PM, MitchAlsup1 wrote:

On Sun, 30 Mar 2025 17:47:59 +0000, BGB wrote:
>

On 3/30/2025 7:16 AM, Robert Finch wrote:

Just got to thinking about stack canaries. I was going to have a special
purpose register holding the canary value for testing while the program
was running. But I just realized today that it may not be needed. Canary
values could be handled by the program loader as constants, eliminating
the need for a register. Since the value is not changing while the
program is running, it could easily be a constant. This may require a
fixup record handled by the assembler / linker to indicate to the loader
to place a canary value.
>
Prolog code would just store an immediate to the stack. On return a TRAP
instruction could check for the immediate value and trap if not present.
But the process seems to require assembler / linker support.
>

>
They are mostly just a normal compiler feature IME:
   Prolog stores the value;
   Epilog loads it and verifies that the value is intact.

>
Agreed.

 I'm glad you, Mitch, chimed in here.  When I saw this, it occurred to me that this could be done automatically by the hardware (optionally, based on a bit in a control register).   The CALL instruction would store magic value, and the RET instruction would test it.  If there was not a match, an exception would be generated.  The value itself could be something like the clock value when the program was initiated, thus guaranteeing uniqueness.
 The advantage over the software approach, of course, is the elimination of several instructions in each prolog/epilog, reducing footprint, and perhaps even time as it might be possible to overlap some of the processing with the other things these instructions do.  The downside is more hardware and perhaps extra overhead.
 Does this make sense?  What have I missed.
 

This would seem to imply an ISA where CALL/RET push onto the stack or similar, rather than the (more common for RISC's) strategy of copying PC into a link register...
Another option being if it could be a feature of a Load/Store Multiple.
Say, LDM/STM:
   6b Hi (Upper bound of register to save)
   6b Lo (Lower bound of registers to save)
   1b LR (Flag to save Link Register)
   1b GP (Flag to save Global Pointer)
   1b SK (Flag to generate a canary)
Likely (STM):
   Pushes LR first (if bit set);
   Pushes GP second (if bit set);
   Pushes registers in range (if Hi>=Lo);
   Pushes stack canary (if bit set).
LDM would check the canary first and fault if it doesn't see the expected value.
Downside, granted, is needing the relative complexity of an LDM/STM style instruction.
Other ISAs use a flag bit for each register, but this is less viable with an ISA with a larger number of registers, well, unless one uses a 64 or 96 bit LDM/STM encoding (possible). Merit though would be not needing multiple LDM's / STM's to deal with a discontinuous register range.
Well, also excluding the possibility where the LDM/STM is essentially just a function call (say, if beyond certain number of registers are to be saved/restored, the compiler generates a call to a save/restore sequence, which is also generates as-needed). Granted, this is basically the strategy used by BGBCC. If multiple functions happen to save/restore the same combination of registers, they get to reuse the prior function's save/restore sequence (generally folded off to before the function in question).
Granted, the folding strategy can still do canary values, but doing so in the reused portions would limit the range of unique canary values (well, unless the canary magic is XOR'ed with SP or something...).
...

 

>

Using a magic number

>
Remove excess words.
>

Nothing fancy needed in the assemble or link stages.

>
They remain blissfully ignorant--at most they generate the magic
number, possibly at random, possibly per link-module.
>

In my case, canary behavior is one of:
   Use them in functions with arrays or similar (default);
   Use them everywhere (optional);
   Disable them entirely (also optional).
>
In my case, it is only checking 16-bit magic numbers, but mostly because
a 16-bit constant is cheaper to load into a register in this case
(single 32-bit instruction, vs a larger encoding needed for larger
values).
>
....