Re: Capabilities, Anybody?

On 3/10/2024 10:42 AM, Theo Markettos wrote:

BGB <cr88192@gmail.com> wrote:

There CHERI designs on silicon in existence
>
https://www.morello-project.org/

 Member of the CHERI team here...
 

Main open question is if they can deliver enough on their claims in a
way that justifies the cost of the memory tagging (eg, where one needs
to tag whether or not each memory location holds a valid capability).

 Crudely speaking, it's 1/128 of memory so 0.78%.  Through use of
a tag cache you don't need to have a speciifc 129th bit on your DRAM
(although that's an option, like ECC) but can just wall off a piece of
regular DRAM and use it for a tag table that's accessed via the tag cache.
You only pay the cost for DRAM you actually have (eg one tag table per DIMM,
sized for that DIMM).
 The tag cache can also be compressed[1] so that if a memory page has no
capabilities you don't need to store tags for it.  This reduces tag cache
bandwidth further.
 

Yeah, I was aware of it being 1 bit per 128 bits.
So, say, if you have a 256MB RAM module, this would be ~ 2MB of overhead.
For an FPGA implementation, there is a certain amount of LUT cost as well.
Also the bigger pointers, ...

As I see it, "locking things down" would likely require turning things
like "malloc()/free()", "dlopen()/dlsym()/...", etc, into system calls
(and generally giving the kernel a much more active role in this process).

 You don't need memory allocations to be system calls, because capabilities
can be manipulated in userspace (which is a key design goal).  ie if
malloc() possesses a capability to 1MB of memory and a client requests 100
bytes, it can take its 1MB capability, offset the base to point to the start
of the allocation, set the offset to zero, and shrink the top to be
base+100, and then return that 100-byte capability.
 In a Unix OS, userspace malloc() does occasionally need to make a syscall -
when it runs out of memory pages on hand it has to ask the kernel to
allocate it some more via mmap().  But this is orthogonal to use of
capabilities (which are virtually addressed in such a system).
 For dynamic linking, you need something that holds capabilities that allow
you to turn read-write memory into read-execute memory.  That can be part of
the dynamic linker or part of the OS.  Since you are typically making
syscalls to read the shared library from disc anyway, as well as marking RW
pages as RX pages in the page table, the OS is already involved.
 

I am not claiming that one needs to have a syscall to make malloc work, but rather, I am not certain if one can make malloc entirely secure without a syscall...

But, say, note that it would not be sufficient, say, for things like
sandboxing hostile code within a shared address space with another
program that needs to be kept protected.
>
>
Granted, the strength could likely be improved (in the face of trying to
prevent hostile code from being able to steal capabilities) through
creative use of ASLR. Along with ABI features, such as "scratch register
scrubbing" (say, loading zeroes into scratch registers on function
return, such as to prevent capabilities from being leaked through via
registers), marking function pointers as "Execute Only" etc.
>
As noted, a capability system would likely still be pretty strong
against things like buffer overflows (but if only being used to mitigate
buffer overflows, is a bit overkill; so the main "interesting" case is
if it can be used to make an "unbreakable sandbox" for potentially
hostile machine code).

 Code capabilities prevent a lot of control flow attacks, because you can
only execute code you have capabilities to.  For example you're in a
function - you possess a return address capability (which has bounds based
on your local environment or compartment you're in) so you can manipulate
the address of that return address, but you can't jump to arbitrary code and
you can't forge return addresses.  So no stack smashing, no ROP/JOP attacks,
etc.
 Through setting the bounds on a code capability you can sandbox small pieces
of code to a function granularity, which is more efficient than typical MMU
sandboxing.
 

Presumably, in addition to the code, one needs some way for the code to be able to access its own ".data" and ".bss" sections when called.
Some options:
   PC-relative:
     Unclear if valid in this case.
   GOT:
     Table of pointers to things, loaded somehow.
     One example here being the ELF FDPIC ABI.
   Reloading a Global Pointer via a lookup table accessed via itself.
     This is what my ABI uses...
I couldn't seem to find any technical descriptions of the CHERI/Morello ABI. I had made a guess that it might work similar to FDPIC, as this could be implemented without needing to use raw addresses (and seemed like a "best fit").
But, an FDPIC style approach would have a weakness: One can get the GOT from a function pointer...
Granted, with an ABI design (like mine) based on daisy-chaining via the Global Pointer, one can also use the global pointer in unorthodox ways.
If capabilities give bounds information, then if there is a way for the program to distinguish valid from invalid capabilities, then a hostile program could perform searches similar to those in a conservative garbage collector for finding capabilities that it may be able to use.
Though, if the program can't tell the difference (probably more secure) then it may be possible to pepper "sensitive areas" with false positive capabilities that look valid, but will crash the program if de-referenced.

*: If it is possible to perform a Load or (worse, Capability Load)
through a function pointer, this is likely to be a significant attack
vector. Need to make it so that function pointers can only be used to
call things. Protecting against normal data loads would be needed mostly
to try to prevent code from being able to gain access to a known pointer
and possibly side-step the ASLR (say, if it can figure out that the
address it wants to access is reachable from a capability that the code
has access to).

 You can't arbitrarily change data to code capabilities - they are different
types.
 

Possible, but:
   How is the C ABI implemented in this case?
...

It is likely that the capability memory tagging would need to be managed
by the L2 cache. Would need some mechanism for the tag-bits memory (say,
2MB for 256MB at 1b per 16B line). Would also need to somehow work this
flag bit into the ringbus messaging.

 AXI has user fields which can be used for sending capabilities across the
interconnect, through third party AXI IPs like muxes, arbiters, etc.
 

I was thinking if I did an implementation of a capability system in my CPU core (as an extension of my existing bounds-checked pointer scheme). Possibly with a special CPU mode that actively enforces this system (vs just treating the bounds-checking system as opt-in at the machine-code level).
In this case, I am not using the AXI bus, rather a custom bus built on passing messages along a circular ring.
The existing bus has an interface where one has input/output connections:
   Data, 128 bit;
   Addr, 48 bit;
   OPM (Request and Flags), 16 bits.
     Low 8 bits holds the request/response/command code;
     High 8 bits holds flags.
   SEQ (Source/Destination and a Sequence Number), 16 bits.
At first, started looking at using different Request/Response IDs for data and capabilities.
Did notice though, that the bit pattern for request/response flags was:
   (7): Random Noise Bit
   (6): Prefetch Hit
   (5): Write Through
   (4): No User
   (3): No Cache
   (2): No Execute
   (1): No Write
   (0): No Read
Had the idea that the Prefetch bit could be overloaded as a capability flag for non prefetch messages.
Though, the other possibility is to use different request/response codes:
   97/A7: Load/Store Data
   AF: Store Capability
   70: Loaded Data (Response)
   60: Stored Data (Response)
   78: Loaded Capability (Response)
   ...
Previously, the low 4 bits of the response mirrored bits (11:8), change was to use exclusively (11:8) and leave the low order bits
Did modify my bus signaling slightly to allow for this, but somewhere on the way my CPU core has partially broken (it has started crashing, and I have yet to figure out what exactly has broken).
Appears that memory Loads and Stores are failing in some edge case, but is being difficult to nail down.
But, yeah, sometimes working on this stuff is like poking an angry bees nest with a stick.
Bug still happens when virtual memory is disabled, but is not reproduced by recreating the same general instruction sequence where it manifests elsewhere:
   LEA.Q   (SP, 208), R46
   MOV.L   (R46, 4), R14
   MOV.L   R14, (R46, 4)
   MOV.L   (R46, 8), R14
   MOV.L   R14, (R46, 8)
After this, the memory value at (R46, 4) is thrashed for some reason.
But, only within the context of the program; does not manifest if I copy the same instruction sequence into the sanity tests.
...
Though, partly reverting the logic for the changes to the bus messaging also did not fix the issue. Behavior is otherwise "rather weird".

Theo
 [1] https://www.cl.cam.ac.uk/research/security/ctsrd/pdfs/201711-iccd2017-efficient-tags.pdf