Re: Memory protection between compilation units?

Mateusz Viste <mateusz@x.invalid> writes:

This might not be a strictly C question, but it definitely concerns all
C programmers.
>
Earlier today, I fixed an out-of-bounds write bug.  An obvious issue:
>
  static int *socks[0xffff];
>
  void update_my_socks(int *sock, int val) {
    socks[val & 0xffff] = sock;
  }
>
While the presented issue is common knowledge for anyone familiar with
C, *locating* the bug was challenging.  The program did not crash at the
moment of the out-of-bounds write but much later - somewhere entirely
different, in a different object file that maintained a static pointer
for tracking a position in a linked list.  To my surprise, the pointer
was randomly reset to NULL about once a week, causing a segfault.
Tracing this back to an unrelated out-of-bounds write elsewhere in the
code was tedious, to say the least.
>
This raises a question:  how can such corruptions be detected sooner?
Protected mode prevents interference between programs but doesn?t
safeguard a program from corrupting itself.  Is there a way to enforce
memory protection between module files of the same program?  After all,
static objects shouldn't be accessible outside their compilation unit.
>
How would you approach this?

The code in question shows several classic error patterns.  In no
particular order:

  * buffer overflow
  * off-by-one error
  * hard-coded constants (rather than symbolic)
  * bitwise operator with signed operand
  * using & to effect what is really a modulo operation
  * two of the above combine to impose a constraint on a
    hard-coded value, and the constraint is never checked

Of course some of these, notably buffer overflow, are hard to find.
But some of them are easy.  The hard-coded constants stand out like a
neon sign, especially because one is duplicated.  Check for any
constant written in open code above the value of, say, 10.  Once the
offending example is found, it can be rewritten, as for example

   static int *socks[0xffff];
  
   void update_my_socks(int *sock, int val) {
      const unsigned N = sizeof socks / sizeof socks[0];
      socks[val % N] = sock;
   }

This revision doesn't fix the program but it does eliminate the bug.
(Presumably fixing the program will happen later.)  Of course the
code should be further revised so that the temptation to use the
hard-coded value elsewhere is reduced, but this revision at least is
a step in the right direction.

Also, whenever a cockroach is seen, you can be sure there are other
cockroaches around.  Each of the types of errors evidenced by the
original code (at least three of the list of six types) represent
bugs waiting to be found;  go through the code and check for all
of them, at least for the ones that can be located easily.  Add
these error classes to the list of potential problems checked
during code review.

I acknowledge that this response isn't exactly an answer to the
original question.  It does illustrate though a kind of thinking
that can be useful when trying to track down hard-to-find bugs.