Re: Good hash for pointers

bart <bc@freeuk.com> writes:

On 24/05/2024 19:57, Malcolm McLean wrote:
>

On 24/05/2024 19:28, Bonita Montero wrote:
>

Am 23.05.2024 um 13:11 schrieb Malcolm McLean:
>

What is a good hash function for pointers to use in portable ANSI C?
>
The pointers are nodes of a tree, which are read only, and I want
to associate read/write data with them.  So potentially a lage
number of pointers,and they might be consecutively ordered if they
are taken from an array, or they might be returned from repeared
calls to malloc() with small allocations.  Obviously I have no
control over pointer size or internal representation.

>
Use FNV.

>
Here's an attempt.
>
/* FNV hash of a pointer */
static unsigned int hash(void *address)
{
  int i;
  unsigned long answer = 2166136261;
  unsigned char *byte = (unsigned char *) &address;
>
  for (i = 0; i < sizeof(void *); i++)
  {
  answer *= 16777619;
  answer ^= byte[i];
  }
  return (unsigned int) (answer & 0xFFFFFFFF);
}
>
Now what will compilers make of that?

>
Compiler, or performance?
>
I tried this function with the test program shown below.  I used it to
populate a hash table of 64K entries with pointers from successive
calls to malloc.
>
Results, in terms of clashes, for different numbers N of entries
populated out of 64K were:
>
 10000     1100
 30000    12000
 50000    67000
 60000   216000
 65535  5500000    (largest N possible)
>
Result were rather variable as malloc produces different patterns of
pointers on different runs.  These were simply the result from the
first run.
>
Was this good?  I'd no idea.  But as a comparison, I used my own hash
function, normally used to hash identifiers, shown below the main
program as the function 'bchash'.
>
If this is subsituted instead, the results were:
>
 10000      230
 30000     3800
 50000    10300
 60000    50300
 65535  2700000
>
Hash tables need a certain amount of free capacity to stay efficient,
so 3/4 full (about 50K/64K) is about right.
>
Again I don't know if these figures are good, they are just better
than from your hash() function, for the inputs I used, within this
test, and for this size of table.
>
No doubt there are much better ones.
>
>
>
------------------------------------------
#include <stdio.h>
#include <stdlib.h>
>
static unsigned int hash(void *address)
{
    int i;
    unsigned long answer = 2166136261;
    unsigned char *byte = (unsigned char *) &address;
>
    for (i = 0; i < sizeof(void *); i++)
    {
        answer *= 16777619;
        answer ^= byte[i];
    }
    return (unsigned int) (answer & 0xFFFFFFFF);
}
>
void* table[65536];
>
int main(void) {
    void* p;
>
    int clashes=0, wrapped;
    int j;
>
    for (int i=0; i<30000; ++i) {
        p = malloc(1);
        j = hash(p) & 65535;
>
        wrapped=0;
        while (table[j]) {
            ++clashes;
            ++j;
            if (j>65535) {
                if (wrapped) { puts("Table full"); exit(1);}
                wrapped=1;
                j=0;
            }
        }
        table[j] = p;
>
    }
    printf("Clashes %d\n", clashes);
}
>
>
------------------------------------------
>
static unsigned int bchash(void *address)
{
    int i;
    unsigned long hsum = 0;
    unsigned char *byte = (unsigned char *) &address;
>
    for (i = 0; i < sizeof(void *); i++)   {
        hsum = (hsum<<4) - hsum + byte[i];
    }
    return (hsum<<5) - hsum;
}

It looks like your hash function was tuned for this testing
setup.  With different choices for testing it does much
worse.

The testing is done with malloc() blocks all of the same
requested size, and that size is 1.  Typical workloads
are likely to be both larger and more variable.

When adding a new entry finds a collision with an old
entry, linear probing is used to find a free slot.  It's
well understood that linear probing suffers badly as the
load factor goes up.  Better to take a few high bits of
the hash value -- as few as five or six is fine -- to
have the reprobe sequences have different strides.

Your hash function is expensive to compute, moreso even
than the "FNV" function shown earlier.  In a case like
this one where the compares are cheap, it's better to
have a dumb-but-fast hash function that might need a
few more looks to find an open slot, because the cost
of looking is so cheap compared to computing the hash
function.