Re: realloc() - frequency, conditions, or experiences about relocation?

On 17/06/2024 08:08, Janis Papanagnou wrote:

In a recent thread realloc() was a substantial part of the discussion.
"Occasionally" the increased data storage will be relocated along
with the previously stored data. On huge data sets that might be a
performance factor. Is there any experience or are there any concrete
factors about the conditions when this relocation happens? - I could
imagine that it's no issue as long as you're in some kB buffer range,
but if, say, we're using realloc() to substantially increase buffers
often it might be an issue to consider. It would be good to get some
feeling about that internal.
 Janis

Consider your target audience and their hardware, the target OS, and the realistic size of your data.  If the target is a PC, you can happily malloc tens of MB at the start without a care, and for systems that do not actually allocate system memory until you try to access the area, there is no cost to this.
So in many situations where you are reading and parsing data from a file, you can just do the initial malloc with more than enough space for any realistic input file.  You might still implement a realloc solution for occasional extreme uses, and because it is nice to avoid artificial limits for programs, but efficiency matters a lot less in those cases.
It may also be the case that even if realloc returns a different address and logically copies a lot of data, that this is done by smarter virtual memory mapping so that only the mapping changes, and the underlying physical ram does not need to be copied.  But I don't know if OS's and realloc implementations are smart enough to do that.