Re: streams and file locks, ancient OS history, ARM is sort of channeling the IBM 360

According to Anton Ertl <anton@mips.complang.tuwien.ac.at>:

An application where one writes via mmap and others read
using stdio or read/write/pread/pwrite simulataneously
is poorly designed.

>
"An application"?  I live in a world where all kinds of programs can
access the same files.  How such a program works internally is often
not known to the designers of the others.
>
In practice your advice boils down to avoiding to write with mmap().

It's more like don't try to write shared files unless you know all of
the writers are using consistent locking. Squinting at the source code
for FreeBSD's stdio, it only flushes writes when it fills the buffer
or when it writes a \n character with the '\n' hard coded in the
source code. It doesn't lock at all other than between threads in the
same task.

You need at least as many locks and flushes to keep things in sync as
you would with mapped files.

In my experience, mmap is most often used when record-granularity
is required, rather than treating the mapped region as a
stream-of-bytes.

>
I usually mmap() the whole file.

Same here, although I have used mmap() to map a whole file as well as
for database record stuff.

* You have to extend the file length with a separate system call, and
 then mmap() the new area, so you might just as well use write().

>
False economy.   ftruncate is a single system call.

>
Yes, that's the one that extends the file length.  ftruncate()
followed by mmap() are two system calls.  And at some point you also
want to msync() (although not for each ftruncate).

Seems to me the sensible thing to do is to use ftruncate and mmap to
extend the file in large chunks, say several megabytes at a time, not
one block at a time. If you have too many system calls your chunks are
not big enough.  I believe the file systems do what they do with write()
and only allocate disk blocks the first time you write to them so there
is little penalty for extending in large chunks.




 
--
Regards,
John Levine, johnl@taugh.com, Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. https://jl.ly