Re: Are We Back to the "Wars" Now ?

On Fri, 22 Nov 2024 19:11:23 -0000 (UTC), Rich <rich@example.invalid>
wrote in <vhql0r$1a0ch$2@dont-email.me>:

vallor <vallor@cultnix.org> wrote:

On Fri, 22 Nov 2024 03:12:43 -0000 (UTC), Lawrence D'Oliveiro
<ldo@nz.invalid> wrote in <vhosra$1171f$1@dont-email.me>:
 

On Thu, 21 Nov 2024 21:55:37 -0500, Phillip Frabott wrote:
 

We had to drop named pipes solely because of the performance hit
because it is writing to a file system so it's being controlled by
the file system, even if that file system is in memory.

 
That doesn’t make any sense, if we were talking about Linux. Is this
on Windows, by any chance?

 
Doesn't the named pipe connection work through the filesystem code?
That could add overhead.

 
Only to the extent that a filesystem lookup has to occur to lookup the
name in order to open() the name.
 
Once you have a file descriptor back from the open() call, there is no
difference at all kernel wise betwenn the two, they are one and the same
block of kernel code.

I stand corrected about that.

 

Can't use named pipes on just any filesystem -- won't work on NFS for
example, unless I'm mistaken.

 
Correct, you need a filesystem that supports storing a 'name' that that
is a reference to a pipe, so windows filesystems are out.
 
Named pipes appear as 'pipe' nodes across NFS (just tested this to be
certian).  And, so long as all the "accessors" of the named pipe are
running on the same Linux machine with the NFS mount containing the pipe
node, the named pipe works as expected (just tested this as well).

I tested it too (with an NFS v4.1 filesystem), and yes, mkfifo makes
a named pipe, and it works as expected.  (Didn't expect it to work
across machines, though that would be a neat trick.)

But a named pipe on NFS does not give you a machine to machine (two
different machines) transmit channel.
 

As the demand grows, we are actually at the limits of performance
that even unnamed pipes gives us. So we are starting to migrate to
UNIX sockets which has about double to bandwidth and performance of
pipes.

 
Not sure how that works, given that Unix sockets are actually a more
complex mechanism than pipes.

 
With Unix sockets, once the connection is made, it's all in-memory
networking.

 
Correct.
 

I suspect (but don't know) that named pipes require the data to pass
through the filesystem for each write.

 
Incorrect.  The only 'filesystem' access for named pipes is during the
open() call to look up the name from the filesystem.  Once you get the
file descriptor back, it is the exact same in-memory FIFO queue as an
anonymous pipe created via pipe() (at least on Linux).

Again, I stand corrected on that.

(Haven't figured out how to increase ulimit -p yet, doesn't seem
to want to increase, even as root...)

--
-v System76 Thelio Mega v1.1 x86_64 NVIDIA RTX 3090 Ti
   OS: Linux 6.12.0 Release: Mint 21.3 Mem: 258G
   "A good hot dog feeds the hand that bites it."