Re: Named pipes vs. Unix sockets

On 11/22/2024 15:52, Lawrence D'Oliveiro wrote:

On Fri, 22 Nov 2024 10:00:31 -0500, Phillip Frabott wrote:
 

At the moment we have to do 4.6GB per 0.5sec to keep up, and
7GB per 0.5sec to stay ahead. We've done everything we can to squeeze
out as much performance from pipes as we could but there just isn't much
we can do anymore.

 Sounds like a job for a shared memory section.

That only works if the entire dataset originates on, and results within, the same system. Unfortunately this is not the case so it wouldn't be possible. This is an intermediate system between the intake and final processing systems which are further down the line.
Unless you can recommend a way to do shared memory across different machines without extra hardware. Currently we ingest data from a (external) /dev (unbuffered data stream) into a process on the machine that buffers the data (currently though an unnamed pipe, which as I said before is in the process of being replaced with UNIX socket) then that data pipes to another process (again, being replaced with UNIX socket soon) which converts the data stream from the buffer to block data and then categorizes those blocks based on data types and depending on those data types sends it to one of 4 /dev devices that leave the machine. The bottleneck is not between the first and second process (buffer process and stream to block data processor) it's between the intake /dev to the intermediate and the intermediate to the 4 outbound /devs. These /dev are not networks so there is no TCP data. They are hard lined between the /dev machine and the intermediate computer and the intermediate computer to the 4 other /dev machines. So how would I share memory between /dev machines from the 80s-90s and the intermediate computer without using any network or TCP/UDP capabilities?
(If it was not apparent. these are manufacturing machines.)
--
Phillip Frabott
----------
- Adam: Is a void really a void if it returns?
- Jack: No, it's just nullspace at that point.
----------