Re: xxd -i vs DIY Was: C23 thoughts and opinions

On 5/30/2024 3:40 AM, Michael S wrote:

On Thu, 30 May 2024 00:40:07 -0400
Paul <nospam@needed.invalid> wrote:
 

On 5/28/2024 6:24 PM, Michael S wrote:

On Tue, 28 May 2024 23:08:22 +0100
bart <bc@freeuk.com> wrote:
 

On 28/05/2024 21:23, Michael S wrote: 

On Tue, 28 May 2024 19:57:38 +0100
bart <bc@freeuk.com> wrote:
   

 

OK, I had go with your program. I used a random data file of
exactly 100M bytes.
>
Runtimes varied from 4.1 to 5 seconds depending on compiler. The
fastest time was with gcc -O3.   

 

>
It sounds like your mass storage device is much slower than aging
SSD on my test machine and ALOT slower than SSD of David Brown.
 

>
>
My machine uses an SSD. 

>
SSDs are not created equal. Especially for writes.
 

>
However the tests were run on Windows, so I ran your program again
under WSL; now it took 14 seconds (using both gcc-O3 and gcc-O2).
>
 

>
3 times slower ?!
I never tested it myself, but I heard that there is a significant
difference in file access speed between WSL's own file system  and
mounted Windows directories. The difference under WSL is not as big
as under WSL2 where they say that access of mounted Windows
filesystem is very slow, but still significant.
I don't know if it applies to all file sizes or only to accessing
many small files. 

>
WSL uses containers, <snip>

 
It seems, you are discussing a speed of access and methods of access
from the host side. My question is opposite - is access from Linux
guest to Windows host files running at the same speed as Linux (WSL,
not WSL2) guest to its own file system?
I heard that it isn't, but it was not conclusive and with insufficient
details. I am going to test our specific case of big files. Now.

WSL Ubuntu20.04 version 2

This is a test of reaching Windows mounts from Linux, the C: drive in particular.
C: is stored on a 3.5GB/sec NVMe in this case. At the very least, this
path is a non-paravirtualized path, just an ordinary hypervisor passing
sectors through.

user@MACHINE:/mnt/c/Users/user/AppData/Local/Packages/
                CanonicalGroupLimited.Ubuntu20.04onWindows_79rhkp1fndgsc/LocalState$
                   time dd if=ext4.vhdx of=/dev/null bs=1048576
4769+0 records in
4769+0 records out
5000658944 bytes (5.0 GB, 4.7 GiB) copied, 20.2495 s, 247 MB/s

real    0m20.268s
user    0m0.000s
sys     0m0.395s

*******

Whereas this is a test of the container ext4.vhdx and materials inside it.
If this is paravirtualization, it's damn fast. The NVMe only does 3.5GB/sec
(and of course that depends on the PCIe buffer size in the hub, where early
PCIe hardwares only ran at 50% link rate due to "tiny buffers"). The test
file is random numbers, in order that "real space" be taken up in the
container ext4.vhdx .

user@MACHINE:/home/user/Downloads$

  219  echo 3 | sudo tee /proc/sys/vm/drop_caches
  220  top
  221  history
user@MACHINE:/home/user/Downloads$ time dd if=random.bin of=/dev/null bs=1048576
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 0.548747 s, 2.0 GB/s

real    0m0.560s
user    0m0.000s
sys     0m0.301s
user@MACHINE:/home/user/Downloads$

Notice I dropped my caches (which on modern Linux is mostly a waste
of time, as there still seem to be caches in there -- benching and
eliminating caches is tough to do now). I verified in "top", that
the command took, and some cached material dropped. But it made
no difference at all to the test results.

Conclusion:  If you're working your 100MB file in ~/Downloads in WSL, things
             should be damn fast, as fast as the media can manage in the
             case of hard drives. Whereas accessing /mnt/c is not nearly as fast.

Note: In the above, the container is being used two ways. In the first test, the
      "outside" of the container is being sampled via a /mnt/c mount. The second test
      is materials inside the container, coming through a different software stack
      in the hypervisor supporting this stuff. WSL is HyperV just as VirtualBox on
      Windows is now forced to be a HyperV client. So whatever properties are exposed
      above, some of them are traceable to HyperV, and might show up if you used
      HyperV for hosting a Linux guest instead.

      There are no block diagrams of the WSL era, the built-in Linux kernel or whatever,
      that I can find on learn.microsoft.com . Only an earlier diagram, before virtualization
      was used in earnest, is available. The "main OS" on your computer, is virtualized.
      It's not actually physical. That's because HyperV is an inverted hypervisor and not
      a conventional one. Can I prove that ? Of course not. Just the single diagram
      I've seen, hints at it. The Windows 11 Task Manager is "trash", to put it lightly,
      and is now an abomination. Worthless. It makes you wonder what the staff at MS use
      for visualizing what is going on. I use the Kill-O-Watt power meter connected to this
      PC, to tell me whether anything is going on under the hood, or whether it is truly idle.

   Paul