WingNews logo WingNews GitHub [2]
top | item 33192152

Digging into a QEMU problem of slow data copying

190 points| todsacerdoti | 3 years ago |linus.schreibt.jetzt

31 comments

order
[+] pabs3|3 years ago|reply
For those who haven't used patches via email before and want to contribute to Linux/qemu/etc that use that, there is a neat site about how to set it up and do it:

https://git-send-email.io/

[+] voidwtf|3 years ago|reply
The idea of using emails for patch management fills me with abject horror.

To each their own, I just don’t understand why anyone would prefer this method.

[+] stabbles|3 years ago|reply
> I opted for a hash table, as conveniently implemented by glib, which QEMU already depends on

I wonder how many C projects use linear scans instead of hash maps simply because C doesn't have it in its standard library.

[+] tubler|3 years ago|reply
I find it odd how all the innovations in getting broadly useful algorithms into a standard library happened in C++ but not C.

It almost feels like the people who keep churning out new standards of C just gave up and decided that part of it couldn't be improved any further.

[+] junon|3 years ago|reply
I've certainly run into this a non-trivial amount of times myself. It's the one thing I'm really missing in C.
[+] MontyCarloHall|3 years ago|reply
I frankly don't understand why projects [*] are still written in pure C. With the exception of a few edge cases, C is pretty much a subset of C++, so you can easily write code that's 100% C, except for when useful data structures from the STL are required. In this specific case, what is the disadvantage of writing C code that pulls in std::unordered_map when a hash table is needed, but otherwise does not use any other C++ language features?

[*] aside from those targeting niche, embedded systems for which C++ is unavailable

[+] tracnar|3 years ago|reply
Nice fix. I wonder if using virtiofs would be more performant: https://virtio-fs.gitlab.io/
[+] linuxhackerman|3 years ago|reply
I've been planning to implement NixOS tests based on virtiofs instead of 9p for a while, but haven't got it working yet (at the time I tried, the available uid mapping strategies didn't cover what was needed for NixOS tests; there's a patch which I haven't got around to trying out yet though.) and this big improvement takes the pain away a bit so my motivation to finish that off is lower. I'll be sure to write about it if I do though!
[+] rwmj|3 years ago|reply
All the development attention is on virtiofs these days and 9pfs is basically deprecated. QEMU developers would certainly recommend switching.
[+] nezirus|3 years ago|reply
Nice story! It also matches my experience with many open source projects. It is not enough to solve a technical problem, you must convince other people to accept the solution.

Having a nice maintainer to guide you through the process is really encouraging and brings more nice and bright people to the comunity. Kudos

[+] ozzy6009|3 years ago|reply
This is awesome, thanks for sharing so much detail. Btw one thing I noticed, the link to "Determinate Systems" was broken. I think this is the correct link: https://determinate.systems/posts/qemu-fix
[+] linuxhackerman|3 years ago|reply
Great job, me. I fixed it in the source but didn't deploy it to the site. Thanks for pointing it out!
[+] lithiumii|3 years ago|reply
I wonder if this will fix the slow IO in WSL2 reading from Windows (/mnt/c), since WSL2 also uses 9p
[+] spijdar|3 years ago|reply
No, since WSL2 is based on the Windows hypervisor/emulator Hyper-V, not QEMU/KVM.
[+] idiocrat|3 years ago|reply
From commit: > The previous implementation would iterate over the fid table for lookup operations, resulting in an operation with O(n) complexity on the number of open files and poor cache locality -- for every open, stat, read, write, etc operation.

Windows's file handling also is very slow at times (especially Windows Explorer has frustrating UX).

We can ask MS to release all their source and we will then fix all O(n) issues for them.

[+] anderspitman|3 years ago|reply
I wonder if this is using slirp networking or a TAP device.
[+] linuxhackerman|3 years ago|reply
It doesn't actually go through a "real network" at all, the VM speaks 9p to the host via a virtio pipe (not sure what the technically correct term for this is) so there's none of the overhead that you'd have with emulating a network device and encapsulating it all in Ethernet/IP/TCP.
[+] tinus_hn|3 years ago|reply
Doctors don’t want you to know this one trick to make me 10 times less likely to avoid your article: don’t use these cringey clickbait titles.
[+] dang|3 years ago|reply
It's clearly a good article otherwise. We've changed the title now—if there's a more representative phrase from the article itself, we can change it again.

Submitters: "Please use the original title, unless it is misleading or linkbait;" - this one was linkbait, so should not have been used. https://news.ycombinator.com/newsguidelines.html