We're a somewhat popular hosting provider that runs Docker containers (as VMs) for our customers and does private networking over IPv6, which expands the size of our DNS requests, and we run into this all the time with Alpine. It's kind of baffling.
TCP DNS is not hard. It's part of the spec. Normally, that argument doesn't mean much to me --- lots of things are parts of specs that I think are silly and not worth doing --- but TCP DNS seems like a basic necessity for DNS to work at all.
What's holding this up? TCP DNS is just UDP DNS, but over a TCP connection, with the packet length sent before the packet itself. It's the simplest thing you could possibly come up with to make TCP DNS work. It's been there since the 1980s. They should add it.
This document updates RFCs 1123 and 1536. This document requires the
operational practice of permitting DNS messages to be carried over
TCP on the Internet as a Best Current Practice.
I mean sure, musl should add TCP DNS. But real world networking says if your DNS response is over the limit for basic UDP DNS, a meaningful proportion of clients won't get that response. Which I guess is okish if it's a TXT for mail servers, cause mail servers probably have a reasonable DNS setup; but if you wanted to return a lot of A records or AAAA records (or A records that a provider DNS64s into AAAA records), you need to be careful about how many you return, before the response gets too large and doesn't arrive.
This is something networks and DNS client libraries should fix, but we live in a world where PathMTU only mostly works, so realism gets your service working.
The moment I saw Alpine Linux in the title, my first guess was "I bet this is something to do with musl libc". Briefly looking through the blog, it looks like my gut feeling was correct.
A while ago I evaluated Alpine Linux. I wanted to like it, I really did, it ticked so many boxes.
But time and time again, I kept on running into issues with their adoption of musl libc.
The last straw for me was when I discovered packages in their package repo (some of which were well-known names) that were compiled against musl when the upstream developers quite clearly wrote in their docs that "if you compile X against anything other than glibc, you're on your own". For me, the fact that Alpine ignored this and compiled against musl anyway, was a big red flag. (And yes I raised some of these as bug reports, but the cases got closed and nothing done about it).
>The last straw for me was when I discovered packages in their package repo (some of which were well-known names) that were compiled against musl when the upstream developers quite clearly wrote in their docs that "if you compile X against anything other than libc, you're on your own".
(I assume you meant "glibc", not "libc".)
That's how every software works. The software developer cares about A, B, C distros at most, so other distros are on their own. The maintainer of distro D takes responsibility themselves to make the package work on D. The maintainer needs to understand the software well enough to be able to assert that it will work on their distro, patch it as necessary to make that happen, and maintain those patches in the light of bug reports from the distro users.
>(And yes I raised some of these as bug reports, but the cases got closed and nothing done about it).
Well, yeah. Unless you find something that is irrecoverably broken against musl such that it can only be fixed by compiling against glibc, your bug report is pointless.
> - Increasing the size of the UPD packet above 512 bytes via the Extension Mechanism for DNS (EDNS)
> - Switching the protocol from UDP to TCP
> Alpine Linux, or rather musl libc, doesn’t support either of those options.
It still seems weird to me that such details are decided by libc. My reflex idea when designing a system would be to put DNS functionality in a system service, while libraries would only query the service, without troubling themselves with system caches, TCP vs UDP etc. Then possibly the service could be even swapped for another with a compatible interface, but making different decisions, without perturbing the applications. It sounds like systemd-resolved is a move in that direction, but I still don't understand why putting all that in libc, essentially making all applications perform their own independent DNS work, was the original choice.
Few days ago, I spent quite a few hours trying to make `apk update` work for alpine on WSL2 on Windows. It didn't want to resolve dl-cdn.alpinelinux.org within alpine. Did resolve on host ubuntu.
In my experience, the storage, bandwidth and time savings using Alpine Linux (even if they were much more significant than they are in practice) are not worth it given the issues you run into every once in a while. Just go with Ubuntu/Debian base images and you'll be much happier that you did in the long run.
[+] [-] tptacek|4 years ago|reply
TCP DNS is not hard. It's part of the spec. Normally, that argument doesn't mean much to me --- lots of things are parts of specs that I think are silly and not worth doing --- but TCP DNS seems like a basic necessity for DNS to work at all.
What's holding this up? TCP DNS is just UDP DNS, but over a TCP connection, with the packet length sent before the packet itself. It's the simplest thing you could possibly come up with to make TCP DNS work. It's been there since the 1980s. They should add it.
[+] [-] AndyMcConachie|4 years ago|reply
https://datatracker.ietf.org/doc/rfc9210/
[+] [-] stonogo|4 years ago|reply
[+] [-] nwmcsween|4 years ago|reply
[+] [-] toast0|4 years ago|reply
This is something networks and DNS client libraries should fix, but we live in a world where PathMTU only mostly works, so realism gets your service working.
[+] [-] traceroute66|4 years ago|reply
A while ago I evaluated Alpine Linux. I wanted to like it, I really did, it ticked so many boxes.
But time and time again, I kept on running into issues with their adoption of musl libc.
The last straw for me was when I discovered packages in their package repo (some of which were well-known names) that were compiled against musl when the upstream developers quite clearly wrote in their docs that "if you compile X against anything other than glibc, you're on your own". For me, the fact that Alpine ignored this and compiled against musl anyway, was a big red flag. (And yes I raised some of these as bug reports, but the cases got closed and nothing done about it).
[+] [-] Arnavion|4 years ago|reply
(I assume you meant "glibc", not "libc".)
That's how every software works. The software developer cares about A, B, C distros at most, so other distros are on their own. The maintainer of distro D takes responsibility themselves to make the package work on D. The maintainer needs to understand the software well enough to be able to assert that it will work on their distro, patch it as necessary to make that happen, and maintain those patches in the light of bug reports from the distro users.
>(And yes I raised some of these as bug reports, but the cases got closed and nothing done about it).
Well, yeah. Unless you find something that is irrecoverably broken against musl such that it can only be fixed by compiling against glibc, your bug report is pointless.
[+] [-] blueflow|4 years ago|reply
Still, Alpine is on top of everyone else when it comes to Docker Images sizes. Thats why it will stick.
[+] [-] amelius|4 years ago|reply
[+] [-] yakubin|4 years ago|reply
> - Increasing the size of the UPD packet above 512 bytes via the Extension Mechanism for DNS (EDNS)
> - Switching the protocol from UDP to TCP
> Alpine Linux, or rather musl libc, doesn’t support either of those options.
It still seems weird to me that such details are decided by libc. My reflex idea when designing a system would be to put DNS functionality in a system service, while libraries would only query the service, without troubling themselves with system caches, TCP vs UDP etc. Then possibly the service could be even swapped for another with a compatible interface, but making different decisions, without perturbing the applications. It sounds like systemd-resolved is a move in that direction, but I still don't understand why putting all that in libc, essentially making all applications perform their own independent DNS work, was the original choice.
[+] [-] brimstedt|4 years ago|reply
First time it took a lot of effort to pinpoint the problem.
Second time too, since it appeared because of a non-relevant code change (which lead to slighty more DNS requests).
In both cases, a simple switch to Debian slim saved the day.
Alpine is since banned from any env I'm working in :-)
[+] [-] h1fra|4 years ago|reply
dnsConfig: options: - name: ndots value: '1'
cc: https://support.cloudbees.com/hc/en-us/articles/360040999471...
There are also plenty of dormant issue, enough so that I won't be using Alpine ever again imo :'(
[+] [-] jve|4 years ago|reply
1. WFH from VPN, firstly I had to lower mtu from 1500 to 1392 (My VPN specific issue) https://github.com/microsoft/WSL/issues/4698
2. Next, I had to run some powershell script that updates /etc/resolv.conf to use my VPN DNS (WSL specific stuff) https://github.com/microsoft/WSL/issues/1350
3. And I still don't know if apk works properly. Kind of works in Docker build, but I have a feeling something not quite right.
See this example. Why does it "hang"? Docker command not exiting
Now, doing it within container itself, works: Can someone shed some light?[+] [-] newman314|4 years ago|reply
[+] [-] rascul|4 years ago|reply
The solution to an intentionally broken resolver is to use a third party library.
[+] [-] blueflow|4 years ago|reply
[+] [-] johnklos|4 years ago|reply
• musl should support EDNS and DNS over TCP/IP without issues
• People should be smart enough to use DNS services that don't have stupid edge cases
For the latter, if you use Google for resolving DNS, you get what you deserve. Run your own resolver if DNS resolution matters.
[+] [-] cpach|4 years ago|reply
[+] [-] richardfey|4 years ago|reply
[+] [-] ComradePhil|4 years ago|reply
[+] [-] unknown|4 years ago|reply
[deleted]