Funny enough, I decided to play with FreeBSD for personal projects in 2020. I gave up and I am reverting all my servers to Linux in 2022, for the opposite of the reasons mentioned in this article.
* Lack of systemd. Managing services through shell scripts is outdated to me. It feels very hacky, there is no way to specify dependencies, and auto-restarts in case of crashes. Many FreeBSD devs praise launchd, well... systemd is a clone of launchd.
* FreeBSD jail are sub-optimal compared to systemd-nspawn. There are tons of tools to create freebsd jails (manually, ezjail, bastillebsd, etc…) half of them are deprecated. At the end all of your jails end up on the same loopback interface, making it hard to firewall. I couldn't find a way to have one network interface per jail. With Linux, debootstrap + machinectl and you're good to go.
* Lack of security modules (such as SELinux) -- Edit: I should have written "Lack of good security module"
* nftables is way easier to grasp than pf, and as fast as pf, and has atomic reloads.
I've grown to love systemd. It solves a lot of my problems with init scripts, particularly the ones involving environment / PATH at boot time. I've made init scripts for things before which work when they are invoked manually, but not at boot time because PATH was different. With systemd I am confident that if it works through systemctl it will work at boot.
Maybe I am not tuning Linux appropriately, but I have been in situations where a Linux system is overwhelmed / overloaded and I am unable to ssh to it. I have never had that experience with FreeBSD -- somehow ssh is always quick and responsive even if the OS is out of memory.
Most of the systems that I deal with are Linux, but I still have a few FreeBSD systems around and they are extraordinarily stable.
My boss graduated from Berkely, and so I occasionally had to administer a FreeBSD box he kept around for the "superior OS philosophy".
My biggest annoyance, apart from the obvious lack of systemd, was a social one: Any time I had to look up how to accomplish some non-trivial task I would inevitably find a thread on the FreeBSD forums by someone else who has had exactly the same problem in the past, together with a response by some dev along the lines of "why would anyone ever need to do that?".
> At the end all of your jails end up on the same loopback interface, making it hard to firewall. I couldn't find a way to have one network interface per jail.
You may want to look at vnet, which gives jails their own networking stack; then you can give interfaces to the jail. If you use ipfw instead of pf, jail id/name is a matchable attribute on firewall rules; although it's not perfect, IIRC I couldn't get incoming SYNs to match by jail id, but you can match the rest of the packets for the connection. And that brings up the three firewalls of FreeBSD debate; maybe you had already picked pf because it met a need you couldn't (easily) meet with ipfw; you can run both simultaneously, but I wouldn't recommend it. Nobody seems to run ipf, though.
Edit: you may also just want to limit each jail to a specific IP address, and then it's easy to firewall.
> Many FreeBSD devs praise launchd, well... systemd is a clone of launchd.
No they are not. Systemd has a way bigger scope then launchd. It's like saying a truck is the same thing as a family car because they both solve the mobility problem.
> FreeBSD jail are sub-optimal compared to systemd-nspawn.
systemd-nspawn isn't a container. For example it doesn't manage resources such as CPU or IO. Again, the scope is way different and in this case there is a whole slew of things systemd-nspawn isn't going to manage for you.
BTW launchd doesn't have a feature like 'systemd-nspawn'.
> Lack of security modules (such as SELinux) -- Edit: I should have written "Lack of good security module"
And how is SELinux a good security module? SELinux with it's design fell in the same pitfall dozens of security systems did before it; ACL-Hell, Role-hell and now with SELinux we also have Label-hell.
> Lack of systemd. Managing services through shell scripts is outdated to me. It feels very hacky, there is no way to specify dependencies, and auto-restarts in case of crashes. Many FreeBSD devs praise launchd, well... systemd is a clone of launchd.
I'm not a fan of systemd personally but I do understand it has some good parts to it (such as the ones you've listed). That all said, you can still specify dependencies in FreeBSD with the existing init daemon. Albeit it's a little hacky compared with systemd (comments at the top of the script). But it does work.
> FreeBSD jail are sub-optimal compared to systemd-nspawn. There are tons of tools to create freebsd jails (manually, ezjail, bastillebsd, etc…) half of them are deprecated. At the end all of your jails end up on the same loopback interface, making it hard to firewall. I couldn't find a way to have one network interface per jail. With Linux, debootstrap + machinectl and you're good to go.
It's definitely possible to have one interface per jail, I've done exactly that. However back when I last did it (5+ years ago?) you needed to manually edit the networking config and jail config to do it. There might be a more "linuxy" way to do this with Jails now though but its definitely possible. eg https://etherealwake.com/2021/08/freebsd-jail-networking/
There's a lot to unpack here. For example, there's certainly other ways to network jails and all three ways you've mentioned to maintain jails are not deprecated.
Security modules do exist, they're different from Linux. Are you sure you're not just expecting FreeBSD-as-Linux?
As for init... What can I say, I've never been anti-systemd, not even remotely, but rc.d is much nicer than sysvinit, and I find it much simpler to understand than systemd. In fact, I think rc.d is an example of how Linux could have alternatively migrated from sysvinit without pissing some people off.
“Managing services through shell scripts is outdated.” By inference, since most things in Unix like systems are built on the notion of shell (and automation using the shell), this is saying large part of the foundation of Unix is outdated.
A tool is a tool, but I would take a shell script from rc.d any time over a binary blob from systemd.
sort of. Systemd took a lot of lessons from launchd but also from sysv and upstart. For anyone who hasn't read Lennart Poettering's "Rethinking PID 1" post[1] I highly recommend it. You'll understand the history, but most importantly you'll understand systemd a ton better.
>At the end all of your jails end up on the same loopback interface, making it hard to firewall. I couldn't find a way to have one network interface per jail.
FreeBSD jails has quite a bit of history (developed in 1998) and made public in the year 2000.
FreeBSD has had VNET for use with jails allowing you to add epair interfaces connected to a bridge, or add a physical interface or VLAN to a jail. This feature has been in FreeBSD since 8.0 (2009) and enabled by default since 2018. It also allows you to run PF in each jail.
Sadly many people a bitten by outdated forum and blog posts when it comes to jails.
I agree that firewalling jails with loopbacks is a pain, but most people don't do it that way anymore.
> At the end all of your jails end up on the same loopback interface, making it hard to firewall.
I suppose you didn't use vnet? It's a vastly better jail networking experience. You can pretend jails are separate machines, connected via ethernet. I don't think anyone who knows about vnet chooses not to use it!
> I couldn't find a way to have one network interface per jail.
I think vnet is what you want.
> nftables is way easier to grasp than pf, and as fast as pf, and has atomic reloads.
Systemd has only one advantage, which is also it's prime disadvantage: It's paws are everywhere in your system. Before there was Systemd, init systems worked okay too - in that space nothing was changed by it.
1.5M lines of code is not an init system, it is a time bomb... and with dodgy wiring.
I think it was a ZFS dev who complained he had to update 150 files to port ZFS to systemd. Simples?
And you have to love all them binary log files, with their dynamic and spontaneus api. Yes, everything is always new and exciting with systemD.
I use devuan. A debian fork with a choice of init systems (well, everything but systemD. ;) It took the dev/devs 2 years just to swap out/revert the init system.
Edit: I will concede that systemD is great for a lot of people. I honestly wish them success. The work that goes in to building, maintaining, and using it, is substantial. It must be a boon for the economy and job creation.
I like how userland and the kernel are sync in freebsd. I disdain how linux works in that. However Linux has its core advantages, some distos are well suited and tested for certain scenarios. I used to have a bunch of freebsd, but nowadays, I use containers, light weight runtimes. With Lambdas and Serverless, most, but not all, business api are streamed line where servers matter not. Its the runtime.
Serverless is killing containers. Its killing the need to care about if its freebsd or linux, does it run my api fast enough?
The reasons are, for a large part, not on the technical side. I was surprised, because this this a lot of work for little visible gain. Here are the reasons, slightly abbreviated:
> The whole system is managed by the same team
Mostly philosophical.
> FreeBSD development is less driven by commercial interests.
Mostly philosophical.
> Linux has Docker but FreeBSD has jails!
IMO, this comparison is a mistake. In the Linux world, systemd's nspawn is very similar to Jails. It's a glorified chroot, with security and resource management. All the systemd tools work seemlessly with nspawn machines (e.g. `systemctl status`). Containers à la Docker are a different thing.
BTW, I thought the last sentence about security issues with Docker images was strange. If you care about unmaintained images, build them yourself. On the other side, the FreeBSD official documentation about Jails has a big warning that starts with "Important: the official Jails are a powerful tool, but they are not a security panacea."
> Linux has no official support for zfs and such
Fair point, though I've heard about production systems with zfs on Linux.
> The FreeBSD boot procedure is better than grub.
YMMV
> FreeBSD's network is more performant.
Is there some conclusive recent benchmark about this. The post uses a 2014 post about ipv6 at Facebook, which I think is far from definitive today. Especially more since it "forgot" to mention that Facebook intended to enhance the "Linux kernel network stack to rival or exceed that of FreeBSD." Did they succeed over these 8 years ?
> Straightforward system performance analysis
The point is not about the quality of the tools, but the way each distribution packages them. Seems very very low impact to me.
> FreeBSD's Bhyve against Linux's KVM
The author reluctantly admits that KVM is more mature.
I have essentially the same take. The sysadmin at my company prefers FreeBSD for all these reasons (as such that's what we're running), and he's engaged me a tonne about FreeBSD but all I see is an operating system that's just as good as the other mainstream server Linux distributions. Except now we've got a system that's more difficult to hire for. "Any competent admin can learn it easily" is something I've been told but how many will want to when they could easily go their whole career without encountering it again?
I like your point about Docker vs Jails, I haven't seen it discussed like that before. I keep hearing Jails are more secure than anything else, I'll have to read more into it.
As far as the networking goes, I haven't seen any recent benchmarks to substantiate those claims either. However, considering Netflix uses FreeBSD on their edge nodes and has put a lot of work into the upstream to improve networking (among other things), it wouldn't surprise me if it's technically superior to the Linux stack. Clearly though Linux's networking isn't an issue for most organizations.
And regarding ZFS, ZFS on Linux and FreeBSD's ZFS implementation are now one and the same. It would be nice to see some of the big distributions(or even the Linux kernel) integrate it more directly. This is probably a solid point in favor of FreeBSD, but it's not like it doesn't work in Linux. I'm not a systems guy, so I'm probably out of the loop on this, but Proxmox is the only distribution I've seen with ZFS as a filesystem out of the box, but I don't know how much production use Proxmox sees. I only run it on my home server.
All that to basically say, I like FreeBSD conceptually. I'm just still not convinced that it's doing enough things better to warrant using it over a common Linux distribution for general computing purposes.
> Is there some conclusive recent benchmark about this. The post uses a 2014 post about ipv6 at Facebook, which I think is far from definitive today. Especially more since it "forgot" to mention that Facebook intended to enhance the "Linux kernel network stack to rival or exceed that of FreeBSD." Did they succeed over these 8 years ?
I would not be surprised if there still were some cases where FreeBSD could do better. The network stack in particular has a vast amount of performance heuristics and tuning where you can just happen to hit the right or wrong side of some performance jump depending on your exact case, for example.
But to go from there to the network stack is more performant is puzzling. There are certainly cases where the Linux network stack is more performant than FreeBSD -- search for linux vs freebsd network performance and you'll find benchmark tests and complaints and anecdotes around the place including freebsd forums and mailing lists where FreeBSD is slower, and yet FreeBSD proponents would (rightly) say these are not proof Linux's network stack is faster and there might be all sorts of reasons for the differences. So you can't have it both ways.
Netflix uses FreeBSD for some things yes. Not sure if number of big internet corporations using an OS would be a favorable metric for FreeBSD either. Again you can't have it both ways. Netflix says FreeBSD is capable of performing well in that kind of environment, it does not say that it outperforms Linux in general or even in that environment. Any more than Google says the opposite.
> Especially more since it "forgot" to mention that Facebook intended to enhance the "Linux kernel network stack to rival or exceed that of FreeBSD." Did they succeed over these 8 years ?
I don't know about Facebook-driven efforts specifically offhand, but I recall attending a talk at LinuxCon 2015 _specifically_ about optimizing the network stack. Given by a Red Hat employee, IIRC.
"Some time ago we started a complex, continuous and not always linear operation, that is to migrate, where possible, most of the servers (ours and of our customers) from Linux to FreeBSD."
I don't really disagree with any of the stated reasons, but I also didn't see a reason that would make me even consider making the move with our servers, or even bother with some small number of servers. At least for me, I'd need a bunch of REALLY GOOD reasons to consider a move like that. A huge cost savings AND some huge time savings in the future might do it.
I agree that the stated reasons don't sound very compelling. Maybe in aggregate, but not individually.
But they left out one of the bigger reasons, IMHO. FreeBSD doesn't tend to churn user (admin) facing interfaces. This saves you time, because you still use ifconfig to configure interfaces, and you still use netstat to look at network statistics, etc; so you don't have to learn a new tool to do the same thing but differently every couple of years. Sure, there's three firewalls, but they're the same three firewalls since forever.
Some people see the bigger picture and recognize that a medium amount of work now is better than lots of small amounts of work stretched over many years.
Likewise, some people and many businesses see the immediate now and aren't always the best at planning for long term, and/or are overly optimistic that their pain points will eventually be fixed.
The characterisation of systemd in this post really bothers me, particularly this:
> 70 binaries just for initialising and logging
It’s just not true. Those 70 binaries provide much more functionality than an init system, they can cover a significant portion of system management, including a local DNS resolver, network configuration or system time management. You can dislike the fact everything is so tightly integrated (which feels ironic given that the post goes on to praise a user space from one team), but let’s at least be correct about this.
> including a local DNS resolver, network configuration or system time management.
Do. Not. Want.
So, I have this pile of 70 binaries that are inexplicably tied to my init system, and (in my, informed enough for me, opinion) they're all garbage. How do I remove them without breaking init?
This month's fresh hell: I have a problem where the systemd replacement for xscreensaver (logind, maybe? Good luck finding the culprit, let alone the manual!) won't accept my password unless I exit the current X session with "switch user" then restore the session using the normal login screen.
There's a whole section on JWZ's xscreensaver page (from over a decade ago) explaining how to avoid this class of bug, but what does he know?!?
That reminds me; I wonder if a *BSD is a good enough daily driver for the pine book pro yet (it's probably easier to port their kernel than to fix Linux userspace, after all...)
> Linux has Docker, Podman, lxc, lxd, etc. but... FreeBSD has jails!
Docker, podman, lxc, lxd, etc are userland components. Linux has cgroups and namespaces.
FreeBSD jails are a bit more complicated because FreeBSD isn't distributed the way Linux is. Linux is distributed as just the kernel, whereas FreeBSD is a base OS. This probably could've been phrased better as, "Linux has no interest in userland and I want some userland consistency". That's fair, Linux was built around the idea that operating system diversity was a good thing long term, FreeBSD was more interested in consistency. I'm reading between the lines, a bit, here because of the critique of SystemD (note: not all linuxes use SystemD)
Personally speaking, I like both Linuxes and FreeBSD but I don't think debating the two is important. Rather, I'd encourage turning your attention to the fact that every other component on a system runs an OS-like interface that we don't make open OS's or "firmware" for.
> Consider systemd - was there really a need for such a system? While it brought some advantages, it added some complexity to an otherwise extremely simple and functional system. It remains divisive to this day, with many asking, "but was it really necessary? Did the advantages it brought balance the disadvantages?"
This is really telling for the level of analysis done: systemd has been the target from a small number of vocal complainers but most working sysadmins only notice it in that they routinely deal with tasks which are now a couple of systemd stanzas instead of having to cobble together some combination of shell scripts and third-party utilities. Confusing noise with numbers is a dangerous mistake here because almost nobody sits around randomly saying “this works well”.
Linux took many markets. The HPC, for example, has been 100% linux in TOP500 for a few years already. Monopoly by FLOSS is still monopoly. Healthy competition is good for users and forces options to improve, see LLVM vs GCC.
To sum up: healthy FLOSS competition is welcome and needed.
> FreeBSD's network stack is (still) superior to Linux's - and, often, so is its performance.
Where is this coming from exactly? The linked article about Facebook is 7 years old. The following benchmark shows the exact opposite: Linux's network stack has long surpassed FreeBSD's. And I would expect nothing else given the amount of work that has gone into Linux compared to FreeBSD.
I ran FreeBSD servers for about a decade. Now all my servers are Linux with systemd. I'm liked FreeBSD then, I'm happy with systemd now. I have commits in both.
I'm glad there are some people who use and prefer FreeBSD and other init system now, because diversity in digital ecosystems is benefits the whole just as diversity in natural ecosystems do.
The shot taking at systemd here was disingenuous though. The author complained about the number of different systemd binaries and the lines of source code, but all these tools provide a highly consistent "system layer" with standardized conventions and high quality documentation-- it's essentially the same argument made to support FreeBSD as a large body of kernel and userspace code that's maintained in harmony.
It feels like the title is wrong. Instead of saying "Linux is bad because I encountered X problem in production, which would have been prevented by BSD" the author goes on to list why BSD is better in general outside his specific use case.
Nothing wrong with the comparison probably, but I got the impression the author just really wanted to do the migration and found some reasons to do so, without actually needing it. Nothing wrong with that as well. It's just the expectations set by the title that are off
>There is controversy about Docker not running on FreeBSD but I believe (like many others) that FreeBSD has a more powerful tool. Jails are older and more mature - and by far - than any containerization solution on Linux.
If FreeBSD jails and Solaris zones were equivalent to Linux containers, we'd have seen them take over the backend already. We haven't. They're really useful, they provided a degree of safety and peace of mind for multi-tenancy but they're not granular enough for what's done with $CONTAINER_RUNTIME these days.
I ran a FreeBSD ZFS NFS server for a cluster for quite a while. I loved it. It was simple and stable. The thing that led me away from FreeBSD (aside from IT not being happy with an "alternative" OS), was that I needed a clustered filesystem. We outgrew the stage where I was comfortable with a single node and where upgrading storage meant a new JBOD.
Are there any FreeBSD-centric answers to Ceph or Gluster or Lustre or BeeGFS?
I don't have enough experience with FreeBSD (outside of FreeNAS seven years ago), but I've never had any success getting it to run on a laptop. Every time I've tried installing it on a laptop I get issues with either the WiFi card not working, issues with the 3D accelerator card not working at all, or the lid-close go to sleep functionality not working.
I've been using Linux since I was a teenager, so it's not like I am a stranger to fixing driver issues, but it seemed like no amount of Googling was good enough for me fix these problems (googling is much harder when you don't have functioning wifi). As a result I've always just stuck with Linux (or macOS semi-recently, which I suppose is kind of BSD?).
This articulates most of my frustrations with the Linux world.
Some of the distros are very good, but some of us who have work to do cringe at the thought of bringing up newer versions of an OS just to check all the things that've broken and changed needlessly.
“ The system is consistent - kernel and userland are created and managed by the same team”
Their first reason is really saying a lot but with few words. For one, there’s no systemd. The init system is maintained alongside the entire rest of the system which adds a lot of consistency. The documentation for FreeBSD is also almost always accurate and standard. Etc etc
I think you also largely don’t need a docker or etc in it since jails have been native to the OS for decades. I’d want to do some cross comparison first though before committing to that statement.
Shouldn’t be lost that the licensing is also much friendlier to business uses. There’s afaik no equivalent to rhel, for that matter. This goes both ways though as how would you hire a FreeBSD admin based on their resume without a rhce-like FreeBSD certification program?
Edit-I’ll posit that since FreeBSD is smaller an entity wishing to add features to the OS might face either less backlash or at least enjoy more visibility from the top developers of the OS. Linus, for instance, just has a larger list of entities vying for his attention on issues and commits.
To be fair all of these reasons come down to personal preference (sans the TCP performance claim). E.g. he prefers FreeBSD’s performance monitoring tools to Linux’s monitoring tools, or he prefers FreeBSD’s user land to Linux’s user land. That’s fine but it’s not very persuasive.
Things I actually care about:
kernel that supports my hardware, ZFS for secure snapshotted data, scriptable tools to manage NICs and ppp and VPNs, a fast optimised C++ compiler, the latest versions of dynamic language runtimes, a shell, a text editor, a terminal multiplexer, a nerdy window manager and an evergreen browser.
On that playing field, the integrated nature of FreeBSD is nice but it’s an asterisk on top of the kernel rather than anything approaching what makes up a the system part of an Operating System. Almost everything else comes from a third party (and I’m fine with that.)
I haven’t used FreeBSD as a daily OS for over a decade though. What’s the new coolness?
"Btrfs is great in its intentions but still not as stable as it should be after all these years of development." may have been true years ago, but doesn't seem to be anymore.
The key point is not Linux vs FreeBSD. It is simply choice. You have a real choice. Do it this way or that - do it your way. I like both Linux and FreeBSD but I deploy them differently.
I slap Linux on my servers and desktops and I deploy FreeBSD via pfSense on firewalls.
Sometimes I do experiments and try out BSD on the desktop which hasn't worked out yet for me but I live in hope because I adore *BSD as much as I do Linux.
If BSD is the way to get your servers to do what you want then lovely. Do it and remember you have choice.
[+] [-] acatton|4 years ago|reply
* Lack of systemd. Managing services through shell scripts is outdated to me. It feels very hacky, there is no way to specify dependencies, and auto-restarts in case of crashes. Many FreeBSD devs praise launchd, well... systemd is a clone of launchd.
* FreeBSD jail are sub-optimal compared to systemd-nspawn. There are tons of tools to create freebsd jails (manually, ezjail, bastillebsd, etc…) half of them are deprecated. At the end all of your jails end up on the same loopback interface, making it hard to firewall. I couldn't find a way to have one network interface per jail. With Linux, debootstrap + machinectl and you're good to go.
* Lack of security modules (such as SELinux) -- Edit: I should have written "Lack of good security module"
* nftables is way easier to grasp than pf, and as fast as pf, and has atomic reloads.
[+] [-] fullstop|4 years ago|reply
Maybe I am not tuning Linux appropriately, but I have been in situations where a Linux system is overwhelmed / overloaded and I am unable to ssh to it. I have never had that experience with FreeBSD -- somehow ssh is always quick and responsive even if the OS is out of memory.
Most of the systems that I deal with are Linux, but I still have a few FreeBSD systems around and they are extraordinarily stable.
[+] [-] HotHotLava|4 years ago|reply
My biggest annoyance, apart from the obvious lack of systemd, was a social one: Any time I had to look up how to accomplish some non-trivial task I would inevitably find a thread on the FreeBSD forums by someone else who has had exactly the same problem in the past, together with a response by some dev along the lines of "why would anyone ever need to do that?".
[+] [-] toast0|4 years ago|reply
You may want to look at vnet, which gives jails their own networking stack; then you can give interfaces to the jail. If you use ipfw instead of pf, jail id/name is a matchable attribute on firewall rules; although it's not perfect, IIRC I couldn't get incoming SYNs to match by jail id, but you can match the rest of the packets for the connection. And that brings up the three firewalls of FreeBSD debate; maybe you had already picked pf because it met a need you couldn't (easily) meet with ipfw; you can run both simultaneously, but I wouldn't recommend it. Nobody seems to run ipf, though.
Edit: you may also just want to limit each jail to a specific IP address, and then it's easy to firewall.
[+] [-] jsiepkes|4 years ago|reply
No they are not. Systemd has a way bigger scope then launchd. It's like saying a truck is the same thing as a family car because they both solve the mobility problem.
> FreeBSD jail are sub-optimal compared to systemd-nspawn.
systemd-nspawn isn't a container. For example it doesn't manage resources such as CPU or IO. Again, the scope is way different and in this case there is a whole slew of things systemd-nspawn isn't going to manage for you.
BTW launchd doesn't have a feature like 'systemd-nspawn'.
> Lack of security modules (such as SELinux) -- Edit: I should have written "Lack of good security module"
And how is SELinux a good security module? SELinux with it's design fell in the same pitfall dozens of security systems did before it; ACL-Hell, Role-hell and now with SELinux we also have Label-hell.
[+] [-] traceroute66|4 years ago|reply
This, this and this again !
After decades of cron, I discovered systemd timers via a passing comment I read here on HN.
My god is it amazing. No more hacky work-arounds in my scripts, systemd now takes care of all the magic such as random timings etc.
I'll never go back to cron.
[+] [-] hnlmorg|4 years ago|reply
I'm not a fan of systemd personally but I do understand it has some good parts to it (such as the ones you've listed). That all said, you can still specify dependencies in FreeBSD with the existing init daemon. Albeit it's a little hacky compared with systemd (comments at the top of the script). But it does work.
> FreeBSD jail are sub-optimal compared to systemd-nspawn. There are tons of tools to create freebsd jails (manually, ezjail, bastillebsd, etc…) half of them are deprecated. At the end all of your jails end up on the same loopback interface, making it hard to firewall. I couldn't find a way to have one network interface per jail. With Linux, debootstrap + machinectl and you're good to go.
It's definitely possible to have one interface per jail, I've done exactly that. However back when I last did it (5+ years ago?) you needed to manually edit the networking config and jail config to do it. There might be a more "linuxy" way to do this with Jails now though but its definitely possible. eg https://etherealwake.com/2021/08/freebsd-jail-networking/
[+] [-] lordgroff|4 years ago|reply
Security modules do exist, they're different from Linux. Are you sure you're not just expecting FreeBSD-as-Linux?
As for init... What can I say, I've never been anti-systemd, not even remotely, but rc.d is much nicer than sysvinit, and I find it much simpler to understand than systemd. In fact, I think rc.d is an example of how Linux could have alternatively migrated from sysvinit without pissing some people off.
[+] [-] throw0101a|4 years ago|reply
* https://www.freebsd.org/cgi/man.cgi?query=rcorder
[+] [-] hestefisk|4 years ago|reply
[+] [-] freedomben|4 years ago|reply
sort of. Systemd took a lot of lessons from launchd but also from sysv and upstart. For anyone who hasn't read Lennart Poettering's "Rethinking PID 1" post[1] I highly recommend it. You'll understand the history, but most importantly you'll understand systemd a ton better.
[1]: http://0pointer.de/blog/projects/systemd.html
[+] [-] mortenlarsen|4 years ago|reply
FreeBSD jails has quite a bit of history (developed in 1998) and made public in the year 2000. FreeBSD has had VNET for use with jails allowing you to add epair interfaces connected to a bridge, or add a physical interface or VLAN to a jail. This feature has been in FreeBSD since 8.0 (2009) and enabled by default since 2018. It also allows you to run PF in each jail.
Sadly many people a bitten by outdated forum and blog posts when it comes to jails.
I agree that firewalling jails with loopbacks is a pain, but most people don't do it that way anymore.
Also PF has always had atomic reloads.
[+] [-] SpaceInvader|4 years ago|reply
[+] [-] wyager|4 years ago|reply
I suppose you didn't use vnet? It's a vastly better jail networking experience. You can pretend jails are separate machines, connected via ethernet. I don't think anyone who knows about vnet chooses not to use it!
> I couldn't find a way to have one network interface per jail.
I think vnet is what you want.
> nftables is way easier to grasp than pf, and as fast as pf, and has atomic reloads.
pfctl allows you to do atomic reloads. `pfctl -f`
[+] [-] krageon|4 years ago|reply
[+] [-] irthomasthomas|4 years ago|reply
I think it was a ZFS dev who complained he had to update 150 files to port ZFS to systemd. Simples?
And you have to love all them binary log files, with their dynamic and spontaneus api. Yes, everything is always new and exciting with systemD.
I use devuan. A debian fork with a choice of init systems (well, everything but systemD. ;) It took the dev/devs 2 years just to swap out/revert the init system.
Also, just learned devuan's website is JS and cookie free. https://www.devuan.org/
Edit: I will concede that systemD is great for a lot of people. I honestly wish them success. The work that goes in to building, maintaining, and using it, is substantial. It must be a boon for the economy and job creation.
[+] [-] JediPig|4 years ago|reply
Serverless is killing containers. Its killing the need to care about if its freebsd or linux, does it run my api fast enough?
[+] [-] hestefisk|4 years ago|reply
[+] [-] yakaccount4|4 years ago|reply
[+] [-] DarylZero|4 years ago|reply
[deleted]
[+] [-] idoubtit|4 years ago|reply
> The whole system is managed by the same team
Mostly philosophical.
> FreeBSD development is less driven by commercial interests.
Mostly philosophical.
> Linux has Docker but FreeBSD has jails!
IMO, this comparison is a mistake. In the Linux world, systemd's nspawn is very similar to Jails. It's a glorified chroot, with security and resource management. All the systemd tools work seemlessly with nspawn machines (e.g. `systemctl status`). Containers à la Docker are a different thing.
BTW, I thought the last sentence about security issues with Docker images was strange. If you care about unmaintained images, build them yourself. On the other side, the FreeBSD official documentation about Jails has a big warning that starts with "Important: the official Jails are a powerful tool, but they are not a security panacea."
> Linux has no official support for zfs and such
Fair point, though I've heard about production systems with zfs on Linux.
> The FreeBSD boot procedure is better than grub.
YMMV
> FreeBSD's network is more performant.
Is there some conclusive recent benchmark about this. The post uses a 2014 post about ipv6 at Facebook, which I think is far from definitive today. Especially more since it "forgot" to mention that Facebook intended to enhance the "Linux kernel network stack to rival or exceed that of FreeBSD." Did they succeed over these 8 years ?
> Straightforward system performance analysis
The point is not about the quality of the tools, but the way each distribution packages them. Seems very very low impact to me.
> FreeBSD's Bhyve against Linux's KVM
The author reluctantly admits that KVM is more mature.
[+] [-] LeonenTheDK|4 years ago|reply
I like your point about Docker vs Jails, I haven't seen it discussed like that before. I keep hearing Jails are more secure than anything else, I'll have to read more into it.
As far as the networking goes, I haven't seen any recent benchmarks to substantiate those claims either. However, considering Netflix uses FreeBSD on their edge nodes and has put a lot of work into the upstream to improve networking (among other things), it wouldn't surprise me if it's technically superior to the Linux stack. Clearly though Linux's networking isn't an issue for most organizations.
And regarding ZFS, ZFS on Linux and FreeBSD's ZFS implementation are now one and the same. It would be nice to see some of the big distributions(or even the Linux kernel) integrate it more directly. This is probably a solid point in favor of FreeBSD, but it's not like it doesn't work in Linux. I'm not a systems guy, so I'm probably out of the loop on this, but Proxmox is the only distribution I've seen with ZFS as a filesystem out of the box, but I don't know how much production use Proxmox sees. I only run it on my home server.
All that to basically say, I like FreeBSD conceptually. I'm just still not convinced that it's doing enough things better to warrant using it over a common Linux distribution for general computing purposes.
[+] [-] throwawaylinux|4 years ago|reply
I would not be surprised if there still were some cases where FreeBSD could do better. The network stack in particular has a vast amount of performance heuristics and tuning where you can just happen to hit the right or wrong side of some performance jump depending on your exact case, for example.
But to go from there to the network stack is more performant is puzzling. There are certainly cases where the Linux network stack is more performant than FreeBSD -- search for linux vs freebsd network performance and you'll find benchmark tests and complaints and anecdotes around the place including freebsd forums and mailing lists where FreeBSD is slower, and yet FreeBSD proponents would (rightly) say these are not proof Linux's network stack is faster and there might be all sorts of reasons for the differences. So you can't have it both ways.
Netflix uses FreeBSD for some things yes. Not sure if number of big internet corporations using an OS would be a favorable metric for FreeBSD either. Again you can't have it both ways. Netflix says FreeBSD is capable of performing well in that kind of environment, it does not say that it outperforms Linux in general or even in that environment. Any more than Google says the opposite.
[+] [-] mcronce|4 years ago|reply
I don't know about Facebook-driven efforts specifically offhand, but I recall attending a talk at LinuxCon 2015 _specifically_ about optimizing the network stack. Given by a Red Hat employee, IIRC.
[+] [-] boring_twenties|4 years ago|reply
> Like all other systemd-nspawn features, this is not a security feature and provides protection against accidental destructive operations only
Doesn't seem very similar to jails to me.
[+] [-] unknown|4 years ago|reply
[deleted]
[+] [-] area51org|4 years ago|reply
But to imply that these are empirical differences between FreeBSD and Linux, well, that's nonsense.
[+] [-] blakesterz|4 years ago|reply
I don't really disagree with any of the stated reasons, but I also didn't see a reason that would make me even consider making the move with our servers, or even bother with some small number of servers. At least for me, I'd need a bunch of REALLY GOOD reasons to consider a move like that. A huge cost savings AND some huge time savings in the future might do it.
[+] [-] toast0|4 years ago|reply
But they left out one of the bigger reasons, IMHO. FreeBSD doesn't tend to churn user (admin) facing interfaces. This saves you time, because you still use ifconfig to configure interfaces, and you still use netstat to look at network statistics, etc; so you don't have to learn a new tool to do the same thing but differently every couple of years. Sure, there's three firewalls, but they're the same three firewalls since forever.
[+] [-] johnklos|4 years ago|reply
Likewise, some people and many businesses see the immediate now and aren't always the best at planning for long term, and/or are overly optimistic that their pain points will eventually be fixed.
[+] [-] raverbashing|4 years ago|reply
Nothing there seems to deliver explicit customer value when switching to FreeBSD.
How will switching help you deliver your service? Or is it just a "nice to have thing"?
[+] [-] embik|4 years ago|reply
> 70 binaries just for initialising and logging
It’s just not true. Those 70 binaries provide much more functionality than an init system, they can cover a significant portion of system management, including a local DNS resolver, network configuration or system time management. You can dislike the fact everything is so tightly integrated (which feels ironic given that the post goes on to praise a user space from one team), but let’s at least be correct about this.
[+] [-] hedora|4 years ago|reply
Do. Not. Want.
So, I have this pile of 70 binaries that are inexplicably tied to my init system, and (in my, informed enough for me, opinion) they're all garbage. How do I remove them without breaking init?
This month's fresh hell: I have a problem where the systemd replacement for xscreensaver (logind, maybe? Good luck finding the culprit, let alone the manual!) won't accept my password unless I exit the current X session with "switch user" then restore the session using the normal login screen.
There's a whole section on JWZ's xscreensaver page (from over a decade ago) explaining how to avoid this class of bug, but what does he know?!?
That reminds me; I wonder if a *BSD is a good enough daily driver for the pine book pro yet (it's probably easier to port their kernel than to fix Linux userspace, after all...)
[+] [-] unknown|4 years ago|reply
[deleted]
[+] [-] kodah|4 years ago|reply
Docker, podman, lxc, lxd, etc are userland components. Linux has cgroups and namespaces.
FreeBSD jails are a bit more complicated because FreeBSD isn't distributed the way Linux is. Linux is distributed as just the kernel, whereas FreeBSD is a base OS. This probably could've been phrased better as, "Linux has no interest in userland and I want some userland consistency". That's fair, Linux was built around the idea that operating system diversity was a good thing long term, FreeBSD was more interested in consistency. I'm reading between the lines, a bit, here because of the critique of SystemD (note: not all linuxes use SystemD)
Personally speaking, I like both Linuxes and FreeBSD but I don't think debating the two is important. Rather, I'd encourage turning your attention to the fact that every other component on a system runs an OS-like interface that we don't make open OS's or "firmware" for.
[+] [-] acdha|4 years ago|reply
This is really telling for the level of analysis done: systemd has been the target from a small number of vocal complainers but most working sysadmins only notice it in that they routinely deal with tasks which are now a couple of systemd stanzas instead of having to cobble together some combination of shell scripts and third-party utilities. Confusing noise with numbers is a dangerous mistake here because almost nobody sits around randomly saying “this works well”.
[+] [-] marcodiego|4 years ago|reply
To sum up: healthy FLOSS competition is welcome and needed.
[+] [-] frankjr|4 years ago|reply
Where is this coming from exactly? The linked article about Facebook is 7 years old. The following benchmark shows the exact opposite: Linux's network stack has long surpassed FreeBSD's. And I would expect nothing else given the amount of work that has gone into Linux compared to FreeBSD.
https://matteocroce.medium.com/linux-and-freebsd-networking-...
[+] [-] markstos|4 years ago|reply
I'm glad there are some people who use and prefer FreeBSD and other init system now, because diversity in digital ecosystems is benefits the whole just as diversity in natural ecosystems do.
The shot taking at systemd here was disingenuous though. The author complained about the number of different systemd binaries and the lines of source code, but all these tools provide a highly consistent "system layer" with standardized conventions and high quality documentation-- it's essentially the same argument made to support FreeBSD as a large body of kernel and userspace code that's maintained in harmony.
[+] [-] gtsop|4 years ago|reply
Nothing wrong with the comparison probably, but I got the impression the author just really wanted to do the migration and found some reasons to do so, without actually needing it. Nothing wrong with that as well. It's just the expectations set by the title that are off
[+] [-] Bayart|4 years ago|reply
If FreeBSD jails and Solaris zones were equivalent to Linux containers, we'd have seen them take over the backend already. We haven't. They're really useful, they provided a degree of safety and peace of mind for multi-tenancy but they're not granular enough for what's done with $CONTAINER_RUNTIME these days.
Jérôme Petazzoni has an old talk where he touches upon container primitives and compared them to jails : https://www.youtube.com/watch?v=sK5i-N34im8
[+] [-] mbreese|4 years ago|reply
Are there any FreeBSD-centric answers to Ceph or Gluster or Lustre or BeeGFS?
[+] [-] tombert|4 years ago|reply
I've been using Linux since I was a teenager, so it's not like I am a stranger to fixing driver issues, but it seemed like no amount of Googling was good enough for me fix these problems (googling is much harder when you don't have functioning wifi). As a result I've always just stuck with Linux (or macOS semi-recently, which I suppose is kind of BSD?).
[+] [-] johnklos|4 years ago|reply
Some of the distros are very good, but some of us who have work to do cringe at the thought of bringing up newer versions of an OS just to check all the things that've broken and changed needlessly.
[+] [-] ianai|4 years ago|reply
Their first reason is really saying a lot but with few words. For one, there’s no systemd. The init system is maintained alongside the entire rest of the system which adds a lot of consistency. The documentation for FreeBSD is also almost always accurate and standard. Etc etc
I think you also largely don’t need a docker or etc in it since jails have been native to the OS for decades. I’d want to do some cross comparison first though before committing to that statement.
Shouldn’t be lost that the licensing is also much friendlier to business uses. There’s afaik no equivalent to rhel, for that matter. This goes both ways though as how would you hire a FreeBSD admin based on their resume without a rhce-like FreeBSD certification program?
Edit-I’ll posit that since FreeBSD is smaller an entity wishing to add features to the OS might face either less backlash or at least enjoy more visibility from the top developers of the OS. Linus, for instance, just has a larger list of entities vying for his attention on issues and commits.
[+] [-] CyberRabbi|4 years ago|reply
[+] [-] gorgoiler|4 years ago|reply
On that playing field, the integrated nature of FreeBSD is nice but it’s an asterisk on top of the kernel rather than anything approaching what makes up a the system part of an Operating System. Almost everything else comes from a third party (and I’m fine with that.)
I haven’t used FreeBSD as a daily OS for over a decade though. What’s the new coolness?
[+] [-] TurningCanadian|4 years ago|reply
[+] [-] gerdesj|4 years ago|reply
I slap Linux on my servers and desktops and I deploy FreeBSD via pfSense on firewalls.
Sometimes I do experiments and try out BSD on the desktop which hasn't worked out yet for me but I live in hope because I adore *BSD as much as I do Linux.
If BSD is the way to get your servers to do what you want then lovely. Do it and remember you have choice.
[+] [-] unknown|4 years ago|reply
[deleted]