top | item 26742588

Run Bhyve in FreeBSD jails – The view from inside the forest (2020)

98 points| rodrigo975 | 4 years ago |neirac.byteswizards.com

14 comments

order
[+] tejado|4 years ago|reply
I wonder how many nested VM inside jail inside a VM inside a jail... are possible from a resource perspective, so that I can still execute a shell
[+] tonoto|4 years ago|reply
For a second you lit up my hope that there would be nested bhyve virtualization, but no. This is a different layer of virtualization/containerization. In order to have nested vm the correct cpu flags (vt-x) have to be exposed in the guest and I believe that such feature is unfortunately not implemented yet.
[+] kazen44|4 years ago|reply
you can nest jails quite a lot. Although i have not tried it (and cannot find proper sources on the max limit of the amount of jails you can inside a freebsd system).

might be fun to give it a try though.

[+] cyberpunk|4 years ago|reply
I am a fan of FreeBSD, and use it for a few tasks.. But I'm not at all clear why someone would want to do this? Does anyone have any idea what this gives you over just running a Bhyve VM outside of a jail?

Is it more secure somehow?

[+] doublerabbit|4 years ago|reply
Before FreeBSD 12, you couldn't expose the bhyve components in a jail so you were forced to use the host as the bHyve hypervisor and/or jails. The new feature of the latest FreeBSD was a network stack within an jail.

With this stack, you can now assign an IP addresses to a jail. No longer requiring you to maintain a rule-set via the firewall while enabling you to operate as if the jail was the bare-metal FreeBSD host.

Knowing that the bHyve VMs are isolated within their own jails you can then hand those off to client as their own hypervisor space. With that they can run their own network, appliances, virtual machines while being isolated from anyone else with confined limits without knowledge of the host they run on.

This enhances public security as at any time I can axe the jail and everything is disabled; nor do I have to worry that the rule-set created will brick anything else. The client also has shell access in case of. And if a hacker breaks in to the bHyve VM, exploits out, the hacker are isolated to jail and not the hypervisor host.

Plus with the added bonus of ZFS, I can make snapshots or backups of clients jails with ease.

[+] UI_at_80x24|4 years ago|reply
It can be MUCH more secure to run it in a jail; but IMHO that's not the point, but it could be.

There's also multiple types of security.

Running it in a jail allows you to do cool things, especially when coupled with ZFS. This is about running & managing VM's.

Imagine you wanted to 'restore from day X', or restart each day with the same config (i.e. testing Windows boxes for exploits), or you wanted solid A/B tests and MS keeps installing updates and moving your goalposts, or some funky abandoned software only works if you have the calendar set to 1993, etc....)

You can do all of these things easily with JAILS + ZFS.

But IMHO the best reason to run something in a jail is to isolate services.

On my file-server right now I have the following running in different jails:

Plex

Windows10

WindowsXP

my Git server

my SAMBA server

DHCP

DNS/PiHole/ad-blocking

IRC server

Mumble server

4 different FAMP servers for friends & family

I like to isolate each jail into the task it'll be handling. That how it stays organized in my mind.

ZFS send & receive + jails makes backups and restoring painless.