This kind of brilliant work makes me feel very very tiny as an engineer. I struggle after work to find time to learn basic Rust. I’m totally in awe of folks who can do this kind of stuff. It’s just so impressive, and maybe one day I can benefit from all this awesome work.
I feel you. Sometimes I browse projects on GitHub and I'm astonished by what people can do and I can't. Example, OpenCore[0], a famous bootloader in the Hackintosh scene. How can people even start to code this.. Awewome work, awesome people.
Re: finding time.
I think this is true for all of life, as we get older our risk and reward profiles change. As we are encumbered by more responsibility, family, mortgage, etc, we stop taking the same risks and start playing life more “safely” or “securely”.
Of course it doesn't have to be that way and I personally advocate for a debt free lifestyle for this reason. Too many times have we heard the story of the midlife crisis: the false deity of security and comfort that robs the well paid executive the majority of his or her life.
Its a sad story. So my advice is to people is to work less for others and work more for your own projects and passions, life is simply too short to give your years to someone else, even if it pays well.
Same. People who can easily straddle the lowest level bits of a machine and also write performant and good c code and then to do all of that and reverse engineer things, amazing.
Building useful and nonporous layers of abstractions and being able to quickly shift between them seems to be a key skill. When each layer leverages another, you can rapidly build something impressive.
This in turn implies apart from technical skill, charting a good course from layer to layer can make a big difference, vs meandering without quite knowing where you are going.
> The silver lining of using this complicated DCP interface is that DCP does in fact run a huge amount of code – the DCP firmware is over 7MB! It implements complicated algorithms like DisplayPort link training, real-time memory bandwidth calculations, handling the DisplayPort to HDMI converter in the Mac mini, enumerating valid video modes and performing mode switching, and more. It would be a huge ordeal to reverse engineer all of this and implement it as a raw hardware driver, so it is quite likely that, in the end, we will save time by using this higher-level interface and leaving all the dirty work to the DCP. In particular, it is likely that newer Apple Silicon chips will use a shared DCP codebase and the same interface for any given macOS version, which gives us support for newer chips “for free”, with only comparatively minor DCP firmware ABI updates.
How interesting: by moving some of the driver to firmware, Apple has effectively made it easier for OSs like Linux to have good support for the hardware, because the OS<->hardware protocol operates at a higher level of abstraction and may work across multiple chips.
The trade-off is that the protocol is not stable and so Linux has to decide which versions it will support, and make sure the OS driver always matches the version of the hardware firmware:
> As a further twist, the DCP interface is not stable and changes every macOS version! [...] Don’t fret: this doesn’t mean you won’t be able to upgrade macOS. This firmware is per-OS, not per-system, and thus Linux can use a different firmware bundle from any sibling macOS installations.
> How interesting: by moving some of the driver to firmware, Apple has effectively made it easier for OSs like Linux to have good support for the hardware, because the OS<->hardware protocol operates at a higher level of abstraction and may work across multiple chips.
If I am remembering right, this was the original idea behind UEFI drivers too — you could in theory write drivers that the firmware would load and they would present a simpler/class-compatible interface to whatever operating system was loaded after that. I think Apple did pretty much this on Intel Macs for a number of things.
I remember hearing about how the M1 Macs were very quick to get displays running when they were plugged in and to change the display configuration. I guess the DCP is why.
> Apple has effectively made it easier for OSs like Linux to have good support for the hardware
Wow, that's a tall stretch of a conclusion, bordering on ignorance, when the reality is that the M1 only supports macOS! Apple fans shouldn't be blind to the fact that developers are trying to reverse engineer the M1 to run Linux on it. (And they have a long way to go before they succeed). Apple has not even published any literature that can help system developers create and run alternative OSes on it. And this is by design - the M1 (and all Arm processor based macs) are designed to be a a closed box like their other iDevices.
(Honestly, except for an academic interest, I believe open source developers are wasting their time trying to support such a useless "closed" piece of hardware, and instead should be boycotting it - or we'll end up losing control over our desktop computers too as other manufacturers move towards this model too).
> On most mobile SoCs, the display controller is just a piece of hardware with simple registers. While this is true on the M1 as well, Apple decided to give it a twist. They added a coprocessor to the display engine (called DCP), which runs its own firmware (initialized by the system bootloader), and moved most of the display driver into the coprocessor. But instead of doing it at a natural driver boundary… they took half of their macOS C++ driver, moved it into the DCP, and created a remote procedure call interface so that each half can call methods on C++ objects on the other CPU! Talk about overcomplicating things… Reverse engineering this is a huge challenge ....
Since there are clearly a lot of people who have been following the asahi development in detail, I would like to hear your takes on this quote from their FAQ:
> No, Apple still controls the boot process and, for example, the firmware that runs on the Secure Enclave Processor. However, no modern device is “fully open” - no usable computer exists today with completely open software and hardware (as much as some companies want to market themselves as such).
I think they're aiming at purism here, but might have forgotten about the MNT Reform, even though it is currently specced at the lower end of "usable".
There's also the Raptor Computing Talos line ... which is interesting, all open firmware and busses, but even more expensive and less practical unfortunately.
https://www.raptorcs.com/TALOSII/
That RPC interface looks very nice for reverse engineering, but what kind of horror is this going to be in the kernel implementing KMS with JSON^2 and pretend-C++-functions?
Oh yes. We're aware, we've had and continue to have debates about how to tackle the monster... :-)
The C++ stuff isn't that bad, in the end you can just pretend it's messages with fixed layout structures. But yes, the serialized dict-of-arrays-of-dicts type stuff can be approached in a few ways, none of which are particularly beautiful.
Can anyone comment on the possibility of going in the opposite direction?
I’m considering buying the frame.work laptop, daily driving pop!_os and then virtualizing OSX on it for the few OSX programs I use (this supports framework and not apple).
It looks like it may be fairly easy and possible with a 10-20% performance hit?
Virtualizing macOS on non-macOS hosts works ok, with a couple of caveats:
* macOS lacks drivers for GPUs commonly emulated by VM software, and as such runs absolutely horribly without graphics acceleration because it assumes the presence of a competent GPU at all times - software rasterization seems to be intended to be used only as an absolute last-resort fallback. As such, passthrough of a supported GPU (generally AMD, though Intel iGPUs might work) is basically a requirement.
* It's technically against the macOS EULA. Apple hasn't gone after anybody for it to date, but it doesn't mean they won't. This is particularly relevant for devs building for Apple platforms.
I'm also considering this. The software renderer should be a non-issue if you successfully passthrough the iGPU. The PopOs won't have any display device, but you can still access it through ssh from the VM Mac. Look into /r/vfio and the discord.
I’ve never passed a GPU to a mac guest but I have done Xcode work in a VM and it worked fine on Arch. If I had the cash I would too get a framework laptop!
I've used this quite a bit for iOS development with Xcode.
It required a patched VMware player or workstation instance. Not sure about other hypervisors. Never had any issues and was running fine on a Ubuntu laptop with 16GB / Intel i7-4720HQ
Re: the installation procedure, why does Asahi Linux have to have its own separate APFS container, rather than being an APFS volume in the pre-existing APFS container (and so sharing the Recovery volume with the pre-existing macOS install, not requiring you to resize the pre-existing APFS container smaller, etc.)?
I actually haven't tried doing multiple installs in the same container, but there's no reason why it wouldn't work as far as I know. It would be easy to change the installer to let you do that.
Though there isn't much of a difference for Linux; you can't actually share the Recovery image (since it's per-OS, even within a single Recovery volume), and since you need to repartition to create standard Linux partitions anyway (since linux-apfs isn't quite at the point where you'd want to use it as your root filesystem as far as I know...) it doesn't make much of a difference if you create a 2.5G stub container too, and makes it much easier to blow away the entire Linux install without affecting macOS. Perhaps in the future, when linux-apfs is solid enough, it would be interesting to support this in the installer in order to have a fully space-sharing dual-boot macOS/linux system, without partitioning.
Also, U-Boot would have to grow APFS support in order to have a truly all-APFS system :) (we plan to use U-Boot as a layer for UEFI services and such, so you don't have to burn kernels into the m1n1 image, which would suck for upgrades).
Amazing work and any developer should work on whatever he pleases.
But there is also something in me that wished development would focus on something more open. What if Apple tomorrow decides only signed kernels are allowed to boot (to protect the children). I know there currently is no ARM laptop equivalent to the M1, and ARM booting is a minefield. So I get that if the M1 becomes a viable Linux target this would be very interesting.
I think Apple has specifically announced at WWDC that ARM macs won't be locked down? I've also read somewhere that they have specifically written some utilities to allow booting arbitrary kernels — they could have very well not done that, so it has to be very intentional.
Apple engineers are also probably reading all this and are very impressed.
Kind of unrelated but I would like to benefit from the knowledge of people hanging out here to ask if anybody is aware of how good and how usable is Linux on Intel MacBook? Is there a specific Intel MacBook that's known to be particularly well supported by Linux ? Been searching but I couldn't find much else than vague information and lists of Linux distributions I could install on a MacBook.
Thanks for working on this! It would be great if making Asahi generally usable was not blocked by the GPU driver. Currently Apple M1 is the only generally-available ARM hardware that can be used for testing other software for compatibility with ARM. So an installable version without desktop
support would be very appreciated.
It's not blocked on the GPU driver; you can already boot a desktop on the boot-time framebuffer (this has worked for months). The issue right now is that as I mentioned at the end, things like USB and storage work but aren't quite there yet and are spread around various kernel branches, so our next step is indeed going to be to get that all in shape so that there is one known good kernel branch for people to use. That will give you Ethernet, USB, NVMe, etc. Then we'll tackle the GPU after that.
Just testing non-desktop software on ARM – you could use a Raspberry Pi 4 for that. Or a Pinebook Pro. Or just an EC2 instance!
For "real workstation" class hardware (i.e. working PCIe/XHCI/AHCI on standard ACPI+UEFI systems with mostly FOSS firmware) nothing beats SolidRun boards (MACCHIATObin/HoneyComb LX2K). Yeah yeah the Cortex-A72 cores are really showing their age, oh well, on the other hand the SoCs are not embedded cursed crap :)
It's several things (yes, we know it's confusing). Asahi Linux is the overall project.
m1n1 is our bootloader, which once installed will present a standard (ish) U-Boot/UEFI boot environment. From there you will be able to boot any distro or USB installer, in principle, once the changes trickle upstream.
Our installer will install m1n1 and then give you the choice of also installing a distro image. We'll be providing our own based on Arch Linux ARM, but we're open to other distros providing images so we can add them to the list. We already have interest from Fedora, for example.
So we expect that people who want to run Linux (or in fact openbsd or anything else) on these Macs will first use our bootloader installer, then either pick one of the built-in images or use their own USB installation media from whatever distro.
They are working on creating a full distro, based on Arch Linux ARM, which will be an easy-to-use package for end-users with bleeding-edge versions of the software they develop.[1]
That said, all components will be upstreamed into their respective projects such that other distributions should eventually be able to add support for Apple's line of desktop computers with Apple Silicon.
It looks like installing a Linux distro on an m1 Mac requires a special installation package, rather than a standard iso. So Asahi is going to be a distro packaged/distributed specifically for m1 Macs, but changes to the kernel etc are being upstreamed, so other distributions should be able to build their own installation media for m1 Macs too in the future.
We need lawmakers to force Apple and other companies to open documentation that could enable writing drivers etc.
Give Apple anti privacy stance, I can see why they don't want a platform they can't control on their hardware.
This should be illegal.
Wow that's some progress. This looks pretty close to useable it seems. Is there some kind of expected timeline? Like in 6 months. Kb, wifi etc work. In 1 year we expect the GPU to work?
What are the licensing implications of using DCP firmware from macOS installations - just having the hardware and OS license is enough to use it within Linux?
[+] [-] SirensOfTitan|4 years ago|reply
[+] [-] hivacruz|4 years ago|reply
[0]: https://github.com/acidanthera/OpenCorePkg
[+] [-] ostenning|4 years ago|reply
[+] [-] gigatexal|4 years ago|reply
[+] [-] ip26|4 years ago|reply
This in turn implies apart from technical skill, charting a good course from layer to layer can make a big difference, vs meandering without quite knowing where you are going.
[+] [-] haberman|4 years ago|reply
How interesting: by moving some of the driver to firmware, Apple has effectively made it easier for OSs like Linux to have good support for the hardware, because the OS<->hardware protocol operates at a higher level of abstraction and may work across multiple chips.
The trade-off is that the protocol is not stable and so Linux has to decide which versions it will support, and make sure the OS driver always matches the version of the hardware firmware:
> As a further twist, the DCP interface is not stable and changes every macOS version! [...] Don’t fret: this doesn’t mean you won’t be able to upgrade macOS. This firmware is per-OS, not per-system, and thus Linux can use a different firmware bundle from any sibling macOS installations.
[+] [-] neilalexander|4 years ago|reply
If I am remembering right, this was the original idea behind UEFI drivers too — you could in theory write drivers that the firmware would load and they would present a simpler/class-compatible interface to whatever operating system was loaded after that. I think Apple did pretty much this on Intel Macs for a number of things.
[+] [-] edeion|4 years ago|reply
[+] [-] skavi|4 years ago|reply
[+] [-] webmobdev|4 years ago|reply
Wow, that's a tall stretch of a conclusion, bordering on ignorance, when the reality is that the M1 only supports macOS! Apple fans shouldn't be blind to the fact that developers are trying to reverse engineer the M1 to run Linux on it. (And they have a long way to go before they succeed). Apple has not even published any literature that can help system developers create and run alternative OSes on it. And this is by design - the M1 (and all Arm processor based macs) are designed to be a a closed box like their other iDevices.
(Honestly, except for an academic interest, I believe open source developers are wasting their time trying to support such a useless "closed" piece of hardware, and instead should be boycotting it - or we'll end up losing control over our desktop computers too as other manufacturers move towards this model too).
> On most mobile SoCs, the display controller is just a piece of hardware with simple registers. While this is true on the M1 as well, Apple decided to give it a twist. They added a coprocessor to the display engine (called DCP), which runs its own firmware (initialized by the system bootloader), and moved most of the display driver into the coprocessor. But instead of doing it at a natural driver boundary… they took half of their macOS C++ driver, moved it into the DCP, and created a remote procedure call interface so that each half can call methods on C++ objects on the other CPU! Talk about overcomplicating things… Reverse engineering this is a huge challenge ....
[+] [-] black_puppydog|4 years ago|reply
> No, Apple still controls the boot process and, for example, the firmware that runs on the Secure Enclave Processor. However, no modern device is “fully open” - no usable computer exists today with completely open software and hardware (as much as some companies want to market themselves as such).
I think they're aiming at purism here, but might have forgotten about the MNT Reform, even though it is currently specced at the lower end of "usable".
[+] [-] ploxiln|4 years ago|reply
[+] [-] marcodiego|4 years ago|reply
[+] [-] stefan_|4 years ago|reply
[+] [-] marcan_42|4 years ago|reply
The C++ stuff isn't that bad, in the end you can just pretend it's messages with fixed layout structures. But yes, the serialized dict-of-arrays-of-dicts type stuff can be approached in a few ways, none of which are particularly beautiful.
[+] [-] zamadatix|4 years ago|reply
https://www.patreon.com/marcan/ https://github.com/sponsors/marcan
[+] [-] rowanG077|4 years ago|reply
[+] [-] jjcon|4 years ago|reply
I’m considering buying the frame.work laptop, daily driving pop!_os and then virtualizing OSX on it for the few OSX programs I use (this supports framework and not apple).
It looks like it may be fairly easy and possible with a 10-20% performance hit?
https://github.com/foxlet/macOS-Simple-KVM
From what I can tell you can pass through a single GPU with a bit of work, even a iGPU? Is that correct?
[+] [-] kitsunesoba|4 years ago|reply
* macOS lacks drivers for GPUs commonly emulated by VM software, and as such runs absolutely horribly without graphics acceleration because it assumes the presence of a competent GPU at all times - software rasterization seems to be intended to be used only as an absolute last-resort fallback. As such, passthrough of a supported GPU (generally AMD, though Intel iGPUs might work) is basically a requirement.
* It's technically against the macOS EULA. Apple hasn't gone after anybody for it to date, but it doesn't mean they won't. This is particularly relevant for devs building for Apple platforms.
[+] [-] raihansaputra|4 years ago|reply
[+] [-] flatiron|4 years ago|reply
[+] [-] adevx|4 years ago|reply
[+] [-] heavyset_go|4 years ago|reply
[+] [-] mixmastamyk|4 years ago|reply
[+] [-] derefr|4 years ago|reply
[+] [-] marcan_42|4 years ago|reply
Though there isn't much of a difference for Linux; you can't actually share the Recovery image (since it's per-OS, even within a single Recovery volume), and since you need to repartition to create standard Linux partitions anyway (since linux-apfs isn't quite at the point where you'd want to use it as your root filesystem as far as I know...) it doesn't make much of a difference if you create a 2.5G stub container too, and makes it much easier to blow away the entire Linux install without affecting macOS. Perhaps in the future, when linux-apfs is solid enough, it would be interesting to support this in the installer in order to have a fully space-sharing dual-boot macOS/linux system, without partitioning.
Also, U-Boot would have to grow APFS support in order to have a truly all-APFS system :) (we plan to use U-Boot as a layer for UEFI services and such, so you don't have to burn kernels into the m1n1 image, which would suck for upgrades).
[+] [-] adevx|4 years ago|reply
[+] [-] grishka|4 years ago|reply
Apple engineers are also probably reading all this and are very impressed.
[+] [-] cassepipe|4 years ago|reply
[+] [-] boris|4 years ago|reply
[+] [-] marcan_42|4 years ago|reply
https://twitter.com/alyssarzg/status/1419469011734073347
[+] [-] swiley|4 years ago|reply
[+] [-] floatboth|4 years ago|reply
For "real workstation" class hardware (i.e. working PCIe/XHCI/AHCI on standard ACPI+UEFI systems with mostly FOSS firmware) nothing beats SolidRun boards (MACCHIATObin/HoneyComb LX2K). Yeah yeah the Cortex-A72 cores are really showing their age, oh well, on the other hand the SoCs are not embedded cursed crap :)
[+] [-] opencl|4 years ago|reply
[+] [-] kzrdude|4 years ago|reply
[+] [-] dm319|4 years ago|reply
[+] [-] fastssd|4 years ago|reply
[+] [-] 2bitencryption|4 years ago|reply
Is this a kernel replacement, as in I could run Manjaro Gnome, and just load the Asahi Kernel and it all Just Works?
Or will Asahi Linux need to be a full distro of its own, in order to be useful?
[+] [-] marcan_42|4 years ago|reply
m1n1 is our bootloader, which once installed will present a standard (ish) U-Boot/UEFI boot environment. From there you will be able to boot any distro or USB installer, in principle, once the changes trickle upstream.
Our installer will install m1n1 and then give you the choice of also installing a distro image. We'll be providing our own based on Arch Linux ARM, but we're open to other distros providing images so we can add them to the list. We already have interest from Fedora, for example.
So we expect that people who want to run Linux (or in fact openbsd or anything else) on these Macs will first use our bootloader installer, then either pick one of the built-in images or use their own USB installation media from whatever distro.
[+] [-] Raqbit|4 years ago|reply
That said, all components will be upstreamed into their respective projects such that other distributions should eventually be able to add support for Apple's line of desktop computers with Apple Silicon.
[1] https://asahilinux.org/about/#is-this-a-linux-distribution
[+] [-] idle_zealot|4 years ago|reply
[+] [-] intricatedetail|4 years ago|reply
[+] [-] rowanG077|4 years ago|reply
[+] [-] mraza007|4 years ago|reply
I can’t wait to use Linux as daily driver on my M1
[+] [-] blinkingled|4 years ago|reply
[+] [-] jaimex2|4 years ago|reply
[+] [-] sim_card_map|4 years ago|reply
also m1 is pretty quick
[+] [-] syntaxing|4 years ago|reply
[+] [-] emodendroket|4 years ago|reply
[+] [-] dilap|4 years ago|reply