Engine diversity is really important for the ecosystem and continued innovation. While building something competitive with the big three engines is a monumental task there's still a lot of value in building alternative engines to try new ideas even if getting "the whole web" implemented is basically impossible.
For example:
- Servo vs Blink vs Cobalt do selector matching and style resolution very differently.
- WebKit has a selector JIT, no one else does though.
- WebKit and Blink do layout super differently (with LayoutNG shipped).
- Firefox's html parser is written in Java and converted to C++ at build time.
The big value in folks trying to write new engines is finding these alternate paths. We can't depend on the big three for all the innovation.
That's what's great about Ladybird. Sure it'll need a lot of sophisticated sandbox improvements to get us to a big four situation, but it's more likely its value will be in finding another road the other engines didn't travel across the huge universe of specs, content and features.
But none of those differences are especially interesting. They certainly don't matter much for the ecosystem, and have no impact on web developers. All the major engines have comparable performance. And the differences are dwarfed by what they must do the same, because the way web tech works basically only allows one implementation architecture.
If you wanted to innovate in browser tech you'd really need to leave the specs behind. SCIter is an example of such an engine. It exposes APIs that are useful for app devs but that regular browsers don't have, for example it provides the vdom diffing algorithm React uses but implemented natively.
Even better if your browser can go beyond HTML and enable fully different ways to do apps and documents. That's the only place where design decisions can start to have a big impact.
"- Firefox's html parser is written in Java and converted to C++ at build time."
This is surprising. Is it really true in the sense that the code that parses the HTML in a regular Firefox install was autoconverted when it was built?
Diversity alone is great but not quite enough. The alternative browsers need to actually see significant usage. Otherwise the developers and corporations will go "this browser has 99% market share so I can just ignore the others", giving the dominant browser makers enormous leverage over the others.
"Fixed-width text to a grid" makes things easier (sometimes), but I think it still qualifies.
On the article itself; it might be better to start with more... basic things than the optimizations it talks about:
* Cascading & layout must be cached and optimized - far from trivial. Layout is one of the hardest parts to get right at all, to make it fast as well is another level... and I'm just talking caching, not even multi-threading.
* The "web platform" contains a very large amount of poorly thought out features harmful for user privacy and/or security. You can choose between convenience (opt out) and privacy (opt in), or try to balance the two as major browsers do. Both is often impossible.
* "Serialize the entire application state" sounds like the most complex thing you can come up with as a distinguishing feature. Much more low hanging fruit exists if you give up on writing a Chromium clone. e.g. a fun side project I've had is making my browser "protocol-agnostic", or adding a bunch small QOL stuff for myself. You can probably find new ideas easily once you start out.
* Older browsers are useful for inspiration too. The coolest features in mine are often just carbon copies of the same things from w3m, just a bit more polished. Reading about what historical engines did - even IE! - is often enlightening too. Sometimes it's just a smaller scale of what engines do now, and easier to implement. Other times it's a whole different approach.
* It's easy to get lost in the details - careful with perfectionism. As you start getting more components interacting, it will slowly become clear when the naive approach is wrong and how to improve it.
Overall, it takes time, but is also fun and rewarding - you will learn a lot, even without replacing Chromium.[0] That said, if you want to learn how "modern" browsers work, getting involved with a "modern" engine itself is probably much more productive.
[0]: To write a Chromium replacement, you may have to reincarnate as awesomekling ;)
> Much more low hanging fruit exists if you give up on writing a Chromium clone.
One idea that’s crossed my mind (may never get around to trying it) is writing an engine that completely ignores everything that’s not modern HTML5 and CSS3. That’s still a lot but it seems like it’d cut down on the scope significantly. This engine wouldn’t be very useful as a browser but it’d probably be enough for displaying bespoke web content in an app or something like that.
I guess one of my points is that layout algorithms are not really part of the "most basic" decisions anymore. Replacing layout algorithms is actually a lot less disruptive to the engine architecture than switching to site isolation, say.
I’m not sure that performance needs to constrain web engine design the way it traditionally has. The world has changed since the browser perf war started.
Just try disabling the JIT in a modern browser and you’ll find your UX is not really any worse.
HW has gotten faster than when the current browser perf wars kicked off. So it might be that to have a meaningfully useful browser engine alternative, perf won’t be the top concern. Compatibility will though, just because of the sheer number of features a modern browser engine supports.
And if you don’t have JIT, then it’s not clear if Spectre is even a meaningful concern.
Those are interesting points, but disabling the JIT doesn't really change anything unless it means you can forgo site isolation, and that would be a very risky bet.
It may be that disabling the JIT is fine for users most of the time. However, that is also a tough call --- so easy for your competitors to beat you up with benchmarks, and there will be real use-cases (e.g. emulators) where it matters to some users.
And of course there's a lot more to perf than just the JIT. I barely mentioned JIT in my blog post.
I'd say a browser is an OS. It is, of course, higher level than a kernel like Linux that can run on the bare metal, but it has most of the aspects of a full OS.
It manages memory, processes and security, it can run user-supplied arbitrary code. It has an API, not unlike system calls that allows programs to access the underlying hardware. It doesn't do thing like writing on the address bus directly, but it does have the equivalent of drivers that are proxies to the underlying OS services for display, audio, video, etc..., which it abstracts away. Like many OSes it comes with a compiler and a shell.
Sure, it doesn't access the hardware directly through the address and data buses, like one may think an OS should do, but now that we have stuff like hypervisors, even what are unquestionably true OSes like Windows and Linux may not access the hardware directly either.
>This makes writing a compiler or writing an OS kernel look like child's play.
Indeed. The modern web browser is the single most advanced, complex, and highly developed piece of software that 99% of people will ever interact with, and we treat it as a given. Having a highly performant fully sandboxed VM that runs on every system imaginable in billions of devices used by nearly every human on earth is the single greatest triumph of software engineering ever, perhaps only second to the Linux kernel.
A compiler by itself is surprisingly easy, especially if you read Abdulaziz Ghuloum's or Jeremy Siek's tutorials. Making it competitive with state-of-the-art compilers like Clang or HotSpot is difficult, but this is true for every kind of software.
Does anyone have the idea of escaping from HTML/CSS? As these specs are too complicated and not friendly for web developers as well. Maybe we could re-invent a browser engine without conforming to HTML/CSS specs?
An (early) alternative spec/engine would be a Figma-compatible vector graphics spec[2] and its rendering engine[3]. It is called VeryGoodGraphics[1].
VeryGoodGraphics has no accessibility tree and no results when searching the documentation for “accessibility”, which makes it broadly immoral (or if you want to disagree with that, at least illegal to use it to build production systems in many locations). If you can’t get that right from the start, or even have plans for it, then you’re obsolete.
My opinion is that the rendering engine should be WASM specified in a header. This way the site provider can choose whatever engine they want, including possibly not even using HTML.
Yeah - there really is a opportunity now to rethink browsers as just sandboxed rendering windows using WebAssembly + WebGPU.
Could still have typical DOM rendering handled with Webassembly delivered by the web sites (ideally cached).
The challenge is though still having standards and accessibility options. That VeryGoodGraphics example allows for no text selection - and doesn't at all handle zooming.
Still though it'd be a good bottom up way for a new browser to disrupt Chrome
I was sad when the creator tried to raise enough funds to open-source Sciter and there wasn't much interest. If 1% of the people who complain about Electron had pledged something, we would have it as a good alternative today.
This post starts with a false dichotomy: You're either making a toy browser for fun, or you're trying to make the next Chrome. There are many points in-between on that spectrum. (Look how long Microsoft IE existed, despite being highly inferior.)
Not at all: at the very end it suggests some in-betweens, like building an alternative to Electron or embedded WebViews.
There are lots of browser engines like that still around. It'd be really interesting to see one that supports the vast majority of web standards but without trying to do everything else a full browser does.
That's only because it was the "Chrome" (i.e. the dominant browser). Look at what happened when Microsoft eventually tried to catch up with Chrome - they gave up.
I've wanted to make my own browser by forking chromium. But then I got confronted with the reality of hacking on a Google scale c++ project.
I've tried doing it by embedding webkit which is much more doable but I've found that many sites I use don't work my embedded webkit.
So now I think if I want to make my own browser it would be better start making my own versions of those sites first and make sure those are good and popular. At which point I'll just make native clients for them and forget about html/js/css.
interesting how oldpersonintx's comment "thinly-veiled passive-aggressive swipe at Ladybird" was dead'd here when that accurately reflects the authors comments (as roc-robert o'callahan) here on LWN.net: https://lwn.net/Articles/977625/
Firefox is also competitive with Chrome when it comes to telemetry and other anti-features.
Really being competitive with Chrome is not enough because that still doesn't give anyone any reason to use your browser over Chrome. You need to actually provide some end-user benifit that Chrome doesn't (and ideally won't) and that is more important than being competitiv with Chrome on the features Google wants to push. This is where Mozilla really dropped the ball. 15 years of Chrome lighting fire under their ass and zero innovation in the way we browse the web. Well, less than zero when you consider all the useful extension broken by browser changes over the years.
Meanwhile Mozilla keeps gaslighting us how they care about privacy while trying numerous ways to integrate ads in the browser.
... yeah I see why someone would rather create a browser from scratch than support Mozilla.
> initially try just being a faster, lighter or lower-power Electron or WebView.
But he mentioned it himself, though maybe someone might want to try this with no intention to become a full browser. Can you skip any of the tricky security requirements if it'll be bundled into an app? Or is that just asking for trouble?
I think sooner or later you're going to want to load lower-trust content --- IFRAMEs of third-party Web content, or sandboxed extensions, or something like that. Building your entire architecture on the assumption you'll never have to do that is very risky.
The average Node project pulls in hundreds of dependencies. While you'd hope these would have some security vetting because of the Many Eyes theory, you have no fucking idea what your project is doing. Even a trivial Electron app is running a ridiculous amount of unreviewed third party code.
Just one module able to exercise some local exploit in your engine because you didn't fix Security Footgun #8176 screws over all of your users.
A browser engine that's been developed with a billion dollars of person hours runs that same untrusted third party code but has security guardrails everywhere.
With the interactions an electron-like app might be doing with external services and the ton of JS third party library it could use, I think it would be indeed risky.
How important is process separation if the browser engine is programmed with a memory safe language like Rust? I am under the impression that things like site separation are to bandage memory safety issues. Is that right, or are there other issues at play?
I was inspired to write it by reading about Ladybird, so I wrote it for Andreas Kling basically :-). But there is also Flow: https://www.ekioh.com/flow-browser/
and Servo. Maybe in the future someone else will try. Also I think it's fun to think about these things even if you're not building a browser engine. It only took about 90 minutes to write.
There's a reason they sometimes on interviews ask "what happens when I fill in a url and hit enter", because almost no one knows. It's good to know how your daily tools work.
As someone who worked with roc for many years at Mozilla, and with 25 years of Mozilla and Gecko experience myself, I think it serves as a solid warning.
I mean, if you want to build a toy browser engine for a CS class or fun or something, then sure.
But the idea that "you want to build an engine that’s competitive with Chromium" is, quite simply, nonsensical.
If you want your own browser engine, you're going to fork Chromium or Gecko (Firefox). I mean, even Microsoft gave up on maintaining its own independent engine and switched to Chromium.
I literally don't understand who the author thinks this post is supposed to be addressed to.
Building an independent browser engine could have made sense in 1998. These days it would cost hundreds of millions of dollars in dev time to catch up to existing engines... just to duplicate something that you already have two open-source versions of?
HTML and CSS are pretty terrible languages. Basic things like form interaction, centering, printing a page of something, overwriting every default UI, etc. often take an odd amount of LOC to make basic things.
We're collectively getting better via following the same spec, which is great, but challenging the browsers and even the spec can shed light on potentially better implementations. Let's not pigeonhole ourselves to a couple vendors just because it's easier and works _good enough_.
My favorite web design is that '90s style where text went from very left to very right, barely any CSS, and Javascript (somehow my favorite language) was for fart buttons.
I'm currently most excited about keeping up with WebRTC. And yeah, I have no interest in writing a new browser engine.
Even Chromium started with WebKit which itself was a fork. This doesn't mean you shouldn't be interested in browser dev but you also don't have to do a totally clean sheet implementation.
esprehn|1 year ago
For example:
- Servo vs Blink vs Cobalt do selector matching and style resolution very differently.
- WebKit has a selector JIT, no one else does though.
- WebKit and Blink do layout super differently (with LayoutNG shipped).
- Firefox's html parser is written in Java and converted to C++ at build time.
The big value in folks trying to write new engines is finding these alternate paths. We can't depend on the big three for all the innovation.
That's what's great about Ladybird. Sure it'll need a lot of sophisticated sandbox improvements to get us to a big four situation, but it's more likely its value will be in finding another road the other engines didn't travel across the huge universe of specs, content and features.
mike_hearn|1 year ago
If you wanted to innovate in browser tech you'd really need to leave the specs behind. SCIter is an example of such an engine. It exposes APIs that are useful for app devs but that regular browsers don't have, for example it provides the vdom diffing algorithm React uses but implemented natively.
Even better if your browser can go beyond HTML and enable fully different ways to do apps and documents. That's the only place where design decisions can start to have a big impact.
weinzierl|1 year ago
This is surprising. Is it really true in the sense that the code that parses the HTML in a regular Firefox install was autoconverted when it was built?
matheusmoreira|1 year ago
shiomiru|1 year ago
"Fixed-width text to a grid" makes things easier (sometimes), but I think it still qualifies.
On the article itself; it might be better to start with more... basic things than the optimizations it talks about:
* Cascading & layout must be cached and optimized - far from trivial. Layout is one of the hardest parts to get right at all, to make it fast as well is another level... and I'm just talking caching, not even multi-threading.
* The "web platform" contains a very large amount of poorly thought out features harmful for user privacy and/or security. You can choose between convenience (opt out) and privacy (opt in), or try to balance the two as major browsers do. Both is often impossible.
* "Serialize the entire application state" sounds like the most complex thing you can come up with as a distinguishing feature. Much more low hanging fruit exists if you give up on writing a Chromium clone. e.g. a fun side project I've had is making my browser "protocol-agnostic", or adding a bunch small QOL stuff for myself. You can probably find new ideas easily once you start out.
* Older browsers are useful for inspiration too. The coolest features in mine are often just carbon copies of the same things from w3m, just a bit more polished. Reading about what historical engines did - even IE! - is often enlightening too. Sometimes it's just a smaller scale of what engines do now, and easier to implement. Other times it's a whole different approach.
* It's easy to get lost in the details - careful with perfectionism. As you start getting more components interacting, it will slowly become clear when the naive approach is wrong and how to improve it.
Overall, it takes time, but is also fun and rewarding - you will learn a lot, even without replacing Chromium.[0] That said, if you want to learn how "modern" browsers work, getting involved with a "modern" engine itself is probably much more productive.
[0]: To write a Chromium replacement, you may have to reincarnate as awesomekling ;)
jwells89|1 year ago
One idea that’s crossed my mind (may never get around to trying it) is writing an engine that completely ignores everything that’s not modern HTML5 and CSS3. That’s still a lot but it seems like it’d cut down on the scope significantly. This engine wouldn’t be very useful as a browser but it’d probably be enough for displaying bespoke web content in an app or something like that.
roca|1 year ago
pizlonator|1 year ago
Just try disabling the JIT in a modern browser and you’ll find your UX is not really any worse.
HW has gotten faster than when the current browser perf wars kicked off. So it might be that to have a meaningfully useful browser engine alternative, perf won’t be the top concern. Compatibility will though, just because of the sheer number of features a modern browser engine supports.
And if you don’t have JIT, then it’s not clear if Spectre is even a meaningful concern.
roca|1 year ago
It may be that disabling the JIT is fine for users most of the time. However, that is also a tough call --- so easy for your competitors to beat you up with benchmarks, and there will be real use-cases (e.g. emulators) where it matters to some users.
And of course there's a lot more to perf than just the JIT. I barely mentioned JIT in my blog post.
amelius|1 year ago
GuB-42|1 year ago
It manages memory, processes and security, it can run user-supplied arbitrary code. It has an API, not unlike system calls that allows programs to access the underlying hardware. It doesn't do thing like writing on the address bus directly, but it does have the equivalent of drivers that are proxies to the underlying OS services for display, audio, video, etc..., which it abstracts away. Like many OSes it comes with a compiler and a shell.
Sure, it doesn't access the hardware directly through the address and data buses, like one may think an OS should do, but now that we have stuff like hypervisors, even what are unquestionably true OSes like Windows and Linux may not access the hardware directly either.
ramesh31|1 year ago
Indeed. The modern web browser is the single most advanced, complex, and highly developed piece of software that 99% of people will ever interact with, and we treat it as a given. Having a highly performant fully sandboxed VM that runs on every system imaginable in billions of devices used by nearly every human on earth is the single greatest triumph of software engineering ever, perhaps only second to the Linux kernel.
Paul-Craft|1 year ago
mepian|1 year ago
bdahz|1 year ago
An (early) alternative spec/engine would be a Figma-compatible vector graphics spec[2] and its rendering engine[3]. It is called VeryGoodGraphics[1].
[1] https://verygoodgraphics.com/, the website is built with its own technology (wasm version).
[2] https://docs.verygoodgraphics.com/specs/overview
[3] https://github.com/verygoodgraphics/vgg_runtime
robin_reala|1 year ago
locallost|1 year ago
https://youtu.be/gGw09RZjQf8?feature=shared
RunSet|1 year ago
zellyn|1 year ago
promiseofbeans|1 year ago
gwbas1c|1 year ago
Vermeulen|1 year ago
Could still have typical DOM rendering handled with Webassembly delivered by the web sites (ideally cached). The challenge is though still having standards and accessibility options. That VeryGoodGraphics example allows for no text selection - and doesn't at all handle zooming. Still though it'd be a good bottom up way for a new browser to disrupt Chrome
djbusby|1 year ago
It's been on HN loads of times.
A "browser" engine but very narrow scope. Works a treat for LOB type apps.
Buttons840|1 year ago
phendrenad2|1 year ago
dmazzoni|1 year ago
There are lots of browser engines like that still around. It'd be really interesting to see one that supports the vast majority of web standards but without trying to do everything else a full browser does.
IshKebab|1 year ago
pootempy|1 year ago
ilrwbwrkhv|1 year ago
bobajeff|1 year ago
I've tried doing it by embedding webkit which is much more doable but I've found that many sites I use don't work my embedded webkit.
So now I think if I want to make my own browser it would be better start making my own versions of those sites first and make sure those are good and popular. At which point I'll just make native clients for them and forget about html/js/css.
cbxyp|1 year ago
account42|1 year ago
Really being competitive with Chrome is not enough because that still doesn't give anyone any reason to use your browser over Chrome. You need to actually provide some end-user benifit that Chrome doesn't (and ideally won't) and that is more important than being competitiv with Chrome on the features Google wants to push. This is where Mozilla really dropped the ball. 15 years of Chrome lighting fire under their ass and zero innovation in the way we browse the web. Well, less than zero when you consider all the useful extension broken by browser changes over the years.
Meanwhile Mozilla keeps gaslighting us how they care about privacy while trying numerous ways to integrate ads in the browser.
... yeah I see why someone would rather create a browser from scratch than support Mozilla.
ZeroGravitas|1 year ago
> initially try just being a faster, lighter or lower-power Electron or WebView.
But he mentioned it himself, though maybe someone might want to try this with no intention to become a full browser. Can you skip any of the tricky security requirements if it'll be bundled into an app? Or is that just asking for trouble?
roca|1 year ago
giantrobot|1 year ago
The average Node project pulls in hundreds of dependencies. While you'd hope these would have some security vetting because of the Many Eyes theory, you have no fucking idea what your project is doing. Even a trivial Electron app is running a ridiculous amount of unreviewed third party code.
Just one module able to exercise some local exploit in your engine because you didn't fix Security Footgun #8176 screws over all of your users.
A browser engine that's been developed with a billion dollars of person hours runs that same untrusted third party code but has security guardrails everywhere.
giancarlostoro|1 year ago
jraph|1 year ago
With the interactions an electron-like app might be doing with external services and the ton of JS third party library it could use, I think it would be indeed risky.
ramon156|1 year ago
9cb14c1ec0|1 year ago
hannob|1 year ago
atum47|1 year ago
notsylver|1 year ago
fabrice_d|1 year ago
purple-leafy|1 year ago
skilled|1 year ago
Nobody builds their own browser engines. And I mean nobody. I didn’t get the feeling that he wrote this for pet projects either.
Andreas over at Ladybird is probably the only one (and Servo?) who is really doing it the way that this post describes.
Still, the last couple of paragraphs made me think that this is more of a reflection of his own time over at Mozilla: could have / would have.
roca|1 year ago
pvg|1 year ago
One such project has pretty regular HN chonkthreads, including last week.
https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...
doublerabbit|1 year ago
Anymore.
https://eylenburg.github.io/browser_engines.htm
jawerty|1 year ago
cjpearson|1 year ago
https://www.ekioh.com/flow-browser/
ramon156|1 year ago
asadotzler|1 year ago
esprehn|1 year ago
There's a fair number of other engines though like Sciter, Flow, Cobalt and Servo.
unknown|1 year ago
[deleted]
oldpersonintx|1 year ago
[deleted]
IncreasePosts|1 year ago
[deleted]
account42|1 year ago
> [bunch of things that are only relevant to an application platform]
Yeah you really don't need any of this for a web browser to be usable for its intended purpose.
Sohcahtoa82|1 year ago
But it would be riddled with security issues.
crazygringo|1 year ago
The only correct answer is, "don't".
I mean, if you want to build a toy browser engine for a CS class or fun or something, then sure.
But the idea that "you want to build an engine that’s competitive with Chromium" is, quite simply, nonsensical.
If you want your own browser engine, you're going to fork Chromium or Gecko (Firefox). I mean, even Microsoft gave up on maintaining its own independent engine and switched to Chromium.
I literally don't understand who the author thinks this post is supposed to be addressed to.
Building an independent browser engine could have made sense in 1998. These days it would cost hundreds of millions of dollars in dev time to catch up to existing engines... just to duplicate something that you already have two open-source versions of?
andirk|1 year ago
We're collectively getting better via following the same spec, which is great, but challenging the browsers and even the spec can shed light on potentially better implementations. Let's not pigeonhole ourselves to a couple vendors just because it's easier and works _good enough_.
My favorite web design is that '90s style where text went from very left to very right, barely any CSS, and Javascript (somehow my favorite language) was for fart buttons.
I'm currently most excited about keeping up with WebRTC. And yeah, I have no interest in writing a new browser engine.
CoolestBeans|1 year ago