top | item 32112846

Advice for the next dozen Rust GUIs

345 points| raphlinus | 3 years ago |raphlinus.github.io

270 comments

order
[+] pornel|3 years ago|reply
The background for this is that most existing UI toolkits are a poor fit for Rust.

Rust doesn't like having shared mutable state, but event-based UIs have a global event loop and can mutate anything at any time.

Rust works best with strictly tree-shaped data structures, but event handlers turn trees of widgets into arbitrary webs with possibility of circular references.

Rust prefers everything thread-safe, but many UI toolkits can't even be touched from a "non-main" thread.

Rust's object-oriented programming features are pretty shallow, and Rust doesn't have inheritance. That makes it awkward to model most toolkits that have deep hierarchies with a base View type and a dozen of Button subclasses.

So instead of retrofitting mutable single-threaded OOP to a functional multi-threaded language, there's a quest to find another approach for UIs. This change of approach has worked for games. Rust wasn't nice for "class Player extends Entity" design, but turned out to be a great fit the ECS pattern.

[+] jchw|3 years ago|reply
The thing is, there do exist UI frameworks that prefer composition over inheritance and strictly tree shaped components where data only flows one way: they're all the rage on the web.

And that's great, because they give plenty of useful insight into things that work and don't work when designing UI frameworks this way.

To be fair, it's not exactly like nobody realized this. More than one Rust desktop UI framework is explicitly React inspired, and it's not like FRP-based UI was non-existent prior to being popularized in web frameworks. Still, all the same... I suspect the best answers for how to do good UI in Rust are not far away from this paradigm.

[+] pohl|3 years ago|reply
Swift is different from Rust, but it's more towards Rust's end of the spectrum. Sure, it has classes, but it's almost as if they're mostly there to aid in compatibility with Objective-C APIs like UIKit. SwiftUI builds UIs with value types and protocols like View.

I wonder if something like SwiftUI would work well for Rust.

[+] nine_k|3 years ago|reply
There's one "UI toolkit" that builds in top of immutable data structures, trees, and unidirectional data flow. It has also taken web UI development by storm. It's called React.

I wonder if React-like approaches are going to work for Rust UI toolkits. They could even wrap some native controls, much like React does with DOM controls, or React Native with Android controls.

[+] epage|3 years ago|reply
I'm glad Rust* is forcing us out of a rut in UI toolkit design. All of these spell hard-to-debug problems.

* Yes, I know elm-derive UIs are also a thing and have laid a lot of ground work in this area.

btw I've had great experience 10 or so years ago doing composition-based UI design in GTK with Python. When Maemo switched from GTK to Qt, the requirements on inheritance were one of my points of frustration and I got away with creating a couple adapter classes to avoid most of it.

[+] amelius|3 years ago|reply
> Rust works best with strictly tree-shaped data structures, but event handlers turn trees of widgets into arbitrary webs with possibility of circular references.

This is also the problem with closures. And why you can't do Haskell/Ocaml style functional programming in Rust.

Rust has the reputation of being fast, but if you force everything into a tree, you are making big compromises from the start.

[+] kelseyfrog|3 years ago|reply
Is there an ECS-compatible ontology for UI components?
[+] cjohansson|3 years ago|reply
Thank you for the context, this also explains why most UIs gets broken / stuck at times. Like spinners that never stops or buttons that stay disabled even though they should be enabled. Those kinds of issues happens every day even in Apple, Google and Meta products (a lot of economical power) so there must be a deeper structural cause for this and I think it boils down to that the UI frameworks used are inherently fragile and has built-in edge case issues because of this
[+] armchairhacker|3 years ago|reply
I believe the solution for this is to have a shared context, batch changes and then run through all of them at once.

So when you e.g. mutate a variable, it doesn’t actually change until the next frame. which you can get by calling some sort of poll function. The poll function has mutable access to your entire widget tree, applies the updates, then you can have more UI with shared access

Rust is fast enough that i think performance won’t be an issue.

[+] dataangel|3 years ago|reply
Rust has no problem expressing things that aren't thread safe. If the UI framework is single threaded all it takes is making its objects neither Send nor Sync, which will likely be taken care of automatically by them having fields that are not.
[+] JediLuke|3 years ago|reply
The ‘Scenic’ GUI library written in Elixir handles all these problems very elegantly, and Elixir is completely functional (no traditional OO, classes or objects)* and immutable.

*of course Elixir/erlang is actually OO done right (actor model) but that’s another story

[+] overgard|3 years ago|reply
I think the web really killed the idea of "native" widgets. I think the rise of web apps just got everyone used to the idea that controls are just going to look different everywhere. Even on macOS, practically every app I use has a very distinct look and set of widgets (vscode, photoshop, blender, spotify -- two of those are electron, but even the non-electron stuff doesn't look very much like a "mac" app anymore). And in windows, microsoft really killed it themselves by constantly updating their own apps with non-standard widgets that everyone else wanted to clone (I remember the longest time it seemed like the only new things of note in every new version of ms office was how the toolbars looked). I suppose it's not a terrible thing, but I do kinda miss the days of windows 2000 and classic macos where the platform had a (somewhat) uniform look and feel.
[+] wmf|3 years ago|reply
I think it's more a desire for branded UI rather than Web apps. Adobe wants their UIs to be instantly identifiable as Adobe, Slack wants to be instantly identifiable as Slack, MS Office as you said, etc. Personally, I doubt this benefits users (do you really need to market to people who already bought your product?) but I'm sure the logic is compelling to decision makers and it's cheaper to "design once, run everywhere". It also makes sense to developers that their app should look the same across OSes even though that "consistency" doesn't affect the 99% of users who only use one OS.
[+] mwcampbell|3 years ago|reply
I'm the main developer of the AccessKit [1] project mentioned in this post. AMA. To preemptively answer one expected question, I know the project has been inactive for a while; I'm back to work on it in earnest starting this month.

[1]: https://github.com/AccessKit/accesskit

[+] password4321|3 years ago|reply
Do you have a Patreon/whatever setup, since "this is the way" per baby Yoda?
[+] favorited|3 years ago|reply
This is a really interesting project. Accessibility support is near and dear to my heart.

The readme mentions a macOS adapter prototype, but I don't see that directory. Does it exist in a different branch? I'd love to check it out – I'm pretty experienced with the Mac's AX implementation, but have basically no experience with AX on Windows...

[+] favorited|3 years ago|reply
> On macOS ... going forward, it’s likely that new capabilities and evolutions of the design language will be provided for the latter [SwiftUI] but not the former [AppKit].

I think this is vastly over-stating the technical churn of UI development on macOS.

While we're definitely seeing new UI toolkits introduced that are Swift-only (like the new Charts.framework[0]), nearly every Mac app Apple ships is written using AppKit. For example, I just verified (`otool -l PATH_TO_APP_BINARY`) that Mail, App Store, Notes, Music, Xcode[1], and Photos all link against AppKit, and none link against SwiftUI (on macOS 12.4 21F79, which I'm running).

There is, in my opinion, approximately zero chance that Apple either:

(A) rewrites all of the UI in all of their apps, replacing AppKit with SwiftUI, in even the medium-term, or

(B) starts treating AppKit apps as second-class citizens by introducing a new design language only available from SwiftUI.

Yes, we're going to keep seeing cool new widgets and features which are only available from Swift. No, the platform's design language is not going change in a way that makes AppKit apps obsolete.

[0]https://developer.apple.com/documentation/Charts

[1]I bet Xcode links SwiftUI somewhere for IDE integration, but I'm specifically referring to the UI implementation, parts of which have been in development since, like, NeXTSTEP and are certainly built using Objective-C & AppKit.

[+] jreese|3 years ago|reply
On the flip-side, it's worth keeping in mind where the puck is heading. Every "new" macOS app from Apple (eg, Shortcuts, System Settings, and even recent refreshes of old apps) has been written in either SwiftUI or Catalyst. A bunch of these still link to AppKit here and there, but it seems pretty clear that no new developement is breaking ground with AppKit.

Similar to the Carbon/Cocoa split, I'm sure we'll see AppKit supported well into the future, especially since we just passed the latest major architecture transition. But that doesn't mean AppKit isn't a dead end.

[+] tacotacotaco|3 years ago|reply
I would remind you that Microsoft disowned MFC and aggressively pushed all developers off of win32api once .net was released. Then they went about rewriting visual studio in .net and possibly the office suite as well. I hope Apple doesn’t attempt this because, in my opinion, I don’t think Microsoft’s products benefited from that churn.
[+] ketralnis|3 years ago|reply
Apple themselves aren't particularly quick at adopting the guidance that they give to developers. iTunes was carbon well after Apple was telling developers to stop using it
[+] pjmlp|3 years ago|reply
Apple clearly stated at WWDC 2022, that the best days of Objective-C and AppKit/UIKit are behind them, they even had a slide for it in case there were any doubts.
[+] jolux|3 years ago|reply
Aren't App Store and Photos Mac Catalyst apps?
[+] c-smile|3 years ago|reply
> The background for this is that most existing UI toolkits are a poor fit for Rust.

Being in GUI business for almost 30+ years I shall admit that this above is true. And not just Rust but C/C++ are there too.

Real (a.k.a. practical) GUIs are multi-paradigm entities. Same GUI implementation may have React'ive alike widgets immersed into purely declarative DOM/widget tree with elements of immediate mode graphics on top or inside that.

It is just that GUI reflects complexity of real life - you cannot say that sickle and hammer are best tools for everything...

Back to Rust... As a language behind UI Rust is the worst language imaginable. That's primarily due to its strictness.

Object ownership graph in GUI is usually quite complex. yet it is dynamic and frequently contain loops. This situation is best handled by GCable languages. On-click-here-highlight-the-thing-there-and-remove-that-one.

Also each part of UI declaration needs it's own DSL: for UI structure definition, style system and logic behind the UI.

Before arriving with Sciter [1] I've tried [2] many things for UI: C++, D, Java, etc.

Conclusion: HTML/CSS/JS is the best (most flexible and multi-paradigm) solution that we've got so far. Not perfect of course but good enough. And considering existence of Sciter it can be fast and lightweight.

Practical example: RustDesk (https://github.com/rustdesk/ - remote desktop access) is using Sciter for UI layer and Rust for app logic layer. So it is using proper tool for each task.

[1] https://sciter.com [2] https://terrainformatica.com/2014/07/17/10-years-road-to-sci...

[+] Animats|3 years ago|reply
I've been using egui->rend3->winit->wgpu->vulkan recently, with the goal of a program that will run on Windows, Mac, and Linux. It all mostly works. There's some dirty laundry revolving around full screen modes and window depth order, but it's not too bad.

egui, an immediate mode GUI on top of winit, is interesting. It works OK, but only does part of the job. It displays the GUI widgets, but doing something with them is your problem. Usually, you need each widget to have some persistent state. Managing that state is the user's problem. So is generating, queuing, and distributing events to and from the GUI elements. There's also something strange which causes some scrolled windows to vibrate between two states.

egui's default widgets are not very good looking. Light grey text on a dark grey background, super-thin scroll bars, that kind of thing. The aesthetics need work.

Overall, though, not bad. This stuff just needs to be used more. It has not had enough attention and polishing.

It's impressive that most of this works cross-platform, even cross-compiled. You don't even need a Windows machine to develop for a Windows target.

[+] __david__|3 years ago|reply
I just happened to build a little project using Slint (https://slint-ui.com/) last week and found it fairly pleasant, even though it's got some rough edges still. I liked it because I was able to cross-compile to Windows from my Mac and from my Linux based CI with extreme ease ("cargo build --release --target x86_64-pc-windows-gnu"). It supposedly has some sort of native widgets available if you have Qt installed, but I haven't pursued that yet.

It has a little meta language for describing the UI that gets compiled when you compile the code and that was nice because it catches type and syntax errors at compile time.

I almost went with Tauri, but the cross-compile story didn't seem quite as good compared to Slint. Tauri does look nice though—web UIs are more flexible than Slint at this point. But for my simple little tool, Slint really hit the spot.

[+] lewisjoe|3 years ago|reply
I like how the ecosystem around Kotlin has approached the problem.

1. The language itself has a way of introducing compiler plugins that lets you express your own tree structures in a succint syntax (comparable to macros in Rust I guess)

2. Then they figured out most UIs are trees and came up with a Flutter-like UI tree declaration API with react-like re-renders. Thus Android Compose was born! This went mainstream because most android devs fell in love with this new way of writing GUI

3. Like flutter, they are slowly exposing Skia (underlying rendering engine) APIs as kotlin wrappers - I guess the project's name is Skiko

4. Now with all these pieces they are building a full-fledged UI toolkit that renders over every platform (including web) - Jetpack Compose

[+] pjmlp|3 years ago|reply
Meaning Android team, no ecosystem to speak of.

There are even ADB episodes speaking about all those steps.

[+] asiachick|3 years ago|reply
> Instead of trying to decide whether a GUI toolkit is native or not, I recommend asking a set of more precise questions: > > * Does text look like the platform native text? > > * To what extent does the app support preferences set at the system level? > > * What subset of expected text control functionality is provided? > > * Complex text rendering including BiDi > > * Support for keyboard layouts including “dead key” for alphabetic scripts > > * Keyboard shortcuts according to platform human interface guidelines > > * Input Method Editor > > * Color emoji rendering and use of platform emoji picker > > * Copy-paste (clipboard) > > * Drag and drop > > * Spelling and grammar correction > > * Accessibility (including reading the text aloud) > > If a toolkit does well on all these axes, then I don’t think it much matters it’s built with “native” technology; that’s basically internal details. That said, it’s also very hard to hit all these points if there is a huge stack of non-native abstractions in the way.

This is it! Browsers handle all of this. It's a ton of work. Many many gui kits start with just ASCII and really don't realize how deep the rabbit hole is for making all of this work.

And as always, if you haven't read them

https://gankra.github.io/blah/text-hates-you/

https://lord.io/text-editing-hates-you-too/

[+] riquito|3 years ago|reply
Raphlinus, great post as usual, thanks for sharing your knowledge, is always an interesting read.

I'm loosely following the progress of slint ui (formerly SixtyFPS), anything interesting in their approach or does it strictly matches one of your examples?

[+] raphlinus|3 years ago|reply
I don't follow slint as closely as maybe I should. I'm definitely happy there's a real product out there, and would be happy to work with them on common infrastructure, but we haven't had much interaction (yet).
[+] malkia|3 years ago|reply
The elephant in the room for big CAD/CAM apps is dockable widgets, where (for me personally), the "standard" is how Visaul Studio does it.

Even Qt, with it's built-in widgets is not enough (e.g. no multiple widgets/windows in non-main window), but there are some extensions/libraries/hacks to go around.

Recently, few years ago, imgui got proper support for these too. And flutter has one or more design docs around it - e.g. multi-window support, which might be the stepping stone to these.

Desktop apps are always going to be important in this space. Our game editor (Radiant) relies on this functionality when you use two, three or even more monitors.

[+] nyanpasu64|3 years ago|reply
Surprisingly enough, Qt's built-in docking system has not one but two actively-maintained open-source third-party alternatives, KDDockWidgets and Qt-Advanced-Docking-System. (QtitanDocking is a commercial third-party docking system.)
[+] PoignardAzur|3 years ago|reply
Can you develop?

I understand the term "dockable widgets", but I'm not sure I'm visualizing the same UI as you.

[+] liquidnitrogen1|3 years ago|reply
Regarding Electron, i don't understand why anyone would want to go this route. I work in finance and desktop native apps (Winforms, WPF) have far more mature libraries, better performance and time to market compared to web GUIs. With Electron, we first write web apps and then wrap inside electron and then reinvent the wheel - this is so stupid.
[+] 1wd|3 years ago|reply
WPF is now open source (MIT licensed [1]), and its XAML control templates provide _as data_ a full declarative description of how a native Windows control is supposed to look like (in multiple Windows themes like Aero for Win7, Aero2 for Win10, Luna + Royale for WinXP, and Classic for Win95 look and feel [2]).

This includes everything like the exact colors and gradient stops and animation timing and vector shapes and accessibility behavior etc. of buttons and scrollbars and everything. Example: [3]

I wonder what one could learn / achieve trying to "port WPF to rust" / implement a XAML control template renderer in Rust. If you can "simply" parse and interpret those XAML files do you instantly get a native-like GUI that supports the exact look and feel of these different Windows themes? (on any OS!)

Somehow I think it is not realized how amazing that is!

[1] https://github.com/dotnet/wpf/blob/main/LICENSE.TXT [2] https://github.com/dotnet/wpf/tree/main/src/Microsoft.DotNet... [3] https://github.com/dotnet/wpf/blob/main/src/Microsoft.DotNet...

[+] speedgoose|3 years ago|reply
You only support windows, electron is multiplatform. About time to market it depends on your team. I don’t think there is a big difference between C#/WPF and TypeScript/React. But the second option works on more platforms and you will find competent developers a lot more easily.
[+] OJFord|3 years ago|reply
This seems to implicitly mean low-level GUIs, or GUI frameworks? The only mention of Tauri is 'tao the fork of winit used by'.

The article opens:

> A few times a week, someone asks on the #gui-and-ui channel on the Rust Discord, “what is the best UI toolkit for my application?” Unfortunately there is still no clear answer to this question.

I would say if by 'UI toolkit for my application' you just want a way to make a GUI app, not a game or some kind of highly native or specific interaction needs thing, just use Tauri. No idea why it's not a 'top contender', it has an order of magnitude more GH stars than Druid, for whatever that counts; I can only assume they mean lower-level toolkits. (No affiliation.)

I just think someone new or unexposed to rust's going to see this and think it's way harder than it needs to be or is just to do something basic.

[+] veidr|3 years ago|reply
I find this omission weird, too.

It seems to me that, maybe because of the situation described in this article, Tauri already has become, or at least is quickly becoming, the clear answer to this question.

This article seems to be implicitly excluding Tauri from consideration, because the UI portion in Tauri is not "native" or even Rust-based. With Tauri, you write the whole app in Rust except the UI.

(Technically, you could use some Rust UI frontend tech, but I think 99% of devs using Tauri are using some web UI technology, such as one of the several excellent built-in vite-powered starting points (Svelte, Vue, Solid, React, Angular...) or something similar.)

This seems to me to be a pretty good solution for the next several years, at least, because implementing a good cross-platform native UI toolkit is very very hard — so hard that, arguably, nobody has ever done it.

[+] raphlinus|3 years ago|reply
That's fair. The main reason Tauri isn't higher on my radar is that if you're going to make an app based on the web technology stack, why not just use Electron? The tooling is mature, there's lots of knowledge, and you can use Rust modules (through wasm) easily enough. I suppose it depends on the actual use case, I'm sure there is a niche for Tauri, but it doesn't intersect use cases I'm interested in particularly.
[+] the_duke|3 years ago|reply
Isn't all the discussion about the top level UX a bit of a red herring?

Iced, slint, sycamore, Dioxus , yew ... they all have quite different but very workable API surfaces. Yes, three of those target the browser, but there's nothing stopping them from building on a lower level Rust widget system instead. Even gtk-rs found a solid way to map a class structure to Rust extension traits.

Isn't the real problem in the weak fundamentals? Solid window and input handling, text rendering, layouting and 2d drawing libraries, ...

I see all the UI attempts struggling with those, and either implementing their own solutions or battling the existing ones (like winit).

I feel like if all those pieces were in place decent solutions could emerge.

[+] rustqt6|3 years ago|reply
Fwiw, I've head a great time with rust qmetaobject for some (basic) GUIs. Qt is more polished than any non-concerted (or even concerted) effort is going to be in at least half a decade
[+] billconan|3 years ago|reply
I attempted to implement my own windowing library after studying winit and SDL2.

The missing feature I need is supporting detachable tabs (like what chrome and sublime text have).

I think for productivity apps, this is very important. But implementing a windowing lib from scratch is not easy.

[+] dmitriid|3 years ago|reply
This advice is applicable to anyone building cross-platform UI toolkits, not just to Rust.
[+] Const-me|3 years ago|reply
On Windows, the graphics story is more or less OK. The OS includes Direct2D and DirectWrite user-facing APIs.

It’s now possible to implement an equivalent on all modern platforms which have a 3D GPU. An open-source example in C# https://github.com/Const-me/Vrmac#vector-graphics-engine That particular one requires GLES 3.1, but I’m sure (did it before) it’s also possible to implement comparable stuff on top of GLES2.

Waiting for target devices to have TFlops of FP32 performance, and good support for compute shaders, is less than ideal. Many currently sold phones, tablets, laptops, and even some desktop PCs, have limited GPGPU capabilities and/or performance. These currently sold devices gonna stay in use for at least couple years in the future.

Rendering things on CPU is less than ideal, because many devices have high-rez displays yet relatively slow CPU and especially RAM.

Am I missing something? What’s the reason there’re no cross-platform GPU-first 2D graphics libraries, despite GLES2 (or better equivalents) is now universally available, and have been for years?

[+] pbsurf|3 years ago|reply
Getting sufficient antialiasing quality for 2D graphics is difficult on GPUs. https://github.com/memononen/nanovg accomplishes this with GL2/GLES2 level hardware for most of the stuff one would want to render as part of a GUI. My project https://github.com/styluslabs/nanovgXC supports rendering arbitrary paths with exact coverage antialiasing, but requires GLES3.1 or GL4 level hardware for reasonable performance.
[+] mwcampbell|3 years ago|reply
> Waiting for target devices to have TFlops of FP32 performance, and good support for compute shaders, is less than ideal. Many currently sold phones, tablets, laptops, and even some desktop PCs, have limited GPGPU capabilities and/or performance. These currently sold devices gonna stay in use for at least couple years in the future.

I'm reminded of how Windows Presentation Foundation relied on a level of GPU-based 3D acceleration that wasn't yet universally available on non-gamer PCs when Vista came out. This affected my brother when he was in seminary and tried to run the WPF-based Logos 4 Bible software on a budget laptop bought around 2007 (edit: actually 2008 IIRC). By 2011 it got bad enough that I just bought him a new laptop. And he wasn't the only one affected by this; there were others talking about it on the forum for that application [1]. This anecdote has stuck with me as a cautionary tale about us developers failing to put our users first in our technology choices (though I admit it hasn't actually stopped me from using Electron when the pressure to ship is on).

[1]: https://community.logos.com/forums/t/6200.aspx

[+] stefan_|3 years ago|reply
Huh? That library exists, it is called Skia, it is the most premier 2D GPU (among other targets) rendering library in the world and it powers the Android UI, Chrome, Firefox and a ton of other things. See also: https://skia.org/
[+] magicalhippo|3 years ago|reply
> What subset of expected text control functionality is provided?

Missing from this otherwise great list, is basic system-standard text navigation and editing. I get so annoyed when basic stuff like ctrl+end or ctrl+v doesn't work like they should.

[+] game-of-throws|3 years ago|reply
Small comment: It's possible to stitch together UI/video/3D without dealing with the compositor, if you use child windows instead (or in wayland terms, a subsurface). On win32 at least, it's a much simpler approach.