[stoev]

I actually just submitted something quite relevant - a side project we soft-launched today that uses LLMs to analyse and group browser tabs across various browsers and devices. It turns out that it’s super demanding for today’s models, but with some optimisations and experimentation, we made it work pretty well.

Running local LLMs might be a heavy order currently to get the job done, but in the near future I don't see a reason why it couldn't be done and it would definitely be quite helpful to people who have either many tabs open or a lot of bookmarks they want to organise.

Ironically, Opera's extension shop is dead and no one reviews new submissions, but for anyone else who is interested in learning more about how we use LLMs to improve the browsing experience or even give the product a shot, here is a link:

https://tabcrunch.com/technical_deep_dive.html

[authorfly]

This is useful. Why the step of involving LLMs though? I note you cluster the tabs and then GPT-4 is involved to name them. But my tab groups don't usually need names - just the icons tells me what I mostly need to know. Could this work locally better using much smaller sentence-transformer models?

[stoev]

That's a good question. If you have many tabs open from the same few websites (depending on what those websites are), maybe just grouping them based on domain names would be enough to provide context.

But LLMs are needed if you want the product to have a deeper understanding of everything that you are reading and really organise it into groups. You might be reading about architecture across three devices, multiple browsers, from a bunch of different websites. This gives you the opportunity to reunite them and really dive into that topic when you need to.

LLMs are also used to create summaries of each page. So, if some content takes 30 minutes to read, you can have them extract all the interesting information for you in bullet points and based on that you can decide if its worth spending the 30 minutes or if you would rather just close that tab.

So, in short, it's about capabilities. You can have just simple statistical models and regex rules filtering similar websites into predefined categories, or you can have a tool that truly organises your reading and shortens it for you. But for the latter, you need complex models handling a lot of context.

[passion__desire]

I think LLMs could replace plugins like DownThemAll and so on with more flexible interface.