I'm reading a lot of criticism here, some I understand, but honestly, this sounds pretty good for a free LLM service: "Orbit doesn't require account creation or save your session data."
Maybe we shouldn't be too critical of Mozilla for providing a privacy-first LLM service - for free. The other big privacy-first LLM provider is Apple, which requires users to have their devices/subscriptions to use, and definitely uses advanced telemetry by default.
Not to speak of all the other providers who are either paid or free-but-mine-your-data.
>Maybe we shouldn't be too critical of Mozilla for providing a privacy-first LLM service
It is just Mozilla have a tendency of chasing hype rather than focusing on what they are doing. During early smartphone era they spend most of the resources trying to write an OS with Javascript ( Firefox OS ) that works on a $35 Smartphone.
Now they are doing it again with AI. Although this time around Firefox is in fairly good shape I guess this isn't too bad. But they need to figure out a way to generate revenue rather than relying on Google. And LLM service isn't it.
This is a local device LLM, outdated of a whole year, which is massive for LLMs, running on servers.
Privacy side, there is little reason to run this on servers, a Pixel 6 could run 7B models at 5 token/s a year ago.
It's bad by incompetence, 7B models of a year ago were terribly bad.
It's not privacy-first enough, as it's possible to run the AI directly in the browser, but for some reason they didnt do it.
It is overly generous to describe this as "privacy first." This looks like it's one ToS change away from being a privacy violating service.
In Apple's case, they are putting some amount of work into making their privacy claims verifiable. Good will is no longer good enough. Verifiability should be the bar for trust in 3P privacy claims.
> Maybe we shouldn't be too critical of Mozilla for providing a privacy-first LLM service - for free. The other big privacy-first LLM provider is Apple, which requires users to have their devices/subscriptions to use, and definitely uses advanced telemetry by default.
There is a very fundamental critique here: a service being offered for free like this is being subsidized, messing with the general market dynamics that really should be making all of these tools cost way more money to begin with.
Of course Apple is also doing similar things, but for Mozilla to be doing it is quite frustrating.
Privacy-first generally entails designing the service on a technical level to be unable to leak personal data. A promise not to store your data is not sufficient.
> The other big privacy-first LLM provider is Apple, which requires users to have their devices/subscriptions to use, and definitely uses advanced telemetry by default.
By definition then, Apple's is most certainly not privacy first by any stretch of mental gymnastics.
That is always how this stuff works. Capture people with nice features and a friendly approach and then rug pull. There is zero chance I ever use this BS. I don't trust Apple either. I do not want LLM integration period because no corporation can ever under any circumstance be trusted with privacy in regards to it.
I guess I'm really not this extension's target audience, because somehow Mozilla managed to make me uninstall it in just a few minutes.
First, they have forced telemetry. Okay, it's an early release, it's very basic information, they want to understand how they're doing - I don't like it, but I can understand it. Sets a wrong vibe, though - I had to check if it was from Microsoft and not Mozilla. ;-)
Then, I figured there is no option to use locally-hosted LLMs, which can be something as minimal as simply allowing to configure custom API URL. Somehow, less and less things about Firefox are tinkerer-friendly than they used to be.
That made me wonder if Mozilla used OpenAI-like API, or if they invented their own unique thing for some reason. So I went to look and according to the extension page, it's proprietary ("All Rights Reserved") and I'm too lazy to bother deminifying code from the xpi or remembering how to debug extensions.
Finally, '00s have called and said they wanted their weird floating round thingy UI back, and so I had to return it to the store. (I'm kidding, I don't really mind the visual style. The real issue with that thingy was how it floats on the page. Like, why on Earth it isn't normal right-click menu option that doesn't obstruct the view until it's needed? Or a menu on that toolbar button? It's not even a paperclip to be worth it.)
And then I realized I somehow missed the big "AI you can trust" header, which should've already been a huge red flag.
> For the current version, we are using a Mistral LLM (Mistral 7B) hosted within Mozilla’s GCP instance.
Based on this I would assume they are using GCP Vertex AI as that's going to be WAY cheaper than rolling it all themselves and hosting the model on a GCP server instance. I would also assume they'd be using the gcloud SDK for Vertex AI/Model Garden, which I believe means they can't just provide for a different endpoint and payload shape if you had a service elsewhere.
Eitherway, at the (presumed) scale they'll probably also be using GCPs API management service, so I would expect further abstraction between what the extension is sending and what the model/Vertex AI expects as a payload. This means providing that kind of "bring your own endpoint" experience would require more bespoke build-out time.
BUT who knows? Maybe this is just straight up hitting the out-of-the-box GCP Vertex AI REST API directly from the extension like some hobby project.
- Currently using Mistral 7B, but ability (by Mozilla) to swap the model used to another open source at any point.
- Hosted "by Mozilla" on their GCP instance.
- No obvious info about what it will cost them to run this since it is free to use.
- No training on user data.
Like others here, I'm very curious about the cost for Mozilla to run this service. It may be less than it initially appears given the 7B model they chose. I do wish they would focus their efforts on creating a very long-term endowment to pay devs for continued Firefox development in lieu of projects like this given the tenuous situation with their Google funding.
I'm not against this kind of thing in theory, but I hope it's being done in a cost-sustainable way.
If you scroll to the bottom of the page it appears like this is made by Fakespot, https://www.fakespot.com/our-mission. They explain that they make money through ads, for their other similar product, wouldn't be far to guess that's the future strategy here as well.
> The Orbit add-on by Mozilla is a new AI-powered tool that summarizes and answers queries about web content, including articles and videos. It uses a Mistral LLM (Mistral 7B) hosted on Mozilla's GCP instance. The add-on is free to use and works on various websites, including Gmail, Wikipedia, NY Times, YouTube, and more. However, some users have raised concerns about the size of the model and its privacy implications, as well as the fact that it requires an internet connection to function. Additionally, some users have suggested that Mozilla should focus on improving the browser itself rather than developing new add-ons.
I installed it and it's a giant floating popup that's permanently on your screen. You can enable the "minimal" theme in the settings to turn it into a smaller-but-still-big pill shape. It doesn't look like there's a way to hide it in a context menu, at least not one I can see.
I don't fancy having a random floaty object in the way of my webpages, no thank you.
Edit: It appears to go away occasionally. This UX is unclear to me.
If you pin the extension in the toolbar, there's an option to "disable" the extension. The floating popup goes away and using the extension to summarize still works.
Yeah, I can't imagine what lunatic thought it would be a good idea to have this stupid orb floating inside the page. Heck, it would be fine if you at least had an option to move it into the toolbar where it belongs...
I'm encouraged that they're actively exploring this and not shying away from experiments. It seems clear to me that there are areas of Mozilla[0],[1] pushing closer and closer to great local AI integrations doing the kinds of things that I, a browser user, find useful. I went to some of the articles on the hn frontpage and had questions (and followups!) that begat reasonable starting points for further learning.
Hopefully they continue to iterate on this with better integration (for instance, moving to a toolbar icon instead of persistent badge on every page) and then make it ~truly privacy respecting by moving locally.
Buried as the last sentence in a collapsed box at the bottom of the page:
> For the current version, we are using a Mistral LLM (Mistral 7B) hosted within Mozilla’s GCP instance.
And why is it "...Mozilla's GCP instance", not "We quietly send all your data to Google servers, and everyone pinkie-swears that's totally privacy-respecting"?
I'm skeptical of a lot of Firefox side quests, but I encourage this one. Orbit could be a stepping stone to an Open Source version of Google Project Mariner. A n un-nerfed built-in AI could turn browsers into true user agents that work on your behalf.
Unfortunately no BYOLLM. Brave supports bringing your own LLM e.g. through Ollama
Besides that I'm using AI Summary Helper plugin for Chromium-based browsers https://philffm.github.io/ai-summary-helper/ which also allows using Ollama (or OpenAI / Mistral), asking questions to articles and inserting summaries right into the DOM (which is perfect for hoarding articles / forwarding them to Kindle)
Sort of funny for the lack of local options considering that Mozilla funds llamafile even. Hopefully they allow some API integration, if they are using standard OpenAI API calls, it should be easy to enable swapping the endpoint.
Also, while it's nice to have a service option for those without any spare compute, I think it's a bit of a shame on the model considering how even at the 7B class, models like Llama 3.1 8B, Qwen 2.5 8B or Tulu 3 8B, Falcon 3 7B, all clearly outclass Mistral 7B (Mistral 7B is also very bad at multilingual, and is particularly inefficient at multilingual tokenization).
The current best fully open weights (Apache 2.0 or similar) small models currently are probably: OLMo 2 7B, Qwen 2.5 7B, Granite 3.1 8B, and Mistral Nemo Instruct (12B)
There's been a recent launch of a "GPU-Poor" Chat Arena for those interested in scoping out some of the smaller models (not a lot of ratings so very noisy, take it with a grain of salt): https://huggingface.co/spaces/k-mktr/gpu-poor-llm-arena
Installing the extension enables a floating widget in the webpage. Horrible implementation. Couldn't it have been integrated into the browser better, like the Reader button?
Looks like there is no way i can use it without giving up on screen real estate. I'd love to have it pinned in the browser toolbar and I click it when I need it. But it wouldn't allow itself to be hidden and needs to have a floating circle taking up page real-estate.
After pinning the extension, click on it and deselect the "enabled" option (the line with the purple X). This will kill the floating orb UI but you can still click on the extension in the toolbar to use it.
Am I the only person who isn't hugely interested in summarizing emails? I don't want emails summarized because that bypasses a tell-tale sign that said email isn't worth reading in the first place.
I kinda want to run some of our CEOs emails through summarizing, because I feel that there's a very real chance that I'll just get a black page back.
Other than that, I'm with you. The need for summarizing is a symptom of our increasingly poor communication and degrading writing skills. That, and SEO optimization which attempts to hit as many keywords as possible.
We're heading in a direction where people will use LLMs to pad their writing, so it will appear more substantial then it actually is. The receiver will then parse it though another LLM, because the writing has now become to convoluted, or they simply don't have the time for a ten page essay (in which case the none padded draft would have sufficed).
Admittedly I have found a few useful cases for LLMs, mostly related to text parsing and information extraction, which can be seen as summarizing I suppose, but mostly I have a pretty negative view of LLMs. Part of it may be me getting older and not fully understanding how they work, partly it's also their deployment in areas where I believe communication should be human to human.
I'm vehemently against the whole Apple Intelligence feature of summarizing personal communications, like the AI-generated break-up text summaries that were floating around. But I've found most other AI summarizing to be pretty useful. The problem is the lack of trust and reliability, so even when those summaries seem to save me time, if the topic is of any actual significance or value I'm forced to read the original materials anyway, which ultimately results in spending more time consuming the same information.
I agree, I've read about lots of LLM-based services for summarising content, and I really wonder whether that's because this is something so many people want, or if it's just because it's something LLMs are good at so they are easy to build and then they get hyped because LLM.
Anything that can be expressed in a few sentences, should be, and I tend not to read media that doesn't abide by that rule. If I'm reading long-form content it's because I am looking for detail and nuance that would be lost in a summary.
I do think a reliable video summary generator could be useful occasionally. Interestingly Orbit seems to work on YouTube, presumably by parsing YT's auto-generated transcript.
Honestly, I don't receive any emails which are even worth summarising. I unsubscribe from marketing emails, and the humans who email me aren't writing more than a couple paragraphs at most. It becomes more effort to hit a summary button, wait for it to generate, and then read something of roughly the same length.
> Orbit is a Firefox add on that uses AI to summarize and answer queries about web content such as articles and videos.
> When a user asks Orbit to summarize or query content, Orbit gathers the context (eg. text, images, videos, etc.) of the page the user is viewing and provides a summary or answer. Orbit works on websites including Gmail, Wikipedia, NY Times, YouTube, and more.
> For the current version, we are using a Mistral LLM (Mistral 7B) hosted within Mozilla’s GCP instance.
It's interesting they're going with Mistral 7B. Is anybody else using Mistral 7B in production? And in what role?
I've considered using it for general knowledge type questions, and as a way to classify information, but would have never considered it for summarization type tasks due to it's limited context size (8k).
The only useful (or even paid) browser-integrated AI service I can imagine using would be a browsing history-aware AI chatbot. Essentially, it would just spit out a link from my history based on the context or prompt I give. Since privacy will be a crucial factor, I can imagine building an extension that reads page contents, stores them in a database, and connects to a self-hosted LLM.
When your organization is so broken that it's easier to buy a separate domain for your team, than to go through the process of putting it under the main company domain.
Yeah, I really don‘t understand why companies on one side try everything in their power to teach people to be vigilant of phishing and then do stuff like this. Azure does it too with www.microsoftazuresponsorships.com I always feel like one of these days I will get phished if those are the domains they force me to use :(
I think someone put it best: make your competitor depend on your evil thing (in this case, google donates to firefox/mozilla and makes deals about default search). In addition, people intentionally go to firefox because it /isn't/ chrome, but that only works SO LONG AS they don't copy everything google does (e.g. opt-out ads in the browser, AI, etc)
- It's clear from user share graphs that Firefox as just a browser is tending towards irrelevance. No amount of "improving the browser" is going to solve the problem.
- More fundamentally, the browser is just one portal to the internet / world wide web. With technology outside the browser getting increasingly sophisticated, Mozilla necessarily needs to expand their mandate beyond Firefox in order to serve user needs and influence the landscape. Otherwise we might easily end up in a future where the browser becomes irrelevant and everybody interacts with proprietary large models.
- As far as innovation in browser features goes, this seems like a breath of fresh air. Internet users at large deserve access to AI services in a secure and privacy-friendly, and as a pillar of the free web Mozilla is well-placed as a distribution channel to serve these needs. Therefore, this seems like a very good stepping stone / experiment for Mozilla.
There will be execution challenges that need to be figured out. AFAIK Mozilla doesn't have the talent+budget for training large AI models, or even for doing intensive product research. So they're going to have to team up with some other AI expertise -- either explicitly or implicitly, by depending on open source models. Regardless, IMHO this is a risk they have to take and figure it out as they go along.
There is a (to me very surprising) typo in the section 'Focus on what matters.' where the AI summary states "... 11 out off[sic] 100 products ..."
I don't think I've ever encountered a typo in any of the LLM output I've seen, seems like the exact sort of thing an LLM would be more or less perfect at.
Am I wrong to take this as an indication that this text is actually written by a human as a concise marketing example?
Can I just be blunt here and ask, why the F? No one asked for this. No one wants this. We just want a good, standards-compliant browser that doesn't eat up 8gb of RAM watching a single YouTube video (something that seems to be quite a challenge, though not entirely the browser engine's fault but rather the runaway train that is "capitalistic motives dictate the browser is now an OS and needs all features thereof"))…
For those wondering, this uses cloud-hosted AI models. But it's completely free to use so Mozilla is just paying the cloud bills out of pocket. Maybe not the best idea given their increasingly precarious financial situation?
I never understood why Mozilla doesn't offer paid privacy respecting services (outside basic sync that already exists) like email, cross platform password storage etc.
I have to ask: why, exactly, does it make sense for Mozilla to invest heavily into running expensive servers to run vanity chat bots for people? What's the path here to something which improves their financial situation or browser market share? How isn't this just yet another random service they'll throw money into for a couple of years before shutting down?
Because if they don't they will not have feature parity with Chrome (Gemini) and Edge (Copilot).
And while not all people are fond of AI, there are shit tons of people out there who do. Which means you automatically diminish your market share if you don't (because your most important competitors do)
Interesting product, very relevant in our time and age.
My guess is this could be useful to many "knowledge workers" who constantly have to crawl, translate and find the meaning of the sugar coated landfill that has become most of the web.
We are right in the middle of the Tower of Babel story.
Seriously if it works reasonably well on legal fine prints I am in.
Mozilla first needs to regain the trust of its users. They aren't privacy-first at all, they run a lot of telemetry on their users. Have you ever checked the number of DNS queries and IP requests that go to Firefox servers every minute? If you haven't, I have, and it is a lot. They literally ping home every minute in the name of network connectivity and other things, if you believe it. I don't mind using Mozilla products, but I just don't trust their motives and data practices.
If you want privacy first AI in the browser here are the tools
In the same area of privacy-aware AI there is Jolla Mind2. It's your own computer, so probably even more trustworthy than Orbit by Mozilla?
I have not studied either product in depth, so I am unable to comment on commonalities or differences.
Jolla has a mixed track record: They supported some phones over 10 years with decently working software (typing on one of those). They also failed at least once to deliver a crowd-sourced tablet to most of the backers. Not a risk-free choice, but at least someone trying to do the right thing.
I wonder of this or something like it can be extended to "unclickbait" videos/article titles by spawning a background crawler that reads the article / watches the video and comes back with a resolution to the curiosity bait they used to get you to click. Would save countless hours and make the web less shitty IMO. Plenty of examples available just scrolling through youtube for training data.
such a random product. uses an old tiny model, they are paying the gcp bill, and it's to summarize content?
why not use their technical expertise to built an in-browser "https://big-agi.com/" of sorts where users can paste in their API key and use bleeding edge models in combination with the browser's data which they could expose and manipulate as the creators of Firefox!
this product seems really random and quite frankly weird.
I really don't get the comments that they should not focus on anything else than the current browser.
If other browsers start adding llms, I bet those same people will start complaining that Firefox is outdated compared to those browsers in about a year.
Just keep it as optional extension and that will be perfectly fine. They should make it compatible with other browsers too. Considering llamafile project, maybe there will be an option for offline assistant, where user will be able to select their preferred model?
Hopefully Mozilla will eventually come up with a local-AI in the browser model, like the one currently being explored by Google Canary called Gemini Nano (which Google of course doesn't seem to want to make available in Chromium though).
It feels very resistant to doing anything other than summarizing. Even when you ask questions for details, the answer is always in the form of a simple summary.
From a product standpoint, I am curious if sidebar is the right way to integrate AI features in the browser? Did anyone see any better integrated solutions?
Tab groups are on by default in nightly, and though I haven’t been on the computer much this past week they seem well realised. This article suggests they can be enabled in other versions, but I only really use nightly so I haven’t tried it..
Their browser is fine. But so is Chrome, and Chrome wins by default because it's installed by default on Android and Android really pushes you to it. Once you're using Chrome on Android and have all your passwords saved there, Firefox is a difficult sell.
I can't think of a single Firefox feature that's better than "I don't have to faff around with passwords". Maybe if they allowed adblock on mobile, but last I checked they were fannying around requiring nightly builds and whitelisting extensions...
In my opinion, the browser has been much better since 2017/2018. Have you used it in the last few years? Complaints about Firefox aside, the Manifest V3/uBlock Origin issue should be a major concern for tech-savvy Chromium-based browser users.
Firefox is genuinely fine, great even, I've used it for years and have no complaints.
But you notice in all these threads, everyone who theoretically ought to use Firefox comes up with their own little list of nitpicks that justify them not using it.
"I can't use it because I was disgusted when they dropped feature x"
"I won't use it because they spent their money on feature y instead of just doing z"
Meanwhile Chrome doesn't give a fuck what you think and does whatever it wants and people keep using it regardless.
Firefox is doomed to be left with the niche audience of people who ignore the 95% of what it does right to focus on the 5% that it does wrong.
It looks like a nice marketing product, but nowhere it acknowledges how bad LLMs are in general for summarization, or what techniques, if any, the devs use to counteract this. (also, their dev page https://orgsoft.org/ claim "NO BS", but again does not provide details, proof, not even an about page or a business address which is a red flag.
I use Librewolf (based on Firefox), but about once a year I open Chrome for some shitty website that only works on Chrome. And I use Chrome for a few minutes.
It shocks me every time just how fast Chrome is. It is legitimately a superb piece of software. Going to Librewolf after feels like going back ten years in hardware.
Can we please start spending some money to make Firefox better? Instead of whatever Mozilla is currently doing?
Firefox Quantum was great. But why stop? Just keep doing that! It's the only thing you should be doing!
In a separate comment, I mentioned how Mozilla should have been more like Proton with their cloud storage, VPN, password manager, and cloud office suite.
In fact, they should have done that a decade a ago.
Mozilla has been around since the late '90s and should have evolved beyond just being a browser company. They launched a VPN service when VPNs were already everywhere, and they did the same with a bookmark manager when others were already offering similar solutions. Mozilla is always catching up, never leading, and that's a common issue with many big open-source and free software companies. They often pretend to be a business that isn't heavily propped up by big tech donations.
If I were leading a browser company, my focus would have been aggressively directed towards small business software. I’d create an internet and privacy-focused affordable minimal business software suite that lives within the browser — a combination of Proton and Zoho. And I’d strongly avoid building things that should be browser extensions.
Why does Mozilla always spin their wheels releasing random side projects? Cloud-hosting an outdated tiny LLM that you can't swap out or run locally, to do basic summarization? This just doesn't feel like an area of strategic focus that makes sense for Mozilla. GPUs are expensive, talent to do inference well is expensive, and the actual product they're shipping seems pretty marginally useful at best.
If they shipped vertical tabs I'd probably switch back to Firefox. But that would require focusing on actual browser UX instead of random offshoots. What's their actual product differentiation from Chrome these days?
For me, the product differentiation of Firefox is a bunch of small convenience features which Chrome in its monolithism refuses to provide, such as:
Allowing Backspace to go back a page.
The built-in screenshot tool.
Being able to turn off smooth scrolling.
Support for a menubar, so that I can navigate the features I want quickly.
Being able to choose page encodings (I use a non-Latin charset language).
A usable/useful bookmark manager with things like sorting, tags/labels, timestamps, etc.
"Restore Previous Session" feature.
These are just a few features off the top of my head, I know there are many more.
Of course, for certain sites like Google Docs and YouTube, I have to use Chrome for obvious reasons, but for most browsing I use Firefox (and qutebrowser.)
They seem to feel that they will be marginalized if Google stops paying them to set their default search engine to Google but, the way they have handled it is to focus on everything else other than their browser. At this point since browser engines are dominated by Blink and Webkit what exactly does Mozilla have? Their market share just keeps on going down.
- continued support for manifest V2 (primarily because ublock origin would stop working if forced to V3 only)
- the firefox address bar is way smarter for any given string i type in than chrome's. it's ability to surface the most relevant deeplinks from my history, vs top level site, vs perform a web search, is night and day difference from the randomness that other browser search bars offer.
- I have the opportunity to use Zen (a Firefox fork) [0] and it's 100% interoperable with my vanilla Firefox instances across devices -- i can even send tabs from my Firefox Nightly on Android to my Zen instances on Windows or Mac. BTW Zen has vertical tab support, (more) first-party multi-profile support, and preserves the address bar behaviors of vanilla Firefox.
Building a browser is hard. Building a proof of concept of the current tech fad is easy and fun. Sometimes developers need to do easy and fun things to keep themselves motivated and happy.
You could build an AI Assistant, or you could spend a month bikeshedding some design details of vertical tabs.
Well, you're in luck, sort of. Mozilla has vertical tabs in a new sidebar experience. It's the worst implementation of vertical tabs and a sidebar I've seen in a browser. Complete with typical Firefox UX, it's completely inconsistent and unintuitive to disable. A complete farce compared to Sidebery.
> Why does Mozilla always spin their wheels releasing random side projects?
Because some project manager had "AI" on his performance goals this year.
Firefox is buggy as hell - which is incomprehensible given its age, but older brother Netscape had the same problem 20-ish years ago. The Netscape 4.x days were absolute hell and you could go hardly a day without the browser crashing.
Despite this, it's packed with nonsense no one asked for like Pocket. Which is a coincidence because "AI assistant for Firefox" is the dictionary definition for redundant things no one asked for, with better alternatives preexisting.
At this point Firefox needs to die and something new and manageable - with energetic developers - needs to take its place. Maybe Microsoft could open source the original Edge engine? The one before they bent over for the long dick of Google Chrome.
I'm sympathetic to the viewpoint but the idea that Mozilla would attract more users via vertical tabs than by an AI assistant strikes me as flat out wrong.
Mozilla's brand is "pro privacy", it does make sense for them to launch an AI product with that brand position. I doubt it'll be successful because I don't think enough people actually care about privacy, but still.
I feel like it's a common HN sentiment to say "why don't Mozilla just focus on the browser?!"... the answer is because barely anyone is using it and there's very little they can do to move the needle on that. IMO they're an organization looking for a purpose.
Firefox needs to be forked and owned by the people. I'd pay $1 a month for a browser run by the people. Management and extension options could be rolled out for government, business, education, etc. There are so many models where the browser thrives, the org that shepherds the browser thrives, and the people thrive.
Firefox is like the shitty best option that camps out in its niche, it sucks but it is really hard to push out of the way.
Just seeing the title as announcing some new mozilla service my first thought was "What personal data does this new mozilla feature send to cloudflare?" -- turns out the answer was emails and documents but to their own google cloud accounts rather than cloudflare.
Of course, no option to use a local model even though the one they're using is small enough that its perfectly reasonable to use locally. Even on a cell phone.
Given the broader Mozilla foundation's political biases, will these assistants be censored or heavily curated? I am reminded of when Mozilla chose to ban Dissenter, a free speech plugin powered by Gab, from its store. I've never used it but I found it distasteful that a company working on a basic utility program decided to become political. I don't understand why they cannot just focus on the basics and get those right. Still waiting for proper vertical tabs.
What is a free speech plugin? were there political discussions in these threads? If so it seems like they did the correct thing removing such political plugin right?
> basic utility program decided to become political
Some comments were deferred for faster rendering.
mentalgear|1 year ago
Maybe we shouldn't be too critical of Mozilla for providing a privacy-first LLM service - for free. The other big privacy-first LLM provider is Apple, which requires users to have their devices/subscriptions to use, and definitely uses advanced telemetry by default.
Not to speak of all the other providers who are either paid or free-but-mine-your-data.
ksec|1 year ago
It is just Mozilla have a tendency of chasing hype rather than focusing on what they are doing. During early smartphone era they spend most of the resources trying to write an OS with Javascript ( Firefox OS ) that works on a $35 Smartphone.
Now they are doing it again with AI. Although this time around Firefox is in fairly good shape I guess this isn't too bad. But they need to figure out a way to generate revenue rather than relying on Google. And LLM service isn't it.
Kuinox|1 year ago
It's bad by incompetence, 7B models of a year ago were terribly bad. It's not privacy-first enough, as it's possible to run the AI directly in the browser, but for some reason they didnt do it.
cdata|1 year ago
In Apple's case, they are putting some amount of work into making their privacy claims verifiable. Good will is no longer good enough. Verifiability should be the bar for trust in 3P privacy claims.
rtpg|1 year ago
There is a very fundamental critique here: a service being offered for free like this is being subsidized, messing with the general market dynamics that really should be making all of these tools cost way more money to begin with.
Of course Apple is also doing similar things, but for Mozilla to be doing it is quite frustrating.
FireInsight|1 year ago
aforty|1 year ago
Sephr|1 year ago
politelemon|1 year ago
By definition then, Apple's is most certainly not privacy first by any stretch of mental gymnastics.
67n76n67n6|1 year ago
kome|1 year ago
throawayonthe|1 year ago
[deleted]
makingstuffs|1 year ago
drdaeman|1 year ago
First, they have forced telemetry. Okay, it's an early release, it's very basic information, they want to understand how they're doing - I don't like it, but I can understand it. Sets a wrong vibe, though - I had to check if it was from Microsoft and not Mozilla. ;-)
Then, I figured there is no option to use locally-hosted LLMs, which can be something as minimal as simply allowing to configure custom API URL. Somehow, less and less things about Firefox are tinkerer-friendly than they used to be.
That made me wonder if Mozilla used OpenAI-like API, or if they invented their own unique thing for some reason. So I went to look and according to the extension page, it's proprietary ("All Rights Reserved") and I'm too lazy to bother deminifying code from the xpi or remembering how to debug extensions.
Finally, '00s have called and said they wanted their weird floating round thingy UI back, and so I had to return it to the store. (I'm kidding, I don't really mind the visual style. The real issue with that thingy was how it floats on the page. Like, why on Earth it isn't normal right-click menu option that doesn't obstruct the view until it's needed? Or a menu on that toolbar button? It's not even a paperclip to be worth it.)
And then I realized I somehow missed the big "AI you can trust" header, which should've already been a huge red flag.
ajcp|1 year ago
Based on this I would assume they are using GCP Vertex AI as that's going to be WAY cheaper than rolling it all themselves and hosting the model on a GCP server instance. I would also assume they'd be using the gcloud SDK for Vertex AI/Model Garden, which I believe means they can't just provide for a different endpoint and payload shape if you had a service elsewhere.
Eitherway, at the (presumed) scale they'll probably also be using GCPs API management service, so I would expect further abstraction between what the extension is sending and what the model/Vertex AI expects as a payload. This means providing that kind of "bring your own endpoint" experience would require more bespoke build-out time.
BUT who knows? Maybe this is just straight up hitting the out-of-the-box GCP Vertex AI REST API directly from the extension like some hobby project.
likeabatterycar|1 year ago
Finally someone admits that BonziBuddy was 25 years ahead of its time.
ericra|1 year ago
- Currently using Mistral 7B, but ability (by Mozilla) to swap the model used to another open source at any point.
- Hosted "by Mozilla" on their GCP instance.
- No obvious info about what it will cost them to run this since it is free to use.
- No training on user data.
Like others here, I'm very curious about the cost for Mozilla to run this service. It may be less than it initially appears given the 7B model they chose. I do wish they would focus their efforts on creating a very long-term endowment to pay devs for continued Firefox development in lieu of projects like this given the tenuous situation with their Google funding.
I'm not against this kind of thing in theory, but I hope it's being done in a cost-sustainable way.
Too|1 year ago
roter|1 year ago
> The Orbit add-on by Mozilla is a new AI-powered tool that summarizes and answers queries about web content, including articles and videos. It uses a Mistral LLM (Mistral 7B) hosted on Mozilla's GCP instance. The add-on is free to use and works on various websites, including Gmail, Wikipedia, NY Times, YouTube, and more. However, some users have raised concerns about the size of the model and its privacy implications, as well as the fact that it requires an internet connection to function. Additionally, some users have suggested that Mozilla should focus on improving the browser itself rather than developing new add-ons.
ChrisArchitect|1 year ago
https://connect.mozilla.org/t5/discussions/try-orbit-by-mozi...
RadiozRadioz|1 year ago
I don't fancy having a random floaty object in the way of my webpages, no thank you.
Edit: It appears to go away occasionally. This UX is unclear to me.
aabiji|1 year ago
TheAceOfHearts|1 year ago
93po|1 year ago
unknown|1 year ago
[deleted]
hu3|1 year ago
Tested on Firefox and Chrome.
Some websites block ChatGPT and can't be summarized this way.
Works in incognito/anonymous mode and doesn't require a ChatGPT account.
You can probably use another AI service with this idea.
simonw|1 year ago
mdaniel|1 year ago
I cannot access external links directly. However, if you provide the text or key points from the page, I can help summarize it for you!
and clicking the model selection drop-down produced "Log in to try advanced features"
bufferoverflow|1 year ago
usernomdeguerre|1 year ago
Hopefully they continue to iterate on this with better integration (for instance, moving to a toolbar icon instead of persistent badge on every page) and then make it ~truly privacy respecting by moving locally.
[0] https://github.com/Mozilla-Ocho/llamafile [1] https://github.com/mozilla/translations
neilv|1 year ago
> Commitment to privacy
Buried as the last sentence in a collapsed box at the bottom of the page:
> For the current version, we are using a Mistral LLM (Mistral 7B) hosted within Mozilla’s GCP instance.
And why is it "...Mozilla's GCP instance", not "We quietly send all your data to Google servers, and everyone pinkie-swears that's totally privacy-respecting"?
BadHumans|1 year ago
ripped_britches|1 year ago
lordofgibbons|1 year ago
Also, does anyone know if we'd be able to point it to our own LLM instance for the guarantee of our data being secure?
JonChesterfield|1 year ago
xnx|1 year ago
ph1lw|1 year ago
Besides that I'm using AI Summary Helper plugin for Chromium-based browsers https://philffm.github.io/ai-summary-helper/ which also allows using Ollama (or OpenAI / Mistral), asking questions to articles and inserting summaries right into the DOM (which is perfect for hoarding articles / forwarding them to Kindle)
lhl|1 year ago
Also, while it's nice to have a service option for those without any spare compute, I think it's a bit of a shame on the model considering how even at the 7B class, models like Llama 3.1 8B, Qwen 2.5 8B or Tulu 3 8B, Falcon 3 7B, all clearly outclass Mistral 7B (Mistral 7B is also very bad at multilingual, and is particularly inefficient at multilingual tokenization).
The current best fully open weights (Apache 2.0 or similar) small models currently are probably: OLMo 2 7B, Qwen 2.5 7B, Granite 3.1 8B, and Mistral Nemo Instruct (12B)
There's been a recent launch of a "GPU-Poor" Chat Arena for those interested in scoping out some of the smaller models (not a lot of ratings so very noisy, take it with a grain of salt): https://huggingface.co/spaces/k-mktr/gpu-poor-llm-arena
JFingleton|1 year ago
It's a shame Brave is so far ahead of the game but no one seems to notice.
binarysneaker|1 year ago
ripped_britches|1 year ago
tangoalpha|1 year ago
Disabled it.
dbmnt|1 year ago
After pinning the extension, click on it and deselect the "enabled" option (the line with the purple X). This will kill the floating orb UI but you can still click on the extension in the toolbar to use it.
oneeyedpigeon|1 year ago
mrweasel|1 year ago
Other than that, I'm with you. The need for summarizing is a symptom of our increasingly poor communication and degrading writing skills. That, and SEO optimization which attempts to hit as many keywords as possible.
We're heading in a direction where people will use LLMs to pad their writing, so it will appear more substantial then it actually is. The receiver will then parse it though another LLM, because the writing has now become to convoluted, or they simply don't have the time for a ten page essay (in which case the none padded draft would have sufficed).
Admittedly I have found a few useful cases for LLMs, mostly related to text parsing and information extraction, which can be seen as summarizing I suppose, but mostly I have a pretty negative view of LLMs. Part of it may be me getting older and not fully understanding how they work, partly it's also their deployment in areas where I believe communication should be human to human.
standardUser|1 year ago
NoboruWataya|1 year ago
Anything that can be expressed in a few sentences, should be, and I tend not to read media that doesn't abide by that rule. If I'm reading long-form content it's because I am looking for detail and nuance that would be lost in a summary.
I do think a reliable video summary generator could be useful occasionally. Interestingly Orbit seems to work on YouTube, presumably by parsing YT's auto-generated transcript.
bluehatbrit|1 year ago
Sateeshm|1 year ago
mhitza|1 year ago
> Orbit is a Firefox add on that uses AI to summarize and answer queries about web content such as articles and videos.
> When a user asks Orbit to summarize or query content, Orbit gathers the context (eg. text, images, videos, etc.) of the page the user is viewing and provides a summary or answer. Orbit works on websites including Gmail, Wikipedia, NY Times, YouTube, and more.
> For the current version, we are using a Mistral LLM (Mistral 7B) hosted within Mozilla’s GCP instance.
It's interesting they're going with Mistral 7B. Is anybody else using Mistral 7B in production? And in what role?
I've considered using it for general knowledge type questions, and as a way to classify information, but would have never considered it for summarization type tasks due to it's limited context size (8k).
pilotneko|1 year ago
mattl|1 year ago
Everything else is a waste of time and money and energy.
wiseowise|1 year ago
goobert|1 year ago
anyfactor|1 year ago
Sephr|1 year ago
thiht|1 year ago
meindnoch|1 year ago
traspler|1 year ago
i_love_retros|1 year ago
What's a better browser that isn't basically advertising spyware (i.e. chrome, edge) ?
pluto_modadic|1 year ago
akimbostrawman|1 year ago
Not Firefox but forks based on it such as Tor Browser, Mullvad Browser or LibreWolf.
https://privacytests.org/
ssivark|1 year ago
- It's clear from user share graphs that Firefox as just a browser is tending towards irrelevance. No amount of "improving the browser" is going to solve the problem.
- More fundamentally, the browser is just one portal to the internet / world wide web. With technology outside the browser getting increasingly sophisticated, Mozilla necessarily needs to expand their mandate beyond Firefox in order to serve user needs and influence the landscape. Otherwise we might easily end up in a future where the browser becomes irrelevant and everybody interacts with proprietary large models.
- As far as innovation in browser features goes, this seems like a breath of fresh air. Internet users at large deserve access to AI services in a secure and privacy-friendly, and as a pillar of the free web Mozilla is well-placed as a distribution channel to serve these needs. Therefore, this seems like a very good stepping stone / experiment for Mozilla.
There will be execution challenges that need to be figured out. AFAIK Mozilla doesn't have the talent+budget for training large AI models, or even for doing intensive product research. So they're going to have to team up with some other AI expertise -- either explicitly or implicitly, by depending on open source models. Regardless, IMHO this is a risk they have to take and figure it out as they go along.
zaep|1 year ago
I don't think I've ever encountered a typo in any of the LLM output I've seen, seems like the exact sort of thing an LLM would be more or less perfect at. Am I wrong to take this as an indication that this text is actually written by a human as a concise marketing example?
amatecha|1 year ago
jsheard|1 year ago
janice1999|1 year ago
koolala|1 year ago
unknown|1 year ago
[deleted]
mort96|1 year ago
mindcrash|1 year ago
And while not all people are fond of AI, there are shit tons of people out there who do. Which means you automatically diminish your market share if you don't (because your most important competitors do)
j45|1 year ago
If they aren’t trying things, they would also then be accused of languishing in obscurity.
AI being built into browsers isn’t new. Summarization isn’t novel. It’s not early in the game where resources are crazy high.
Summarization could run with a basic low powered model privately hosted.
Market share changes based on what browsers do well.
knallfrosch|1 year ago
sunshine-o|1 year ago
My guess is this could be useful to many "knowledge workers" who constantly have to crawl, translate and find the meaning of the sugar coated landfill that has become most of the web.
We are right in the middle of the Tower of Babel story.
Seriously if it works reasonably well on legal fine prints I am in.
andro_dev|1 year ago
If you want privacy first AI in the browser here are the tools
https://ollama.com
https://github.com/n4ze3m/page-assist
meindnoch|1 year ago
andhuman|1 year ago
timtom123|1 year ago
usr1106|1 year ago
I have not studied either product in depth, so I am unable to comment on commonalities or differences.
Jolla has a mixed track record: They supported some phones over 10 years with decently working software (typing on one of those). They also failed at least once to deliver a crowd-sourced tablet to most of the backers. Not a risk-free choice, but at least someone trying to do the right thing.
xnx|1 year ago
Is that the right flow? FAQ link is broken so I can't tell.
pyaamb|1 year ago
sionisrecur|1 year ago
wwqrd|1 year ago
sergiotapia|1 year ago
why not use their technical expertise to built an in-browser "https://big-agi.com/" of sorts where users can paste in their API key and use bleeding edge models in combination with the browser's data which they could expose and manipulate as the creators of Firefox!
this product seems really random and quite frankly weird.
breakingcups|1 year ago
ilaksh|1 year ago
unknown|1 year ago
[deleted]
legacynl|1 year ago
I really don't get the comments that they should not focus on anything else than the current browser.
If other browsers start adding llms, I bet those same people will start complaining that Firefox is outdated compared to those browsers in about a year.
butz|1 year ago
mentalgear|1 year ago
unknown|1 year ago
[deleted]
jamieplex|1 year ago
andypants|1 year ago
bearjaws|1 year ago
freediver|1 year ago
Dilettante_|1 year ago
unknown|1 year ago
[deleted]
todotask|1 year ago
poppafuze|1 year ago
unknown|1 year ago
[deleted]
quickslowdown|1 year ago
beretguy|1 year ago
easygenes|1 year ago
https://github.com/mbnuqw/sidebery
stuartd|1 year ago
https://www.ghacks.net/2024/12/03/how-to-enable-tab-groups-i...
gorbachev|1 year ago
I've been using Simple Tab Groups [1] for many, many years.
1. https://addons.mozilla.org/en-GB/firefox/addon/simple-tab-gr...
navs|1 year ago
jckahn|1 year ago
tonymet|1 year ago
timtom123|1 year ago
block_dagger|1 year ago
nipperkinfeet|1 year ago
PeterStuer|1 year ago
yabatopia|1 year ago
silisili|1 year ago
IshKebab|1 year ago
I can't think of a single Firefox feature that's better than "I don't have to faff around with passwords". Maybe if they allowed adblock on mobile, but last I checked they were fannying around requiring nightly builds and whitelisting extensions...
nophunphil|1 year ago
dageshi|1 year ago
But you notice in all these threads, everyone who theoretically ought to use Firefox comes up with their own little list of nitpicks that justify them not using it.
"I can't use it because I was disgusted when they dropped feature x"
"I won't use it because they spent their money on feature y instead of just doing z"
Meanwhile Chrome doesn't give a fuck what you think and does whatever it wants and people keep using it regardless.
Firefox is doomed to be left with the niche audience of people who ignore the 95% of what it does right to focus on the 5% that it does wrong.
whilenot-dev|1 year ago
4k93n2|1 year ago
i_love_retros|1 year ago
mattvr|1 year ago
mentalgear|1 year ago
serbuvlad|1 year ago
It shocks me every time just how fast Chrome is. It is legitimately a superb piece of software. Going to Librewolf after feels like going back ten years in hardware.
Can we please start spending some money to make Firefox better? Instead of whatever Mozilla is currently doing?
Firefox Quantum was great. But why stop? Just keep doing that! It's the only thing you should be doing!
Qem|1 year ago
shmerl|1 year ago
benatkin|1 year ago
Mistral has released Codestral under a new license, but that's not the one used here. https://mistral.ai/news/mistral-ai-non-production-license-mn...
sitkack|1 year ago
dankobgd|1 year ago
sharmi|1 year ago
[deleted]
BadHumans|1 year ago
raincole|1 year ago
anyfactor|1 year ago
In fact, they should have done that a decade a ago.
Mozilla has been around since the late '90s and should have evolved beyond just being a browser company. They launched a VPN service when VPNs were already everywhere, and they did the same with a bookmark manager when others were already offering similar solutions. Mozilla is always catching up, never leading, and that's a common issue with many big open-source and free software companies. They often pretend to be a business that isn't heavily propped up by big tech donations.
If I were leading a browser company, my focus would have been aggressively directed towards small business software. I’d create an internet and privacy-focused affordable minimal business software suite that lives within the browser — a combination of Proton and Zoho. And I’d strongly avoid building things that should be browser extensions.
stiltzkin|1 year ago
[deleted]
omolobo|1 year ago
[deleted]
reissbaker|1 year ago
If they shipped vertical tabs I'd probably switch back to Firefox. But that would require focusing on actual browser UX instead of random offshoots. What's their actual product differentiation from Chrome these days?
forgotmypw17|1 year ago
Allowing Backspace to go back a page.
The built-in screenshot tool.
Being able to turn off smooth scrolling.
Support for a menubar, so that I can navigate the features I want quickly.
Being able to choose page encodings (I use a non-Latin charset language).
A usable/useful bookmark manager with things like sorting, tags/labels, timestamps, etc.
"Restore Previous Session" feature.
These are just a few features off the top of my head, I know there are many more.
Of course, for certain sites like Google Docs and YouTube, I have to use Chrome for obvious reasons, but for most browsing I use Firefox (and qutebrowser.)
brokencode|1 year ago
I think it’s pretty clear why they keep on doing this type of side project. They are trying anything they can think of to diversify revenue.
zitterbewegung|1 year ago
Duwensatzaj|1 year ago
Providing monopoly protection to Google.
They have no interest in actually competing for market share.
denismi|1 year ago
about:config, sidebar.revamp = true, sidebar.verticalTabs = true
It's getting there.
pier25|1 year ago
keerthiko|1 year ago
- continued support for manifest V2 (primarily because ublock origin would stop working if forced to V3 only)
- the firefox address bar is way smarter for any given string i type in than chrome's. it's ability to surface the most relevant deeplinks from my history, vs top level site, vs perform a web search, is night and day difference from the randomness that other browser search bars offer.
- I have the opportunity to use Zen (a Firefox fork) [0] and it's 100% interoperable with my vanilla Firefox instances across devices -- i can even send tabs from my Firefox Nightly on Android to my Zen instances on Windows or Mac. BTW Zen has vertical tab support, (more) first-party multi-profile support, and preserves the address bar behaviors of vanilla Firefox.
[0]: https://zen-browser.app/
notatoad|1 year ago
You could build an AI Assistant, or you could spend a month bikeshedding some design details of vertical tabs.
hussi|1 year ago
nixosbestos|1 year ago
askvictor|1 year ago
mouse_|1 year ago
NotYourLawyer|1 year ago
likeabatterycar|1 year ago
Because some project manager had "AI" on his performance goals this year.
Firefox is buggy as hell - which is incomprehensible given its age, but older brother Netscape had the same problem 20-ish years ago. The Netscape 4.x days were absolute hell and you could go hardly a day without the browser crashing.
Despite this, it's packed with nonsense no one asked for like Pocket. Which is a coincidence because "AI assistant for Firefox" is the dictionary definition for redundant things no one asked for, with better alternatives preexisting.
At this point Firefox needs to die and something new and manageable - with energetic developers - needs to take its place. Maybe Microsoft could open source the original Edge engine? The one before they bent over for the long dick of Google Chrome.
add-sub-mul-div|1 year ago
Not being the progenitor and linchpin of surveillance capitalism.
bee_rider|1 year ago
Agreed that it would be nice if they had better focus though.
wpm|1 year ago
yonatan8070|1 year ago
afavour|1 year ago
Mozilla's brand is "pro privacy", it does make sense for them to launch an AI product with that brand position. I doubt it'll be successful because I don't think enough people actually care about privacy, but still.
I feel like it's a common HN sentiment to say "why don't Mozilla just focus on the browser?!"... the answer is because barely anyone is using it and there's very little they can do to move the needle on that. IMO they're an organization looking for a purpose.
devvvvvvv|1 year ago
batata_frita|1 year ago
DeepYogurt|1 year ago
JBiserkov|1 year ago
Dalewyn|1 year ago
sitkack|1 year ago
Firefox is like the shitty best option that camps out in its niche, it sucks but it is really hard to push out of the way.
HKH2|1 year ago
CatWChainsaw|1 year ago
nullc|1 year ago
Of course, no option to use a local model even though the one they're using is small enough that its perfectly reasonable to use locally. Even on a cell phone.
blackeyeblitzar|1 year ago
JaDogg|1 year ago
> basic utility program decided to become political
looks like they wanted to avoid it, isn't it?