I wish this kind of low-effort lazy AI "content" would stop making it to the top of HN day after day. In any case, the piece is clearly an ad for Apricot. The examples cited in the article are so pedestrian, it's hardly even worth discussing.
How can you even seriously think that asking GPT for a function that parses OPML is a "realistic" task. I can just Google it and get like 100 pages of Python functions that do exactly that.
Personally I found the comparison useful and relatable. I could imagine wanting to accomplish these exact same tasks and wanting to know which language model would be best. And in general I like this qualitative analysis rather than the metrics we get in official releases and research papers which often don’t capture real world use very well. I can’t exactly begrudge the author for killing two birds with one stone here, it’s better than some completely made up use case that’s completely theoretical.
I think this comparisons cover use cases of 99.9999999% of people. As you said, if you are using bard, you will be forced to Google it anyway. Bard is just designed to be not helpful.
I’ve been comparing bard and ChatGPT in most my tasks since bards release. Bard is infuriating. It claims it can’t answer most things, although if I prompt tweak things it’ll eventually answer. It has terrible contextual awareness - it literally can’t piece a thread between one prompt and the next - each prompt is self contained as far as I can tell. There’s no history of prior discussions other than in Google activity, but you can’t resume those. In theory they all form a continuum, but I don’t really want that - I want the context from one session to be distinct from the other.
I don’t think ChatGPT is a particularly amazing interface or experience. But Bard is so far off the mark it makes me realize Google still won’t be able to make a product. LLM as they mature and integrate into a better ecosystem of feedback mechanisms and interfaces will eat Google’s search business alive, and I see no indications they can do anything about that.
I feel the same way about bing chat. gpt4 is still the king, however phind.com may be better for coding, though I have the browser plugin now and not really vetted.
codeium and genie plugins for vscode and code whisper replaced my need for copilot.
though I try to use codeium first since it's free and genie is using my GPT4 API key.
> Yes, Bard can get yesterday’s stock returns, but so can Yahoo Finance
Off putting when the author puts down a significant and fundamental improvement (having up to date information instead of point in time snapshot) with such a remark that so completely misses the point.
I agree that the author is too dismissive of that, but I also think people are way too quick to oversimplify to "Bard has up to date information"
From what I can tell, Bard's access to recent information is through augmentation, not continuous training, and that's a really important difference. ChatGPT also has its "web browsing" alpha, which is a similar concept. The problem is that being able to search the web isn't the same as having a piece of knowledge integrated into your model of the world.
So, for example, if you ask contrived questions like "Why might Elon Musk have more time to focus on Tesla soon?", both Bard and ChatGPT+Browsing get a clue that they should search for Musk in the news, and they'll tell you about Twitter's new CEO pick. They'll then apply that competently to your question, and give you a reasonable answer.
But if you ask a question that requires more indirect inference, you immediately see the shortcomings of this kind of augmentation. For example, if you try "Is there hope for LGBTQ rights improving in Turkey?", neither model finds the extremely relevant point that Turkey's homophobic current president stands a significant chance of losing reelection. I can't see what Bard searched for in collecting its information (in fact, based on the answer I got, I'm not even sure it tried to go beyond its training data). I can see that ChatGPT searched for "LGBTQ rights in Turkey 2023". It's not surprising that that search didn't clue it in.
Now, obviously, I can't actually examine what would happen if each model was actually trained on the most recent news about Turkey's politics. But I'd have a very high expectation that GPT-4 would make that connection, and I would be fairly surprised if 3.5 didn't as well. But there's a bottleneck because the information isn't actually integrated.
Which isn't to say that that kind of augmentation is useless, of course. But the distinction is important.
phind.com can probably do it way better than bard, it's basically gpt4 on steroids with Internet access. I find it easy better than bing, I use it for coding more than gpt4 now because it knows things like server actions for next 13.4+ which just launched.
so far I'm very unimpressed with bard and even bing chat, and I just got gpt4 browser plugin, so that might be better eventually but it's really slow.
> OK, this is not the biggest thing, but why on earth does Bard use indentation with 2 spaces instead of the standard 4? This drove me up the wall, so I tried all sorts of prompts to get Bard to use indentation with 4 spaces but they all failed.
This is also interesting with GPT-4. I've been using it to generate tons of code and sometimes it indents by 2 spaces, sometimes it mimics what I give it first, but often times to reverts to 2 spaces regardless even if told to explicitly indent differently. Rather annoying indeed.
Two spaces is less tokens, might actually be a good thing overall to limit token usage so you can fit more code in the context window. It’s easy enough to have your IDE convert it to your preferred format after generation. But I am a diehard fan of tabs over spaces, so I guess I’m used to converting incorrectly formatted code anyway ;-)
You can just ask it to use your preferred indentation style. Personally I prefer 2 spaces so even if 4 spaces is apparently “standard” I’d like it to use 2
The construction of the summarizing task is a bit odd. The prompt asks for a single sentence of output, but then the author complains that the Bard output is too terse. If I'm asking for a single sentence, that's about the amount of text I'd expect, not a paragraph of text mushed into a run-on sentence.
Especially if the problem is something as easily fixable as the wrong level of detail, I would have thought that exploring some alternative prompts would make sense. Given this is the prompt that the author is already using for a ChatGPT-based app, it's obvious that this specific prompt would work well there. If it didn't, they would have iterated on the prompt until it produced acceptable results for their app.
ok, the title should probably be more like "One person's anecdotal, totally unscientific but realistic comparison. Still, amidst all the breathless hype, I thought a little actual data might help people evaluate the two tools side by side.
While I don't doubt that chatGPT is better than the current alternatives, the fact that the author use the 20$/month version and not the free version (or both) to compare to bard is not very great. The same for giving the same value to paying 20$/month and knowing the model used
well, being as this is a comparison of bard using palm2 and gpt4, the free version which is gpt3.5 wouldn't suffice. I'd like to see them add in bing chat, phind.com and perplexity, and one of the open source models.
Bard's also not available in Canada yet! I feel a bit like a broken record. But it's something a bit shocking and I believe needs to be addressed. Why not google? What's the real answer?
i use them day to day last week for various matters. That includes work, leisure, life. ChatGPT's $20 is a good deal, 1 month ago Bard was a junk right now it can be ok for many reasons, I'd rate it as free GPT-3.5
When you ask Bard "What do you know about Edgar Allan Poe?" it answers "I'm a text-based AI and can't assist with that", some people have managed to extract information about Edgar Allan Poe from Bard but it does require some effort. With ChatGPT you just need to ask the question to get information about Edgar Allan Poe.
Good summary and test cases. Not too surprised to see the result – I feel most commentators have preferred ChatGPT to Bard. In fact, on Bard's initial launch, I remember a lot of discussion expressing surprise that Google was as far behind as it seemed. Maybe there is more hype for Bard in other circles than mine?
One thing that Bard doesn't seem to get credit for is speed. I am a paying ChatGPT customer and only tried out Bard after GA. I was pretty shocked at the speed.
Considering that my $20 only gets me 25 messages every 3 hours, I'll probably be turning to Bard when I run out (which I haven't yet, to be fair).
Are those gpt4 caps even real? I've never hit one even though I swear sometimes my use is higher. Does it only mean new chat threads or each individual message?
yesterday I coded qrpwd - a command line tool to password protect information in a QR code (as a backup for my 2 factor auth backup codes). https://github.com/franzenzenhofer/qrpwd
i say coded, i mean, i directed chatgpt to do it
chatgpt had some issues with the QR content extraction using zxing so I tried Bard.
Bard destroyed totally working code again and again while ignoring the issue on hand.
In the end I googled some stackoverflow posts with working code, fed them to chatgpt and it worked it out.
In my experience, BARD is not up to any coding tasks, while chatgpt - while not perfect - is magic.
dvt|2 years ago
How can you even seriously think that asking GPT for a function that parses OPML is a "realistic" task. I can just Google it and get like 100 pages of Python functions that do exactly that.
OkGoDoIt|2 years ago
mkrishnan|2 years ago
fnordpiglet|2 years ago
I don’t think ChatGPT is a particularly amazing interface or experience. But Bard is so far off the mark it makes me realize Google still won’t be able to make a product. LLM as they mature and integrate into a better ecosystem of feedback mechanisms and interfaces will eat Google’s search business alive, and I see no indications they can do anything about that.
gremlinsinc|2 years ago
codeium and genie plugins for vscode and code whisper replaced my need for copilot.
though I try to use codeium first since it's free and genie is using my GPT4 API key.
ilaksh|2 years ago
magoghm|2 years ago
zmmmmm|2 years ago
Off putting when the author puts down a significant and fundamental improvement (having up to date information instead of point in time snapshot) with such a remark that so completely misses the point.
mistercow|2 years ago
From what I can tell, Bard's access to recent information is through augmentation, not continuous training, and that's a really important difference. ChatGPT also has its "web browsing" alpha, which is a similar concept. The problem is that being able to search the web isn't the same as having a piece of knowledge integrated into your model of the world.
So, for example, if you ask contrived questions like "Why might Elon Musk have more time to focus on Tesla soon?", both Bard and ChatGPT+Browsing get a clue that they should search for Musk in the news, and they'll tell you about Twitter's new CEO pick. They'll then apply that competently to your question, and give you a reasonable answer.
But if you ask a question that requires more indirect inference, you immediately see the shortcomings of this kind of augmentation. For example, if you try "Is there hope for LGBTQ rights improving in Turkey?", neither model finds the extremely relevant point that Turkey's homophobic current president stands a significant chance of losing reelection. I can't see what Bard searched for in collecting its information (in fact, based on the answer I got, I'm not even sure it tried to go beyond its training data). I can see that ChatGPT searched for "LGBTQ rights in Turkey 2023". It's not surprising that that search didn't clue it in.
Now, obviously, I can't actually examine what would happen if each model was actually trained on the most recent news about Turkey's politics. But I'd have a very high expectation that GPT-4 would make that connection, and I would be fairly surprised if 3.5 didn't as well. But there's a bottleneck because the information isn't actually integrated.
Which isn't to say that that kind of augmentation is useless, of course. But the distinction is important.
gremlinsinc|2 years ago
so far I'm very unimpressed with bard and even bing chat, and I just got gpt4 browser plugin, so that might be better eventually but it's really slow.
jacooper|2 years ago
cardosof|2 years ago
drcode|2 years ago
moffkalast|2 years ago
This is also interesting with GPT-4. I've been using it to generate tons of code and sometimes it indents by 2 spaces, sometimes it mimics what I give it first, but often times to reverts to 2 spaces regardless even if told to explicitly indent differently. Rather annoying indeed.
ChatGPT needs a plugin that automatically sends the output through this site lol: https://www.browserling.com/tools/spaces-to-tabs
OkGoDoIt|2 years ago
circuit10|2 years ago
jsnell|2 years ago
Especially if the problem is something as easily fixable as the wrong level of detail, I would have thought that exploring some alternative prompts would make sense. Given this is the prompt that the author is already using for a ChatGPT-based app, it's obvious that this specific prompt would work well there. If it didn't, they would have iterated on the prompt until it produced acceptable results for their app.
brianpk|2 years ago
burlesona|2 years ago
jimsimmons|2 years ago
You can’t have the cake and eat it too
poulpy123|2 years ago
gremlinsinc|2 years ago
theshrike79|2 years ago
Bard: Bard isn’t currently supported in your country.
ChatGPT: Is available and I can pay for GPT4.
Winner: ChatGPT
quickthrower2|2 years ago
marketingLizard|2 years ago
8note|2 years ago
siva7|2 years ago
maxdo|2 years ago
magoghm|2 years ago
IanCal|2 years ago
npilk|2 years ago
Geee|2 years ago
unknown|2 years ago
[deleted]
zamalek|2 years ago
Considering that my $20 only gets me 25 messages every 3 hours, I'll probably be turning to Bard when I run out (which I haven't yet, to be fair).
kevviiinn|2 years ago
franze|2 years ago
i say coded, i mean, i directed chatgpt to do it
chatgpt had some issues with the QR content extraction using zxing so I tried Bard.
Bard destroyed totally working code again and again while ignoring the issue on hand.
In the end I googled some stackoverflow posts with working code, fed them to chatgpt and it worked it out.
In my experience, BARD is not up to any coding tasks, while chatgpt - while not perfect - is magic.
dgeiser13|2 years ago
porkbeer|2 years ago
rochak|2 years ago
jacooper|2 years ago