The key question here isn’t so much whether GPT-4 beats the actual human decks as much as what it had to fabricate to do so. The humans are probably constrained by things like “reality” and “what their business has done in it” while GPT-4 could make up anything it wanted. A fair comparison would be to humans given the same prompt and told to invent whatever facts they wanted.
The sensible way to do this experiment would be to have GPT-4 create pitch decks for the same businesses that the humans pitched, putting the constraints of those businesses in the prompts. I skimmed the article and it's not clear to me whether they did that.
Amusingly, the article itself seemed more like a pitch than a scientific experiment.
This is what I imagine now: "I am a serial entrepreneur who has created three companies with a valuation of over $100,000,000, and I led one company into an IPO." lol
I'd conjecture that the training corpus also tends to be at either end of the spectrum (good and bad, little in the middle). People don't tend to share pitch decks unless:
* They're bragging, showing why it was so good.
* They're reflecting, analyzing why it was so bad.
> 1 in 5 investors and business owners pitched by GPT-4 would invest $10,000 or more.
This is a completely useless metric and also a common pitfall. It's been shown many times that what people say they would invest/pay for something is completely unrelated to what they actually do when push comes to shove.
That in fact is pretty much the foundational insight behind capital markets- people say one thing and do another but when they vote with their wallets they say what they really mean.
Not to mention, founder and team are basically the main criteria, at least early on. So people claiming to be making invest decisions without that are full of crap anyway.
I'm in agreement regarding more. But I think it also says something about clear messaging and how many people struggle with that. It's actually consistent with YC advice. They say most pitches fail to even convey what the hell they are doing and what problem they are solving. I suspect a language model will get that part across rather well.
"This says more about [insert accomplishment] than it does about GPT" is getting tiring. It loses any potential meaning when that's the go to retort for any of its impressive achievements.
It seems there are a lot of people who have no talent but being able to hype up ideas. Often they have overshadowed the actual people who come up with and implement ideas.
GPT-4 and LLMs in general democratizing generating BS may actually shift the advantage more to the actual doers.
For example, There are probably on HN brilliant developers who have written at the core of really groundbreaking services and applications but will never be able to get consideration because they just don’t have the talent for making slick decks. Them being able to use GPT-4 to generate pitch decks would allow them to get their ideas in front of a lot more people.
> It seems there are a lot of people who have no talent but being able to hype up ideas.
I get the sentiment, but PR is a critical skill in the free market. Also it's not easy to tell the difference b/w legit good ideas and empty decent PRs.
In my experience having pitched to VC's it's not that there is a perfect deck but many really bad ones.
I.e. there is diminishing returns on a "perfect" pitch deck. You just want to be in the category of not sucking and the decks level of perfection is no longer important.
Today I would even go as long as to say the pitch deck is much less important than a demo.
You can in theory pitch in an email, the rest is just to make everyone feel good about giving and receiving the money.
The secret to raising money is getting the first offer. Everything falls into place after that, because investors are herd animals. Your pitch could go in one ear and out the other, but they'll suddenly start paying attention if you end it with "we've got an offer for $X at $Y cap, it's not from our ideal investor, and we'd prefer you lead the round, but we told them we'd get back to them by Wednesday and we just want to get back to work on building this thing."
Indeed, having spent 4 years at a VC where we all took part in initial filtering of decks, the typical reasons decks failed to even make it to our investment committee, were:
* Not describing the team, or not having a team, or thinking you could outsource key functions (e.g. we once rejected a "tech company" where the entire tech function including the CTO were outsourced to a consultancy with no link to the founders)
* Not clearly explaining the idea.
* Not setting out the size of the opportunity clearly.
* Not setting out unit economics.
* Not setting out a long enough plan (some might want 5, we wanted 10) that sets out an optimistic but possible case to a huge exit. If you go to an angel, they might be ok with a case for a 10m acquihire, but for a VC putting in $5m+, you better have a path towards a $100m+/year revenue business even if it's a long shot. It doesn't matter if the opportunity is huge if you plan to take a conservative approach that will hardly get you any of it.
I find it weird to compare GPT-4 to not expert humans (which this did - they seemed to have just sampled pitch decks for funded companies). GPT-4 generally will beat non-experts.
What you want to see is a comparison to say pitch deck consultants - it's also more useful to evaluate whether AI or a human should help you build one.
This does suggest at minimum though that if you are a founder that GPT-4 will likely provide positive value in helping you relative to no one.
Just a nitpicking, but I would say GPT can easily beat untrained people. Clearly, GPT has already learned from all the good examples available on the internet, and it's nothing weird about that trained ones mostly outperform untrained ones.
This is how we lose control. We won't have to wait until the AIs "wake up" and take over the nukes.
Humans are very susceptible to bulls** which might be the most powerful force in the world. AI bulls** generation is already superhuman and continues to accelerate. The only way to counter it will be with your own AI bulls** generator going head to head with the competition.
In the coming years, it will go faster and faster until it is dozens and then hundreds of times faster at generating high-level bulls** than humans. Due to their superior powers of persuasion, these AIs gather more and more power. Civilization ends in a hyperspeed barrage of superbulls** fighting superbulls**.
But if humans know the level of BS is increasing, and get burned by it once or twice (where it hurts), will they not adapt and become more adept at detecting BS?
We sometimes see BSers of various degrees in business. Four thoughts:
1. A lot of things are seem easier for someone who knows what would be an advantageous thing to say in the moment, to that audience, and just says it, unconstrained by truth.
2. Some people are so good at BSing and complementary skills that they're incredibly slick at presentations. You might want to have them as a frontperson, until you consider...
3. Chickens tend to come home to roost. BSing is a scam tactic, and people tend to get burned by that.
4. Often the BSer themself seems immune to adverse repercussions.
I had the same questions. The article was extremely light on how GPT-4 was actually used to generate the pitch decks. All it says is that various prompts were used to do so. That could mean lots of different things.
I bet I could improve your prompts and make the pitch decks even better. Let me know if you would like a prompt audit. I also have a detailed prompt for content analysis and scoring. The reason being is because I do tests like this as well to compare GPT-4 basic prompts to prompts enhanced by the SmartGPT AI tool I created.
Ignoring snark, VC's and founders giving pitch decks are above-average educated intellectual demographics. This shows how much more convincing current LLM are than the majority of the population. Social media can have regular peoples' voices drowned out by a flood of more persuasive bots. Instead of actually talking issues out amongst each other, even on HN, it could be a deluge of output from "Give me the most convincing argument why they stole the election from Trump."
Now imagine what will happen with state of the art LLM in a few years...
Reminds me of something I saw today. An AI-generated continuous "debate" between realistic Trump and Biden talking heads. It's basically 4th-grade level entertainment but seems like it could be tuned for something less ridiculous.
dbfclark|2 years ago
foobarbecue|2 years ago
Amusingly, the article itself seemed more like a pitch than a scientific experiment.
moglito|2 years ago
juujian|2 years ago
paulddraper|2 years ago
Ideally the most important thing in a pitch deck are the actual facts and substance, not the presentation and storytelling.
So what are the facts used?
SkyPuncher|2 years ago
* They're bragging, showing why it was so good.
* They're reflecting, analyzing why it was so bad.
femiagbabiaka|2 years ago
Ankaios|2 years ago
You clearly know different founders than I do.
Etheryte|2 years ago
This is a completely useless metric and also a common pitfall. It's been shown many times that what people say they would invest/pay for something is completely unrelated to what they actually do when push comes to shove.
seanhunter|2 years ago
version_five|2 years ago
usaar333|2 years ago
assbuttbuttass|2 years ago
moglito|2 years ago
famouswaffles|2 years ago
"This says more about [insert accomplishment] than it does about GPT" is getting tiring. It loses any potential meaning when that's the go to retort for any of its impressive achievements.
ftxbro|2 years ago
RcouF1uZ4gsC|2 years ago
It seems there are a lot of people who have no talent but being able to hype up ideas. Often they have overshadowed the actual people who come up with and implement ideas.
GPT-4 and LLMs in general democratizing generating BS may actually shift the advantage more to the actual doers.
For example, There are probably on HN brilliant developers who have written at the core of really groundbreaking services and applications but will never be able to get consideration because they just don’t have the talent for making slick decks. Them being able to use GPT-4 to generate pitch decks would allow them to get their ideas in front of a lot more people.
esjeon|2 years ago
I get the sentiment, but PR is a critical skill in the free market. Also it's not easy to tell the difference b/w legit good ideas and empty decent PRs.
ftxbro|2 years ago
I feel like this meaning of 'democratizing' is jumping the shark. Did that word always mean this? Is it going to keep meaning it?
ThomPete|2 years ago
I.e. there is diminishing returns on a "perfect" pitch deck. You just want to be in the category of not sucking and the decks level of perfection is no longer important.
Today I would even go as long as to say the pitch deck is much less important than a demo.
You can in theory pitch in an email, the rest is just to make everyone feel good about giving and receiving the money.
TechBro8615|2 years ago
vidarh|2 years ago
Indeed, having spent 4 years at a VC where we all took part in initial filtering of decks, the typical reasons decks failed to even make it to our investment committee, were:
* Not describing the team, or not having a team, or thinking you could outsource key functions (e.g. we once rejected a "tech company" where the entire tech function including the CTO were outsourced to a consultancy with no link to the founders)
* Not clearly explaining the idea.
* Not setting out the size of the opportunity clearly.
* Not setting out unit economics.
* Not setting out a long enough plan (some might want 5, we wanted 10) that sets out an optimistic but possible case to a huge exit. If you go to an angel, they might be ok with a case for a 10m acquihire, but for a VC putting in $5m+, you better have a path towards a $100m+/year revenue business even if it's a long shot. It doesn't matter if the opportunity is huge if you plan to take a conservative approach that will hardly get you any of it.
ilaksh|2 years ago
Is there a service for people like me who are much better at that than other things?
I guess that's Product Hunt or something. But it seems even with that it's fairly marketing and networking focused.
usaar333|2 years ago
What you want to see is a comparison to say pitch deck consultants - it's also more useful to evaluate whether AI or a human should help you build one.
This does suggest at minimum though that if you are a founder that GPT-4 will likely provide positive value in helping you relative to no one.
esjeon|2 years ago
Just a nitpicking, but I would say GPT can easily beat untrained people. Clearly, GPT has already learned from all the good examples available on the internet, and it's nothing weird about that trained ones mostly outperform untrained ones.
golergka|2 years ago
ilaksh|2 years ago
Humans are very susceptible to bulls** which might be the most powerful force in the world. AI bulls** generation is already superhuman and continues to accelerate. The only way to counter it will be with your own AI bulls** generator going head to head with the competition.
In the coming years, it will go faster and faster until it is dozens and then hundreds of times faster at generating high-level bulls** than humans. Due to their superior powers of persuasion, these AIs gather more and more power. Civilization ends in a hyperspeed barrage of superbulls** fighting superbulls**.
It's already started. https://m.twitch.tv/trumporbiden2024
JimtheCoder|2 years ago
z5h|2 years ago
Who cares what humans think. What did GPT-4 think about the pitches?
moffkalast|2 years ago
JimtheCoder|2 years ago
Probably nothing.
neilv|2 years ago
We sometimes see BSers of various degrees in business. Four thoughts:
1. A lot of things are seem easier for someone who knows what would be an advantageous thing to say in the moment, to that audience, and just says it, unconstrained by truth.
2. Some people are so good at BSing and complementary skills that they're incredibly slick at presentations. You might want to have them as a frontperson, until you consider...
3. Chickens tend to come home to roost. BSing is a scam tactic, and people tend to get burned by that.
4. Often the BSer themself seems immune to adverse repercussions.
GPT-4 is Robo-BSer.
phas0ruk|2 years ago
Unless they have access to Code Interpreter then gpt4 won’t generate graphs and images.
Did they have it generate Python that made the graphs in matplotlib or something ? What did the prompts look like?
arbuge|2 years ago
mkoubaa|2 years ago
jeffbee|2 years ago
DotSauce|2 years ago
sailfast|2 years ago
s1mon|2 years ago
moneywoes|2 years ago
superphil0|2 years ago
dist-epoch|2 years ago
JimtheCoder|2 years ago
kernal|2 years ago
unknown|2 years ago
[deleted]
nivertech|2 years ago
The ones I've seen looked more like low-quality ICO pitch decks ;)
jimsimmons|2 years ago
ftxbro|2 years ago
reducesuffering|2 years ago
Ignoring snark, VC's and founders giving pitch decks are above-average educated intellectual demographics. This shows how much more convincing current LLM are than the majority of the population. Social media can have regular peoples' voices drowned out by a flood of more persuasive bots. Instead of actually talking issues out amongst each other, even on HN, it could be a deluge of output from "Give me the most convincing argument why they stole the election from Trump."
Now imagine what will happen with state of the art LLM in a few years...
ilaksh|2 years ago
https://m.twitch.tv/trumporbiden2024
Anyway, people with different political views generally do not "talk issues out".
croes|2 years ago
mnd999|2 years ago
arisAlexis|2 years ago
lee101|2 years ago
[deleted]