I've done a lot of stage demos. I've done pre-recorded ones, I've done live ones.
I prefer live ones, by a significant margin, even for non-deterministic AI things. In general I find live ones better, but it does depend on your circumstance. It's worth asking yourself three questions when you're doing a demo:
What is the outcome when it goes great?
What is the outcome when it goes wrong?
What percentage of the time do each of those occur?
With a pre-recorded demo, when they go great it's usually a 9/10. Things are solid, but they aren't adaptive to the situation, which means you might have lost an opportunity to insert a quip from something that happened earlier. When they go wrong though, it's a 0/10. You've told people that you don't have enough faith in your product to show it live - why should they have faith in it?. Now you might say that the demo goes right 100% of the time(ish), but there's also a human element to it. Someone might play the demo early, someone might not act in time with the live demo, any of these can give away that it's pre-recorded. I'd put their success rate at 95%.
With a live demo, when they go great they're 10/10. You can adapt them, you can show it live, things react instantly instead of slightly-out-of-sync due to scripted content. It's incredible. When they go wrong, it's a 3/10. You've got egg on your face. People make fun of you after. But they know at that point that you ran a live demo. That does count for something. We know Zuck did that demo live. They do fail a bit more, especially in the age of AI. 10% failure rates are pretty good for live demos. To me though, the higher failure rate is worth it for the better PR if you do fail.
100% agree. It's not a BINARY choice though! I prefer to do the demo live, and if it fails, have a one-push switch over to a recording and quickly declare that live projects are a work in progress. I think this is ultimately how you show both strength of live demo preparation, but can humbly recover the show if the live version fails by switching to recorded.
But, NEVER TELL ME IT'S A LIVE DEMO WHEN ITS PRE-RECORDED. You will lose my business forever if I catch it.
I don't agree that a bad live demo is a 3/10. The recent demo showed me conclusively that the Meta Ray Bans are not at the quality level where I would buy them, especially not for $800. That's pretty much a 0/10 in my book. Since it was live usage, it's indicative of the real quality of the product.
IMO a perfect prerecorded demo would have to be shockingly good to get better than like a… B- for me. It’s just too easy to cut or reshoot the interaction to be perfect.
I'm a sales engineer and demoing is one of the core parts of the job. I think this is applicable if it's an internal demo or a demo to a large audience, i.e a conference where you have a very fixed window of time to present.
I would never recommend doing this in a sales context though, where you're selling to prospects. Mainly because it gives you no flexibility to pivot and adjust the content based on what they're interested in, and also because it gives off the impression you don't have a lot of faith in your product to perform live.
I'd still 100% recommend having pre-recorded backups however. I was once doing a demo to our largest customer in region and the second we joined the call our core product had a SEV0. I was very relieved to have something to show while the engineers fixed the issue.
If you pre-record your live demo then it isn't a live demo...
What I dislike is presenting something as if it is live when it isn't. Demos are fantastic things and it is even perfectly fine to have imaginative aspects to them. But there's a division line between demoing and lying. Take Rabbit R1 as an example[0]. This whole thing was faked yet it was presented as a live demonstration, not an illustration. In other words, fraud.
The difference is if you're trying to show your customers (or potential customers) what you're imagining the device works like vs what the device actually works like. The difference is how you're communicating the difference to them.
I'm not sure why this is a contentious opinion, but just don't lie to people?
Absolutely. I don't mind a recording, as long as it's presented as a recording and anything misleading is disclosed. If you say you're going to demo something, demo it. We have enough erosion of truth in this world already -- don't contribute to it!
I do live demos at work all the time and the best part is that when someone asks "can it frobulate with encapulation?" I can actually press the frobulate button with encapulation enabled and show them. Can't do that with a recording.
I had a professor who had taken courses from Paul Halmos, who authored one of my favorite math books, Naive Set Theory. I regarded the book so highly that I was somewhat star struck to learn my professor had taken courses from him.
I asked my professor what it was like, was he an incredible professor? He told me the lectures were useless! I was shocked and he said they were so well prepared they were effectively a recording of the material - and no advantage over reading the book. At that point I reflected and realized that, by far, my favorite professor was one who constantly made mistakes while solving problems on the board. He'd have to notice something wrong and track down the error, and it was wonderful and highly instructional to watch the process.
I can even pause the replay mid-demo, take over, and resume again later. People are always impressed at how flawless my live demos are, and with some judicious audience participation, the illusion is perfect.
I love this idea. I remember giving a live demonstration of a configuration management tool (SaltStack circa 2012) and some unknown networking conflict between my Virtual Machines and the local wifi subnet caused my commands to timeout. I was too junior to resolve it live and was left with boring slides explaining what ought to be happening.
Luckily it was a small group of devs and not a large venue. Still embarrassing...
If you're launching some fragile and finicky AI thing before a crowd like you're Steve Jobs, but you're not Steve Jobs, then, sure, pre-record it.
But when the standards are lower, people appreciate the live demo. Videos are much less compelling, and they could've watched that at home in a few minutes, rather than fly out to you.
In grad school -- in a lab where, instead of "publish or perish", our mantra was "demo or die" -- we were giving demos most weekdays, usually to only a few people at a time. Always live. I think once I had to play a video of someone else's stuff, and it fell flat.
My best demo story is when I was showing a new thing (just a class project), to a few people crowded around a computer in my lab area at an open house event, typing into my software agent in a chat window (pre-LLMs), ad-libbing and speaking aloud what I was typing, "I'm interested in... thinks Microsoft and... thinks Oracle..." Then I turned my head around to look at the people behind me, to see that they were following, and I swear to gosh, Larry Ellison himself was standing in the doorway of my lab area, giving me this fiercely aggressive stare. Whether he was glaring because I said Oracle or Microsoft, I do not know.
My worst demo story was at the very start of my demo career. I had just arrived on campus, and I was borrowing a computer in an open area. A research scientist comes up, and sees on my screen this Web-scraping personalized newspaper thing I had made. So I'm showing it to them, and they're interested, then all of a sudden they say, sadly, "Mother Theresa died..." Which was what my "demo" was showing.
So maybe don't include an uncontrolled live data feed of depressing or distracting news in your demo.
Also, implementing early software agents in Java apparently summons Larry Ellison.
I agree that you should always have a pre-recorded version as backup, but live demos communicate a confidence in your product, and that can be worth something. Whenever I see a pre-recorded demo I wonder how many takes it took them, or how many pauses were taken out in editing, etc.
It helps if you have a master of ceremonies capable of running three rings simultaneously. Jobs could do it. Looking at what Apple is doing, they feel Cook is not. You lean into your strengths and avoid your weaknesses. Meta should realize that theZuck is not Jobs and is much closer to Cook
This article reminded me of a talk I gave a couple times that went very smoothly because I pre-recorded the typing parts. Something like this:
- record the screen as you type out all of your code as you would do live
- don't need the audio, just the video
- edit it down to remove typos and pauses
- speed it up to a nice fast typing pace
Now you can drop the video into Keynote or whatever, and Play/Pause the video as you speak.
A crucial part of this is that the video has effectively no pauses. Maybe just a few short ones between sections, to leave time to hit Pause before the next keystrokes appear.
Instead of a 30 minute video that I had to perfectly keep pace with, it was like 5 minutes of rapid typing that I could speak along with and pause when it was time to explain something.
At this point in time, it is absolutely trivial to pre-record whatever you'd like, and it will look glorious, and it doesn't need any basis in reality.
As a customer, when you show me a video, I default-assume you're lying.
Also, as a presenter: If your live demo fails and you can't turn that into a win, maybe you're not good at presenting. Your audience wants you to succeed, it takes a minimal amount of charm to make it work. (Heck, even Mark turned this into a win, and he's not exactly a charm bomb)
Thought at first this was maybe just being applied to external product demos, but this reads like dev demos, too:
> Since most of the time you're demoing something you previously worked on, this prevents you from working on current stuff right up to the demo, because you're worried you might break your local setup for the demo by installing different node modules, switching to different Git branches, etc.
This isn't the case for me generally... probably more of problem for web-apps things I guess.
I keep multiple workspaces locally for the multiple things I'm working on, but that hasn't been my pain point. Usually it's that I'm demoing a work in progress (not my choice generally) that has rough edges.
I enjoy the internal demos where questions can be asked about doing XYZ, and that can be shown if it's quick. Recordings tend to take away from that, and be less memorable/engaging I guess.
First time I recorded a demo was when it became too hard to return things to a known good state after every demo. In this case that meant multiple RAID arrays, 3 servers, an application with DB, and monitoring software.
All so that I could demonstrate that when I turned off a RAID controller a red light came on and the application just…continued.
So much work for something largely unspectacular meant that a recorded demo that could be run anytime from a laptop was far more appealing. My junior colleagues were now doing demos they otherwise wouldn’t, and the seniors turned off the sound and edited the video into snippets they could introduce and talk around
A company I worked for switched over to pre-recorded demos and everyone talked about how clever it was for the first few larger audiences. Then they made a mistake and replayed a clip during the same session and the audience chat blew up. You could see a dip in new users for days after the demo.
Apple can pull it off, but if you’re at a mid market tech company you just sunk like weeks people’s time into choreographing a live movie. Super stressful time sync.
IMO live demo is better in some cases, but if there are complicated or time-consuming steps where the prerequisite can not be met quickly, a recording would be better
> So here’s the story behind why yesterdays live #metaconnect demo failed - when the chef said “Hey Meta start Live AI” it activated everyone’s Meta AI in the room at once and effectively DDOS’d our servers
> That’s what we get for doing it live!
And if you believe that, I've got a bridge to sell you...
It makes me question the rest of this article when the author takes that at face value. A DDOS made your LLM say the same thing over and over? Yeah, I don't think so. Also, this new tech, how many device that support “Hey Meta start Live AI” even exist or were "listening" to the stream.
This is about a valid as Zuck saying that it was "Wifi" issues. How often do you have Wifi issue where you can talk to remote servers just fine?
or not like the recent FB AR glasses. if your live demo doesn't work in rehearsal, there's a good chance it's not going to work during the actual presentation. there's a reason the phrase "demo hell" is well understood.
I always do half and half, a mostly live demo with some cooking show style start and results. It does depend on your audience, and there is charm in a live demo.
No one wants to see a terminal, that's for sure, so don't display code or start pecking away at a terminal unless you really think you'll need it.
jjcm|5 months ago
I prefer live ones, by a significant margin, even for non-deterministic AI things. In general I find live ones better, but it does depend on your circumstance. It's worth asking yourself three questions when you're doing a demo:
What is the outcome when it goes great?
What is the outcome when it goes wrong?
What percentage of the time do each of those occur?
With a pre-recorded demo, when they go great it's usually a 9/10. Things are solid, but they aren't adaptive to the situation, which means you might have lost an opportunity to insert a quip from something that happened earlier. When they go wrong though, it's a 0/10. You've told people that you don't have enough faith in your product to show it live - why should they have faith in it?. Now you might say that the demo goes right 100% of the time(ish), but there's also a human element to it. Someone might play the demo early, someone might not act in time with the live demo, any of these can give away that it's pre-recorded. I'd put their success rate at 95%.
With a live demo, when they go great they're 10/10. You can adapt them, you can show it live, things react instantly instead of slightly-out-of-sync due to scripted content. It's incredible. When they go wrong, it's a 3/10. You've got egg on your face. People make fun of you after. But they know at that point that you ran a live demo. That does count for something. We know Zuck did that demo live. They do fail a bit more, especially in the age of AI. 10% failure rates are pretty good for live demos. To me though, the higher failure rate is worth it for the better PR if you do fail.
jadbox|5 months ago
But, NEVER TELL ME IT'S A LIVE DEMO WHEN ITS PRE-RECORDED. You will lose my business forever if I catch it.
tschwimmer|5 months ago
bee_rider|5 months ago
qwertytyyuu|5 months ago
oggadog|5 months ago
I would never recommend doing this in a sales context though, where you're selling to prospects. Mainly because it gives you no flexibility to pivot and adjust the content based on what they're interested in, and also because it gives off the impression you don't have a lot of faith in your product to perform live.
I'd still 100% recommend having pre-recorded backups however. I was once doing a demo to our largest customer in region and the second we joined the call our core product had a SEV0. I was very relieved to have something to show while the engineers fixed the issue.
godelski|5 months ago
What I dislike is presenting something as if it is live when it isn't. Demos are fantastic things and it is even perfectly fine to have imaginative aspects to them. But there's a division line between demoing and lying. Take Rabbit R1 as an example[0]. This whole thing was faked yet it was presented as a live demonstration, not an illustration. In other words, fraud.
The difference is if you're trying to show your customers (or potential customers) what you're imagining the device works like vs what the device actually works like. The difference is how you're communicating the difference to them.
I'm not sure why this is a contentious opinion, but just don't lie to people?
[0] https://www.youtube.com/watch?v=22wlLy7hKP4
foobarbecue|5 months ago
I do live demos at work all the time and the best part is that when someone asks "can it frobulate with encapulation?" I can actually press the frobulate button with encapulation enabled and show them. Can't do that with a recording.
furyofantares|5 months ago
I asked my professor what it was like, was he an incredible professor? He told me the lectures were useless! I was shocked and he said they were so well prepared they were effectively a recording of the material - and no advantage over reading the book. At that point I reflected and realized that, by far, my favorite professor was one who constantly made mistakes while solving problems on the board. He'd have to notice something wrong and track down the error, and it was wonderful and highly instructional to watch the process.
stavros|5 months ago
https://gitlab.com/stavros/itsalive
I can even pause the replay mid-demo, take over, and resume again later. People are always impressed at how flawless my live demos are, and with some judicious audience participation, the illusion is perfect.
Syntonicles|5 months ago
Luckily it was a small group of devs and not a large venue. Still embarrassing...
unknown|5 months ago
[deleted]
godelski|5 months ago
Seriously... just call it a demo or pre-recorded. As soon as you call it "live" you've crossed into the territory of fraud.
neilv|5 months ago
But when the standards are lower, people appreciate the live demo. Videos are much less compelling, and they could've watched that at home in a few minutes, rather than fly out to you.
In grad school -- in a lab where, instead of "publish or perish", our mantra was "demo or die" -- we were giving demos most weekdays, usually to only a few people at a time. Always live. I think once I had to play a video of someone else's stuff, and it fell flat.
My best demo story is when I was showing a new thing (just a class project), to a few people crowded around a computer in my lab area at an open house event, typing into my software agent in a chat window (pre-LLMs), ad-libbing and speaking aloud what I was typing, "I'm interested in... thinks Microsoft and... thinks Oracle..." Then I turned my head around to look at the people behind me, to see that they were following, and I swear to gosh, Larry Ellison himself was standing in the doorway of my lab area, giving me this fiercely aggressive stare. Whether he was glaring because I said Oracle or Microsoft, I do not know.
My worst demo story was at the very start of my demo career. I had just arrived on campus, and I was borrowing a computer in an open area. A research scientist comes up, and sees on my screen this Web-scraping personalized newspaper thing I had made. So I'm showing it to them, and they're interested, then all of a sudden they say, sadly, "Mother Theresa died..." Which was what my "demo" was showing.
So maybe don't include an uncontrolled live data feed of depressing or distracting news in your demo.
Also, implementing early software agents in Java apparently summons Larry Ellison.
wanderingstan|5 months ago
Meta took that risk and failed.
dylan604|5 months ago
dceddia|5 months ago
- record the screen as you type out all of your code as you would do live
- don't need the audio, just the video
- edit it down to remove typos and pauses
- speed it up to a nice fast typing pace
Now you can drop the video into Keynote or whatever, and Play/Pause the video as you speak.
A crucial part of this is that the video has effectively no pauses. Maybe just a few short ones between sections, to leave time to hit Pause before the next keystrokes appear.
Instead of a 30 minute video that I had to perfectly keep pace with, it was like 5 minutes of rapid typing that I could speak along with and pause when it was time to explain something.
groby_b|5 months ago
As a customer, when you show me a video, I default-assume you're lying.
Also, as a presenter: If your live demo fails and you can't turn that into a win, maybe you're not good at presenting. Your audience wants you to succeed, it takes a minimal amount of charm to make it work. (Heck, even Mark turned this into a win, and he's not exactly a charm bomb)
unknown|5 months ago
[deleted]
psyklic|5 months ago
gertlex|5 months ago
> Since most of the time you're demoing something you previously worked on, this prevents you from working on current stuff right up to the demo, because you're worried you might break your local setup for the demo by installing different node modules, switching to different Git branches, etc.
This isn't the case for me generally... probably more of problem for web-apps things I guess.
I keep multiple workspaces locally for the multiple things I'm working on, but that hasn't been my pain point. Usually it's that I'm demoing a work in progress (not my choice generally) that has rough edges.
I enjoy the internal demos where questions can be asked about doing XYZ, and that can be shown if it's quick. Recordings tend to take away from that, and be less memorable/engaging I guess.
JSR_FDED|5 months ago
All so that I could demonstrate that when I turned off a RAID controller a red light came on and the application just…continued.
So much work for something largely unspectacular meant that a recorded demo that could be run anytime from a laptop was far more appealing. My junior colleagues were now doing demos they otherwise wouldn’t, and the seniors turned off the sound and edited the video into snippets they could introduce and talk around
unknown|5 months ago
[deleted]
robertclaus|5 months ago
Aperocky|5 months ago
https://github.com/sloria/doitlive
unknown|5 months ago
[deleted]
drewcon|5 months ago
Apple can pull it off, but if you’re at a mid market tech company you just sunk like weeks people’s time into choreographing a live movie. Super stressful time sync.
evadne|5 months ago
Audience knows when something is live
j45|5 months ago
But anywhere the demo gods can show up, for example in live coding during a demo, can be partially or fully recorded, but narrated in person.
collinmcnulty|5 months ago
If you aren’t doing a live demo, it’s not a demo, it’s just a sizzle reel.
joshstrange|5 months ago
> That’s what we get for doing it live!
And if you believe that, I've got a bridge to sell you...
It makes me question the rest of this article when the author takes that at face value. A DDOS made your LLM say the same thing over and over? Yeah, I don't think so. Also, this new tech, how many device that support “Hey Meta start Live AI” even exist or were "listening" to the stream.
This is about a valid as Zuck saying that it was "Wifi" issues. How often do you have Wifi issue where you can talk to remote servers just fine?
m11a|5 months ago
dylan604|5 months ago
oftenwrong|5 months ago
hartator|5 months ago
BoorishBears|5 months ago
infiniteregrets|5 months ago
anarticle|5 months ago
No one wants to see a terminal, that's for sure, so don't display code or start pecking away at a terminal unless you really think you'll need it.
kevinmchugh|5 months ago
zahlman|5 months ago