[noen]

I worked on this project at my former employer for nearly 8 months. We worked as a good faith partner with McDonald’s, only to find out eight months in that McDonald’s had never had any intention of working with us and we’re only using us as a negotiating tactic for the acquisition they eventually made for this technology.

I learned two things from this project. Our team built a complete English language model for every possible permutation of ordering every menu item. No LLMs, no gpt required.

The hilarious part is that McDonald’s stopped responding to our team and abandoned the project only about a month before we were going to tell them that it was not technically feasible to do it scale. The problem is not bad AI nor a lack of data. The problem is all of the real world interface challenges, keeping speakers and microphones working outdoors across many different climates and weather conditions, and temperatures and humidity is incredibly expensive. And on top of that, the hardware needed per store at the edge and the permanent network infrastructure required to keep everything running makes the whole system substantially more expensive than just having a single human being running drive-through even have much higher wages than minimum wage.

So McDonald’s never got this conclusion from us and instead spent several billion more dollars and another six years of R&D to come to the same conclusion.

My other take away from this project was that I will never give McDonald’s another dollar of my money. Working with over 200 fortune 500 companies in my career McDonald’s is far away the most evil heartless and ruthless company I’ve ever dealt with.

They don’t care about their customers, their users, their franchisees or their employees. The only thing they care about at all is their stock price.

[_nist]

Doesn't surprise me in the least. McDonald's is such a bad company on many levels, even from its founding roots. They make fast food (an industry notorious for craping all over their employees) and it was usurped by a greedy business man (Ray Kroc) from its original founders. Their food is okay, but I would never, ever do work for them.

[WorldMaker]

> usurped by a greedy business man (Ray Kroc) from its original founders

It's a weird aside, but maybe worth noting that Ray Croc was probably not just greedy. Ray Kroc was probably also racist against the Irish. A tell-tale sign is using a clown as the company mascot. The US strain of clowns were heavily influenced by "pale white face" racism jokes about Irish immigrants [0] from some of the same minstrel shows notorious for "black face" and "yellow face" and "red face". Ray Kroc was from a generation that would have easily been aware of that and would have been "entertained" by it. Ray Kroc's behavior to the actual McDonald's founders is rather easier to explain assuming it included quite a bit of Old Fashioned Racism than assuming just pure greed. Sometimes it is useful to remind ourselves that past isn't as clean as corporate memos want to paper over it.

[0] Notably, among other things: red hair, big feet, freckles, red drunken noses, loutish drunken behavior. Even the "clown car joke" is the exact same "joke" as "Mexican pickup truck" transposed across a couple of decades and about a different working class immigrant population. (Racists seem pretty lazy in how they reuse old material.) So yeah, if you ever wondered why clowns don't seem all that funny in the modern era, congratulations you probably aren't a racist. Also, now that this past horror is in your head I'm sorry for ruining Disney's Dumbo which uses all the worst of clown stereotypes and "jokes" all in the same place and eats up a lot of runtime with it, if you weren't already concerned about the "black face" crows in the movie or thought you could dismiss them as not central characters or contributing that much to the runtime.

[tracker1]

I don't even consider the food okay... I liked a few breakfast options, but the prices of even that have gone up so much the past 3 years, I won't even go then. It wasn't that long ago that the breakfast burrito was $1 (then 2 for $2/3) and they had some sandwiches 2 for $2/3/3.33/4/5 ... When I'm paying over $10 for a couple breakfast sandwiches and a drink, I'm out.

They definitely seem to be squeezing more out of their franchises than ever at this point. They will push technology optimizations, but they're at a point where service and price just isn't there. If I'm spending $15+ for lunch, I may as well go to Applebees/Chili's, etc.

[tehwebguy]

Thanks for sharing, everything makes sense except this:

> The problem is all of the real world interface challenges, keeping speakers and microphones working outdoors across many different climates and weather conditions, and temperatures and humidity is incredibly expensive.

I thought this was a solved problem already, by McDonald’s in fact?

[mvid]

I am not the poster, but my guess here is that a point to point or radio setup which just needs to be good enough for humans to understand it (and even then just barely) is likely way worse quality than what would be needed for recording and feeding to a program

[josefresco]

> The problem is all of the real world interface challenges, keeping speakers and microphones working outdoors across many different climates and weather conditions, and temperatures and humidity is incredibly expensive.

This sounds like a problem for humans too - are humans just better equipped to deal with bad audio?

[noen]

Yes, but not in the ways you think. Humans can do about as good a job as a computer in understanding poor audio quality in context, but the compute needed for the latter, in realtime, is pretty substantial.

As the commenter says below, a human can intervene in many more ways when equipment malfunctions or customers have special needs that an AI just gets blocked by.

There are literally hundreds of edge cases where a voice powered drive through just stops working, from high winds to pouring rain, to thick accents, broken equipment, out of stock or seasonal items not available. Just a few of the ones I encountered personally in the wild tagging 15,000+ orders.

[lazide]

Humans have had to evolve to be able to communicate with each other effectively in this and far worse conditions - on battlefields, in driving hurricanes, while being stalked by other humans (and animals), yelling across vast distances, over raging rivers, while wounded, sick, etc.

Humans can generally figure it out eventually.

[torginus]

Yeah this sounds strange to me - every drive thru has had an intercom system like this working outside for decades. Doesn't seem like an insurmountable challenge to me.

[nunez]

> The problem is not bad AI nor a lack of data. The problem is all of the real world interface challenges, keeping speakers and microphones working outdoors across many different climates and weather conditions, and temperatures and humidity is incredibly expensive.

That's a great point. Shazam, which has a "simpler" challenge, still trips up with identifying music in challenging environmental conditions.

[ethbr1]

Having worked for a variety of corporate structures, I'd agree that franchise models tend to have... a bit more ruthless approach to everything. (Even unintentionally)

When "the business" is literally outside your company, you get the same dynamics as when it's in another org structure, except even worse.

[andrewf]

> keeping speakers and microphones working outdoors across many different climates

Pretty common for two humans to have trouble understanding each other through the microphone setups, too.

[robotwizard]

Why can't the hardware problem solved with just a phone and an app?

Also what ML model did your company use?

[FrustratedMonky]

I can answer that. Because I'm sick of installing a hundred apps for every single company, setting up accounts, verification, so I can spend 10$, and never use it again.

And, when you are traveling, and just taking the highway exit for a quick order, what, now I have to stop, park, download an app then order.

Or do you mean, like mount Ipads in kiosk mode? I think that gets back to the expense and weather.

[noen]

This is why McDonalds literally spends tens of millions of dollars per quarter giving away free food to attempt to get more people to just use the app.

Their plan longterm is exactly this - have a robot box with a window that only accepts orders through your phone and removes all humans from the process.

You might be surprised how many people absolutely hate using apps, and do not want to interact with a business through a mobile phone. Older folks especially, but also there is a large portion of the population that only use technology when necessary.

We build a heuristic model from scratch. We used BART for some NLP bits and Azure speech to text for the basic mic -> raw text. My memory is very muddy on this bit as I didn’t work on the algo portion of the project, I was working on the UX workflows and interaction / conversational workflow design and validation.

Fun fact, McDonalds menu is a graph database that tracks every single ingredient as purchasable entity. You can order damn near any combination of elements and they will sell it to you - want 32 pickles and the bottom piece of a bun with a chocolate chip cookie on top? No problem.

And every promotional item and name ever put on sale is persisted in the menu forever.

[astronads]

Please share more about this!

[frithsun]

McDonald's has screwed up my order less than any other restaurant, provided better, warmer food than any other restaurant, is cheaper for the quality than any other restaurant, and their Ronald McDonald House charity rescued my family during our darkest hour.

Sorry your AI partnership with them failed, guy.

[ethbr1]

For those unfamiliar,

>> There are over 380 Ronald McDonald Houses in 64 countries. These accommodate families with hospitalized children under 21 years of age (or 18 or 26, depending on the House), who are being treated at nearby hospitals and medical facilities. [...] Ronald McDonald Houses allow families to stay free of charge.

https://en.m.wikipedia.org/wiki/Ronald_McDonald_House_Charit... https://www.charitynavigator.org/ein/362934689

As far as targeted charities go, they're near the top in terms of delivering something everyone agrees is good.

[tracker1]

That hasn't been my experience at all. Not sure where you are, but in the Phoenix area, I have gone several multi-year periods not spending anything at McDonald's. They had messed up my order more times than getting it right.

Worst was going ketovore... how the hell hard is it to put two pieces of meat in a box without a bun and nothing on it? OR getting 3x "2 strips" of bacon and counting to 6.