top | item 41659324

DoNotPay has to pay $193K for falsely touting untested AI lawyer, FTC says

315 points| Brajeshwar | 1 year ago |arstechnica.com | reply

201 comments

order
[+] ziddoap|1 year ago|reply
>DoNotPay also did not "hire or retain any attorneys" to help verify AI outputs or validate DoNotPay's legal claims.

Wow, that's brave. Create a wrapper around ChatGPT, call it a lawyer, and never check the output. $193k fine seems like peanuts.

Sometimes I think about where I would be in life if I had no moral or ethical qualms. I'd probably be running a company like this.

[+] ryandrake|1 year ago|reply
> Sometimes I think about where I would be in life if I had no moral or ethical qualms. I'd probably be running a company like this.

I think about this all the time. If I didn’t have a conscience, I would be retired by now.

[+] exe34|1 year ago|reply
> Sometimes I think about where I would be in life if I had no moral or ethical qualms. I'd probably be running a company like this.

I invented smaller variants of deliveroo, airbnb and uber in my mind, around 2008, but I thought no, the only way to make any money would be to exploit people and break laws. honestly, what held me back was more the hassle of lawyers to make it all work. I didn't think I could stomach the effort.

[+] debarshri|1 year ago|reply
Donotpay should have used donotpay to fight FTC ruling back. That would have been the ultimate outcome if they would have won.
[+] alexey-salmin|1 year ago|reply
Given how bad an average attorney is, I wonder if chatgpt would actually be an improvement.
[+] lancesells|1 year ago|reply
> Sometimes I think about where I would be in life if I had no moral or ethical qualms. I'd probably be running a company like this.

And that's why you're not a criminal. This is a criminal act and approved and implemented by criminals. Main thing is in the US it usually pays to do white collar crime as long as it's not too big or too embarassing to politicians.

[+] 1vuio0pswjnm7|1 year ago|reply
Is "brave" a euphemism for stupid.
[+] bloodyplonker22|1 year ago|reply
To prove that that DoNotPay does not work, they will use DoNotPay on themselves to defend against this case.
[+] jlarocco|1 year ago|reply
I'm not sure they mean "LLM based AI".

To me it looks like they automated some boilerplate legal forms and marketted it as "AI" to capitalize on the hype.

[+] datavirtue|1 year ago|reply
What is morally or ethically wrong with what they did?

Maybe it's morally or ethically wrong to prosecute them and take their belongings?

[+] Narhem|1 year ago|reply
Crazy people take any advice from people without citing the exact legal clause.
[+] Palmik|1 year ago|reply
Did they help regular people defend themselves while saving on legal costs or not?

Most of these cases wouldn't be defended at all otherwise.

[+] refurb|1 year ago|reply
This is my issue with AI. We know it spits out nonsense sometimes. Not just random questions, but even code generation.

It will no doubt improve, but somebody has to confirm that there are no errors in the output. If it has 1 error in 10,000 now, let's say it has a 3 log improvement, so now it's 1 in 1 million.

Would that be ok for legal or medical decisions? I don't think so. How about business decisions? Nope.

As long as AI is generating output with errors, it's going to have limited use.

[+] MangoCoffee|1 year ago|reply
Another Silicon Valley startup looking to get rich quick, following in the footsteps of Uber, Airbnb, DoorDash, WeWork...etc, which have all played in the legal grey areas
[+] entreunemployed|1 year ago|reply
Wow buddy. I guess we could say the dollar amount you were willing to forego is the price of your sense of superiority.
[+] guluarte|1 year ago|reply
lmao at least the should have finetuned gpt4
[+] gosub100|1 year ago|reply
> if I had no moral or ethical qualms. I'd probably be running a company like this.

You mean you'd be like any other corporation or property manager or attorney? If you operate in the confines of the law that's all that matters. If you give normal people the same power to litigate as a billionaire, that's a feature, not a bug.

[+] gradyfps|1 year ago|reply
"In 2021, Browder reported that DoNotPay had 250K subscribers; in May 2023, Browder said that DoNotPay had “well over 200,000 subscribers”.

To date, DoNotPay has resolved over 2 million cases and offers over 200 use cases on its website. Though DoNotPay has not disclosed its revenue, it charges $36 every two months. Given this, it can be estimated that DoNotPay is generating $54 million in annual revenue, assuming that all 250K users subscribe for 1 year."[1]

$193K seems like a pittance compared to the money they're making off of this.

[1]: https://research.contrary.com/company/donotpay

[+] xoa|1 year ago|reply
>$193K seems like a pittance compared to the money they're making off of this.

I don't have any special knowledge of this specific case, but it's important to note as a general principle that often the point of these fines is as the start of a process. It creates a formal legal record of actual damages and judgement, but the government doesn't see massive harm done yet nor think the business should be dead entirely. They want a modification of certain practices going forward, and the expectation is that the company will immediately comply and that's the end of it.

If instead the business simply paid the fine and flagrantly blew it off and did the exact same thing without so much as a fig leaf, round 2 would see the book thrown at them. Defiance of process and lawful orders is much easier to prove and has little to no wiggle room, regardless of the complexity that began an action originally. Same as an individual investigated for a crime who ends up with a section 1001 charge or other obstruction of justice and ends up in more trouble for that than the underlying cause of investigation.

So yes, not necessarily a huge fine. But if there weren't huge actual damages that seems appropriate too, so long as the behavior doesn't repeat (and everyone else in the industry is on notice now too).

[+] ahmeneeroe-v2|1 year ago|reply
This is founder-raising-funds math (or VC looking for liquidity math). 200k subscribers might not mean paid subs and it certainly doesn't mean 1-year of paid subs. This could be $9M (a single-month of 250 paid subs) or lower.
[+] mgraczyk|1 year ago|reply
The main product actually works, this is for additional claims that were misleading. It isn't right to compare the settlement to the entire company revenue. Better to compare to the benefit gained by wrongdoing, or the amount of harm caused.
[+] Mordisquitos|1 year ago|reply
I love the quote they included in their ads, purportedly from the Los Angeles Time but "actually from a high-schooler’s opinion piece in the Los Angeles Times’ High School Insider":

> "what this robot lawyer can do is astonishingly similar—if not more—to what human lawyers do."

[+] gradyfps|1 year ago|reply
To be fair if legal paperwork follows a standard process with standard information, a "robot" can complete many orders of magnitude more than any human lawyer. (I'm also not a lawyer and have no idea if this line of thinking is applicable.)
[+] whoisjuan|1 year ago|reply
This is a very sneaky ethically gray company. Their app is not only of terrible quality but also full of dark patterns. I'm convinced that any revenue they make comes from people who can't figure out how to cancel. Stay away from it.
[+] ec109685|1 year ago|reply
Ironic given the app’s purpose.
[+] Palmik|1 year ago|reply
The legal system has a great sense of self preservation. They will surely fight anything that possibly encroaches on their domain, especially things that give non lawyers the tools to defend themselves without feeding the machine.
[+] lolinder|1 year ago|reply
Sometimes the enemy of my enemy is still my enemy. Many organizations that purport to be helping the little guy are actually just exploiting them for profit.
[+] winddude|1 year ago|reply
'''"None of the Service’s technologies has been trained on a comprehensive and current corpus of federal and state laws, regulations, and judicial decisions or on the application of those laws to fact patterns," the FTC found'''

Wow!! That seems so simple, and literally a few weeks to do in today's ecosystem, now thoroughly testing make take a little more time, but wow, I wonder if it was evening attempting to do RAG.

[+] mcheung610|1 year ago|reply
DoNoPay has to pay a fine is just an irony.
[+] abixb|1 year ago|reply
Haha, my thoughts precisely. The first 4 words of the title were bewildering for me for a few seconds.
[+] dcchambers|1 year ago|reply
> initially was advertised as "the world's first robot lawyer" with the ability to "sue anyone with the click of a button."

There is no world in which allowing that to happen is a good idea.

[+] praveen9920|1 year ago|reply
I understand the sentiment but to be fair this is currently happening everywhere but only the rich people have access as they are the only one who have lawyers on retainer.

Access to legal services to poor will change things. In short term, judicial system will be overwhelmed and are forced to adopt new and efficient procedures.

[+] Habgdnv|1 year ago|reply
What a title: "DoNotPay has to pay."

Can't they ask ChatGPT to write some objection and, at the end of the prompt, put "but make it look like it was written by a lawyer" and send that to the court to waive the fine?

Please read my comment as a joke. The title really sounds funny!

[+] swalsh|1 year ago|reply
The problem with today's technology is it is indistinguishable from magic. Sometimes the magic is real, sometimes it's an illusion. It's nearly impossible as a regular consumer not deeply knowledgable of the current capabilities of models to know which is which.
[+] tonygiorgio|1 year ago|reply
There’s a lot of hate for the AI marketing aspects, and that AI isn’t up to par for full lawyer replacements, but they’ve been around with a very working and usable app before way before the AI hype.

Lawyers at huge firms or companies automate the hell out of their legal actions against normal citizens and get things wrong all the time. But it’s okay if they do it because they’re part of the same cabal keeping the legal system afloat. Say what you want negatively about some dark patterns and marketing BS, they’re making legal things affordable to the every day person.

The fine seems fair for overhyped marketing claims, but I hope they keep going and improving.

[+] Crackula|1 year ago|reply
1. Amusing title 2. Yea it sounds like a smug and shitty company But 3. If I get some parking ticket or some small wrong doing I am totally going to consult with GPT before I go ahead and hire a lawyer. A well trained AI could definitely do the work of most lawyers better than they could. Lawyers and judges usually just recite rules, previous cases and known loopholes. They are a human search engine and they cost quite a bit.
[+] dboreham|1 year ago|reply
Ok well it seems my test for whether we really have AI yet (are there self-driving lawyers) remains unsatisfied. For me lawyers work is significantly easier to automate with some proto-AI than is software development or driving a car. So although recent AI progress is highly impressive, I'm not retiring until it takes over the lawyers.
[+] BarryMilo|1 year ago|reply
Truly one of the worst takes on AI I've seen. What is the purpose of a lawyer? It's not to read the law, the text is free (or should be). It's to advise you, based on the law but also on the immediate, historical and sociopolitical coontext, as well as on their understanding of the characters of all the humans involved.

If that gets automated, we are in AGI territory.

[+] zulban|1 year ago|reply
It's not a question of how easy it is to automate it. It's about how frequent and costly the mistakes are. Lawyer AI is a high bar - plus the people watching you are human lawyers, exactly the kind of people who can make your mistakes more costly.
[+] pnw|1 year ago|reply
Even before AI, that website has been making overly optimistic claims for many years. It was never clear to me how real or effective it was. The Wikipedia article has more detail but it seems like this is the first time the government has actually called them out?

Of course this didn't stop them raising $10m from credulous investors in 2021.

[+] popcalc|1 year ago|reply
In 2021 a literal discord group raised several million dollars so take that into consideration.
[+] shahzaibmushtaq|1 year ago|reply
I think DoNotPay and find out is the vision of this company.
[+] system2|1 year ago|reply
I am stopping myself from releasing shitty chatgpt wrappers which I see at every trade expo. I am just not doing it because I don't want to add more shit to the shitshow.
[+] beerandt|1 year ago|reply
Has the chance to make their best marketing material yet.