top | item 44967796

A statistical analysis of Rotten Tomatoes

234 points| m463 | 7 months ago |statsignificant.com | reply

151 comments

order
[+] jackero|7 months ago|reply
I find RT scores very accurate but not the raw score.

What I mean is that a 70% score is meaningless to me. I need to know the movie genre, the audience score, the age of the movie and then I basically do a “lookup table” in my head. And I have that lookup table because I’ve looked up every movie I’ve watched on RT for 15 years so I know how the scores correlate to my own personal opinions.

As an example: the author said that critic scores should align with audience scores but no that’s not true at all. Critics tend to care more about plot continuity, plot depth and details while the audience tends to care about enjoyability. Both are important to me so I always look at both scores. That’s why a lot of very funny comedies have a 60-69% critic score but a 90%-100% audience score — because it’s hilarious but the plot makes no fucking sense and has a million holes. And if you see a comedy with 95% critic but 70% audience, it will be thought-provoking and well done but don’t expect more than occasional chuckles.

[+] uoaei|7 months ago|reply
Plex shows you both critic and audience scores from RT (IMDB also) and they indeed diverge consistently on the lines you suggest. In general I trust the audience scores a lot more because I'm trying to have fun watching movies rather than analyze their plot/pacing/cinematography/etc.

The audience can be trusted to know how to have fun. The discrepancy between critic and audience scores is also a valuable signal to judge how fun campy/schlocky/B-movie horror films particularly from the 80s and 90s.

[+] SwtCyber|7 months ago|reply
Rotten Tomatoes becomes way more useful once you treat it as a tool rather than a verdict
[+] Jarmsy|7 months ago|reply
I often enjoy movies that are unexpected and don't fit neatly into one established genre, but I think these tend to get lower audience ratings, while films that deliver to expectations do better, even if most of a randomly selected audience would dislike them. If a movie is a comedy, with a poster with big red letters and a white background, people know it's a certain kind of movie, and mostly those who enjoy those movies will go see it. Likewise with documentaries about some niche interest - those who watch it mostly sought it out because they're into that.
[+] waerhert|7 months ago|reply
My immediate thought after seeing the first chart was that it is inversely related to my own experience with movies in the last 20 years. Maybe there's an idea in there for a 'score normalizing' browser extension.
[+] Finbel|7 months ago|reply
Always wondered why Rings of Power have 84% critics score but just 49% audience.
[+] eastbound|7 months ago|reply
Critics have a political agenda, they overrate movies with “a message”, the message being always leaning Californian. The movie industry is a massive sector with lobbies, and paid critics are no stranger to that.

And as the sibling says, audience pays to see a movie. The audience, the people, are more politically balanced. There is no bias or selection: It’s the democratic components, including people that the “in” lobbies don’t like.

If only we could get rid of this damn audience!

[+] neogodless|7 months ago|reply
Something I thought you might get into would be series, whether movies or TV / sequels. Sometimes they get devoted fans who love the whole series, and those sequels or later season have great scores, but you might not enjoy them whatsoever.

My example would be a TV show, A Discovery of Witches which is overall well-received, but I couldn't enjoy at all. Perhaps if you read the books, you'll like the show, but for me, it was such an empty show, devoid of excitement or intrigue or entertainment value.

[+] atoav|7 months ago|reply
Additionally there are movies who just have something unique to them that a niche audience may love, but both critics and the general audience treat them more harshly.

The truth is that other peoples opinion may or may not be a good proxy for your own taste in movies, even if it was uncorrupted and independent.

[+] freeopinion|7 months ago|reply
Too bad the two ratings categories are "critic" and "audience" instead of "plot", "humor", "characters", "suspense", ...
[+] baxtr|7 months ago|reply
What I find weird is that no one has solved the "people like you also liked this" problem for ratings/reviews.

All ratings on these platforms are average values through the entire cross-section of people.

Yet I am sure that they are people who have a very similar taste like me. I want to read their reviews, see their ratings, and recommendations.

Social media platforms do that pretty well these days.

[+] jv22222|7 months ago|reply
Please list a few more insights like this for picking good movies, thanks!

Additionally, I think someone could build an interesting RT browser based on these kinds of insights.

[+] testdelacc1|7 months ago|reply
The last comedy that I saw that matches your description is American Fiction. It didn’t feature too many laugh out loud moments, but it was thought provoking and well done. And yet, 93% from critics and 95% from audiences.

I wonder if audiences can appreciate these movies more than you give them credit for?

Let’s try a few more

- Death of Stalin (94%, 79%) has the pattern you’ve predicted.

- O Brother Where Art Thou? (78%, 89%) has the opposite of the pattern.

- Grand Budapest Hotel (92%, 87%) was appreciated by both, like American Fiction.

I’m just not seeing a pattern here. Looking at comedies that fit your description the critics and audience scores don’t follow a predictable 95%, 70% pattern.

[+] kelseydh|7 months ago|reply
I generally find IMDB user scores far more reliable and granular for movies. There is a noticeable jump in a movie's quality when it gets a 6.x rating (okay), versus a 7.x (great) versus an 8.x (a Top 500 of all time).

Metacritic is the next most useful, while Rotten Tomatoes is easily the least useful. High critical and user RT reviews often does not provide a good intensity barometer of how good the film actually is. The last ten years I went from being a loyal RT user to completely ignoring their scores altogether.

[+] mhh__|7 months ago|reply
You basically want a cross sectionally standardised score
[+] rubzah|7 months ago|reply
High critic score / high audience score = Good

High critic score / low audience score = Paid-for hype, or politically motivated reviews

Low critic score / high audience score = Possibly a good movie

Low critic score / low audience score = Bad

[+] rurban|7 months ago|reply
> Humanity Has Stopped Producing Bad Art: After a century of trial and error, mankind has perfected the art of cinema, as proven by recent masterworks like Cats, Space Jam: A New Legacy, the live-action Snow White, Red One, and Joker: Folie à Deux. Critics, who were once joyless automatons thriving on takedowns of human creativity, now bask in this golden age of moviemaking, lavishing praise upon the timeless artistry of The Walt Disney Company and Warner Bros. Discovery.

This really should appear in professional film reviews.

[+] neilv|7 months ago|reply
> To account for this influx of reviewers, Rotten Tomatoes has created a "Top Critic" designation reserved for established media outlets, such as The New York Times and The Atlantic. However, this label has no special bearing on a film's top-line Tomatometer score and is largely incorporated into ancillary aspects of the site.

Just last night, I noticed that I could access the two percentage scores for critic reviews.

If you go to "https://www.rottentomatoes.com/m/the_dilemma", and click on the critic reviews percentage (25%), you get a popup that lets you select between seeing the All Critics score (25%) and Top Critics score (28%).

(And if I'd thought to check Rotten Tomatoes first, when selecting what looked like a fun light comedy on Netflix, I wouldn't have wasted an hour of my life before I said WTF, checked RT, and continued to be in a bad mood.)

Incidentally, I'd love to have the Tomatometer score integrated into my UI for video streaming services. The services seem to instead like to use their own very generous scoring instead. (When they show any score at all. Some like to suppress the ratings for new shows they produced, presumably to avoid shooting down their own poor shows before people watch them by default.) But Rotten Tomatoes is a much better predictor of how I'll like a show than the streaming service scores are. But maybe the streaming services don't want to expose that the majority of the movies and series offered at any time now range from mediocre to outright bad.

[+] bruce511|7 months ago|reply
>> But maybe the streaming services don't want to expose that the majority of the movies and series offered at any time now range from mediocre to outright bad.

There is no "now" necessary in that sentence.

All media production in all eras is mostly terrible. Music wasn't better in the 80s, or 70s, or 60s, its just that the 80s music you hear today is heavily curated to the good stuff.

It seems like streaming has made it worse, but only because you're watching so much more. In the past movies took effort to watch. You went to the cinema, or video shop. By the time they made it to TV they were curated, or at the very least you knew about them.

There was plenty of dross that made it direct to video that never made it to cinema or TV. (In 1989 I lived for a year at a place with no broadcast TV. We watched 2 videos a night from the local blockbuster-type store. They had a LOT of very crap movies.

To blame streamers for delivering a lot of mediocre content is to miss the root cause. Most new content is mediocre. Or bad. It has always been the way. Streaming just makes it easier to watch.

[+] michaeljx|7 months ago|reply
I am doing my movie selection via Plex, which has both tomatoes' and IMDb scores in the movie description
[+] daft_pink|7 months ago|reply
I will say that when I used to go to the theaters, which was before the pandemic and I started a family I used metacritic.

I found that any time I went to something that was red, I absolutley regretted it and it was terrible. Yellow was more hit or miss and top green scores were pretty good.

Exceptions were comedy where a lower score could still mean a good film, and politics oriented films, where a bad film with a media approved message could get a really good score even if it sucked.

It’s sad to not get a reliable indicator of that and someone should just resurrect the old score and call it Bad Apples. Since the actual score seems transparent, why not develop a competitor.

[+] gboss|7 months ago|reply
Theaters still exist you should go back. I go two or three times a month. If your kid is 4 or older they’ll have a great time. It’s good and healthy to get out of the house!
[+] SwtCyber|7 months ago|reply
Some genres just don't track well with critic metrics
[+] mxxx|7 months ago|reply
Regardless of the introduction of sycophantic reviewers, the 3/5 = fresh thing has always been a pretty half-ass threshold imo, and that a fact that a film can be "100% fresh" on RT on the basis of every single reviewer saying "yeah it's nothing special but it's fine, 3 stars" is fairly easy to misinterpret.
[+] padraigfl|7 months ago|reply
I think it's fine as a metric if you read it correctly.

100% means a film is extremely agreeable with whatever audience it has managed to get to. For major releases this can ultimately mean it's actually lacking anything particularly bold or interesting. This results in things like Frost/Nixon or Knives Out having higher ratings than broadly acclaimed films like Mulholland Drive or even There Will Be Blood; I know which ones I'd be more likely to put on with my extended family even if I don't especially like them.

But yeah, it's amazing how many people still don't grasp it after decades of getting angry about it.

[+] cubefox|7 months ago|reply
Yeah. Pixar movies are often close to 100%. IMDb ratings are usually far more reasonable.
[+] pbsds|7 months ago|reply
a mean rating of 3 can only be 100% fresh if the variance is 0
[+] zdc1|7 months ago|reply
Yeah, I'd love a personalised Tomatometer. I'd only get out of bed for something 4 stars (8/10) or above, so I'd love to know the percentage of audience/reviewers that scored like that.

This would also give "cult classics" and interesting/creative films that are more love-it-or-hate-it a bit more of an edge in ratings over the lukewarm Marvel slop we see these days.

[+] FuturisticLover|7 months ago|reply
I find IMDb better than RT. RT even though it has an Audience score. It tends to give priority to the Critics' score, which in most cases we know is influential in one way or another.

IMDb score doesn't rely on a group of people, but on all users. It may give a little more weightage to the US users, but that's fine. Its top movies and TV shows make a lot more sense, unlike RT.

[+] cantor_S_drug|7 months ago|reply
Tangentially related : It is possible to deanonymize users from kaggle dataset or netflix competition.

https://medium.com/@EmiLabsTech/data-privacy-the-netflix-pri...

Compared to the example of the medical records, Netflix had been very careful not to add any data that could identify a user, like zip-code, birthdate, and of course name, personal IDs, etc. Nevertheless, only a couple of weeks after the release, another PhD student, Arvind Narayanan, announced that they (together with his advisor Vitaly Shmatikov), had been able to connect many of the unique IDs in the Netflix dataset to real people, by cross referencing another publicly available dataset: the movie ratings in the IMDB site, where many users post publicly with their own names.

https://www.cs.utexas.edu/~shmat/shmat_oak08netflix.pdf

https://courses.csail.mit.edu/6.857/2018/project/Archie-Gers...

[+] Gareth321|7 months ago|reply
I agree. I stopped relying on RT scores years ago. There are far fewer occasions when the audience score doesn't align with my preferences. Today I am so far away from most critics in taste and preference, it's like they live on another planet.
[+] defrost|7 months ago|reply
There are many metrics that can be sampled, many challenges to normalization.

eg: https://ext.to/browse/?sort=seeds&order=desc&cat=1&q=2019

is a listing of 2019 movie torrents ranked by seeds (number of clients holding full copies of a torrent version).

A normalization challenge is to group torrent variations (1080p rips and 720p rips and WEB-DL's and BluRay and etc.) and tally up and rank interest in various films over time.

Clearly Ne Zha (2019), a Chinese Animation, Fantasy, Adventure movie was a global pirate star of that year .. should it be "normalized" by population of country of origin to smooth out the home team having a billion+ in population "bias" ?

One advantage of ranking films by year and pirate copies is it provides a pragmatic measure of "staying power"

https://ext.to/browse/?sort=seeds&order=desc&cat=1&q=1964

My Fair Lady, Dr Strangelove, Rudolph the Red-Nosed Reindeer, Mary Poppins, A Fistful of Dollars, and Goldfinger are still being hoarded 60 years after their release.

* https://www.themoviedb.org/movie/615453

* https://www.rottentomatoes.com/m/ne_zha

[+] seemaze|7 months ago|reply
I just attend my local independent cinema. Sometimes I’m blown away. Sometimes the film sucks, but there are only 3-4 options at any given time. Simplifies the decision, and at the very least, they serve beer.
[+] ildon|7 months ago|reply
I only rely on IMDb scores and they're still reliable if enough time has passed since the movie release. A movie with more than 6 avg is usually enjoyable, while below 6 is not worth watching. Over 7 is usually very good, over 8 is a masterpiece
[+] kelseydh|7 months ago|reply
I agree wholeheartedly. What none of the review sites have figured out is how to adjust TV show user scores so that they don't score so much higher than movies.
[+] mjamil|7 months ago|reply
It’s interesting that people pay close attention to one-size-fits all number (regardless of the pros and cons of the methodology used to derive said number). I find RT really useful for collating the reviews from “top” reviewers in one place: over the years, I know how my interests align with the tastes of particular reviewers, and I don’t have to look in multiple places to get a snapshot view of their opinions.
[+] lordhumphrey|7 months ago|reply
The idea that something like a movie's "good"-ness (or whatever) could be measured accurately, reliably or even usefully, by experts or laypeople, should be absurd on its face to any right-thinking person.

What a movie does or does not do for you will depend on a host of things which couldn't possibly be blamed on the movie itself - there's no objective experience of a movie.

The lighting, what you had for dinner, the comfort of your chair, if the baby starts crying, who you're with in the room, how they're feeling, whether you actually really have time for a movie or should be doing something else, if you're 16 and just had your first break up 3 days before, whether you saw the original movie 30 years ago or totally missed the boat, if you've smoked a few spliffs with your friends in the car in the parking lot of the cinema beforehand, etc etc.

People who think differently are, I think, simply going along with a very peculiar recent trend.

It's a general effect the Internet has had on our aesthetics. We no longer think our experience matters or even really exists, instead we think our take on the thing in its social context amongst all the other people's takes is all that matters and exists.

Liking something means liking it exactly how the average person liked it, and sharing a "take" means describing how we deviated, ever so slightly, from the agreed-upon-reading of the agreed-upon-thing. In reality, if you watch a movie ten times, you've had ten different experiences. If one hundred people watch a movie, there have been one hundred different experiences.

It's a great shame that we've forgotten this fact, and lost ourselves in a culture of "reacting".

[+] ChrisMarshallNY|7 months ago|reply
It's simple: When there's money to be made, the folks with the money to make, will pull out the stops to game the reviews. We have seen this in just about every review system ever (except Consumer Reports). They game it, milk some money, until everyone figures out it sucks, then abandon it. The reviewers count on getting brought.

I really miss Mr. Cranky[0, 1]. Here's his review of Battlefield Earth[2]:

[0] https://en.wikipedia.org/wiki/Mr._Cranky

[1] http://web.archive.org/web/20120503072554/http://www.mrcrank...

[2] http://web.archive.org/web/20120926010236/http://www.mrcrank...

[+] s_dev|7 months ago|reply
I've always found IMDBs rating to be far far better.

Anything that's 7+ is generally good, anything below that is flawed. The Tomato meter just comes off as random and an unreliable indicator for me.

[+] bazmattaz|7 months ago|reply
Yes I was going to say something similar. I often use a blend on IMDb score and rotten tomatoes to judge whether I should watch a movie.

The only thing I will say is that IMDB scores are also (likely) gamed by movie studios to artificially inflate their scores. You also need to be careful with certain genres and series on IMDB that attract positive ratings from the archetypal IMDB user. This is why marvel movies and such receive such sky high ratings.

Basically if the movie is not a mainstream superhero type blockbuster full of cgi you can use the IMDB score if your judgement

[+] kelseydh|7 months ago|reply
Rotten Tomatoes is the First Past the Post of movie reviews. Incredibly distorted and unreliable in capturing the sentiment of the majority.
[+] retox|7 months ago|reply
Obvious to anyone using the site or aware of is ratings, including the author, but it is good to see some analysis as evidence.
[+] sirnicolaz|7 months ago|reply
I still miss the times when I would decide to watch a movie based on the cover at the VHS store or based on a recommendation. Much faster, way more serendipity.
[+] bmacho|7 months ago|reply
Netflix shows movie posters and Reddit has recommendation threads.
[+] rfwhyte|7 months ago|reply
Its not just Rotten Tomatoes, the vast majority of so called "Reviews" the average consumer sees these days are being manipulated in service of corporate interests.

Amazon's full of paid reviews for scam products, so called "Independent" review sites like Wirecutter are basically just advertorial hosting platforms now, or are even secretly owned by the companies who's products are being "Reviewed," 99% of YouTube reviews are nothing more than sponsored content that regurgitates press release talking points from companies who provide the reviewers free products, Google and Yelp reviews of local businesses have an entire manipulation industry built up around them (Just Google "Yelp review service" to see what I'm talking about," and the sad reality we live in now is most anywhere you go, most so called "Reviews" you see these days are either from bots or corporate shills.

We officially live in the "No trust" era and its only going to get so, so much worse from here.

[+] DantesKite|7 months ago|reply
Highly recommend MovieLens if you have eclectic or niche movie tastes. It can be a bit of a nerd-snipe though. One of my favorites activities is rating movies and watching the recommended ratings (what rating it thinks I'll give a movie) update overnight.

It's at the very least, better than average chance at predicting which movies you will like.

[+] jonathaneunice|7 months ago|reply
Movie reviews (stars) given on Amazon Prime's streaming service always seemed incredibly sus. All the bad movies from the long-tail back catalog all seemed remarkably highly-rated. Typical pattern: Forth-rate sci-fi action movie amalgamation of the 12 other well-known sci-fi action hits? Yes!! Four stars!

Just assumed Prime's claimed ratings were untrustworthy and must be based on some corporate "we'll promote your dreck, no problem-o!" data feed. But a bit sadder that Rotten Tomatoes, once at least reasonably legit and crowd-sourced, now poisoned by the same "everything's good! watch it! watch it!" inflation. Not surprising. Anti-surprising, really. Still a bit sad.

Thank you for the excellent historical, statistical deep dive.

[+] Joel_Mckay|7 months ago|reply
In my experience of film reviews, Rotten Tomatoes high positive scores are not always representative of how entertaining some content will be.. However, the negative skew is almost always accurate for how bad something will be for all viewers.

The bimodal distribution of professional critics versus community opinions obviously describes what is happening behind the data. Recycled AstroTurf for 1980's cult films have little appeal to modern viewers even with maximal pandering for nostalgia.

Good Hollywood writers likely starved to death, and were replaced with LLM interpreted Nielsen Media Research data. Most video games offer better writing now... lol =3