[stromgo]

YouTube is partly to blame for this. Since YouTube reported the average stars, if you saw a 4-star video and thought that it should be 3-star, then the optimal strategy was to vote 1-star to pull it down as fast as possible. If instead YouTube had reported the majority option (and publicized that fact) then it would have incentivized people to vote what they truly think.

[kartickv]

Your suggestion has its own problems. Imagine two videos with a majority rating of three stars. Say 60% of raters have rated it that. But one video has the remaining 40% one-star, and the other has the remaining 40% five-star. Your algorithm will ignore this data, showing a completely wrong result that both videos are equally good.