top | item 41490143

Project Hammer: reduce collusion in the Canadian grocery sector

361 points| surprisetalk | 1 year ago |jacobfilipp.com | reply

241 comments

order
[+] huitzitziltzin|1 year ago|reply
‘’’ Reach out to me (email “jacob” at this website) if:

You can do economic analysis of pricing data, and especially the interaction/correlation of multiple streams of prices across time ‘’’

The item in that quote is significantly, significantly harder than the author is giving it credit for. Canada has a competition policy agency. They are (almost surely) entitled to demand data from firms as part of an investigation. Their data will be better than the data here.

You can rarely if ever prove these cases on the basis of data analysis alone. (Indeed if you could the world’s antitrust agencies would just monitor such data! And anyone engaging in collusion would attack the monitoring adversarially - not that hard to predict!)

In grocery data you will be looking at thousands and thousands of prices of different goods from different suppliers with different costs who are exposed to different shocks due to variation in the costs of their inputs or god knows what else.

Not to be negative bc the idea is nice, but this is a complete waste of time.

~ an actual antitrust guy.

[+] surgicalcolor|1 year ago|reply
Conversely, I think you're overestimating the technical capailities of Canadian federal or provincial regulatory organizations.

Government departments notoriously have serious technical debt, so while they might entitled to demand this kind of data, I sincerely doubt they are doing it with anything resembling modern data analysis. These orgs aren't going to have working data lakes, spark clusters or data warehouses on cloud infrastructure. They maybe have legacy SQL databases being pulled into spreadsheets.

So, while their data access might be better theoretically, I doubt they truly are able to even remotely analyze the data effectively. Data science in govt is notoriously poor outside of Statscan or other tech heavy departments.

[+] jfil|1 year ago|reply
Hey man, I'm just creating the hammer - you can throw it in the lake, make a treehouse for your kid, smash a window, up to you :-D
[+] the_cat_kittles|1 year ago|reply
isnt the difference that when you have an open dataset, you get many many more eyeballs, and some very passionate ones. and they can make it their hobby, unconstrained by day to day needs of a job. eh?
[+] __MatrixMan__|1 year ago|reply
Is it necessary to prove it? I think it's sufficient to come up with an unbiased metric for which players are the most likely to be colluding at any given time. That would be a starting point for encouraging them to behave such that they're obviously not colluding: compete louder please, or you'll be at the top of this list.
[+] ActionHank|1 year ago|reply
They're still trying to resolve the bread price fixing from years before the pandemic.

The agency is likely underfunded, understaffed, and disincentivized from actually fixing the issues which are so pervasive across the country that it's just business as usual for most people..

[+] patcon|1 year ago|reply
> You can do economic analysis of pricing data, and especially the interaction/correlation of multiple streams of prices across time

Would this be the sort of analysis that measured transfer entropy or granger causality?

[+] transcriptase|1 year ago|reply
Yeah we have a competition bureau that’s known for never doing its actual job.

The fact that they keep approving mergers and acquisitions in telecom, banking, and groceries despite those being some of the most comically oligopoly-esque industries within a country on earth shows just how tragically wrong you are.

If there was 2 telecom companies left standing in Canada, our competition bureau would pretend to study their desire to merge for a year before approving it, under the condition that their collection of subsidiary brands pretend to be different companies while advertising.

[+] tazu|1 year ago|reply
> ~ an actual antitrust guy

So, unfirable? Essentially useless and beholden to little if any performance metrics?

[+] a13n|1 year ago|reply
Love this project. I just moved from Toronto, and compared to the US was frustrated by how every major industry in Canada was basically an oligopoly.

In Canada if you look at telecoms, banks, insurance, grocers, airlines, etc – there are a few major competitors and that's about it. It's very difficult to start a competitor, often for regulatory reasons, and most smaller competitors end up getting bought out by the big guys.

As a result, they have crazy shitty experiences. Telecoms are frustratingly expensive for cable and mobile services. Banks are dreadful and charge fees left and right, for basic things that are free in the US. Customer support with any of these companies is terrible.

I'm not surprised that they are colluding on pricing. It's quite obvious in the telecom market at least.

It seems tricky from the gov's perspective because this oligopoly/collusion behavior likely fuels higher GDP and more tax revenue... but ultimately more competition and consumer protection would make for a better country to live in.

[+] jlos|1 year ago|reply
Canada is intentionally setup to produce oligopolies as a defense against large American companies:

"Canada was, in a lot of ways, built on monopolies — think about the Hudson’s Bay Company or Canadian Pacific Rail. Canada has always feared that if we don’t let our homegrown companies get huge, we’ll get swamped by American competitors. That’s why there’s a tension between Canadian politicians, who often say they’re pro-competition, and the law, which incentivizes consolidation."

I think this strategy work well-enough until about 20 years ago. And by well enough I mean Canadian consumers weren't in an ideal situation, but things were good enough for most Canadians. Now the oligopolies have become basically predatory, gobbling up goverment funds and market capture wherever possible.

Case in point: our Temporary Foreign Worker program (who now make up 7% of the Canadian population) have not only strained housing, healthcare, and the job market it has even been called a "breeding ground for slavery" by the U.N. [1].

[0] https://www.wealthsimple.com/en-ca/magazine/canada-monopolie... [1] https://documents.un.org/doc/undoc/gen/g24/120/97/pdf/g24120...

[+] jfil|1 year ago|reply
There's a fascinating cultural history of accepting cartels/oligopoly in Canada. I've heard it described as an "Us Against the Vast North" attitude that prevailed as the British settled Canada.

Also, I believe Canada's merger laws are very different from the USA's. We (in Canada) didn't check whether a merger is better for customers, the standard was to check whether it was good for shareholders. I believe this is changing with a modification to merger laws in the last year or two. It's slow, but change is coming.

[+] FredPret|1 year ago|reply
From your description it sounds as if we're stuck in a local optimum. It's tough to break out of that, but there's a lot of technological change coming our way and that tends to disrupt the status quo.
[+] Fervicus|1 year ago|reply
> It seems tricky from the gov's perspective because this oligopoly/collusion behavior likely fuels higher GDP and more tax revenue... but ultimately more competition and consumer protection would make for a better country to live in.

Or because the government is working for these oligopolies and not for the people, regardless of the color of their party's logo.

[+] sofixa|1 year ago|reply
> In Canada if you look at telecoms, banks, insurance, grocers, airlines, etc – there are a few major competitors and that's about it. It's very difficult to start a competitor, often for regulatory reasons, and most smaller competitors end up getting bought out by the big guys.

> As a result, they have crazy shitty experiences. Telecoms are frustratingly expensive for cable and mobile services. Banks are dreadful and charge fees left and right, for basic things that are free in the US. Customer support with any of these companies is terrible.

This is funny to me from across the pond, because that's how I'd describe the US as well.

Small amount of telecoms and banks that charge exorbitant fees sounds like the US too.

High barrier to entry for physical infrastructure and banking are the reality all around the world, and for good reasons. There's more to it than that.

[+] rectang|1 year ago|reply
Interesting history! For the record, it's not like telecoms aren't an oligopoly in the USA, with the attendant "crazy shitty" customer service.
[+] ChumpGPT|1 year ago|reply
I'm originally from Canada and I think what you failed to realize is the Oligopoly is the Gov and it doesn't matter which one is in power. Canada is controlled by a few very rich families and they basically lay out how everything is going to work. The Gov puts on a good show but as you and I have seen, nothing changes. It is all theater for the masses and none of their promises come to fruition.

There is only one way to bring true competition to the Canadian market and that is to let American Companies compete instead of barring them from Canadian markets.

If you recall during NAFTA 2, Trudeau went to great efforts to allow Rogers, Bell and Telus to be re-classified as Canadian Media Companies to protect them from the American businesses. Canadians cheered higher prices and protectionism because fuck America and fuck Trump, but in the end cut off their nose to spite their face.

Canadians deserve their high prices and lack of competition and a market that is not consumer friendly, they made that loud and clear in their support of the Liberals during NAFTA2 Negotiations.

Glad I left a long time ago....

ExPat

[+] 3eb7988a1663|1 year ago|reply
Not sure if the author is here, but the downloadable SQLite database significantly benefits from applying compression (~75% with gzip).

Also, is there a write-up of how they collected the prices? I have wanted to do a similar analysis for years, but immediately gave up realizing I would be spending 95% of my efforts scraping and entity matching. By and large manufacturers seem to go out of their way offer unique SKUs intentionally to avoid comparisons.

[+] IncreasePosts|1 year ago|reply
I was going to mention that your browser almost certainly sends an Accept-Encoding: gzip header, but it appears the server doesn't care to sent a Content-Encoding: gzip back!
[+] danudey|1 year ago|reply
With 7z compression level 9, the .sqlite archive comes down to 61 MiB, which is about a 92% file size reduction.
[+] xp84|1 year ago|reply
If Canada is anything like the U.S. the snag I think you'll hit in establishing answers to questions like "what does this pound of butter cost at X chain" is that supermarkets now play so many games with pricing that there isn't one answer!

1. Prices may be regionally or locally varying based on, I assume, either inventory management needs or their idea of what people in one area will tolerate.

2. Pricing gimmicks. At my local supermarket, a standard bag of potato chips is $6.99 or so, but during some phases of the moon it becomes "$1.99 WHEN YOU BUY 4." Other products, a lot of packaged goods, go on sale in bundles: An item that may be $4.99 will be "$1.99 WHEN YOU BUY 5" participating products which may be chosen from certain unrelated stuff, like crackers, Tide, and Pillsbury biscuits. It's up to you to scrutinize the circular, the shelves don't always enumerate the eligible products. And finally, the best one, the "with digital coupon" prices. All you have to do is get on a sluggish app in terrible cell service, get properly signed in (oh, be sure to check your email because we needed to send you a 'security code') and locate and 'clip' the coupon and then you get a sale price. If you forget to do that you'll pay full price.

So in the above situations I'm concerned what kind of data integrity one would get because different consumers will pay different amounts. For instance, not everyone has room to transport 4 bags of chips home, so they may end up spending $7 for 1 instead of $8 for 4. Therefore the chips are simultaneously $2 and $7.

[+] juujian|1 year ago|reply
I'm sure there is a theory of how this would reduce prices - what is it? Real estate markets became infinitely more transparent over the last couple of years, to the extent that the data is used to train data scientist. The result? Speculators entered the market at it's messed up.
[+] 486sx33|1 year ago|reply
It’s pretty simple 1. Land area is huge in Canada, you can’t build a startup distribution network at scale and be efficient (ask target) 2. Interprovincial trade barriers 3. Everything has to have the French language printed on it - even if the closest supplier to Alberta is Montana, you can’t just import stuff without different inspection and packaging. This centralizes everything imported in Canada to things like the Toronto distribution center of which most foodstuffs only increase in cost the further across Canada from Toronto you get 4. Smaller grocery stores have tried, and the reasons above, plus the massive amounts of corporate welfare that companies like sobeys and loblaws get make the market unbalanced and unfriendly to competition.
[+] delichon|1 year ago|reply
Say I set my apple prices at 0.5 standard deviations below the average prices I can find in the vicinity. Is that an attempt to undercut the market or collude with it?

If I set the price to the average, is that collusion or just trying to maximize my profit on apples?

If I set it higher than average, is that collusion, or me telling the market that I think I have premium apples?

How do you get from correlation to collusion? This project seems to build on an assumption that it is collusive to set prices based on other market prices, or else how could it be a hammer to bash collusion? Is it required to be ignorant of market prices when you set a price so that regulators can assume that correlation is evidence of an agreement?

[+] seabass-labrax|1 year ago|reply
Does that matter except as a philosophical question? The point of a free market in practice is to ensure that sale values are correlated with, and reasonably close to, production costs.

If an ostensibly free market is not producing competition which causes that to happen, it's a problem for society; then, it is society's prerogative to use whatever means appropriate to fix it. That could include, depending on the ideology of the society, loans and grants for new participants in the market, legislation that regulates pricing, specific taxation policies, or even rejecting the free market entirely and moving to central control.

By analogy, a mysterious violent crime case might need to be solved for justice to be attained, but it doesn't need to be solved in order for a society to improve and make itself safer generally.

[+] bogwog|1 year ago|reply
> This project seems to build on an assumption that it is collusive to set prices based on other market prices, or else how could it be a hammer to bash collusion?

Did you read that on the website, or are you making that assumption?

This is just a dataset, and on its own isn't going to reveal something like collusion. But if you are researching collusion, you can use this data to help you understand behaviors of businesses.

Also, collusion doesn't necessarily have to involve prices.

[+] maronato|1 year ago|reply
> Collusion is a deceitful agreement or secret cooperation between two or more parties to limit open competition by deceiving, misleading, or defrauding others of their legal right. [1] If you decide, on your own, to sell a product at a lower price than your competitors, you are doing exactly what capitalism expects from a free market.

If you call the neighboring stores and say "hey, let's all sell apples for 2x our current price so consumers don't have a choice but to buy it for that price", then you are colluding.

Your point does stand though. From the outside it's very hard to distinguish between collusion and not by mere correlation. Prices may converge for various legitimate reasons, and without insider knowledge, reliably identifying collusion is often impossible.

The best-case scenario for the project might involve comprehensively modeling a single product category to demonstrate that only collusion could explain the pricing patterns. Success would then hinge on litigation uncovering direct evidence like incriminating communications.

[1] https://en.m.wikipedia.org/wiki/Collusion

[+] motohagiography|1 year ago|reply
canada's attitude toward competition is different than that of the US though. our agricultural products (dairy, wheat, maple syrup, and to a lesser extent retail beer and alcohols) are controlled by state monopolies who set prices.

the reason food is so expensive is because of fuel costs from the last few years, which were further not-helped by both an increase in federal taxes on fuel and the semi-official policy of weakening the CAD against the USD to support exports that weakens purchasing power. the effort appeals to economically illiterate constituents who support "price controls."

if you want to know where canada is on its de-kulakization spiral, the state is blaming "price gougers" for its policy failures, next are hoarders, speculators and probably "international bankers," on the list of clichés.

[+] adamomada|1 year ago|reply
There was a website a few years back that posted the prices of Brewers Retail (aka The Beer Store, single buyer of beer in Ontario) in a nicer, spreadsheet like format where you could sort by total price, price per mL, price per case size, etc etc (can’t recall the name of it at the moment). It was great. And transparent, and just data.

And were shut down by a threat of a lawsuit. Apparently you can’t do that in Canada. They had some stupid fine print that said the data was theirs and you can’t use it.

And I seem to recall similar fine print in grocery flyers.

[+] mistermann|1 year ago|reply
> And were shut down by a threat of a lawsuit.

As is the will of people, since Canada is a democracy, and that's what democracies do. Not like those terrible communist governments. Everyone should well know what they do (and how your friendly local neighborhood democracy is so much better, you see), you can read about it in your newspaper quite literally every single day for the last several years/decades.

Threads like these are diamonds in the rough, I love collecting them.

[+] magicalhippo|1 year ago|reply
Top three grocery companies in Norway, which have almost the entire grocery market here between them, just got fined[1] for price collusion.

They'd signal rising prices for a category by rising the price on certain products within that category, was one accusation. Extensive use of price scouts aided in this.

Initially the fine was much larger, but ended up at about 450M USD total between the three.

For comparison, the larger company had roughly the same amount as profit before taxes[2] in 2023.

The recent heavy inflation in grocery prices has been far greater than what say the farmers got for the raw products.

That said, here in Norway we have a ridiculous amount of smaller, local grocery stores, rather than fewer larger ones here and there.

As I sit here, in the outskirts of Oslo, within a 15 minute walk I have 8 grocery stores, all from the top three.

[1]: https://www.nrk.no/norge/daglegvare-etterforskinga_-4_9-mill...

[2]: https://www.dn.no/handel/resultathopp-for-norgesgruppen-tjen...

[+] willvarfar|1 year ago|reply
Here is an app idea:

Users use a smartphone app to zap the price labels on shelves in shops.

The app can provide basic price history and comparison prices etc.

Furthermore, users could organise into little community groups. The app is tracking what is a good price and what is a good shelf life etc and what is in demand and what is on specific members shopping lists etc and tells the zapper how many to buy for a common stockroom.

Of course the supermarkets will not like people wandering about their shop with an app, but they have a pretty ugly PR problem telling people they aren't allowed to use the app in their store etc?

[+] sys32768|1 year ago|reply
I live 70 miles from the Canada border where we whine about Canadians crowding our Costco.

I roughly counted their plates in one section of the parking lot and about every fifth plate was from Alberta.

Edit: I have also witnessed Canadians in awe at how inexpensive was the stack of chewing tobacco they purchased.

[+] islewis|1 year ago|reply
Big fan of any data aggregation projects like this, especially with such a relatable theme.

However it feel like the conclusion might be jumping the gun a bit. Instead of "Think there is collusion" -> finding the data top support the claim, maybe run the numbers first and see what they say? I think coming up with a strong position (Canadian stores are colluding) before looking at the data makes it enticing to find numbers that back up the claim, whether or not they are taken out of context.

[+] smeej|1 year ago|reply
Doesn't making this data available also provide it to the grocers? Is it still "collusion" if they all just start raising their prices to public benchmarks on any items where they're low? They're not "working with each other" now. They're just "analyzing the publicly available data to remain competitive."
[+] Lalabadie|1 year ago|reply
Canadian grocers already have sophisticated price watch tools all pointed at each other and augmented by human labor.

This data set is from scraping online grocery store prices over a period of time, it's not live. It wouldn't improve grocer's price data harvesting, but it does slightly reduce the gap between what they have in hand, and what the public and press can have in hand.

[+] tomComb|1 year ago|reply
Oh man, we really need this for telecom too. And even more important than their collusion is their political corruption, so the government protects them from competition and funnels endless billions in taxpayer money to them.
[+] arcticbull|1 year ago|reply
You know what actually works great for reducing the price of cell plans in Canada is a public option.

There's one province where people pay roughly half the national average for all there telecom needs - and that's Saskatchewan. SaskTel's mere existence keeps all the majors in line.

Manitobans should be absolutely up in arms over the privatization of Manitoba Telecom.

Canada has a ton of land and just not that many people. There's 3 + Videotron major telecom operators servicing a population the size of California spread out over a land area the size of the United States. The Capex and payback period for spinning up a new, competitive, national carrier is wild (there's a pile of bodies there proving my point) which makes this a perfect job for the government.

[+] nazcan|1 year ago|reply
Prices have come down a lot. I have reasonable plans from black Friday at like $30ish/month.
[+] iambateman|1 year ago|reply
A lot of people are mentioning how hard collusion is to prove.

Perhaps the government should offer a $500,000 incentive to anyone offering information which demonstrates collusion of a sufficient size.

If consumers are getting systematically ripped off at a scale of many billions of dollars, it seems like we should try to just buy that information rather than pay a few dozen Ph.D’s to build models which demonstrate collusion.

Knowing that employees are one email away from the biggest payday of their lives would have a deterrent effect on executive behavior, too.

[+] openrisk|1 year ago|reply
You will need a Project Mjölnir for that, as the real ideology, moral stance and practice of our era is that competition is for losers. Or: Those who can't collude, compete.
[+] constantlm|1 year ago|reply
This could be useful for New Zealand too, where things are even worse and a duopoly effectively controls the whole market. Might put this on my ever-growing list of projects I want to do.
[+] kwar13|1 year ago|reply
Never forget that these !@#$* price fixed bread for years, and got away by offering a $25 gift card. No one went to jail.

https://en.wikipedia.org/wiki/Bread_price-fixing_in_Canada

And that's just what could be proven in court.

[+] jfil|1 year ago|reply
Last year, Canada Bread paid $50mil for its role in price fixing:

https://www.cbc.ca/news/business/canada-bread-price-fixing-1...

The government hasn't taken anyone to court yet, but justice and progress is possible. We will get there by continuing to talk about this and asking fellow Canadians to reflect on the state of our society.

[+] 1970-01-01|1 year ago|reply
I read this title as "reduce collision in a Canadian grocery store" and thought this was an app for parking.
[+] winddude|1 year ago|reply
or shopping carts in the store, need to cut down on the number of sorries.