The profit motive is being blamed here for optimizing for engagement.
But how do you assess progress when you remove all desire to monetize?
I run r/WallStreetBets and "quality" is an extremely nebulous term.
I look for things like "novelty", "thought-provoking / well reasoned commentary", "original content", "authenticity", "self-awareness" or "primary research". But these are human assessed metrics.
Some more easily measurable metrics might include, "length of submissions", or "number of outbound links excluding blacklisted domains". Or even "number of tickers or quality-correlated keywords mentioned".
All these metrics have very clear downsides, and if generally well-known, become useless. Interestingly, a score too high can also result in something being unlikely to be authentic.
Another challenge is your relationship with users. Surprisingly, moderators are not innately adversarial to users, they can also promote content through other channels (discord, twitter) or sticky threads for a viewership boost.
> But how do you assess progress when you remove all desire to monetize?
If you ignore the profit motive, presumably you are doing it for other reasons. You assess "progress" based on those motives. Or perhaps you don't assess "progress" at all, beyond "I like what I'm doing here and where this is going".
I've run numerous internet services over the years without the goal of them generating an income at all. I've run MUDs, websites, discussion groups, etc. I didn't objectively measure anything about any of them, because my goals were not ones that could be objectively measured. And honestly, even if there were some metric that could be used, I would have avoided using it because then it becomes about maximizing the metric rather than the purpose I started the activity in the first place. If I was happy with how they were going, that counted as "success" to me.
> So, even without a profit motive, what do you do?
I honestly don't understand this question. Without a profit motive, you do it for other motives. If you have no other motives, then why are you doing it at all?
Historically, the immortals on MUDs used to optimize for thing like "fun" and "clout" - how much do you enjoy interacting with the players and being seen as the person in charge of their game? How much do you enjoy telling people you volunteer your time to work on it?
If WSB members were doing bad stuff like, making mean jokes about the leadership of companies Ryan Cohen divests from in; or coordinating real-life harassment of family members of employees of the Depository Trust Company or something; you'd probably want to put a stop to that and focus them somewhere else. Because it wouldn't be fun.
If Fox News interviewed you and you sounded silly and all the community members got mad at you, you'd probably also quit the volunteer job.
But as long as you make choices that make the community a fun and engaging place to hang out and feel like you're part of a big secret treasure hunt, that seems awesome, and you would probably want to make choices that maximize that.
TBH most MUDs were not data-driven and kinda govern by feeling -- they'd sort of add steering they thought was interesting and keep it in place unless there's a backlash. The tedious administrative stuff (bans, moderation, player requests for item reimbursement) would always have a backlog and you'd recruit junior imms to help out and they would feel like they were part of the fun too.
OK, and? I could guess what the problem with this is, but let's spell it out... maybe that they are subjective so different assessors can disagree... and this is a problem why?
What if there are some cases where it's actually important to use human-assessed metrics as one component, or even as the entire thing, cases where no appropriate 'objective' metrics are available? (In scare quotes, because these quantitative metrics are seldom _quite_ as 'objective' or independent from human judgement as assumed. There is usually human judgement involved in how the metrics are defined and measured, where different people might define and measure a metric different ways resulting in different numbers...)
Measuring community engagement makes sense when your primary mission / core goal is to build a strong community. When I was building a large online third-party Discord community for a university during COVID, I was very glad that Discord exposed a lot of metrics for admins/mods to track. It's obviously important to not "game" the metrics by falling into short-term gain traps like over-pinging users, or encouraging high-volume but low-quality participation, etc. But as long as you are laser-focused on "providing real value" to members, then long-term or multi-cyclical trends of community engagement is a useful benchmark.
But it doesn't make sense to focus on these metrics if your mission is to create a great product -- focusing on community engagement should be just one small aspect of your overall marketing strategy, which should be a relatively constrained part of your overall budget (both in labor and money). And even within marketing, there are many other important metrics besides community engagement.
Soylent had great community engagement but not enough focus on scientifically-guided product engineering. Tons of businesses that get this wrong.
Generally I interpret overcommitment to community engagement as a signal that the founders are overly narcissistic and usually apply a discount the expected future value/evolution of their product.
1. If the community that you have does a "foom" and grows like crazy BEFORE it has a set of values, then you're at the whims of the mob and where certain vocal and rallying members decide they want to take the community. Usually this ends in internecine conflict and the inevitable "break offs" into other smaller groups
2. If your community starts with a value vector then you structure, nudge and contain the community to a MDP (Markov Decision Process) with the value vector as the reward, for which the moderators are the critics and the community members are the actors in the actor-critic model.
It seems like the vast majority of organizations/movements etc... fall into category 1
For me the learning here is, if you are a leader in your community it's never too early to nail down what the core value and benefit of your community is and stay obsessed with just that.
Hard to scale, but farming out content samples to a panel of your target market for them to rate on the attributes of "novelty", "thought-provoking / well reasoned commentary", "original content", "authenticity", "self-awareness" or "primary research" might be an approach
Mark Cuban recently launched a community at https://biztoc.com as sort of a WSB for business people. I wonder if that will/could work without an umbrella topic.
Good article, though I think it missed explicitly making the point that the reason "engagement" is the metric is because that's what platforms monetize. Engagement is a euphemism for attention, and social platforms exist to sell your to advertisers.
Since the article is written for "community builders," understanding that engagement is a metric to quantify the advertising potential of a platform should make it obvious that people who aren't in the business of selling attention to advertisers don't need to copy social media and optimize for engagement; instead they should target better measurements of value creation for their community. Of course Goodhart's law[1] still applies.
Exactly. It took me a while to understand that this article was written for community managers of brands, not people developing social networks. They keep saying engagement has no value, but to social networks and to "influencers" who both make their money off of ads, that is exactly what has value.
> the reason "engagement" is the metric is because that's what platforms monetize
...and the implication of that is that if we change the monetization model, then the metrics also change.
For a company selling a paid product, metrics become things like the effectiveness of the sales funnel, how much support the average user requires, the rating of the product on app stores, etc - which are not ideal, but still far better than "engagement".
>> Good article, though I think it missed explicitly making the point that the reason "engagement" is the metric is because that's what platforms monetize.
In the case of "community" around a product, the purpose of the community is usually to provide free support. That's not so much monetization as it is cost savings. For that to work, people need to be "engaged" or kept around so they are present to field those support issues.
Engagement is just a more sophisticated version of 'eyeballs'.
For community building you want successful events, you want interesting questions to be asked, and interesting answers to be offered. Someone could write a book on how to make communities work. Probably several books.
You are completely right on Goodhart's Law. I know this from experience.
I put the customer first in the belief that 'the rain follows the plough' and, if you put the customer first, then all else follows.
I see metrics as there to be cracked. If you are measuring the right things then you can see where the problems are, fix the problems and move on to measuring different metrics that are more specific and focused on the remaining problem areas. For me it is a campaign where initial reactive firefighting gives way to calm, proactive problem solving, fixing problems before they are of consequence. Along the way many unexpected things are learned about the customer.
I do all this with a focus on the customer, meanwhile my colleagues in marketing are out for themselves. They are the dead weight keeping progress back as they have to do things like buy traffic and hold on to keyword optimised URLs that are cargo cult SEO. Their reports for their micromanagers are always measuring the same stuff, e.g. 'engagement', yet they could be engaging customers solely for the purpose of pissing them off for them to never come back.
My favourite is the pop up shown as someone leaves a website begging them to stay, or the sign up to the newsletter popup. These could get one extra sale at the cost of pissing off a million people but if you only measure the former then it all seems good.
In the capitalist world only one metric matters to the shareholders. Once I worked for a highly successful company that sent out sales newsletters with no measurement whatsoever of open rates or click throughs. None of that was needed. The problem was emptying the warehouse too quickly to have no stock left. Customers would be waiting for our email, not ignoring them. We had plenty of engagement with the customers as they actually bought stuff.
Subsequently I have worked for agencies where they spend a fortune on some email service and they get these fancy graphs to show the clients of these open rates the world over. Which had wow value at the time and 'engagement' stats. But I had come from somewhere where we had no need to measure that stuff, we weren't looking at these charts, we were coping with a deluge of orders and had no time for that.
I enjoyed the Monsters Inc reference and the turning of it on its head central to this piece. But we've said/argued this for years and at the end of the day engagement = ad reach = profits and until the economics there change any for-profit social media will gravitate towards engagement. Hell ANY media gravitates toward engagement and outrage bait. An actual community built around Mastodon seems possible, and there's community discussions around private group chats but they will always remain niche and not profitable.
I don't get what https://savannahhq.com is and what the value add is there but that seems to be what he's interested in, niche + small communities.
I think the problem is that while engagement may be the source of profits that is short-term thinking. If joy based engagement drives profits and keeps users on your site long-term it may be worth it to take the short-term revenue boost that anger-based engagement brings you if it slowly drives people away from your site.
Of course it isn't completely clear if anger-based engagement actually drives most people away. In that case it is actually Facebook's fiduciary duty to capitalize on it which seems to be a failing on our society.
You don't advertise to your community. If you do you're actively killing it.
Community is made up of people making connections and building relationships with other people. Savannah CRM helps you manage those connections and relationships.
Well said! Outrage sells and it is easy to keep it stoked. Everyone is addicted to free. If we get rid of ads, how does (even the barest bones) Twitter keep the lights on?
I wonder if there is a cross-over between Patreon, Substack and Twitter. This would probably work for niches but not writ large. But at the end of the day, we need to find a way to pay for what we value because that is how we get more of it. Interested in new ideas in this space....
The article dances around a topic that is very relevant across technology companies- toxic metrics.
I worked for many years at a company that prides itself on being metrics driven. To the point that they celebrated having many significant figures after the decimal point when measuring length of negative events in seconds.
The good side of metrics driven culture is that having a metric gives you a concrete goal.
The downside is that really good metrics are often very hard. By really good, I mean metrics that measure the desired outcome.
It’s much easier to measure, for example, how many operation X your service performed, rather than the value the service delivered to your business or your customers. So we assume that each operation X contributes a uniform, positive quantum to those outcomes and we count X’s.
The software industry has made me very aware that being metric-driven can be a terrible thing as easily as a good thing. Software has been getting substantially worse over time (in my opinion) in large part because of the use of metrics.
I don't think they dance around the topic so much as outright state that it is bad. They mostly talk about engagement because it is the primary metric but even state that we should not value what we measure but measure what we value. Which it is harder to do that, but hey, you got billions of dollars to work with, it seems like the resources are there.
Nicely written post but can't help noticing the elephant in the room being tiptoed around, which really makes me question the sincerity of the author.
The monsters need energy to power their civilisation. Why do social media companies measure engagement? What are they extracting? There is not one single mention of the "p word" in the article.
The last two sections are delivered with such heavy doses of naivety that this ends up just coming across as Orwellian in tone.
> “We should measure what we value, not value what we measure”
Who is "we"? Is "we" the shareholders? What do "they" value do you think?
Early on in a community, there often aren't enough data points to cover other possible metrics. With that in mind, starting somewhere is better than not starting at all — measuring engagement is a useful way to begin understanding what's resonating in your community. As you continue building your community, there are many different angles that you should use to evaluate its health.
Measuring engagement then becomes one piece of data that's valuable, but shouldn't be the only piece. When engagement data is combined with qualitative community surveys, enthusiasm from members to contribute to a community, clear and timely responsiveness to community needs, membership growth over time and geographies, depth of member interactions (whether across community channels or within specific channels), what's topically important to members and why, and overall sentiment and change in sentiment over time - that's when community builders can begin to better understand the health of their communities and their impact on their business.
Engagement is an important piece, but just one of the pieces, that helps paint the full picture.
Disclaimer: I’m a co-founder of Common Room [1] and a we’ve invested a lot of energy in solving for this exact problem. You’re welcome to check out the product (it’s free to sign up) and would love to hear of ideas or feedback.
Hot take: stop building communities around everything.
People are highly tribalist and banding together like that produces a lot of ugly outcomes. Communities form and likely won't go away, particularly the more people internalize them emotionally and integrate them into their identity.
It is exhausting to see a community for literally everything, even the smallest products.
You're raising an interesting point, but you're coming close to saying that communities are bad, and in my experience they don't have to be. Even ones based around a product.
Near where I used to live in the Valley, there was a real-world club for enthusiasts of a particular weird antique car. Not toxic, just eccentric. They just liked getting together and drinking in a nearby pub and showing off how they'd restored this or that aspect of the car.
I think our goals should be to replicate that kind of success.
I think the other posters in this thread have it right, that the reason why we usually can't is because our metrics are all about selling audiences to advertisers.
This sounds great, but I see a challenge. The author recommends measuring e.g. user happiness instead of engagement. How to measure user happiness? IMO, some type of survey could be used. But the “best” metrics are based on observable behavior. Not what users self-report, because they may not know how they actually feel about the experience. In general it’s better to see what users do rather than what they say.
You must have an expert or experts. That expert will then make decisions, and use metrics and surveys and intuition to drive their new world view, but it's the expert that is always at the top of the stack.
If you then take that model and try to use them in a new domain, that is never going to work out. The environment adapts to the incentives that the current system produces, and will instantly make your pre-created model worthless[0]. The only way to work out long-term is by having an expert stay ahead of the pack creating what they think are systems with better signal to the noise.
Observables are better, but self reported metrics do still have value. It's an aggregate version of the problem devs encounter when doing usability testing. Users will tell you what their experience is, but it's up to you as a software designer to distill that feedback into actual needs. Which may not at all be what they actually ask for. Survey responses are similar in that they don't tell you exactly what to do, but they do contain actionable information if you have the skill to interpret it.
Unfortunately what is really needed here is a suggestion of something else to measure that is better. I fight this a lot with bad metrics (code test coverage), nobody wants to give them up unless I can suggest a replacement to measure.
I believe this falls into the 'Results Oriented' mindset of delivering value. This means figuring out your desired outcome ahead of time, then measure it over time. In addition, overtime, you may determine that your desired outcome has changed and you need to measure differently. Yes, iterating is not just for software.
IMHO pure quantitative metrics are of little value for customer experience. I believe metrics should be a little squishy and subjective. The value in the subjectivity is that it spurns conversation and actual thought instead of simple counting.
That's a great post, and I plan to share it with others.
Sadly, it's pretty difficult to measure for some of the "values" that my communities run on, but "engagement" isn't even on the map.
I suspect that "engagement" does, indeed, provide real monetary value for advertising-based sites, as clicks == money. Since that isn't a factor in my communities, we don't worry about that. In fact, we try to minimize "engagement."
I don't get it. What is a "community manager"? Is it someone that runs a company's social media page? And do they think that people care about their brand so much that they would join a community of people whose only common trait is liking the same brand? And in the first place, how do you even build a "community" manually and artificially? It's almost a bastardization of the word, as if a community is not a set of people with similar goals but instead yet another way to convert people's time and desire for social interaction into profit.
The article implies that community building should be some kind of altruistic purpose, where your only goal is to maximize the amount of "meaningful relationships" created? But building an artificial community in the first place can never be altruistic because the end goal of it all is to guide people to your product or conference or whatever. If you were an actual altruistic community builder, you would be telling people to get off the Internet and go make "meaningful relationships" with people in real life.
> I don't get it. What is a "community manager"? Is it someone that runs a company's social media page?
No, that's a social media manager, completely different role.
> And do they think that people care about their brand so much that they would join a community of people whose only common trait is liking the same brand?
Yes! It happens all the time. Most of the biggest brands have a community of people who share that common trait.
> And in the first place, how do you even build a "community" manually and artificially?
Again yes, just like you build a garden. The gardener doesn't make the plants grow, but they do make sure they have the right environment and resources to grow in.
> The article implies that community building should be some kind of altruistic purpose, where your only goal is to maximize the amount of "meaningful relationships" created?
That's not altruism. Those meaningful relationships help the company/project the community was built around. They provide feedback, help improve processes, make connections outside the company/project, provide support to other users, there are too many things that communities do to list them all here.
This article completely undermines itself by arguing in the conclusion ("Measure what you value") that we can and should measure "meaningful relationships" instead.
> They’re not as easy to measure as engagement, sure, but they can be measured.
I'm afraid that a citation is sorely needed here. There are no real measurements for "meaningful relationships."
In fact, the only proxy metric we have for "meaningful relationships" is engagement!
And, yes, engagement is not a very good proxy metric for meaningful relationships, but since there's no alternative, it's all we've got, so this article is pointless.
Here's the little discussion I have in my head when this issue comes up.
"Engagement isn't a good metric. Optimize for having a good community."
"But engagement is easy to measure AND it means people are using the product. If people are choosing to use the product, doesn't that mean the product is giving them value?"
"Not necessarily. Drugs like tobacco and heroin have fantastic 'engagement' but we consider them bad for us."
"But how do you tell the difference between something that engages because it's good and something that engages because it's some kind of drug?"
"I guess one important signal would be when people know something is bad for them but they're doing it anyway?"
"Is that the case for social media? Do people feel addicted - feel that it's bad for them but they can't stop?"
I think we've heard a lot of anecdata on this last question, but I can't recall seeing any big survey on it.
I think people enjoy this argument because it's counter intuitive, but I also think it's counter intuitive because it's wrong. If you build a playground and everyone avoids the teeter-totter, there's probably something wrong with the teeter totter. Oddly the monsters inc example seems counter to his point since it seems like they were measuring something to specific and the proper solution would be to something more general (like engagement) instead. It seems to me like trying to start with something other than engagement would be premature optimization.
Of course engagement is insufficient and of course if you want to know something else, measure that other thing, and of course you should interrogate your metrics, and maybe it's just this thing that bugs me about modern writing where you have to take this ridiculous extreme stance to get eyeballs but yeah.
So please tell me, what should I optimize for when moderating a local OpenStreetMap community. I personally feel it's crucial people internalize that's a project with a bottom-up organization. A "civil society", if you will. Alas, despite 200-250 daily active mappers in Poland [1] only a fraction ever posts on the forum, even if just to ask something. Meeting IRL is also futile except 2-3 largest cities. We have a forum, Facebook group and Discord.
Whatever I'd come up with, I'd like it to be a self-healing organism that would continue to thrive even if some important members become inactive. And to feel it's not a Sisyphean task.
I'm confused. I see OSM as a huge success. Firms like Mapbox and Strava have an incentive to update that wiki with you. The maps are amazing. Any local sub-community will be challenging to organize, but maybe the metric should be making it fun for you, and if you're having a good time others will join.
It's still very early for me, but I see myself in this post. It's going to be extremely slow going, as the main consumers of the content created have no desire to contribute, but since golf courses are mostly static, as long as i plug away, adding a little bit all the time, the site will be a success. It costs almost nothing to operate. Generating traffic is challenging, and i've been trying to find more people willing to contribute, but again, I see my product as best placed if i do exactly the opposite of what most of the golf course info sites are doing. Don't base my success on engagement, no aggregation, no overly-prodding to contribute. Just one day at a time, with the assumption that the task is endless anyway, having people share their passion on their own time.
If i can make that profitable at all, and it shouldn't be too difficult as the site grows, i'll see it as a total win.
>Now imagine if this played out the other way around. Imagine that the monsters originally focused on maximizing energy by making children laugh as much as possible. Then one day, in a horrible twist, someone discovers that a child’s scream was a much easier and more powerful way of meeting their goals. Their company, which built a platform that gave them instant access to children around the world, suddenly realizes that they could reach their goals by spreading fear rather than joy.
Then they would instantly start using screams? Nobody making business decisions in social media companies will prioritize "user happiness" over increased production of their main product (your attention) for their customers (advertisers).
Some will and some won't. In the absence of rules that penalize that type of behavior, usually the ones that will end up winning. It has nothing to do with morals, those just seem to be the rules of the game as we currently play it.
I like the premise of the post. I think it paints a rather overly idealistic view of social networks, who exist to make money first, not to create value (if they could make money without creating value, they certainly would do that).
They chose to pay higher prices for shares, divorced from actual revenue
The companies, in-turn, found non revenue metrics to show quarter over quarter
and thousands of startups copied the model
Just don’t rely on the A/B test, be aware of it, but don’t rely on it. Put the human control back into the process. Users aren't staying on your platform 5 seconds longer after you hid the escape hatch because they love it, they are frustrated and lost! The A/B test doesn't tell you why the engagement is longer, only that it is. It takes an empathic human to say “okay, B got us that result but not for the reason we want”
[+] [-] wsb_mod2|3 years ago|reply
But how do you assess progress when you remove all desire to monetize?
I run r/WallStreetBets and "quality" is an extremely nebulous term.
I look for things like "novelty", "thought-provoking / well reasoned commentary", "original content", "authenticity", "self-awareness" or "primary research". But these are human assessed metrics.
Some more easily measurable metrics might include, "length of submissions", or "number of outbound links excluding blacklisted domains". Or even "number of tickers or quality-correlated keywords mentioned".
All these metrics have very clear downsides, and if generally well-known, become useless. Interestingly, a score too high can also result in something being unlikely to be authentic.
Another challenge is your relationship with users. Surprisingly, moderators are not innately adversarial to users, they can also promote content through other channels (discord, twitter) or sticky threads for a viewership boost.
So, even without a profit motive, what do you do?
[+] [-] JohnFen|3 years ago|reply
If you ignore the profit motive, presumably you are doing it for other reasons. You assess "progress" based on those motives. Or perhaps you don't assess "progress" at all, beyond "I like what I'm doing here and where this is going".
I've run numerous internet services over the years without the goal of them generating an income at all. I've run MUDs, websites, discussion groups, etc. I didn't objectively measure anything about any of them, because my goals were not ones that could be objectively measured. And honestly, even if there were some metric that could be used, I would have avoided using it because then it becomes about maximizing the metric rather than the purpose I started the activity in the first place. If I was happy with how they were going, that counted as "success" to me.
> So, even without a profit motive, what do you do?
I honestly don't understand this question. Without a profit motive, you do it for other motives. If you have no other motives, then why are you doing it at all?
[+] [-] empathy_m|3 years ago|reply
If WSB members were doing bad stuff like, making mean jokes about the leadership of companies Ryan Cohen divests from in; or coordinating real-life harassment of family members of employees of the Depository Trust Company or something; you'd probably want to put a stop to that and focus them somewhere else. Because it wouldn't be fun.
If Fox News interviewed you and you sounded silly and all the community members got mad at you, you'd probably also quit the volunteer job.
But as long as you make choices that make the community a fun and engaging place to hang out and feel like you're part of a big secret treasure hunt, that seems awesome, and you would probably want to make choices that maximize that.
TBH most MUDs were not data-driven and kinda govern by feeling -- they'd sort of add steering they thought was interesting and keep it in place unless there's a backlash. The tedious administrative stuff (bans, moderation, player requests for item reimbursement) would always have a backlog and you'd recruit junior imms to help out and they would feel like they were part of the fun too.
[+] [-] jrochkind1|3 years ago|reply
OK, and? I could guess what the problem with this is, but let's spell it out... maybe that they are subjective so different assessors can disagree... and this is a problem why?
What if there are some cases where it's actually important to use human-assessed metrics as one component, or even as the entire thing, cases where no appropriate 'objective' metrics are available? (In scare quotes, because these quantitative metrics are seldom _quite_ as 'objective' or independent from human judgement as assumed. There is usually human judgement involved in how the metrics are defined and measured, where different people might define and measure a metric different ways resulting in different numbers...)
[+] [-] runnerup|3 years ago|reply
But it doesn't make sense to focus on these metrics if your mission is to create a great product -- focusing on community engagement should be just one small aspect of your overall marketing strategy, which should be a relatively constrained part of your overall budget (both in labor and money). And even within marketing, there are many other important metrics besides community engagement.
Soylent had great community engagement but not enough focus on scientifically-guided product engineering. Tons of businesses that get this wrong.
Generally I interpret overcommitment to community engagement as a signal that the founders are overly narcissistic and usually apply a discount the expected future value/evolution of their product.
[+] [-] AndrewKemendo|3 years ago|reply
I think there are two options here:
1. If the community that you have does a "foom" and grows like crazy BEFORE it has a set of values, then you're at the whims of the mob and where certain vocal and rallying members decide they want to take the community. Usually this ends in internecine conflict and the inevitable "break offs" into other smaller groups
2. If your community starts with a value vector then you structure, nudge and contain the community to a MDP (Markov Decision Process) with the value vector as the reward, for which the moderators are the critics and the community members are the actors in the actor-critic model.
It seems like the vast majority of organizations/movements etc... fall into category 1
For me the learning here is, if you are a leader in your community it's never too early to nail down what the core value and benefit of your community is and stay obsessed with just that.
[+] [-] hammock|3 years ago|reply
[+] [-] wolfofcrypto|3 years ago|reply
[+] [-] burlesona|3 years ago|reply
Since the article is written for "community builders," understanding that engagement is a metric to quantify the advertising potential of a platform should make it obvious that people who aren't in the business of selling attention to advertisers don't need to copy social media and optimize for engagement; instead they should target better measurements of value creation for their community. Of course Goodhart's law[1] still applies.
1: https://en.wikipedia.org/wiki/Goodhart%27s_law
(edited for readability)
[+] [-] cyborgx7|3 years ago|reply
[+] [-] throw10920|3 years ago|reply
...and the implication of that is that if we change the monetization model, then the metrics also change.
For a company selling a paid product, metrics become things like the effectiveness of the sales funnel, how much support the average user requires, the rating of the product on app stores, etc - which are not ideal, but still far better than "engagement".
[+] [-] phkahler|3 years ago|reply
In the case of "community" around a product, the purpose of the community is usually to provide free support. That's not so much monetization as it is cost savings. For that to work, people need to be "engaged" or kept around so they are present to field those support issues.
[+] [-] hinkley|3 years ago|reply
For community building you want successful events, you want interesting questions to be asked, and interesting answers to be offered. Someone could write a book on how to make communities work. Probably several books.
[+] [-] Theodores|3 years ago|reply
I put the customer first in the belief that 'the rain follows the plough' and, if you put the customer first, then all else follows.
I see metrics as there to be cracked. If you are measuring the right things then you can see where the problems are, fix the problems and move on to measuring different metrics that are more specific and focused on the remaining problem areas. For me it is a campaign where initial reactive firefighting gives way to calm, proactive problem solving, fixing problems before they are of consequence. Along the way many unexpected things are learned about the customer.
I do all this with a focus on the customer, meanwhile my colleagues in marketing are out for themselves. They are the dead weight keeping progress back as they have to do things like buy traffic and hold on to keyword optimised URLs that are cargo cult SEO. Their reports for their micromanagers are always measuring the same stuff, e.g. 'engagement', yet they could be engaging customers solely for the purpose of pissing them off for them to never come back.
My favourite is the pop up shown as someone leaves a website begging them to stay, or the sign up to the newsletter popup. These could get one extra sale at the cost of pissing off a million people but if you only measure the former then it all seems good.
In the capitalist world only one metric matters to the shareholders. Once I worked for a highly successful company that sent out sales newsletters with no measurement whatsoever of open rates or click throughs. None of that was needed. The problem was emptying the warehouse too quickly to have no stock left. Customers would be waiting for our email, not ignoring them. We had plenty of engagement with the customers as they actually bought stuff.
Subsequently I have worked for agencies where they spend a fortune on some email service and they get these fancy graphs to show the clients of these open rates the world over. Which had wow value at the time and 'engagement' stats. But I had come from somewhere where we had no need to measure that stuff, we weren't looking at these charts, we were coping with a deluge of orders and had no time for that.
[+] [-] fullshark|3 years ago|reply
I don't get what https://savannahhq.com is and what the value add is there but that seems to be what he's interested in, niche + small communities.
[+] [-] kevincox|3 years ago|reply
Of course it isn't completely clear if anger-based engagement actually drives most people away. In that case it is actually Facebook's fiduciary duty to capitalize on it which seems to be a failing on our society.
[+] [-] mhall119|3 years ago|reply
You don't advertise to your community. If you do you're actively killing it.
Community is made up of people making connections and building relationships with other people. Savannah CRM helps you manage those connections and relationships.
[+] [-] snarf21|3 years ago|reply
I wonder if there is a cross-over between Patreon, Substack and Twitter. This would probably work for niches but not writ large. But at the end of the day, we need to find a way to pay for what we value because that is how we get more of it. Interested in new ideas in this space....
[+] [-] puchatek|3 years ago|reply
> We still want our members to be happy, right?
... we might want that, but do the people running the platforms?
[+] [-] efitz|3 years ago|reply
I worked for many years at a company that prides itself on being metrics driven. To the point that they celebrated having many significant figures after the decimal point when measuring length of negative events in seconds.
The good side of metrics driven culture is that having a metric gives you a concrete goal.
The downside is that really good metrics are often very hard. By really good, I mean metrics that measure the desired outcome.
It’s much easier to measure, for example, how many operation X your service performed, rather than the value the service delivered to your business or your customers. So we assume that each operation X contributes a uniform, positive quantum to those outcomes and we count X’s.
[+] [-] JohnFen|3 years ago|reply
[+] [-] godelski|3 years ago|reply
[+] [-] hinkley|3 years ago|reply
[+] [-] lucideer|3 years ago|reply
The monsters need energy to power their civilisation. Why do social media companies measure engagement? What are they extracting? There is not one single mention of the "p word" in the article.
The last two sections are delivered with such heavy doses of naivety that this ends up just coming across as Orwellian in tone.
> “We should measure what we value, not value what we measure”
Who is "we"? Is "we" the shareholders? What do "they" value do you think?
[+] [-] unknown|3 years ago|reply
[deleted]
[+] [-] slackerIII|3 years ago|reply
Measuring engagement then becomes one piece of data that's valuable, but shouldn't be the only piece. When engagement data is combined with qualitative community surveys, enthusiasm from members to contribute to a community, clear and timely responsiveness to community needs, membership growth over time and geographies, depth of member interactions (whether across community channels or within specific channels), what's topically important to members and why, and overall sentiment and change in sentiment over time - that's when community builders can begin to better understand the health of their communities and their impact on their business.
Engagement is an important piece, but just one of the pieces, that helps paint the full picture.
Disclaimer: I’m a co-founder of Common Room [1] and a we’ve invested a lot of energy in solving for this exact problem. You’re welcome to check out the product (it’s free to sign up) and would love to hear of ideas or feedback.
1: https://www.commonroom.io/
[+] [-] pdntspa|3 years ago|reply
People are highly tribalist and banding together like that produces a lot of ugly outcomes. Communities form and likely won't go away, particularly the more people internalize them emotionally and integrate them into their identity.
It is exhausting to see a community for literally everything, even the smallest products.
Perhaps we should let products simply exist.
[+] [-] neilk|3 years ago|reply
Near where I used to live in the Valley, there was a real-world club for enthusiasts of a particular weird antique car. Not toxic, just eccentric. They just liked getting together and drinking in a nearby pub and showing off how they'd restored this or that aspect of the car.
I think our goals should be to replicate that kind of success.
I think the other posters in this thread have it right, that the reason why we usually can't is because our metrics are all about selling audiences to advertisers.
[+] [-] BrianOnHN|3 years ago|reply
And we know those platforms are tools for manipulation.
How much of the desire for these communities might be manufactured in the first place?
[+] [-] jpster|3 years ago|reply
[+] [-] ItsMonkk|3 years ago|reply
If you then take that model and try to use them in a new domain, that is never going to work out. The environment adapts to the incentives that the current system produces, and will instantly make your pre-created model worthless[0]. The only way to work out long-term is by having an expert stay ahead of the pack creating what they think are systems with better signal to the noise.
[0]: https://dilbert.com/strip/1995-11-13
[+] [-] Bjartr|3 years ago|reply
[+] [-] bluGill|3 years ago|reply
[+] [-] mhall119|3 years ago|reply
Connections between people: https://www.savannahhq.com/2022/09/06/measuring-your-communi...
Support: https://www.savannahhq.com/2022/09/08/measuring-your-communi...
[+] [-] JAA1337|3 years ago|reply
I believe this falls into the 'Results Oriented' mindset of delivering value. This means figuring out your desired outcome ahead of time, then measure it over time. In addition, overtime, you may determine that your desired outcome has changed and you need to measure differently. Yes, iterating is not just for software.
IMHO pure quantitative metrics are of little value for customer experience. I believe metrics should be a little squishy and subjective. The value in the subjectivity is that it spurns conversation and actual thought instead of simple counting.
[+] [-] ChrisMarshallNY|3 years ago|reply
Sadly, it's pretty difficult to measure for some of the "values" that my communities run on, but "engagement" isn't even on the map.
I suspect that "engagement" does, indeed, provide real monetary value for advertising-based sites, as clicks == money. Since that isn't a factor in my communities, we don't worry about that. In fact, we try to minimize "engagement."
[+] [-] nmilo|3 years ago|reply
The article implies that community building should be some kind of altruistic purpose, where your only goal is to maximize the amount of "meaningful relationships" created? But building an artificial community in the first place can never be altruistic because the end goal of it all is to guide people to your product or conference or whatever. If you were an actual altruistic community builder, you would be telling people to get off the Internet and go make "meaningful relationships" with people in real life.
[+] [-] mhall119|3 years ago|reply
No, that's a social media manager, completely different role.
> And do they think that people care about their brand so much that they would join a community of people whose only common trait is liking the same brand?
Yes! It happens all the time. Most of the biggest brands have a community of people who share that common trait.
> And in the first place, how do you even build a "community" manually and artificially?
Again yes, just like you build a garden. The gardener doesn't make the plants grow, but they do make sure they have the right environment and resources to grow in.
> The article implies that community building should be some kind of altruistic purpose, where your only goal is to maximize the amount of "meaningful relationships" created?
That's not altruism. Those meaningful relationships help the company/project the community was built around. They provide feedback, help improve processes, make connections outside the company/project, provide support to other users, there are too many things that communities do to list them all here.
[+] [-] unknown|3 years ago|reply
[deleted]
[+] [-] dfabulich|3 years ago|reply
> They’re not as easy to measure as engagement, sure, but they can be measured.
I'm afraid that a citation is sorely needed here. There are no real measurements for "meaningful relationships."
In fact, the only proxy metric we have for "meaningful relationships" is engagement!
And, yes, engagement is not a very good proxy metric for meaningful relationships, but since there's no alternative, it's all we've got, so this article is pointless.
[+] [-] civilized|3 years ago|reply
"Engagement isn't a good metric. Optimize for having a good community."
"But engagement is easy to measure AND it means people are using the product. If people are choosing to use the product, doesn't that mean the product is giving them value?"
"Not necessarily. Drugs like tobacco and heroin have fantastic 'engagement' but we consider them bad for us."
"But how do you tell the difference between something that engages because it's good and something that engages because it's some kind of drug?"
"I guess one important signal would be when people know something is bad for them but they're doing it anyway?"
"Is that the case for social media? Do people feel addicted - feel that it's bad for them but they can't stop?"
I think we've heard a lot of anecdata on this last question, but I can't recall seeing any big survey on it.
EDIT: according to this, people will pay to have their social media restricted: https://www.washingtonpost.com/outlook/2021/07/19/social-med...
[+] [-] mikkergp|3 years ago|reply
Of course engagement is insufficient and of course if you want to know something else, measure that other thing, and of course you should interrogate your metrics, and maybe it's just this thing that bugs me about modern writing where you have to take this ridiculous extreme stance to get eyeballs but yeah.
[+] [-] swayvil|3 years ago|reply
All 3 are concerned with attention. Are hungry for it.
If social media could be optimized to deliver engaging conversation to everybody then the demon would be slain. Because the hunger would be satisfied.
Does that sound right?
[+] [-] RicoElectrico|3 years ago|reply
Whatever I'd come up with, I'd like it to be a self-healing organism that would continue to thrive even if some important members become inactive. And to feel it's not a Sisyphean task.
[1] https://osmstats.neis-one.org/?item=countries&country=Poland
[+] [-] scoofy|3 years ago|reply
I am also starting a wiki, a golf course wiki: https://golfcourse.wiki
It's still very early for me, but I see myself in this post. It's going to be extremely slow going, as the main consumers of the content created have no desire to contribute, but since golf courses are mostly static, as long as i plug away, adding a little bit all the time, the site will be a success. It costs almost nothing to operate. Generating traffic is challenging, and i've been trying to find more people willing to contribute, but again, I see my product as best placed if i do exactly the opposite of what most of the golf course info sites are doing. Don't base my success on engagement, no aggregation, no overly-prodding to contribute. Just one day at a time, with the assumption that the task is endless anyway, having people share their passion on their own time.
If i can make that profitable at all, and it shouldn't be too difficult as the site grows, i'll see it as a total win.
[+] [-] alexb_|3 years ago|reply
Then they would instantly start using screams? Nobody making business decisions in social media companies will prioritize "user happiness" over increased production of their main product (your attention) for their customers (advertisers).
[+] [-] egypturnash|3 years ago|reply
[+] [-] catherd|3 years ago|reply
[+] [-] forbiddenvoid|3 years ago|reply
[+] [-] yieldcrv|3 years ago|reply
They chose to pay higher prices for shares, divorced from actual revenue
The companies, in-turn, found non revenue metrics to show quarter over quarter
and thousands of startups copied the model
Just don’t rely on the A/B test, be aware of it, but don’t rely on it. Put the human control back into the process. Users aren't staying on your platform 5 seconds longer after you hid the escape hatch because they love it, they are frustrated and lost! The A/B test doesn't tell you why the engagement is longer, only that it is. It takes an empathic human to say “okay, B got us that result but not for the reason we want”