Here's an idea for how to maintain Go moving forward: keep making the damn tool however you want. The thing never would have existed in the first place if they had started with an industry survey. It was created to address a perceived need, by the people with the need, for themselves. This model is perfectly fine moving forward. Some industrial user wants a new Go feature or bugfix? Great. If it's enough of a problem, they can fix it and upstream a patch. That's how open source software is always supposed to work. Telemetry does nothing to improve this situation. If the Go team at google no longer has any ideas for what to work on (as must be the case if they're wasting their time on dumb crap like forced google spyware in a compiler) then they should just stop. Maybe focus on accepting PRs from people with ideas and strong enough motivation to work on them. I mean, there are 5000+ issues and 330 open PRs on the go github right now, so that should be plenty to keep them occupied. On top of that, how can literal thousands of issues not be a strong enough source of actionable usage information that they saw fit to try to get more? Do they plan on wrapping up all the open issues before looking at telemetry?
I really appreciate this point, and I think it applies very broadly. Good design comes from a coherent, individual (or group) vision. Citing examples will only incite discussions, because everyone has a different needs and ideas, but the things I most appreciate in tech and art share this common aspect.
As I see it, relying on excessive telemetry, surveys, focus-groups, etc is a bit of an indication that nobody at the top has a strong idea of where to take the project.
> keep making the damn tool however you want. The thing never would have existed in the first place if they had started with an industry survey. It was created to address a perceived need, by the people with the need, for themselves. This model is perfectly fine moving forward.
I disagree, doing what you feel like is a great approach when you have no users, your building a tool that solves your needs and basically saying "Try this it, it works great for me, maybe it'll be good for you too!"
Once you have millions of users and billions of lines of code you need to think about how your changes affect those users. If you get it wrong there's a huge cost to that (c.f. early forays in to go packaging and dependency management). This is why go's comparability promise is such a big deal and a big part of why go is a very popular language.
> If the Go team at google no longer has any ideas for what to work on (as must be the case if they're wasting their time on dumb crap like forced google spyware in a compiler) then they should just stop.
They did stop. "Go 1" is complete.
Since 2017, the project renewed itself by moving to what is dubbed "Go 2"[1], being built around community feedback. Decisions are made based on data collected from the community. Adding telemetry to increase the amount of data available is in line with the new project scope. Go 1.13 was the first "Go 2" release[2].
Indeed, "Go 1" is still there if you want to use the tool that was built for what was needed at Google and nothing more. You don't have to move into the "Go 2" ecosystem. But if others want to put their time into that, why not? That's their prerogative.
> If the Go team at google no longer has any ideas for what to work on (as must be the case if they're wasting their time on dumb crap like forced google spyware in a compiler) then they should just stop.
They want to add telemetry. So they should just add it. If you dislike it, then patch it out yourself. As you say:
>That's how open source software is always supposed to work.
I think that the author has kind of a Stockholm syndrome.
"Users are liars, so let's spy them directly to know what we want to know".
It is mind blowing how, as an user/the target, you can support that.
Nothing is really anonymous and your anonymous data can say a lot about you.
Telemetry coming from this IP, so company x is using go. A pattern of data coming every 2 days, so their build nodes rebuild every 2 days. That kind of build pattern is there, so they are using the xxx crypto library...
And when they say, let's trust Google, I would propose to Google to accept the opposite:
Now they will transmit to the public telemetry of their internal systems: how many users, what do they do, how many users they block, for what reason, how many build nodes they have, how many commits, how long the go team is spending looking at telemetry reports, which website are the more visited by Google employees,...
And let's see if they will accept. It's for the good of the world, why they would refuse?
I find the attitude on display here utterly gross and it makes me wonder where this person got it. Is this the common attitude inside Google or Silicon Valley proper these days?
Privacy used to be kind of sacrosanct in the industry. What happened?
I guess when surveillance driven advertising pays all the bills it has an extreme warping effect on everyone’s mentality. It’s hard to care about privacy when violating it is how you get paid.
I’m increasingly wondering if we might vastly improve our society by banning or taxing the hell out of all advertising. It’s an industry that makes the world a worse place in deep and pervasive ways that go much further than just making everything ugly and obnoxious. The industry pushes a toxic relationship between people and businesses at every level.
Look, I hate large corps too but this paranoia hinders open source's ability to self-cooperate.
> Telemetry coming from this IP, so company x is using go. A pattern of data coming every 2 days, so their build nodes rebuild every 2 days.
Not true. Even if Google lied about collecting IPs, the data would be sent only every ~ year with aggregated counts so no real time usage data. And even if one could see the patterns from the data, everyone will be able to, not just Google.
Let's not trust Google. But let's not shoot ourselves in the foot by refusing any automated cooperation.
Additionally, majority of distros use package managers, so if any of the major promises of the upstream would be broken, distro packages could patch it out. This isn't forced, there are several points where anyone can stop the telemetry.
> For every kilobyte of Go code available on GitHub, GitLab, and other Git forges, there are unknowable megabytes of private Go code that will never see the light of day (or maybe they will if LAPSUS$ decides to make that company a target).
> Without knowing what ports of Go are used, the Go team can’t make sure that the right time is spent on maintaining those ports.
Am I being overly pragmatic if not selfish for thinking this is absolutely fine? If you use a free tool behind closed doors to make money and don't want to opt-in to telemetry, then I couldn't care less if the go team doesn't make go work better for your use case. Meanwhile I'm developing my open source project with telemetry turned on, ?????, PROFIT.
People really need to accept that they can't and shouldn't have full introspection into everything they're ever interested in. Even if it's a thing they care about (like their own homepage or programming language). Sure it's nice to get feedback. Sure it's nice to know that somebody uses it. But putting a tracker on everything with the argument of "will help me provide a better service", forgoing informed consent.. It's wrong when it's done on an institutional level, whether it's companies or the goverment, and it's wrong when individuals do that.
> Unfortunately, the proposal came from a Google employee. Google has a reputation of taking user data and feeding it into piles of linear algebra in order to skew its search results to match your existing biases or whatever the hell else people do with linear algebra.
Right. We live in an age now where gigantic, powerful, organizations are ruining everything to privilege a fantastically small few. They have consistently shown a complete lack of any moral compass and a willingness to do absolutely anything necessary to increase their power or continue to enrich themselves.
If we lived in any other age, people might tolerate this, but we don't, and therefore everyone is well within their right to mutiny, regardless of how interesting a technical solution this is.
Reminded of a post I saw the other day referencing Clausewitz to say "if your ideal military strategy is politically unachievable, it's not the ideal strategy". If going for opt-out telemetry makes your customers hate you and you're forced to retreat under a hail of fire, going opt-in is not a mistake.
There's also a philosophical problem in going too deep into "customers don't know what they want" and A/B testing everything to death: you end up using it as a substitute for dialogue. Ultimately customers are making conscious decisions. You have to make the case rather than assert that it's "better" from behind your metrics dashboard.
Yes, the author of the article makes the mistake of... not listening to or responding to what the other side thinks, just because she disagrees with them. No concern was addressed in her article.
In that scenario, not going opt-out is not a mistake. But going opt-in probably is a mistake, because the data will be useless, so it’s a waste of time to implement.
While I agree that going with what the users want is the correct approach, but "a few digits less of accuracy" really undersells how much the data quality will suffer.
It will pretty much become useless (how many people are going to opt-in, realistically?) - and that's fine if it's what the community wants.
> For sure it may have a few digits less of accuracy in some dashboards but who cares.
You could argue that it's ultimately the users who suffer. Telemetry helps guide development, so it's harder for developers to know what to focus their efforts on.
Good job Russ Cox. Despite initial concerns around this the right decision was made. Opt in is the morally correct choice for any software community. Notably I will opt into this where it is appropriate.
To be honest, the fact that Google is behind Go, is whats holding me back from getting into it. Same with React and FB. I dont really have any sympathy points left for them.
> Developers using macOS are used to just installing Xcode to make the error messages go away, so they just installed Xcode and assumed it was normal behavior.
So, they don’t test go in CI on clean Mac OS installs? If this sort of issue is the rationale for the telemetry system, it sounds like they’re trying for a older, slightly-less-evil Microsoft approach where you fire QA, and just treat user machines as your test environment.
Google: "we don't want to talk to people, or hear their opinions, so we'll add telemetry to make decisions we want without including anyone else!"
Also Google: removed "do no evil" so they could... Do evil... Without sounding like hypocrites.
There are bugs reports with 100s of people complaining, about simple features, or broken code, and the attitude is WONTFIX.
If you won't listen to users, seething, upset, annoyed over inane issues you created? If you won't fix bugs for years, because it's your pet project/feature?
Then how are you even paying attention to telemetry correctly?
Opt-in telemetry is only useful if you will, no matter what, implement and fix what it tells you. Mozilla, predicated upon their behaviour, clearly only uses data when it agrees with prior opinions.
opt-in is useless to such an org, and opt-out isn't why it is happening.
If they want to add telemetry without asking the user, then they have to fix an entire industry including Google itself. They trained the people to hard disagree with such approaches, because in the past there was way too many incidents where data was unnecessary collected and misused.
Just trying to add this telemetry with opt-out in the first place shown one thing: They don't know their audience. Because if they would, if someone would have come up with that idea, everybody in the team should have screamed: "No, hell no!". You don't have the trust of your users.
I get it, getting trust back is much more effort than collecting more data (even if it's with good intent). But hey, the industry put itself into this.
They should simply allow for people to pick their default at installation. (so at download)
Even better if the go tool were to have a `go update` with a `telemetry-off` flag or something. (and possibly a prompt to remind people otherwise they will complain again)
`telemetry-on` flag. Sending data to third parties should _never_ be implicit or the default. Every PM, marketer, and mid-level manager needs to be firmly reminded that this is grossly invasive, user-hostile, impolite, and creepy.
Without an explicit agreement, it could even run afoul of wiretapping laws (unresolved in courts, as far as I’m aware).
Honestly, I think they should go with opt-in and make it clear to the community that the public dataset will be the sole source of decision making for future deprecations. If you want your favorite pet feature still enabled and no one using it is sending them telemetry then too bad.
As someone who was considering getting some Go and Rust under his belt, they've saved me a lot of time. This whole telemetry nonsense has ensured I won't waste my time investing it in Go. Great job.
Uh, what? The Go team introduced an idea, opt out telemetry with a one week delay before it was sent. Users said it wasn’t great, overwhelmingly. The Go team heard the feedback and made it opt in. What’s the problem, that’s a good thing.
yalue|3 years ago
oplaadpunt|3 years ago
As I see it, relying on excessive telemetry, surveys, focus-groups, etc is a bit of an indication that nobody at the top has a strong idea of where to take the project.
remus|3 years ago
I disagree, doing what you feel like is a great approach when you have no users, your building a tool that solves your needs and basically saying "Try this it, it works great for me, maybe it'll be good for you too!"
Once you have millions of users and billions of lines of code you need to think about how your changes affect those users. If you get it wrong there's a huge cost to that (c.f. early forays in to go packaging and dependency management). This is why go's comparability promise is such a big deal and a big part of why go is a very popular language.
randomdata|3 years ago
They did stop. "Go 1" is complete.
Since 2017, the project renewed itself by moving to what is dubbed "Go 2"[1], being built around community feedback. Decisions are made based on data collected from the community. Adding telemetry to increase the amount of data available is in line with the new project scope. Go 1.13 was the first "Go 2" release[2].
Indeed, "Go 1" is still there if you want to use the tool that was built for what was needed at Google and nothing more. You don't have to move into the "Go 2" ecosystem. But if others want to put their time into that, why not? That's their prerogative.
[1] https://go.dev/blog/toward-go2
[2] https://go.dev/blog/go2-next-steps
em-bee|3 years ago
just for the sake of argument, wouldn't telemetry be useful to know which of these issues are most likely to benefit the majority of users?
whether that's worth the cost of telemetry is another issue.
guipsp|3 years ago
> keep making the damn tool however you want.
But then you follow up with this:
> If the Go team at google no longer has any ideas for what to work on (as must be the case if they're wasting their time on dumb crap like forced google spyware in a compiler) then they should just stop.
They want to add telemetry. So they should just add it. If you dislike it, then patch it out yourself. As you say:
>That's how open source software is always supposed to work.
rdxm|3 years ago
[deleted]
mseepgood|3 years ago
They already do regular surveys.
majorhelmet|3 years ago
Inbefore I go to a new job and find out that they are using outdated, custom patched go compiler.
> I mean, there are 5000+ issues and 330 open PRs on the go github right now
How do they know which ones are affecting the most users?
> forced google spyware in a compiler
go is open source, feel free to compile it yourself without the telemetry. Which distros will do if any major promises would be broken
greatgib|3 years ago
"Users are liars, so let's spy them directly to know what we want to know".
It is mind blowing how, as an user/the target, you can support that.
Nothing is really anonymous and your anonymous data can say a lot about you.
Telemetry coming from this IP, so company x is using go. A pattern of data coming every 2 days, so their build nodes rebuild every 2 days. That kind of build pattern is there, so they are using the xxx crypto library...
And when they say, let's trust Google, I would propose to Google to accept the opposite:
Now they will transmit to the public telemetry of their internal systems: how many users, what do they do, how many users they block, for what reason, how many build nodes they have, how many commits, how long the go team is spending looking at telemetry reports, which website are the more visited by Google employees,...
And let's see if they will accept. It's for the good of the world, why they would refuse?
api|3 years ago
Privacy used to be kind of sacrosanct in the industry. What happened?
I guess when surveillance driven advertising pays all the bills it has an extreme warping effect on everyone’s mentality. It’s hard to care about privacy when violating it is how you get paid.
I’m increasingly wondering if we might vastly improve our society by banning or taxing the hell out of all advertising. It’s an industry that makes the world a worse place in deep and pervasive ways that go much further than just making everything ugly and obnoxious. The industry pushes a toxic relationship between people and businesses at every level.
majorhelmet|3 years ago
> Telemetry coming from this IP, so company x is using go. A pattern of data coming every 2 days, so their build nodes rebuild every 2 days.
Not true. Even if Google lied about collecting IPs, the data would be sent only every ~ year with aggregated counts so no real time usage data. And even if one could see the patterns from the data, everyone will be able to, not just Google.
Let's not trust Google. But let's not shoot ourselves in the foot by refusing any automated cooperation.
Additionally, majority of distros use package managers, so if any of the major promises of the upstream would be broken, distro packages could patch it out. This isn't forced, there are several points where anyone can stop the telemetry.
iforgotpassword|3 years ago
> Without knowing what ports of Go are used, the Go team can’t make sure that the right time is spent on maintaining those ports.
Am I being overly pragmatic if not selfish for thinking this is absolutely fine? If you use a free tool behind closed doors to make money and don't want to opt-in to telemetry, then I couldn't care less if the go team doesn't make go work better for your use case. Meanwhile I'm developing my open source project with telemetry turned on, ?????, PROFIT.
jonhohle|3 years ago
str3wer|3 years ago
groestl|3 years ago
People really need to accept that they can't and shouldn't have full introspection into everything they're ever interested in. Even if it's a thing they care about (like their own homepage or programming language). Sure it's nice to get feedback. Sure it's nice to know that somebody uses it. But putting a tracker on everything with the argument of "will help me provide a better service", forgoing informed consent.. It's wrong when it's done on an institutional level, whether it's companies or the goverment, and it's wrong when individuals do that.
junon|3 years ago
spicyusername|3 years ago
If we lived in any other age, people might tolerate this, but we don't, and therefore everyone is well within their right to mutiny, regardless of how interesting a technical solution this is.
pjc50|3 years ago
There's also a philosophical problem in going too deep into "customers don't know what they want" and A/B testing everything to death: you end up using it as a substitute for dialogue. Ultimately customers are making conscious decisions. You have to make the case rather than assert that it's "better" from behind your metrics dashboard.
lozenge|3 years ago
Uvix|3 years ago
mplanchard|3 years ago
speedgoose|3 years ago
The consensus was that opt-in was the best solution, and thankfully Google went with the best solution.
abmackenzie|3 years ago
It will pretty much become useless (how many people are going to opt-in, realistically?) - and that's fine if it's what the community wants.
remus|3 years ago
You could argue that it's ultimately the users who suffer. Telemetry helps guide development, so it's harder for developers to know what to focus their efforts on.
majorhelmet|3 years ago
zajio1am|3 years ago
Not really. People hate opt-out telemetry regardless of who runs it.
unxdfa|3 years ago
tacker2000|3 years ago
hedora|3 years ago
So, they don’t test go in CI on clean Mac OS installs? If this sort of issue is the rationale for the telemetry system, it sounds like they’re trying for a older, slightly-less-evil Microsoft approach where you fire QA, and just treat user machines as your test environment.
StreetChief|3 years ago
waweic|3 years ago
"Telemetry shows people aren't using this (privacy / internet freedom related) feature, so we can just remove it"
b112|3 years ago
There are bugs reports with 100s of people complaining, about simple features, or broken code, and the attitude is WONTFIX.
If you won't listen to users, seething, upset, annoyed over inane issues you created? If you won't fix bugs for years, because it's your pet project/feature?
Then how are you even paying attention to telemetry correctly?
Opt-in telemetry is only useful if you will, no matter what, implement and fix what it tells you. Mozilla, predicated upon their behaviour, clearly only uses data when it agrees with prior opinions.
opt-in is useless to such an org, and opt-out isn't why it is happening.
It's them. They are broken as an org.
Scubabear68|3 years ago
If the Go team wants to see what features are used, go audit open source Go code.
Since Go is also a Google thing, they could likely audit the presumably large sums of Google Go.
Yes, you will miss unknowable amounts of code this way. Who cares?
patchymcnoodles|3 years ago
Just trying to add this telemetry with opt-out in the first place shown one thing: They don't know their audience. Because if they would, if someone would have come up with that idea, everybody in the team should have screamed: "No, hell no!". You don't have the trust of your users.
I get it, getting trust back is much more effort than collecting more data (even if it's with good intent). But hey, the industry put itself into this.
aatd86|3 years ago
Even better if the go tool were to have a `go update` with a `telemetry-off` flag or something. (and possibly a prompt to remind people otherwise they will complain again)
Problem solved.
jonhohle|3 years ago
Without an explicit agreement, it could even run afoul of wiretapping laws (unresolved in courts, as far as I’m aware).
kardianos|3 years ago
Knowing the goal allowed rsc to make this judgement.
zaphar|3 years ago
fortyseven|3 years ago
Operyl|3 years ago
anon23432343|3 years ago
croes|3 years ago
Patrickmi|3 years ago