> It’s important to realise this isn’t a customer-side issue; they shouldn’t have to consider the impact of every configuration option we choose to put in front of them. They don’t have the full context and knowledge, and expecting them to be experts in the nitty gritty of Canarytoken discoverability
Yes!
> Going forward, we will show multiple examples of prefixes. A user looking to add a custom domain will see a variety of example zones when they visit the page, and the examples will cycle each time they open the configuration page. We want to convey that they have options in choosing the name, and we show them a variety of sample options. Our hope is that this will prompt customers to pick their own names, and if they do rely on our examples then those are now spread over a large list of examples.
No! They were so close and yet it sounds like they've still missed the point. The issue is that users don't understand the "why" behind the prefix. Just randomizing the prefix that they're shown does nothing to change that.
IMO, a better solution would be.
1. The shortest possible explanation under the field of why you shouldn't use "someprefix".
2. Prevent users from using "someprefix" as the prefix and show them the warning again. By eliminating the default option as an option, you force your users to leave auto-pilot mode and actually consider their choice.
> 2. Prevent users from using "someprefix" as the prefix and show them the warning again.
Don't do this - examples in documentation should be valid. Having an example that doesn't work when the user tries it out will just lead to frustration.
Reminds me of a large company I worked at, I had some documents for developers how to set up some local environment thing they had to do exactly 1 time and never again. It was just a handful of terminal commands, all starting with the traditional shell notation like:
$ (some command)
Over the course of a year I got periodic complaints that it "wasn't working" and I tried to find issues on my end and couldn't. One particularly vocal dev came to me directly and insisted it was broken, so I went on a shared session with him, it turns out they were pasting the "$" into the terminal causing it to say: "$: command not found."
That was the source of all the complaints, once I removed it, they stopped.
Another trouble source I've had has been been fixed by some wikis. When I first ran into one that supported this about a dozen years ago it made my life a lot easier.
And that is having a function that injects the current user's username into the wiki page. Any instructions that require you to log in as you can then be blindly cut and pasted, without people coming and telling me "it doesn't work" when they try to log in as me. Yeah of course that doesn't work, mate.
People reading runbooks aren't fully engaged. They may actually be in a war room. The moral of this story is that you are not entitled to carve out a chunk of your coworker's attention. Those periods should be brief, and tied to an active initiative/epic. After that 'work' is done, everything you wrote should be something that they can operate at 2:00 am.
Is there a reason websites put that $ in front of commands you are fully expected to mindlessly copy and paste? I've seen it happen more and more and it simply baffles me.
I gave what my company calls a “lunch and learn” presentation once of some interesting tools. People liked it and shared my deck around which was cool. But then my quick/dirty examples started showing up in best practice (I loathe that term) decks shared to very large teams with my name at the bottom. A security guy, who I greatly respect, raised some questions and I had to go through the whole story with him and then find all references to my examples and fix them. It was pretty embarrassing.
I did a similar thing as part of a "lunch and learn". NodeJS + Express was super fresh and I did a small example app. When returning the user profile, I just queried the database and returned the entry displaying some properties on the frontend.
The team lead was like "show us the request in the console", and I opened it up and there was the non-encrypted password, createdAt date and basically all the not-needed properties.
Maybe, but you did learn a lot too. A follow-up that explains all those fixes might be even more interesting than original presentation, as far as I'm concerned.
Me too :( One of my coworkers keeps talking about “best practice” all the time. And he writes the most broken garbage of anyone I’ve met in a long time.
I’m often feel like quitting, because I do not enjoy working with him. But I like the company I work for. And I don’t want to spend time trying to find a new job at the moment.
Can we please use `example.com` for an example domain name instead of like `somedomain.com`? It can create accounts with emails that someone can actually intercept.
There is also an entire TLD, .example so that you can put multiple names in a TLD and distinguish big-corp.example from my-local-store.example and it's clear that those aren't related, they just share a registry the same way as letsencrypt.org and wikipedia.org do
Another "can we please" is stop illustrating (or providing as defaults) insecure configuration examples.
A large percentage of people following the examples will use them verbatim.
Had a case recently where someone was setting up a postgres container and the example was something like:
docker run --name some-postgres -e POSTGRES_PASSWORD=mysecretpassword -d postgres
You guessed it, "mysecretpassword" ended up being the password. And the service predictably got compromised, because that example was one that was pretty high up in seasrch results for "how to run postgres in docker" and attackers probing for port 5432 will try "mysecretpassword."
Another one is can we please stop posting code examples that are illustrating some language feature or how to do thing X in language Y, where the code has a footnote "this isn't secure, don't do this in production code." That footnote will be ignored. If you're going to bother answering a question, answer it with a proper implementation and not something that is a gaping code injection vulnerability.
This is also an opportunity to think about the value of a piece of configuration. If an example configuration value works for 40% of users without modification, should that value even exist? Think Bash's `HISTCONTROL=erasedups`, which shouldn't be necessary to set in the 21st century. Or should it be auto-generated, like Docker's container names?
In the very best case, the defaults are so good that an empty configuration does what most people want. Think ripgrep, …, welp, I can't really think of many good examples. Browsers need extensions, Bash needs a decent prompt, even many pro cameras need to be configured to save raw images by default.
> Think Bash's `HISTCONTROL=erasedups`, which shouldn't be necessary to set in the 21st century
As in that it should be the default? Or only option? I personally do not have (and want to have) that set, and I am hardly alone. Any change in defaults fucks someone over, especially in things like bash, where you ssh into many machines with many different versions of bash...
How do we combine evolution and development with backwards compatiblity? I think it's quite natural that we end up with this conundrum. Like say for example Vim having outdated defaults, because changing them could disrupt existing users. A reboot/fork of it can reset and start fresh but will eventually in its own development hit the same problem.
Are there examples of projects who solve this well?
Ripgrep's author is also very careful about breaking changes - I think that means it will also one day have outdated defaults!
Especially for software which is mod-able, malleable, composable, configurable... defaults have a disproportionate impact on UX, DevEx etc. A reason why TLS 1.3 got rid of a laundry list of options, or why WireGuard is simply a joy to work with (as opposed to IPsec / OpenVPN), or why middleboxes on the Internet are a big hurdle to protocol upgrades, or how NewCloud companies like Cloudflare, Replit, Flyio, and Vercel have devex beyond what the Big 3 can muster up.
> Think ripgrep ... welp, I can't really think of many good examples.
Apple has got this spot on, across decades. Their products "just work", as they say so in their own marketing.
> Browsers need extensions
Now you see why Chrome is defaulting to Manifest v3 ;)
> This is also an opportunity to think about the value of a piece of configuration. If an example configuration value works for 40% of users without modification, should that value even exist?
This sounds completely insane. If a majority of people need something different, they shouldn't be allowed to have it?
Oh, absolutely. If you give people an example (and you should), the overwhelming majority will copy the example exactly and then only change what they are forced to change when it doesn’t work otherwise. Therefore, prepare your examples accordingly.
About 5 years ago I made a blog post detailing how to use Traefik/LE with PHP. For about one day, I realized I had my personal email in the template for the warning email for when the Lets Encrypt cert is expiring. I still get emails warning random people that their domain is going to expire.
This also applies to code. I come across the situation A LOT in OOP codebases that they have completely superfluous interfaces and/or abstract classes, that are made pointless by the fact that every single implementation extends some example implementation. The most egregious example I ever found was a Minecraft mod which had an interface, implemented by an abstract class, which had another abstract class that extended it, which was implemented by a an example class.
Every single mod I could find (sample size of ~1000) just extended the example.
This may be an actual answer to the problem. If you show "someprefix" as an example, then give users a helpful error message if they actually type in "someprefix". Something like "Dear user, someprefix is only an example, please replace this with a name appropriate to your business."
I'm thinking if invalid characters in the examples given would be an acceptable solution.
For DNS records this could be XML-like tags like <someprefix>.<yourdomain>.<tld>
On one hand, it prevents blind copy-pasting but on the other hand, your example is invalid.
For procedure documentation I often use ${service_namr:?} For similar reasons. If the variable isn't set you get a clear error. And it provides an easy way to use the template without modification.
It isn't perfect, common variable names may already be set or have been set in a previous execution of this playbook on a different problem. But it catches common issues while being convenient.
Yeah I think this is the only good solution, really. Otherwise it's just often unclear which things are required to have a specific value, and which things can be replaced. In this example, I'm sure plenty of people thought it was possible that "someprefix" was required, rather than just an example.
A few years back, I recall reading about some automotive manufacturers who had just copied an example "airbag arming authorization" code/value that appeared in a shared spec document (IIRC) for their vehicles. There was a Metasploit module created (for the Hardware Bridge) that would send CAN bus messages to just check/verify if a particular vehicle uses this insecure arming code. For vehicles using this known code, an attacker with CAN bus access could deploy airbags on an unsuspecting target during vehicle operation. https://www.rapid7.com/blog/post/2017/12/22/metasploit-wrapu...
I've done some commercial software SDKs and this strikes me as the least surprising thing in the world. MOST programmers will copy and paste example code into production applications without really thinking about how well it fits into what they're doing.
The takeaway is similar to the article: think very, VERY hard about your examples and sample code. It doesn't just have to be correct and demonstrate the features, it also needs to be fairly robust so that customers don't hurt themselves with it.
An interesting thought is that the examples in your documentation don't necessarily need to be static and the same for everyone.
For example, if a user is logged in, you can autofill the appropriate accounts/domains/ids/etc to make the example work out of the box; and if some ID needs to be essentially random, then you can make it actually random when you generate the example.
They don't have to be static but making them dynamic might not worth the cost.
From a simple static page, now you need an API service, most probably connected to a DB or somehow integrated to the rest of your backend. So markdown suddenly isn't enough and you need some server-side logic.
For random strings, you can do this with client-side logic, which in some cases might be easier than server-side logic. But you are still moving from no-logic (static) to somewhere-logic.
Why is "use more examples" the solution? If the users are copy pasting the code, why not just generate random strings thereby showing an example and also fulfilling their own requirement of non-identifiable strings?
Very true. Good examples consume a lot of time. I was bitten a couple of times when the customer nailed me down with "But this example can never occur" and my futile attempt to justify "But it's an example!"
Good examples make documentation worthwhile to read.
Good defaults make an application worthwhile to use.
The opposite is also true. It happens rarely, but I have been bitten by trying to configure something to be what I would like it to be, only to discover it had to be what was in the documentation for it to work, generally with nothing in the documentation itself to clarify.
Can't think of any examples now though I'm afraid.
This isn't quite the point of the article, but we allow people to apply for student discounts for our service, and provide the following example that we ask users to send to us over Intercom:
> Hello, could I please apply for the student discount?
>
> [PLEASE READ AND DELETE THIS – After sending this initial message, please attach a proof of your student status, such as a photo of your valid Student ID so we can process this quicker!]
I don't think any of the countless people that have asked for the discount have ever removed the "PLEASE REMOVE" part, and many don't bother to send the proof until we ask for it either.
I've run into a number of networks in my area (private businesses, a couple municipalities, a couple law enforcement agencies) all using the 192.9.1.0/24 subnet.
There was some overlap in these sites w/ respect to IT service companies involved in their setup. Best as I can guess it came down to one person who floated between the employ of a couple (or three) IT service companies leaving a swath of 192.9.1.0/24 in their wake (or maybe training other technicians during their time at these companies). It seems like this work might have been done pre-RFC 1597 (which is, I think, the first place that what is today's RFC 1918 address space shows up) but I think they were just following examples.
I'd love to know what examples motivated the us of this address space. I find some old Sun docs[0] referencing this address space, and RFC 2328[1] makes reference to it.
Any template strings are ambiguous unless there is more than one example.
For example, let's imagine that there is an instruction saying that in a config file, there should be:
PASSWORD=[password]
Let's say our password is "admin". Then it could be that:
PASSWORD=admin
PASSWORD=[admin]
or even
PASSWORD=[password]
as it is not a place to actually store the password, but to select an authentication method.
Sure, sometimes (but not always!), it is possible to deduce how to fill the pattern.
If the field has some canonical value, go with a sane default e.g. "canary.their-company.com", with a note that any other suffix works instead of "canary". Sensible defaults save us a lot of brainpower (vide https://en.wikipedia.org/wiki/Convention_over_configuration).
In-band signalling often seems to have this issue where it's not 100% clear what is part of the message and what is part of the meta-message.
Edit: I think it's mildly amusing and further drives the point hom that some people in this thread missed endnote 1, where you say it was not the actual prefix.
> Frankly it’s a reason enterprise software is often so terrible; tons of options you barely understand or know about, and are configured according to tutorials/examples rather than understanding.
This article stresses that it's not a 'customer-side problem', and what they'll do to try to address it on their end.
But is there anything that enterprises can do in order to encourage people not to work blindly from tutorials? What do companies where workers avoid this pitfall look like?
This is such a great story and an important one. I always optimise examples for people copying and pasting, trying to make it as safe and meaningful by default as possible. It doesn't matter why you're copying and pasting - you may not have a lot of skills in this specific area, or you might be in a hurry. If you know what you're doing, you can probably improve the code, but if you use it as-is, it shouldn't come back to bite you!
[+] [-] superfrank|2 years ago|reply
Yes!
> Going forward, we will show multiple examples of prefixes. A user looking to add a custom domain will see a variety of example zones when they visit the page, and the examples will cycle each time they open the configuration page. We want to convey that they have options in choosing the name, and we show them a variety of sample options. Our hope is that this will prompt customers to pick their own names, and if they do rely on our examples then those are now spread over a large list of examples.
No! They were so close and yet it sounds like they've still missed the point. The issue is that users don't understand the "why" behind the prefix. Just randomizing the prefix that they're shown does nothing to change that.
IMO, a better solution would be. 1. The shortest possible explanation under the field of why you shouldn't use "someprefix". 2. Prevent users from using "someprefix" as the prefix and show them the warning again. By eliminating the default option as an option, you force your users to leave auto-pilot mode and actually consider their choice.
[+] [-] cmovq|2 years ago|reply
Don't do this - examples in documentation should be valid. Having an example that doesn't work when the user tries it out will just lead to frustration.
[+] [-] Animats|2 years ago|reply
Yes. That's why https://example.com exists.
[+] [-] JohnMakin|2 years ago|reply
$ (some command)
Over the course of a year I got periodic complaints that it "wasn't working" and I tried to find issues on my end and couldn't. One particularly vocal dev came to me directly and insisted it was broken, so I went on a shared session with him, it turns out they were pasting the "$" into the terminal causing it to say: "$: command not found."
That was the source of all the complaints, once I removed it, they stopped.
[+] [-] hinkley|2 years ago|reply
And that is having a function that injects the current user's username into the wiki page. Any instructions that require you to log in as you can then be blindly cut and pasted, without people coming and telling me "it doesn't work" when they try to log in as me. Yeah of course that doesn't work, mate.
People reading runbooks aren't fully engaged. They may actually be in a war room. The moral of this story is that you are not entitled to carve out a chunk of your coworker's attention. Those periods should be brief, and tied to an active initiative/epic. After that 'work' is done, everything you wrote should be something that they can operate at 2:00 am.
Because sooner or later, they well.
[+] [-] hiAndrewQuinn|2 years ago|reply
[+] [-] cduzz|2 years ago|reply
:$! $CWD $HOSTNAME $;
: (exit status, current directory, hostname, $ or # and a ;
the idea being -- you could just cut and paste and it'd mostly just work (except megabozos who like to make directories named ; or whatnot)
[+] [-] chasd00|2 years ago|reply
[+] [-] ThalesX|2 years ago|reply
The team lead was like "show us the request in the console", and I opened it up and there was the non-encrypted password, createdAt date and basically all the not-needed properties.
I still cringe thinking about it.
[+] [-] bornfreddy|2 years ago|reply
Maybe, but you did learn a lot too. A follow-up that explains all those fixes might be even more interesting than original presentation, as far as I'm concerned.
[+] [-] codetrotter|2 years ago|reply
Me too :( One of my coworkers keeps talking about “best practice” all the time. And he writes the most broken garbage of anyone I’ve met in a long time.
I’m often feel like quitting, because I do not enjoy working with him. But I like the company I work for. And I don’t want to spend time trying to find a new job at the moment.
[+] [-] andirk|2 years ago|reply
[+] [-] tialaramex|2 years ago|reply
[+] [-] SoftTalker|2 years ago|reply
A large percentage of people following the examples will use them verbatim.
Had a case recently where someone was setting up a postgres container and the example was something like:
You guessed it, "mysecretpassword" ended up being the password. And the service predictably got compromised, because that example was one that was pretty high up in seasrch results for "how to run postgres in docker" and attackers probing for port 5432 will try "mysecretpassword."Another one is can we please stop posting code examples that are illustrating some language feature or how to do thing X in language Y, where the code has a footnote "this isn't secure, don't do this in production code." That footnote will be ignored. If you're going to bother answering a question, answer it with a proper implementation and not something that is a gaping code injection vulnerability.
[+] [-] l0b0|2 years ago|reply
In the very best case, the defaults are so good that an empty configuration does what most people want. Think ripgrep, …, welp, I can't really think of many good examples. Browsers need extensions, Bash needs a decent prompt, even many pro cameras need to be configured to save raw images by default.
[+] [-] gray_-_wolf|2 years ago|reply
As in that it should be the default? Or only option? I personally do not have (and want to have) that set, and I am hardly alone. Any change in defaults fucks someone over, especially in things like bash, where you ssh into many machines with many different versions of bash...
[+] [-] kzrdude|2 years ago|reply
Are there examples of projects who solve this well?
Ripgrep's author is also very careful about breaking changes - I think that means it will also one day have outdated defaults!
[+] [-] ignoramous|2 years ago|reply
More generally, defaults (including, default examples), matter: https://news.ycombinator.com/item?id=25646180
Especially for software which is mod-able, malleable, composable, configurable... defaults have a disproportionate impact on UX, DevEx etc. A reason why TLS 1.3 got rid of a laundry list of options, or why WireGuard is simply a joy to work with (as opposed to IPsec / OpenVPN), or why middleboxes on the Internet are a big hurdle to protocol upgrades, or how NewCloud companies like Cloudflare, Replit, Flyio, and Vercel have devex beyond what the Big 3 can muster up.
> Think ripgrep ... welp, I can't really think of many good examples.
Apple has got this spot on, across decades. Their products "just work", as they say so in their own marketing.
> Browsers need extensions
Now you see why Chrome is defaulting to Manifest v3 ;)
[+] [-] thaumasiotes|2 years ago|reply
This sounds completely insane. If a majority of people need something different, they shouldn't be allowed to have it?
[+] [-] teddyh|2 years ago|reply
[+] [-] jmondi|2 years ago|reply
Prepare your examples accordingly.
[+] [-] OkayPhysicist|2 years ago|reply
Every single mod I could find (sample size of ~1000) just extended the example.
[+] [-] bradley13|2 years ago|reply
[+] [-] nicexe|2 years ago|reply
On one hand, it prevents blind copy-pasting but on the other hand, your example is invalid.
[+] [-] kevincox|2 years ago|reply
It isn't perfect, common variable names may already be set or have been set in a previous execution of this playbook on a different problem. But it catches common issues while being convenient.
[+] [-] sanderjd|2 years ago|reply
[+] [-] croes|2 years ago|reply
[+] [-] recursivetech|2 years ago|reply
[+] [-] eschneider|2 years ago|reply
The takeaway is similar to the article: think very, VERY hard about your examples and sample code. It doesn't just have to be correct and demonstrate the features, it also needs to be fairly robust so that customers don't hurt themselves with it.
[+] [-] PeterisP|2 years ago|reply
For example, if a user is logged in, you can autofill the appropriate accounts/domains/ids/etc to make the example work out of the box; and if some ID needs to be essentially random, then you can make it actually random when you generate the example.
[+] [-] nicexe|2 years ago|reply
From a simple static page, now you need an API service, most probably connected to a DB or somehow integrated to the rest of your backend. So markdown suddenly isn't enough and you need some server-side logic.
For random strings, you can do this with client-side logic, which in some cases might be easier than server-side logic. But you are still moving from no-logic (static) to somewhere-logic.
[+] [-] personjerry|2 years ago|reply
[+] [-] albinowax_|2 years ago|reply
[+] [-] jhoechtl|2 years ago|reply
Good examples make documentation worthwhile to read.
Good defaults make an application worthwhile to use.
[+] [-] DoingIsLearning|2 years ago|reply
[+] [-] alex-moon|2 years ago|reply
Can't think of any examples now though I'm afraid.
[+] [-] gjvc|2 years ago|reply
Dodged a meta-bullet there...
[+] [-] mh_|2 years ago|reply
[+] [-] tetrep|2 years ago|reply
[+] [-] fastball|2 years ago|reply
> Hello, could I please apply for the student discount?
>
> [PLEASE READ AND DELETE THIS – After sending this initial message, please attach a proof of your student status, such as a photo of your valid Student ID so we can process this quicker!]
I don't think any of the countless people that have asked for the discount have ever removed the "PLEASE REMOVE" part, and many don't bother to send the proof until we ask for it either.
[+] [-] im3w1l|2 years ago|reply
[+] [-] EvanAnderson|2 years ago|reply
There was some overlap in these sites w/ respect to IT service companies involved in their setup. Best as I can guess it came down to one person who floated between the employ of a couple (or three) IT service companies leaving a swath of 192.9.1.0/24 in their wake (or maybe training other technicians during their time at these companies). It seems like this work might have been done pre-RFC 1597 (which is, I think, the first place that what is today's RFC 1918 address space shows up) but I think they were just following examples.
I'd love to know what examples motivated the us of this address space. I find some old Sun docs[0] referencing this address space, and RFC 2328[1] makes reference to it.
[0] http://bitsavers.informatik.uni-stuttgart.de/pdf/sun/sunos/3...
[1] https://www.ietf.org/rfc/rfc2328.txt
[+] [-] stared|2 years ago|reply
For example, let's imagine that there is an instruction saying that in a config file, there should be:
PASSWORD=[password]
Let's say our password is "admin". Then it could be that:
PASSWORD=admin
PASSWORD=[admin]
or even
PASSWORD=[password]
as it is not a place to actually store the password, but to select an authentication method.
Sure, sometimes (but not always!), it is possible to deduce how to fill the pattern.
If the field has some canonical value, go with a sane default e.g. "canary.their-company.com", with a note that any other suffix works instead of "canary". Sensible defaults save us a lot of brainpower (vide https://en.wikipedia.org/wiki/Convention_over_configuration).
[+] [-] im3w1l|2 years ago|reply
Edit: I think it's mildly amusing and further drives the point hom that some people in this thread missed endnote 1, where you say it was not the actual prefix.
[+] [-] Lammy|2 years ago|reply
[+] [-] pxc|2 years ago|reply
This article stresses that it's not a 'customer-side problem', and what they'll do to try to address it on their end.
But is there anything that enterprises can do in order to encourage people not to work blindly from tutorials? What do companies where workers avoid this pitfall look like?
[+] [-] lornajane|2 years ago|reply