Some lessons in software engineering take a long, long time to sink in. (For me, at least):
- Don't generalize too early, which seems to address the original question pretty directly.
- Don't optimize too early.
- Maintainable > Correct > Fast. Maintainable = comprehensible. If it isn't maintainable, you won't know if it's correct. If it isn't correct, there's no point in making it fast.
- Minimize state. Deriving something from scratch every time it's needed leads to far fewer bugs, and the bugs there are probably far more obvious. Cache only when you need to for performance.
- Simplify as much as you can. This is the hardest one, because it can take a very long time to figure out all the ways in which a design that you thought was simple, turns out not to be.
- Simplify some more: The usual statement of the "simplify" rule is "as simple as possible, but no simpler". I don't think that's right. If you see a nice simplification at odds with requirements, push back on requirements.
I'm starting to see this more clearly as time passes. I've spent years developing the ability to "engineer" things and only now am I realizing that I've learned to "over-engineer" things. Writing software has taken on a life of its own that is actually counter to its purpose.
I don't think in terms of simplification though, I see it as "write what you need, when you need it." But I absolutely agree that you should avoid generalization aggressively and optimization should rarely be on the table. I also agree about minimizing state. Do it all "on the fly" unless there's known performance hit.
The moderation on programmers.stackexchange.com is completely overbearing. How can a question about analysis paralysis be closed for off-topic? If stackoverflow is for technical question/answer, what is the point the linked stackexchange if not for discussion of a common programmer problem like this? I see stuff like this closed all time time and it's really annoying.
This is a rambling discussion, which Stack Exchange is not really intended for. There are lots of good places to have rambling discussions. We focus on answerable questions with specific answers, not long-winded, open-ended conversation starters. There are plenty of other sites for long-winded, open-ended conversation starters, including Reddit and Hacker News.
After a decade or so of operating sites for programmers on the Internet, I've learned that, left by themselves, ALL programmer online sites will rapidly deteriorate into:
* I hate my job, woe is me ("I hate my job too!")
* I'm not productive any more, oh dear ("Oh my god you took the words right out of my mouth!")
* And some nonsense about H1B visas.
These kinds of conversations are endless because, I'm sad to say, they represent a common problem among programmers. Discussing it as a means of therapy is a wonderful thing, but DO IT SOMEWHERE ELSE, because Stack Exchange is 100% focused on questions with specific factual answers.
This hard and fast rule is actually WHY you love stack overflow so much. It's WHY the network works. It's WHY you click on the stackoverflow.com link in the Google results even if it's third. If we weren't strict about this stuff we wouldn't have such consistently high quality ANSWERS.
Just because "moderators" (actually, users) have deemed to close a question on Stack Exchange, doesn't mean it's a bad question, or that the person who asked it is bad. It just means that Stack Exchange is an edited, curated environment, not your livejournal. The very moderation that people complain about every time a popular question is closed is what has been making the site work so well.
I looked it up now[1] and it seems that this kind of questions are supposed to be asked on programmers.etackexchange.com!
This is what FAQ says. 2nd, 4th, 7th and 8th are exactly what this question is about!
""" Programmers — Stack Exchange is a site for professional programmers who are interested in getting expert answers on conceptual questions about software development. If you have a question about... """
1 - algorithm and data structure concepts
2 - design patterns <------
3 - developer testing
4 - development methodologies <------
5 - freelancing and business concerns
6 - quality assurance
7 - software architecture <------
8 - software engineering <------
9 - software licensing
So, Jarrod Roberson, GrandmasterB, gnat, Walter and ChrisF♦, you are all bozos, you shouldn't have closed this question.
The sad part is it's predictable. Upon seeing the link on HN, my thought process went like this: "Ahh, an interesting and thought-provoking discussion on a StackExchange site? It will be closed."
click
sigh
So, a question to our StackExchange overlords: you clearly do not believe that the popular StackExchange sites are the appropriate place for this content. With that in mind, what alternative would you recommend?
While I agree that very often interesting topics get closed, they DO HAVE a good reason for doing so.
The intent of all the stack-exchange sites is to provide a high-quality resource of _answers_ for specific _questions_. If "the answer" to a question ends up being a divergent discussion, then not closing responses invites more of these types of questions. Over time that will subvert the intent of stack-exchange.
The POV of the stack-exchange folks is that there are plenty of forums for discussion, and that is NOT what they want to do. Hacker-news, however, is more free-form and definitely much better for "need advice" type questions with highly subjective answers.
That said, it is true that moderators sometimes get persnickety about what they close. I wish that at least they would be less smug and judgmental about how they do that. But oh well, it is still an awesome resource.
It's really frustrating how they keep doing this. Their users are begging, literally fighting with the mods over this, and they still don't allow it. If they are opposed to diluting the technical questions there, why don't they just make an programmer opinion stack exchange site?
This reminds me of the founding of Adobe, when Steve Jobs told them what he wanted, as did two other companied, and the refused, saying "We have this business plan, we raised $2.5 million, and this is what we said we're going to do." Thankfully, a man named QT Wiles explained to them that "You guys are nuts. Throw out your business plan. Your customers-or potential customers-are telling you what your business should be. The business plan was only used to get you the money. Why don't you rewrite a business plan that is focused just on providing what your customers want?"
This is the same position. SE needs to focus on what the customers want.
Ditto for StackOverflow itself? I just can't understand why so often a really valuable topic that collects insightful answers gets closed up because of not fitting some template.
Also, I don't get the idea of "protected questions" - the ones marked by:
"This question is protected to prevent "thanks!", "me too!", or spam answers by new users. To answer it, you must have earned at least 10 reputation on this site."
What does this even mean? Is the assumption that a new user will always say "thanks!" / "me too", while older user won't, in any way correct? Do statistics suggest something like that? It always struck me as offensive to new users.
It might be off-topic for StackOverflow,but how on earth is this off-topic on programmers.stackexchange?! If such discussions are off-topic, then why did they create programmers.stackexchange in the first place?
Hey frankc! You're right Analysis paralysis is on topic on Programmers, the question however is not constructive, in the Stack Exchange sense. The close reason was a bit misleading (now changed), but at the end of the day closed means closed.
As to why it is not constructive: We don't like discussions. Ok, that's a lie, we love discussions, especially heated flamewars, after all we are programmers. However we haven't yet found a good way of moderating and curating such questions on Stack Exchange, we have an array of excellent tools at our disposal, none of which was designed with free flowing open ended discussions in mind.
This is more of a "it's not you, it's us" problem and it's a discussion we seem to be having at least once a week but the bottom line is, open ended free flowing discussions don't work on the platform, and unfortunately we have to close them.
Feel free to bring up any issues you have with Programmers on Meta, especially when it's about closed questions. Sometimes all it takes is a small edit to make the question a bit more focused for it to be re-opened...
86 people upvoted the question and 5 votes is all it takes to close the question ? So 5 people hold the truth on the validity an pertinence of a question ? How many people does it take to undesrstand that this moderation system is broken ?
This is one of the most pernicious forms of procrastination, because it's hard for smart people to accept that perfectionism is a negative trait. The more self-worth you derive from your previous successful outcomes, the more susceptible your ego becomes to fear of future failures. Your amygdala is telling you that you are successful already, so don't shake the boat. Tread carefully and you'll always have an excuse for why things don't work out. You'll never have to admit you just couldn't do it.
The solution is to divorce your self-worth from outcomes. Instead, derive self-worth from the process. We can have complete control over our actions, but never complete control over our outcomes.
Arrogance is probably the trait that most limits our intelligence. I've seen many smart people do extremely stupid stuff just out of fear of admitting they are prone to mistakes and failure. Humbleness is hard to master, but it's very important for us to try and practice it as much as possible.
Potential of success = (Intelligence / arrogance) + luck
This is a real thing for sure. The first 80% of a project is the fun/creative part. The last 20% is painful and usually where you make a product out of a project. Experienced developers know that starting a project means that ultimately you will have to do that 20% and they question whether it's worth the investment (before starting the fun part). New developers are usually blissfully ignorant and simply enjoy the first 80%. Some of them will take it beyond that... most will leave it as nothing more than an unfinished project.
The fondness of directing others is really just the enjoyment of not dealing with the minutia of developing a robust solution.
Very sad that the moderators are closing such interesting questions, because they do not conform to their opinion of what should go on the site, even though the topic is obviously hugely popular and appealing to a lot of users.
It's particularly outrageous because the given reason is "off topic for software development". It's quite clear from that judgment that the moderators don't know much about the topic and therefore are not qualified to hold a position of moderator on a board devoted to software development topics.
I submitted it because it's exactly how I feel lately, and would like to know what others think about it. My current (personal/academic) project involves Node.js, (Iced)CoffeeScript, MongoDB (with Mangoose), PhantomJS, Mocha, Neo4j, Nutch, Solr, Hadoop and Mahout.
All of them are new technologies and are changing rapidly and daily, and I find new stuff every day that forces me to change my APIs and the way I do things, and it's getting frustrating... It's the price you pay by using cutting-edge technologies and I accept that, but I can't help thinking that maybe Java+MySQL was a better/easier choice!
Experience has taught me that if you want to get something done, you need to stay off the bleeding edge, or at the very least only drag yourself to the bleeding edge at controlled increments.
The only thing that should be changing often is YOUR code. Everything else should be as stable as possible.
What I find works best is playing leap frog:
* your code is unstable / upgrade nothing
* your code is stable, stop changing it and upgrade things one at a time
I suffer from the same trouble on occasions. I primarily deal with the Microsoft ecosystem which is volatile and fragmented at the best of times.
I would kill to do something in good old java/mysql.
A lot of the new ideas and principles are a mess. Don't get too absorbed - just use what lets you get the job done quickly and efficiently. a lot of it is just noise.
I tend to find that a complete break from everything for a day helps. That and cider :)
A month ago I finished a project for university, involving Node.js - nothing more. At the beginning, I've started with Node.js, mongoDB, socket.io. express and connect. Half way through the project, I realized I would never be able to deliver in time. So I started to cut out technologies... first mongoDB, socket.io, express, and in the end, even connect. Even dealing with Node alone, I had a hard time finishing my project... but I did. I still got quite a good grade because in contrast to my fellow students, I was able to deliver.
My solution to this is a simple kanban board. Trello.com is a good on-line one, but when possible I'll make them out of index cards or post-it notes on the wall.
People think that a kanban board is a to-do list, but there's a subtle difference. The real heart of is limiting the number of work-in-progress items. That means that the backlog is less a to-do list and more a things-I-am-definitely-not-doing-now list.
For me, every time I have one of those "It would be nice" thoughts, I write it down on a card for the kanban, throw it in the backlog, and forget it until the next time I make a prioritization pass. Most of the cards end up in what I think of as "the pit of nice ideas", the bottom segment of the board that never moves up because more important things always come up.
For the curious, you can see a photo and a little more explanation here:
A situation I have seen with well-educated novice developers, is that they think of "technical debt" as universally bad. That's not true. It's not even true for actual debt.
Debt is only bad if it's your own long-term debt. One can also use short-term debt as a powerful tool.
For example, it might be simpler for one to write an initial iteration of a program that's badly factored. Then, once you get the program running correctly, refactor it. It's much easier to know you're on the right path if your program runs correctly, and you have verification data-sets or unit tests.
Modestly off topic: Can I say how much I have grown to hate StackOverflow and its cousins? Just like this article, half of my favorite threads on there are closed or have been deleted due to violating some sort of rule that is not unreasonable in theory but leads to obvious idiocy in practice.
A great example of people losing track of the forest because of all the damned trees in the way.
this is easier said than done, but the key to avoiding analysis paralysis is to worry about building the right abstractions and only that. if the software is abstracted well -- ie it correctly represents the underlying problem at an adequate resolution -- then it becomes easy to say YAGNI in the short run with the confidence that refactoring or adding functionality will be straight forward later.
The ideal to strive for is what we call the "Goldielocks effect".
Not knowing too little. Not knowing too much. Knowing just enough to get the job done.
In computing there will always be multiple solutions, and tradeoffs. What you want to aim for is "The simplest solution possible."
In my experience, this is really hard for most developers.
Maybe the smart thing to do is find someone who makes it look easy and trust their judgment. Again, my experience is that most developers are reluctant to do this.
The question is not how much someone knows, it's whether they know "enough" to get the job done. Goldielocks.
[+] [-] geophile|14 years ago|reply
- Don't generalize too early, which seems to address the original question pretty directly.
- Don't optimize too early.
- Maintainable > Correct > Fast. Maintainable = comprehensible. If it isn't maintainable, you won't know if it's correct. If it isn't correct, there's no point in making it fast.
- Minimize state. Deriving something from scratch every time it's needed leads to far fewer bugs, and the bugs there are probably far more obvious. Cache only when you need to for performance.
- Simplify as much as you can. This is the hardest one, because it can take a very long time to figure out all the ways in which a design that you thought was simple, turns out not to be.
- Simplify some more: The usual statement of the "simplify" rule is "as simple as possible, but no simpler". I don't think that's right. If you see a nice simplification at odds with requirements, push back on requirements.
[+] [-] mbenjaminsmith|14 years ago|reply
I don't think in terms of simplification though, I see it as "write what you need, when you need it." But I absolutely agree that you should avoid generalization aggressively and optimization should rarely be on the table. I also agree about minimizing state. Do it all "on the fly" unless there's known performance hit.
[+] [-] sklivvz1971|14 years ago|reply
[+] [-] frankc|14 years ago|reply
[+] [-] spolsky|14 years ago|reply
After a decade or so of operating sites for programmers on the Internet, I've learned that, left by themselves, ALL programmer online sites will rapidly deteriorate into:
* I hate my job, woe is me ("I hate my job too!")
* I'm not productive any more, oh dear ("Oh my god you took the words right out of my mouth!")
* And some nonsense about H1B visas.
These kinds of conversations are endless because, I'm sad to say, they represent a common problem among programmers. Discussing it as a means of therapy is a wonderful thing, but DO IT SOMEWHERE ELSE, because Stack Exchange is 100% focused on questions with specific factual answers.
This hard and fast rule is actually WHY you love stack overflow so much. It's WHY the network works. It's WHY you click on the stackoverflow.com link in the Google results even if it's third. If we weren't strict about this stuff we wouldn't have such consistently high quality ANSWERS.
Just because "moderators" (actually, users) have deemed to close a question on Stack Exchange, doesn't mean it's a bad question, or that the person who asked it is bad. It just means that Stack Exchange is an edited, curated environment, not your livejournal. The very moderation that people complain about every time a popular question is closed is what has been making the site work so well.
[+] [-] pooriaazimi|14 years ago|reply
I looked it up now[1] and it seems that this kind of questions are supposed to be asked on programmers.etackexchange.com!
This is what FAQ says. 2nd, 4th, 7th and 8th are exactly what this question is about!
""" Programmers — Stack Exchange is a site for professional programmers who are interested in getting expert answers on conceptual questions about software development. If you have a question about... """
1 - algorithm and data structure concepts
2 - design patterns <------
3 - developer testing
4 - development methodologies <------
5 - freelancing and business concerns
6 - quality assurance
7 - software architecture <------
8 - software engineering <------
9 - software licensing
So, Jarrod Roberson, GrandmasterB, gnat, Walter and ChrisF♦, you are all bozos, you shouldn't have closed this question.
[1]: http://programmers.stackexchange.com/faq
[+] [-] breckinloggins|14 years ago|reply
click
sigh
So, a question to our StackExchange overlords: you clearly do not believe that the popular StackExchange sites are the appropriate place for this content. With that in mind, what alternative would you recommend?
[+] [-] angdis|14 years ago|reply
The intent of all the stack-exchange sites is to provide a high-quality resource of _answers_ for specific _questions_. If "the answer" to a question ends up being a divergent discussion, then not closing responses invites more of these types of questions. Over time that will subvert the intent of stack-exchange.
The POV of the stack-exchange folks is that there are plenty of forums for discussion, and that is NOT what they want to do. Hacker-news, however, is more free-form and definitely much better for "need advice" type questions with highly subjective answers.
That said, it is true that moderators sometimes get persnickety about what they close. I wish that at least they would be less smug and judgmental about how they do that. But oh well, it is still an awesome resource.
[+] [-] michael_fine|14 years ago|reply
This reminds me of the founding of Adobe, when Steve Jobs told them what he wanted, as did two other companied, and the refused, saying "We have this business plan, we raised $2.5 million, and this is what we said we're going to do." Thankfully, a man named QT Wiles explained to them that "You guys are nuts. Throw out your business plan. Your customers-or potential customers-are telling you what your business should be. The business plan was only used to get you the money. Why don't you rewrite a business plan that is focused just on providing what your customers want?"
This is the same position. SE needs to focus on what the customers want.
[+] [-] TeMPOraL|14 years ago|reply
Also, I don't get the idea of "protected questions" - the ones marked by:
What does this even mean? Is the assumption that a new user will always say "thanks!" / "me too", while older user won't, in any way correct? Do statistics suggest something like that? It always struck me as offensive to new users.[+] [-] benjaminwootton|14 years ago|reply
It's a great site that fills a necessary niche but the moderation is killing it!
[+] [-] pooriaazimi|14 years ago|reply
[+] [-] yrizos|14 years ago|reply
As to why it is not constructive: We don't like discussions. Ok, that's a lie, we love discussions, especially heated flamewars, after all we are programmers. However we haven't yet found a good way of moderating and curating such questions on Stack Exchange, we have an array of excellent tools at our disposal, none of which was designed with free flowing open ended discussions in mind.
This is more of a "it's not you, it's us" problem and it's a discussion we seem to be having at least once a week but the bottom line is, open ended free flowing discussions don't work on the platform, and unfortunately we have to close them.
Feel free to bring up any issues you have with Programmers on Meta, especially when it's about closed questions. Sometimes all it takes is a small edit to make the question a bit more focused for it to be re-opened...
[+] [-] droithomme|14 years ago|reply
[+] [-] chmike|14 years ago|reply
[+] [-] Zarathust|14 years ago|reply
[+] [-] joshklein|14 years ago|reply
The solution is to divorce your self-worth from outcomes. Instead, derive self-worth from the process. We can have complete control over our actions, but never complete control over our outcomes.
This is far easier said than done.
[+] [-] vibrunazo|14 years ago|reply
Potential of success = (Intelligence / arrogance) + luck
[+] [-] hoopism|14 years ago|reply
The fondness of directing others is really just the enjoyment of not dealing with the minutia of developing a robust solution.
[+] [-] MortenK|14 years ago|reply
[+] [-] droithomme|14 years ago|reply
[+] [-] Aqua_Geek|14 years ago|reply
1. Make it work.
2. Make it good (visually or otherwise).
3. Make it fast.
[+] [-] arethuza|14 years ago|reply
[+] [-] betadj|14 years ago|reply
[+] [-] chmike|14 years ago|reply
[+] [-] slantyyz|14 years ago|reply
[+] [-] pooriaazimi|14 years ago|reply
All of them are new technologies and are changing rapidly and daily, and I find new stuff every day that forces me to change my APIs and the way I do things, and it's getting frustrating... It's the price you pay by using cutting-edge technologies and I accept that, but I can't help thinking that maybe Java+MySQL was a better/easier choice!
[+] [-] engtech|14 years ago|reply
The only thing that should be changing often is YOUR code. Everything else should be as stable as possible.
What I find works best is playing leap frog: * your code is unstable / upgrade nothing * your code is stable, stop changing it and upgrade things one at a time
test suites / regressions are a must.
[+] [-] gouranga|14 years ago|reply
I would kill to do something in good old java/mysql.
A lot of the new ideas and principles are a mess. Don't get too absorbed - just use what lets you get the job done quickly and efficiently. a lot of it is just noise.
I tend to find that a complete break from everything for a day helps. That and cider :)
[+] [-] gfosco|14 years ago|reply
Node being in its infancy is a pretty good reason in itself to not tie yourself to the frameworks which are likewise infantile.
I've got several production back-ends and a full-stack e-commerce site running with Node and MySQL only.
[+] [-] helper-method|14 years ago|reply
[+] [-] cheald|14 years ago|reply
A flawed-but-functional system that ships is empirically better than a perfect system that never ships.
[+] [-] TazeTSchnitzel|14 years ago|reply
[+] [-] wpietri|14 years ago|reply
People think that a kanban board is a to-do list, but there's a subtle difference. The real heart of is limiting the number of work-in-progress items. That means that the backlog is less a to-do list and more a things-I-am-definitely-not-doing-now list.
For me, every time I have one of those "It would be nice" thoughts, I write it down on a card for the kanban, throw it in the backlog, and forget it until the next time I make a prioritization pass. Most of the cards end up in what I think of as "the pit of nice ideas", the bottom segment of the board that never moves up because more important things always come up.
For the curious, you can see a photo and a little more explanation here:
http://www.quora.com/What-are-some-uncommon-ways-to-work-sma...
[+] [-] swalsh|14 years ago|reply
[+] [-] stcredzero|14 years ago|reply
Debt is only bad if it's your own long-term debt. One can also use short-term debt as a powerful tool.
For example, it might be simpler for one to write an initial iteration of a program that's badly factored. Then, once you get the program running correctly, refactor it. It's much easier to know you're on the right path if your program runs correctly, and you have verification data-sets or unit tests.
EDIT: Changed one short to long.
[+] [-] wpietri|14 years ago|reply
A great example of people losing track of the forest because of all the damned trees in the way.
[+] [-] rguzman|14 years ago|reply
[+] [-] Morg|14 years ago|reply
Analysis Paralysis is inevitable as long as you want to write the RIGHT thing.
As soon as you accept "good enough, we'll upgrade later if necessary", you start moving towards the right thing.
Anything that says perfect, adequate, right, correct, etc. is wrong and leads to the dark side.
[+] [-] Tloewald|14 years ago|reply
[+] [-] yrizos|14 years ago|reply
[+] [-] h84ru3a|14 years ago|reply
Not knowing too little. Not knowing too much. Knowing just enough to get the job done.
In computing there will always be multiple solutions, and tradeoffs. What you want to aim for is "The simplest solution possible."
In my experience, this is really hard for most developers. Maybe the smart thing to do is find someone who makes it look easy and trust their judgment. Again, my experience is that most developers are reluctant to do this.
The question is not how much someone knows, it's whether they know "enough" to get the job done. Goldielocks.
[+] [-] powertower|14 years ago|reply