I have a degree in computer science, but at the end of the day it's just a piece of paper - I learned programming mainly through the internet. I talked to people who cared enough about programming enough to hang out on IRC, or write technical blog posts, or write their own libraries, etc.
What has shocked me since entering the "real world" is how little most other programmers I meet care about programming, their craft, code quality, or learning new things. I was legitimately shocked at how good I was compared to the real world population, as opposed to the "internet programmer" population where I was mediocre at best.
This isn't a value judgment - not everyone is passionate about their day job. But it might resonate with someone out there.
One time, when leaving a company, I helped interview candidates for my old job, since I could best tell what skills are needed and to what extent.
I reckoned at the time, from most interactions on the Internet, that I am (globally) a mediocre, if not below-average, developer - in each discussion I participated online, there were always people above my level, from the ones having a slightly better understanding of the subject to the people whose depth of knowledge appeared somewhat otherworldly.
This belief was particularly strengthened by the fact that I didn't have much contact with real-world developers in real life, as my friends tended to come from different backgrounds.
When the interviews came, I was astonished how many of them couldn't code their way out of a paper bag. The ones who did, wrote code with messed up indentation, inconsistent style, ugly and hackish approaches, and the like. I couldn't imagine how someone could leave their code like that and say "there, this is my finished product." But here they were, and it seemed like they didn't really see anything particularly wrong, or just not important enough to justify the added effort of cleaning the mess up. It was a very strange moment for me.
I believe this to be an example of the classical selection bias: people inclined to participate in online discussions, answer questions on StackOverflow, write blogs and type out detailed technical comments on HN are on average a lot more skilled than the average just-earning-my-living developer fresh out of some half-assed "be a software engineer and make tons of money!" three-month course.
Yeah, I don't even have a CS degree. I was considering myself below average as long as I was working mostly alone or on specific domains, for I could not compare with others IRL. I could only read people on Usenet or on the Web and they often looked 10 times as knowledgeable as I was. Also, I could see that other professionals listed impressive job titles and definitions, so I was, yes, really impressed and couldn't see how I could ever match those advertised skills.
Then I got a job in a bigger company, where because of the nature of the job, I could easily compare with other people in a large team, in the rest of company, and in other companies; and I quickly noticed I was in the top 10% or perhaps even 5%. That was at the same time very surprising and... very appalling, because that meant that all the big titles of those guys were just kind of fake, empty. it felt a bit as if I had been deceived. Even though they had CS degrees, they had vast holes in basic programming know-how an in computer knowledge, and they didn't give a fuck about it, as little as they cared about doing a clean work.
So the people I used to read on Usenet or the Web were indeed exceptions and not the norm, and sometimes pretty much gurus in their domain. Anyway, they were not at all representative of the majority of developers that populate companies.
I agree with everything you say, but would also like to add that the level of passion and 'pride' which programmers have seems to vary by specialization. I work in embedded systems, where having well-factored code, with well thought-out scope is the anomaly, not the rule.
I have honestly seen the exact same thing. I was rating myself quite average or just below average. I found actually that as a web developer, if you have a general culture of lower level computing and can produce readable and maintainable code, you are already in the top 10%.
This agrees with my own experience. I learned programming with people that would do things like write their own graphics library, in Assembly language, or device drivers, just because they wanted to see if they could. I thought every programmer was like that in the professional world - with rare exceptions, they were not.
When I came out of college, some real life examples of people who do not seem to care at all which shocked me:
- CTO of a company I worked for (brother in law of the company director) typed with 2 fingers. No Joke
- Senior Software developer to nullify a list, he just creates a "new list" instead of setting it to "null"
End result from practical point of view is the same. But the lack of computer science insight baffled me.
- Team lead implements lines of code as a tool to measure productivity.
Being a new kid out of school, I felt too young to call bullshit but intuitively, I felt the metric was flawed.
This is interesting. I do think there can be a problem around the concept of "competent" vs "average". For instance, consider a poll like this: On a scale of 1-10, where 1 is inexperienced, 2 is competent, 3 is above average, and 4-10 are various levels of good-brilliant, where do you rank yourself?
My guess is that even with these explicit instructions, people are reluctant to rate themselves a 2. I can see a few reasons why higher scores would actually indicate that people see themselves as average. For instance, some people might view anything under 5 is incompetent. In this case, a 7.5 would indicate the middle of the range, which is actually 5-10. Alternatively, people might think of grades in US based schools, where 75% is a C, technically "satisfactory", but essentially failing in an era of grade inflation. In many PhD programs, an A- (91%), is a shot across the bow, and a B+, (88%), is meant as a vote of low confidence. This could lead many people to rate themselves as an 8 or 9 simply to indicate their averageness.
Lastly, programming is still a young field with a wide variance of talent, but I think we need to be more of a "paper hat" distribution than a "pyramid". To explain, I don't think we're a field that works well with a large, unskilled base, that slowly narrows to a tiny elite at the top. I think the baseline to be a competent developer is quite high. I know, I read about these "CRUD" apps, and what can I say? The framework churn, the challenge in communicating with clients who understand business but aren't familiar with the unforgiving logic of a program, the need to deal with deadline pressures under uncertainty, the challenges of data integrity and testing coverage... honestly, simply reliably deploying a relatively simple web app is considerable, in my opinion. This is why I go for the "paper hat" distribution - devs are largely on a long, flat curve, with a small little mini-pyramid in the middle, where the true innovators live. In short, maybe we are a field represented by wide, short rectangle of 1's-3's, with a minuscule steep and thin little triangle of 4-10s popping up at the top. However, the 1's really are smart people who are learning, and 2-3 represents quite a valuable and talented developer.
This seems like a really significant issue. First, because "5 as median" isn't a universal grading scheme, and 50% is often failing. Second, because this poll didn't specify what the numbers mean.
The bounds of "programming ability" are basically undefined. Does 1, being the lowest, mean "cannot program"? Does 10, pain-scale style, mean "best programmer imaginable"?
Or do the numbers represent deciles of skill? If so, do they include everyone who has written code? Only people who have completed a substantive project? Only people who have been "programmers" in a non-hobby setting? Only in a non-hobby, non-academic setting? Only people taking this quiz?
And worse, it's a meta question. We aren't guessing what the survey meant here, we're guessing what the respondents thought it meant - which means that the skew may be totally accurate under their chosen boundaries! Keynesian beauty contest and all that.
So this is an interesting result, but I can't really use it as data about self-reported skill, only about interpretation of surveys.
My own perceived rating significantly lowered with the rise of social networks. As you start to follow more closely some really great programmers out there, you stop comparing yourself with your peers and start comparing against the best in the game... and that is self-confidence killer :)
Move jobs a few times, inherit some crappy application and look at the code that others write, that has done the opposite for me.
Though a lot depends on what you define as being a good programmer. I used to be a lot better at clever algorithmic stuff. Now my focus is on clean maintainable code and getting the higher level architecture right, so that I don't need clever algorithmic stuff.
I'm always worried when I feel too good at something, I can't help but wonder whether there's an entire world of knowledge out there I'm not aware of (there usually is).
I also noticed my peers tend to consistently rate me 1-2 points higher than I do rate myself. I'm unsure if this is a good thing, if my peers are overestimating me, or if I'm underestimating myself.
I approach your last point differently however; I find it inspiring to look at people being at the top of their fields; makes it easier to plan a route to ramp up your knowledge to that level as the work has already been done.
Social media is deceiving in both directions: I have worked with a lot of nobodies and with big names with 10K+ twitter followers. Some of those well known programmers know a lot of trivia about obscure parts of computer science, but that doesn't mean that they are really all that productive when you look at their job contributions as a whole.
Other times the fame comes from being big external communicators: Giving a lot of talks about programming doesn't meant that your can code quickly, accurately, or that you can come up with good designs.
When we look at what it is for someone to be a great programmer and a good teammate, there's a whole lot of characteristics that are extremely hard to measure in a job interview. Do we really think it's easier to measure them by just hearing someone give talks or reading some articles? Do you really know that your favorite speaker writes good tests, cares about code readability, or will be there for you whenever you have a technical problem? Will they even work as much as you do, or do they have a contract that says that their speaking is their job, therefore leaving your team behind to do things less famous people have to do, like being on call for cyber monday, or making sure the new shiny system is actually operable in production?
Therefore, my personal experience working with a lot of the socially famous has given me a lot of respect for the great developers I have met that spend less time in self promotion, and more time working on their own craft and take care of their job and family obligations. I've worked with many that are as good, if not better, than the average famous developer I've work with. You just don't know about them.
If you're a little self-aware you can use the comparison as a source of inspiration and drive to better yourself instead of turning it into a self-confidence killer.
Even more importantly than that... is you are comparing yourself (presumably new/newish) to those who have spent years? decades? getting where they are at.
Behind every successful person is years of hard work. Behind every success is an untold number of failures and learning experiences.
You can't compare yourself to someone who has been making games for 20 years when you're learning.
I normally dislike working with survey data since there
is a high possibility of selection bias among
the respondents.[..] For this reason, I will
show confidence intervals whenever possible to
reflect the proportionate uncertainty for
groupings with insufficient data [..]
That... is not how statistics work(?). I mean – confidence intervals help with small sample sizes, but they do nothing for systematic errors such as those introduced by selection bias.
[continued from above] and to also account for
possibility that a minority of respondents may
be dishonest and nudge their programming ability
a few points higher than the truth.
There's a surprising amount of assumptions that went into this sentence. I'd question the assertions that:
- people are "dishonest" (My intuition would point to a subconscious bias more than actual dishonesty)
- It's a minority (The second chart shows that >50% of respondents with one year of experience or less rate themselves as better than average).
- That subconscious biases only work in one direction
... and once again I have no idea how confidence intervals can help. A large interval may indicate bad measurement. It may also indicate high variability in the actual data.
Also, keep in mind that these groupings alone
do not imply a causal relationship between
the two variables.
... someone paid attention in his middle-school statistics club...
Employing traditional regression analysis to
build a model for predicting programming ability
would be tricky: does having more experience
cause programming skill to improve, or does having
strong innate technical skill cause developers to
remain in the industry and grow?
... but failed statistics 101. A regression analysis doesn't care about causality. If "Mac users are more likely to be college-educated" it doesn't matter that "buying a Mac may not actually make you smarter". I can still make the prediction "a given Mac user is more likely to have a college degree".
Microaggressions aside, these are fair counterpoints. I spent far less time editing the body of the post than optimizing the visualizations/Jupyter Notebook (especially in this particular post). I've taken more care in future posts since.
There are many domains of expertise. Dealing with networking, graphics, sound, threads, filesystems, embedded platforms, compilers, cryptography, operating systems (which is in itself another world), distributed systems, databases, drivers, high performance computing, machine learning, etc.
Ask a senior network programmer to write you a shader and ask a graphics programmer to write you a production-strength multithreaded socket server...
Ask a senior web designer to write you a compiler from scratch, and ask a senior compiler engineer to write you a responsive website that renders correctly on all browsers.
Then, some programming abilities are rarer. There are more web developers than compiler engineers, the amount of jobs and the requirements for them are different.
What I am trying to get at is that software development comes on many flavors and it's really hard to touch all those surfaces in your career.
Looks exactly like driving competency. Over 93% of US drivers self-rate their driving as above average (even significantly above average). It's very frightening to see that 36% rate themselves as above average drivers whilst sending text messages.
I get this asked a lot and if I tell people "If Carmack and Torvalds are a 10 and someone who can write a hello world is a 1, then I would rate myself 5 max" and most just think I must be a bad programmer because of that rating.
Well even with those bounds, is that scale in terms of the population as a whole, some measurable notion of productivity, or in terms of some abstract concept of absolute skill? Depending on the definition, 5 could mean many different things.
Not to mention that there are different aspects of skill. Is Torvalds a good functional programmer? (I actually have no idea, so this might be a bad example, but hopefully I make my point.)
The consistent trend in unemployed / part time / full time is a little strange considering that many programmers value time to work on their own projects, and can gain new skills that are valuable to an employer. Seems like HR and cultural attitudes about employment are large factors in programmers view of skill.
The number of people that picked "never check in or commit" or only do it "once or twice a month" but that also rate themselves as a 7+ makes me a combination of sad and irritated.
Interesting analysis. I do wonder what a self rating for developers actually means: how do you define the difference between a high rating and a low rating? What is a great developer vs an average one?
The definition of competence wildly varies based upon who you ask. There are many experienced developers in the work force, whom I am sure would rate themselves high, who cannot do their jobs (at all) without their favorite frameworks and abstractions. Epic fail. They might rate themselves with an 8, but I would give them a 3.
Just the hick in the salary/ability plot at 160k-170k makes me wonder what's going on there. It might just be inaccuracy due to small sample size, as this range is probably an unusual salary. I guess people tend to gravitate more towards round numbers like 150k and 180k when negotiating salaries.
i'd like to see the same group of people asked to pick a random integer between 1 and 10 inclusive, and compare that distribution to what we see in the first graph.
Nice data analysis. Definitely a selection bias with respect to self-assessment - I am sure many are comparing to 'programmers' they know in the workplace who are not particularly strong and would seldom use a resource like S.O. (and consequently not respond to a survey like this).
This was the first thing that I thought of also - surveying Stack Overflow survey-takers about their place in the larger order of programming threatens sample bias in several ways (experience, community engagement, etc).
> As it turns out, there is no correlation between programming ability and and the frequency of Stack Overflow visits, as the averages and distributions are virtually identical across all groups.
As an infrequent SO participator, this is somewhat comforting to hear.
> As an infrequent SO participator, this is somewhat comforting to hear.
I'd assume that people who visit infrequently are better on average than those who visit often because it means you don't have to look things up. Why would you assume the opposite?
[+] [-] lacampbell|9 years ago|reply
What has shocked me since entering the "real world" is how little most other programmers I meet care about programming, their craft, code quality, or learning new things. I was legitimately shocked at how good I was compared to the real world population, as opposed to the "internet programmer" population where I was mediocre at best.
This isn't a value judgment - not everyone is passionate about their day job. But it might resonate with someone out there.
[+] [-] thegeomaster|9 years ago|reply
I reckoned at the time, from most interactions on the Internet, that I am (globally) a mediocre, if not below-average, developer - in each discussion I participated online, there were always people above my level, from the ones having a slightly better understanding of the subject to the people whose depth of knowledge appeared somewhat otherworldly.
This belief was particularly strengthened by the fact that I didn't have much contact with real-world developers in real life, as my friends tended to come from different backgrounds.
When the interviews came, I was astonished how many of them couldn't code their way out of a paper bag. The ones who did, wrote code with messed up indentation, inconsistent style, ugly and hackish approaches, and the like. I couldn't imagine how someone could leave their code like that and say "there, this is my finished product." But here they were, and it seemed like they didn't really see anything particularly wrong, or just not important enough to justify the added effort of cleaning the mess up. It was a very strange moment for me.
I believe this to be an example of the classical selection bias: people inclined to participate in online discussions, answer questions on StackOverflow, write blogs and type out detailed technical comments on HN are on average a lot more skilled than the average just-earning-my-living developer fresh out of some half-assed "be a software engineer and make tons of money!" three-month course.
[+] [-] wott|9 years ago|reply
Then I got a job in a bigger company, where because of the nature of the job, I could easily compare with other people in a large team, in the rest of company, and in other companies; and I quickly noticed I was in the top 10% or perhaps even 5%. That was at the same time very surprising and... very appalling, because that meant that all the big titles of those guys were just kind of fake, empty. it felt a bit as if I had been deceived. Even though they had CS degrees, they had vast holes in basic programming know-how an in computer knowledge, and they didn't give a fuck about it, as little as they cared about doing a clean work.
So the people I used to read on Usenet or the Web were indeed exceptions and not the norm, and sometimes pretty much gurus in their domain. Anyway, they were not at all representative of the majority of developers that populate companies.
[+] [-] nickff|9 years ago|reply
[+] [-] realusername|9 years ago|reply
[+] [-] SeanDav|9 years ago|reply
[+] [-] kayman|9 years ago|reply
- CTO of a company I worked for (brother in law of the company director) typed with 2 fingers. No Joke
- Senior Software developer to nullify a list, he just creates a "new list" instead of setting it to "null" End result from practical point of view is the same. But the lack of computer science insight baffled me.
- Team lead implements lines of code as a tool to measure productivity. Being a new kid out of school, I felt too young to call bullshit but intuitively, I felt the metric was flawed.
[+] [-] k__|9 years ago|reply
I'm really bad at Math, but CS helped me to understand FP and algorithms a bit more.
[+] [-] geebee|9 years ago|reply
My guess is that even with these explicit instructions, people are reluctant to rate themselves a 2. I can see a few reasons why higher scores would actually indicate that people see themselves as average. For instance, some people might view anything under 5 is incompetent. In this case, a 7.5 would indicate the middle of the range, which is actually 5-10. Alternatively, people might think of grades in US based schools, where 75% is a C, technically "satisfactory", but essentially failing in an era of grade inflation. In many PhD programs, an A- (91%), is a shot across the bow, and a B+, (88%), is meant as a vote of low confidence. This could lead many people to rate themselves as an 8 or 9 simply to indicate their averageness.
Lastly, programming is still a young field with a wide variance of talent, but I think we need to be more of a "paper hat" distribution than a "pyramid". To explain, I don't think we're a field that works well with a large, unskilled base, that slowly narrows to a tiny elite at the top. I think the baseline to be a competent developer is quite high. I know, I read about these "CRUD" apps, and what can I say? The framework churn, the challenge in communicating with clients who understand business but aren't familiar with the unforgiving logic of a program, the need to deal with deadline pressures under uncertainty, the challenges of data integrity and testing coverage... honestly, simply reliably deploying a relatively simple web app is considerable, in my opinion. This is why I go for the "paper hat" distribution - devs are largely on a long, flat curve, with a small little mini-pyramid in the middle, where the true innovators live. In short, maybe we are a field represented by wide, short rectangle of 1's-3's, with a minuscule steep and thin little triangle of 4-10s popping up at the top. However, the 1's really are smart people who are learning, and 2-3 represents quite a valuable and talented developer.
[+] [-] Bartweiss|9 years ago|reply
The bounds of "programming ability" are basically undefined. Does 1, being the lowest, mean "cannot program"? Does 10, pain-scale style, mean "best programmer imaginable"?
Or do the numbers represent deciles of skill? If so, do they include everyone who has written code? Only people who have completed a substantive project? Only people who have been "programmers" in a non-hobby setting? Only in a non-hobby, non-academic setting? Only people taking this quiz?
And worse, it's a meta question. We aren't guessing what the survey meant here, we're guessing what the respondents thought it meant - which means that the skew may be totally accurate under their chosen boundaries! Keynesian beauty contest and all that.
So this is an interesting result, but I can't really use it as data about self-reported skill, only about interpretation of surveys.
[+] [-] ivanhoe|9 years ago|reply
[+] [-] collyw|9 years ago|reply
Though a lot depends on what you define as being a good programmer. I used to be a lot better at clever algorithmic stuff. Now my focus is on clean maintainable code and getting the higher level architecture right, so that I don't need clever algorithmic stuff.
[+] [-] jeremiep|9 years ago|reply
I also noticed my peers tend to consistently rate me 1-2 points higher than I do rate myself. I'm unsure if this is a good thing, if my peers are overestimating me, or if I'm underestimating myself.
I approach your last point differently however; I find it inspiring to look at people being at the top of their fields; makes it easier to plan a route to ramp up your knowledge to that level as the work has already been done.
[+] [-] hibikir|9 years ago|reply
Other times the fame comes from being big external communicators: Giving a lot of talks about programming doesn't meant that your can code quickly, accurately, or that you can come up with good designs.
When we look at what it is for someone to be a great programmer and a good teammate, there's a whole lot of characteristics that are extremely hard to measure in a job interview. Do we really think it's easier to measure them by just hearing someone give talks or reading some articles? Do you really know that your favorite speaker writes good tests, cares about code readability, or will be there for you whenever you have a technical problem? Will they even work as much as you do, or do they have a contract that says that their speaking is their job, therefore leaving your team behind to do things less famous people have to do, like being on call for cyber monday, or making sure the new shiny system is actually operable in production?
Therefore, my personal experience working with a lot of the socially famous has given me a lot of respect for the great developers I have met that spend less time in self promotion, and more time working on their own craft and take care of their job and family obligations. I've worked with many that are as good, if not better, than the average famous developer I've work with. You just don't know about them.
[+] [-] Cyph0n|9 years ago|reply
[+] [-] rezashirazian|9 years ago|reply
[+] [-] wernercd|9 years ago|reply
Behind every successful person is years of hard work. Behind every success is an untold number of failures and learning experiences.
You can't compare yourself to someone who has been making games for 20 years when you're learning.
[+] [-] matt4077|9 years ago|reply
- people are "dishonest" (My intuition would point to a subconscious bias more than actual dishonesty)
- It's a minority (The second chart shows that >50% of respondents with one year of experience or less rate themselves as better than average).
- That subconscious biases only work in one direction
... and once again I have no idea how confidence intervals can help. A large interval may indicate bad measurement. It may also indicate high variability in the actual data.
... someone paid attention in his middle-school statistics club... ... but failed statistics 101. A regression analysis doesn't care about causality. If "Mac users are more likely to be college-educated" it doesn't matter that "buying a Mac may not actually make you smarter". I can still make the prediction "a given Mac user is more likely to have a college degree".[+] [-] ASpring|9 years ago|reply
[+] [-] minimaxir|9 years ago|reply
Microaggressions aside, these are fair counterpoints. I spent far less time editing the body of the post than optimizing the visualizations/Jupyter Notebook (especially in this particular post). I've taken more care in future posts since.
[+] [-] partycoder|9 years ago|reply
There are many domains of expertise. Dealing with networking, graphics, sound, threads, filesystems, embedded platforms, compilers, cryptography, operating systems (which is in itself another world), distributed systems, databases, drivers, high performance computing, machine learning, etc.
Ask a senior network programmer to write you a shader and ask a graphics programmer to write you a production-strength multithreaded socket server...
Ask a senior web designer to write you a compiler from scratch, and ask a senior compiler engineer to write you a responsive website that renders correctly on all browsers.
Then, some programming abilities are rarer. There are more web developers than compiler engineers, the amount of jobs and the requirements for them are different.
What I am trying to get at is that software development comes on many flavors and it's really hard to touch all those surfaces in your career.
[+] [-] pascalmemories|9 years ago|reply
There's even a name for it, Illusory Superiority https://en.wikipedia.org/wiki/Illusory_superiority
edit: added some of the latest figures from the Wikipedia article.
[+] [-] emodendroket|9 years ago|reply
[+] [-] k__|9 years ago|reply
[+] [-] aninhumer|9 years ago|reply
Not to mention that there are different aspects of skill. Is Torvalds a good functional programmer? (I actually have no idea, so this might be a bad example, but hopefully I make my point.)
[+] [-] irq-1|9 years ago|reply
[+] [-] gregmac|9 years ago|reply
[+] [-] coldcode|9 years ago|reply
[+] [-] sytelus|9 years ago|reply
As it turns out, there is no correlation between programming ability and and the frequency of Stack Overflow visits
[+] [-] callesgg|9 years ago|reply
I wonder if Indians and people from other lower wage countries feel the same about my own salary.
[+] [-] hood_syntax|9 years ago|reply
[+] [-] austincheney|9 years ago|reply
[+] [-] tbrownaw|9 years ago|reply
[+] [-] edblarney|9 years ago|reply
But then - in their context, they wouldn't see knowing other languages as meaningful.
So it really depends on a whole lot of context.
[+] [-] blauditore|9 years ago|reply
Just the hick in the salary/ability plot at 160k-170k makes me wonder what's going on there. It might just be inaccuracy due to small sample size, as this range is probably an unusual salary. I guess people tend to gravitate more towards round numbers like 150k and 180k when negotiating salaries.
[+] [-] skylark|9 years ago|reply
[+] [-] pixeloution|9 years ago|reply
The engineer with a compensation package worth 150k in Silicon Valley isn't the same as the guy with the same package in Austin.
[+] [-] sigstoat|9 years ago|reply
[+] [-] alexpetralia|9 years ago|reply
[+] [-] Bartweiss|9 years ago|reply
[+] [-] unknown|9 years ago|reply
[deleted]
[+] [-] unknown|9 years ago|reply
[deleted]
[+] [-] Terr_|9 years ago|reply
As an infrequent SO participator, this is somewhat comforting to hear.
[+] [-] rifung|9 years ago|reply
I'd assume that people who visit infrequently are better on average than those who visit often because it means you don't have to look things up. Why would you assume the opposite?