top | item 1992433

Web Programming Is Hard

258 points| grayhairmomma | 15 years ago |shubharamani.com | reply

154 comments

order
[+] patio11|15 years ago|reply
I periodically think "Man, I really shouldn't be getting paid for this. Any idiot could stick these two APIs together."

In my more sensible moments, I remember that that idiot only needed to know HTML, CSS, the DOM model, Javascript, jQuery, HTTP (mostly headers and status codes), Ruby, Rails, basic MVC design, Oauth, SQL, SQL performance, five-ish APIs to support the two that needed integration, how to configure and maintain a server, system architecture, a bit of security, etc.

[There's also object oriented programming, class inheritance, polymorphism, algorithms, time complexity, discrete math, data structures, imperative programming, and a few other things I forgot the first time around. Lest we forget, we might take these for granted because we've been programming since we were $ARBITRARILY_YOUNG, but to most people these are just black magic. My girlfriend, a smart cookie, asked to see "how I made the phone ring" and, after I showed her the code that did that, told me it had never even occurred to her that every program she's ever seen was once a collection of special words placed in a particular order.]

After that, it was pretty much done, except for the marketing.

On the plus side: you can learn one or a few of these things at a time, and get more comfortable with how deep the rabbit hole goes as you go along. I coded my website in static HTML written in Notepad with no JS or CSS, crikey, only four years ago. (And my brain still recoils at how much better I'd have to get to do e.g. web scale work.)

[+] mechanical_fish|15 years ago|reply
It's like maintaining a skyscraper built out of Tinker Toys.

What I wonder is: Does the following sentence run through the head of every expert in every field?

Man, I really shouldn't be getting paid for this. Any idiot could [do X].

It certainly felt true of most of my day-to-day work in graduate semiconductor electronics. It felt true of a lot of biology research. My conclusion is that there are a mere handful of days in your career when you will have to do something so hard that you will feel smart about it. But most of your career as an expert will be spent doing stuff that you know so well that it's kind of routine, even if it is totally esoteric stuff that only you know how to do.

Feynman said it well when he teased the mathematicians: "Mathematicians can only prove trivial theorems, because any theorem, once proved, is immediately seen to be trivial."

[+] davidw|15 years ago|reply
I think the comparison was with embedded programming, which can be pretty hard in its own right.

Also, with lots of embedded stuff, you don't get the "do over" of being able to update your server with the latest code. Once it has shipped, it had better work. In some cases, people's lives may directly depend on it, something that's probably fairly rare in the world of web programming.

I'd hire someone good with C pretty quickly, if I had the time to let them get ramped up, and they demonstrated the flexibility and willingness to learn.

[+] kellishaver|15 years ago|reply
My husband's just starting to get into web development in the hopes of being able to do it with me in a few months. He's a C programmer from years back and has a great head for sorting out and designing how an app should run. His theory is solid, and he has real programming experience to back it up.

I've been bringing him up to speed on the web stuff, though, and he's said the main thing that gets him is that he can't just learn a language and build an app. He's got to learn HTML, CSS, JavaScript (the DOM, jQuery), SQL, PHP/Ruby/Python and any accompanying framework we choose to use, etc. and piece it all together.

It's not that he finds any one of them particularly difficult (he's picking up on them quickly), it's just that there are so many pieces to the puzzle that all need to be understood and fitted together (and they often inter-mingle so much that you can't always separate them out and do one, then the other). It's just a large and varied volume of things to process.

[+] mcantor|15 years ago|reply
I bet 90% of the time, any "expert" is working on things that any idiot can do. But that isn't why people pay experts!

I think experts are paid so much because of the other 10% of the time. When convoluted edge conditions have made a mockery of all things abstract and sensible, they are capable of diving into the black boxes and figuring out what's really going on to save the day.

[+] kingsidharth|15 years ago|reply
Being a web-programmer - those C and GUI stuff looks hard to me. The grass looks greener on the other side, the tech looks harder on the other side. I am humbled too.reply
[+] okaramian|15 years ago|reply
It's interesting, I definitely see myself thinking the same thing (which may lead to myself undervaluing my skills because "any old dummy!" can learn this stuff). There are days where I've joked where I'm a "glorified typist" :)

I think in order to become more and more skilled you tend to abstract certain things out and take them as a given. Basically you know how things work, you just never think about them.

I remember when my company had an intern, and his only real education was college based (he hadn't really worked on any of his own projects). I had to explain how our stack worked, and generally these are a given when you're working with the stuff. But to someone who has never worked on this stuff it's very odd and unfamiliar. I also think that in order to build a good product for someone it helps to get to that point. I don't really need to spend a lot of time thinking about implementation these days and can focus on what would make clients happy.

To users, this stuff just works and I forget that sometimes because I'm in the weeds.

[+] kmfrk|15 years ago|reply
I think the people I have the most respect for are the people who master cross-browser CSS going back to IE7.

Is there another equivalent when it comes to a somewhat popular technology where quirky fixes are actually the main part of the work process?

[+] wisty|15 years ago|reply
You forgot the architecture needed to sustain a development enviornment - the VCS, bug tracking, deployment (ssh? shell scripts? something more modern?), and so on. But that's probably because it's all so obvious.
[+] mistermann|15 years ago|reply
"only needed to know HTML, CSS, the DOM model, Javascript, jQuery, HTTP (mostly headers and status codes), Ruby, Rails, basic MVC design, Oauth, SQL, SQL performance, five-ish APIs to support the two that needed integration, how to configure and maintain a server, system architecture, a bit of security, etc."

You should talk to my new boss, who just tore a strip off my ass for not getting somethings done in 3 days, that took another team in the company 3 months. Oh, you just have to "do this", no problem. Invoke Magic, upon 10 year old hardware.

[+] dasil003|15 years ago|reply
As a web developer I appreciate the concession from an embedded systems programmer that I am in fact not simply a monkey, but the quoted section probably goes a bit too far.

For instance, terrible UIs on the web are just as common as terrible UIs in GUI applications. And making a cross-platform GUI application that looks good on all platforms requires familiarity with tools specific to each platform, which is much more involved than learning browser quirks.

The hardest thing about the web is probably how fast it moves, but that's also what makes it exciting.

[+] andywood|15 years ago|reply
I did C and C++ programming for more than 10 years, before suddenly delving into serious web development only 3 years ago. I soon realized that I had vastly underestimated how deeply technical web development is.

But the reason was simply that I had formed my opinion back in ~1998, when "web development" really did mean just HTML, and maybe some roll-over script, most of the time! Being a systems programmer, I simply never had occasion to update my perception, as "web development" dramatically evolved while I wasn't looking.

[+] ibagrak|15 years ago|reply
I am not sure which particular flavor of "embedded" the author is referring to, but the argument, in my view, is completely off the mark, and here is why:

The main challenge of embedded is that you are in charge of controlling and managing everything. The developer is closer to the hardware and the cost of things breaking is far higher because you cannot for one second forget about some part of the technology stack that's below you. Everything is suspect, and nothing can be ignored.

On the web, you get used to your automatic garbage collection, your GUI toolkits, your nice browser sandbox with infinite memory that gets automagically replenished, your in-browser JavaScript debugger and all that jazz. You get to enjoy none of it in the embedded systems I've worked with.

Fixing embedded systems in the field is freaking hard, so the quality of code matters is in a completely different dimension from what's on the web today. Now, I am not talking about Linksys router or your iPhone. That's just a fraction of "embedded" devices. I am talking about things that don't have a TCP/IP stack (or any connection to the outside world), don't have a GUI, and are installed at the contractor rate of $1,000/hr and must exist in the field for 10-20 years. There are millions of these devices shipping every month, and they are all around us.

Are you telling me these systems are easier to design than a webpage that can be twiddled with at your whim a million times a day?

Don't downvote just yet! Just so you know where I am coming from:

I am a product manager for a 100K LOC embedded stack that runs in 128K of flash and 8K of RAM. It's all C code, no OS, no toolkits, no MMUs, no garbage collection and no dynamic memory allocation. These devices get 15 years on a single battery and go inside your house. I've also had good exposure (not an expert) to the online technologies the author mentions. Yes, things may be tough to learn (I don't actually believe they are), but the web is a lot more forgiving of mistakes too.

[+] Murkin|15 years ago|reply
You won't get much upvotes as the vast majority of HN readers appear to come from the Web world.

And as someone who made the switch from embedded to web,I find web much easier in 90% of the time.

Sure its hard to learn CSS, but I want to see you learn how to handle Linux's buffer-heads or program a custom dma chip.

[+] smutticus|15 years ago|reply
I work for an embedded company doing web development and tool chain support for the embedded product. So I kind of have my feet in both buckets as it were.

You're right in that the main difference is control. In the embedded world the developer has total control over everything. And the specifications for the product are very well defined.

The problem with the web world is that the specs are terribly defined or ignored and you depend on APIs that constantly change. Not having to worry about memory management is great in the web world. But instead you get to worry about how your page will render in 5 different browsers that each have decided to only partially implement some standard. At least in the embedded world if a piece of HW doesn't conform to spec it's defective and gets replaced. And you know exactly what that spec is.

I don't know which is worse. Being forced to write everything yourself or depending on the sometimes mediocre code of others?

[+] grayhairmomma|15 years ago|reply
No doubt, your embedded world presents incredibly difficult problems -- arguably much trickier ones than most web jobs out there. Keep in mind, as the OP and author, I'm comparing the "average" embedded job or desktop software job to the "average" web job. Perhaps I should have made this distinction clearer in the blog article. Most employed embedded engineers do not work on problems as tough as the ones which you describe.

But to me, the biggest challenge with web is the unforgiving avalanche of emerging new technologies a practicing professional must keep abreast of, literally during every spare second of their life, if he or she is to remain employed !! This unrelenting blizzard of new algorithms, new languages, new frameworks, new design methodologies is just another typical day in the life of a highly competent and employable web engineer.

The resource-limited embedded problems at your company are "old school". I'm not saying they're easy, but do the tools and technologies needed to solve such problems change much through the years ?

[+] verysimple|15 years ago|reply
One common mistake when one tries to tackle the web is to do it all at once. What you need:

- get familiar with HTML and css. That is, learn the basics, their purpose and how they interact.

- get familiar with some basic JavaScript and how you use it on the browser.

- learn one of the prominent web languages for the server-side. Python, Ruby, PHP, etc. I used to recommend PHP as a first language because compared to other languages, it was ubiquitous amongst hosting providers. Nowadays, I recommend against, especially if you already have some programming experience. Python and Ruby also have a decent offering and they have the added benefit of a community that generally promotes better programming practices than PHP.

Having an overview of the entire development process, you should now be able to pick one area where you'd like to expand. Being a programmer I suspect you might pick either server-side or browser scripting (JavaScript). Stick to one at first and learn it well. When I started the web I rarely did any front-end at all. I concentrated on the server-side and was aided by some CSS and JavaScript coders. Likewise, I often worked with JS programmers who didn't want to know anything beyond the realm of what they were doing. It's a symbiotic relationship. - It may be tempting to do everything vanilla at first, but quickly switch to using a framework, they're often packed with lots of best practices. They're like training wheels, you can always take them off later when you feel confident.

As you get comfortable with one field you can expand on others. After years playing in the server, I'm only now expanding my client side skills. Also, beyond technologies, other areas of interest that can expand your overall understanding and web expertise, are interface architecture, usability and various other optimizations. As you go, you'll stumble upon many.

[+] alanh|15 years ago|reply
That said, you need to have a firm grasp of all the basic security concepts before rolling out production code. You need to know how to avoid all the various injections.
[+] nathanwdavis|15 years ago|reply
I would add one thing: get to know HTTP. Even a basic understanding will prevent you from making costly mistakes.
[+] jemfinch|15 years ago|reply
Yes, it's hard. Unnecessarily hard, which is why I hate when I have to do it.

The complexity of web development is not intrinsic to the problem, but an extrinsic reality imposed by widely differing implementations of a number of overengineered technologies.

[+] jordanlev|15 years ago|reply
Well, if you expand your definition of "the problem" to "how do we enable people all over the world to interact with this information on any computing platform over an untrustworthy connection running on low-end hardware", then I think the chaos and technology soup is a little more necessary to things being where they're at now. So no, it's not technically necessary, but politically/economically it is.
[+] newobj|15 years ago|reply
First, she thought implementing a linked list during an interview was hard. Then, she says that web programming is hard. And I'm not saying it's "easy", but talk about blogging yourself into a professional grave?
[+] alex_c|15 years ago|reply
Personally, I find it refreshing to read a blog post that isn't - either directly or indirectly - about how amazing the author is.

Programming blogging has a strong culture of focusing on the newest, shiniest, and cleverest; the only negative blog posts tend to be "why X is wrong". No one blogs about their difficulties, weaknesses, doubts, or the huge amount of effort it takes to get really good at something. I think that's unfortunate, because it paints an inaccurate image of programming.

[+] grayhairmomma|15 years ago|reply
I never said linked lists were hard. I said that it's not necessary to know how to reverse a linked list from memory -- from scratch. I can look it up and get to it right away.
[+] vegai|15 years ago|reply
Talking like that attracts employers who are not idiots, many people want to work with people like that.
[+] edw519|15 years ago|reply
But upon studying CSS, HTML and Javascript over the last month, I can honestly say I’m humbled.

About half the time, I feel humbled whenever I encounter some new technology.

The other half, I wonder why anyone bothered.

The trick is knowing which half is which.

[+] PhrosTT|15 years ago|reply
Web Programming is nigh impossible for 1 person any more.

To build a legit website:

1. Mockup a pretty design in photoshop. Use color theory, design principles, UX theory, typography skills, etc. 2. Convert the design into HTML/CSS. Make it degrade gracefully, it should be cross-browser compatible, validate, be lightweight, meet accessibility standards. 3. Add unobstrusive javascript if you want, site should work without it. 4. Run YSlow, convert all images into sprites where possible, condense/minify your JS/CSS. 5. Do SEO tweaks and best practices. 6. Oh wait, does the site display well on iPads? build a mobile version and a tablet version.

...

We haven't even gotten past the front end. Learn good db design, code your middle tier. Choose or roll your own framework. TONS of work.

Once it works, go back and secure everything for the OWASP10 and other potential holes. Also make sure it will scale gracefully.

Maybe you should optimize your cacheing scheme? Maybe tweak your php config so it runs faster.

AGhghghsdhdshf%@!$^#$ it never ends!

[+] gbog|15 years ago|reply
With all due respect, putting OWASP at the end is a big mistake, you may have to refactor a lot to secure your code.
[+] pocoloco|15 years ago|reply
I'm currently working on a set of SOAP web services using tomcat, cxf, mysql, and hibernate among others.

I decided to do a switch from embedded to web development about a year and a half ago after being laid off. My paycheck is smaller but I now work on a whole new set of problems. After working about 10 years in embedded and everything around it, I felt that I needed a change. It was as if I was solving the same problems over and over. And don't get me started in the state of the tools. I remember thanking the heavens when we switched platforms to PowerPC and ELDK.

At first I did not know were to start in web development. But I did decide to concentrate on the back end quite early. At first I approached each technology separately, mostly because of my ignorance. For example, I saw that tomcat was very popular, so I decided to take a look into it. But I quickly realized that I needed a birds eye view of the whole web service stack and not its individual components. At least not yet.

I started to look into frameworks. After realizing that there are lots of those and that I learn about a new one almost every week, I had to narrow my search. I've been working and learning Groovy on Grails which is all based on the JVM since then. What sold it to me was the fact that Groovy is a language very similar to Python, which I know, and that Grails is a web framework that integrates all the necessary technologies to get a decent site up and running thanks in big part to the amount of plugins available for it.

Like grayhairmomma says, it's a humbling experience.

[+] rbranson|15 years ago|reply
Working for any primarily software driven company is going to be challenging. Working in an expensed IT department is what most developers do, and the work is pretty pedestrian. This seems to provide the most clear line of delineation in software development.
[+] MarkPNeyer|15 years ago|reply
They're hard for different reasons. In my experience, games, scientific computing, trading, and 'real' web backend work (i.e. not 'move this shit into the database' but 'make the database scale to a million users') are hard because they challenge you to solve difficult problems. Web frontend and simple backend work are hard because of the pain in the ass of managing all the different languages and paradigms involved.
[+] arohner|15 years ago|reply
Yes, Web Programming can be hard, but it's probably harder than it needs to be. The whole ecosystem of HTML + CSS + JS + IE6 is a mess.

Something along the lines of Cappuccino will help a lot. Then you're "only" left with the problems of UI/UX, scaling, security, A/B testing, big data and marketing.

[+] jordanlev|15 years ago|reply
Until you want to do something new that Cappuccino developers didn't think of... now your ecosystem has expanded by 1 to HTML + CSS + JS + IE6 + Cappuccino :)
[+] iamwil|15 years ago|reply
It's not the sort of thing we should have pride in--that web programming is hard. For me, it goes to show that web front-end programming has a long way to go to make it better for developers.

We're still looking for X to do to front end web programming that Rails did for back end web programming.

[+] wh-uws|15 years ago|reply
I believe Jquery is doing at least some of that
[+] tibbon|15 years ago|reply
Conversely, I think traditional programming seems really hard compared to RoR. I have to think about memory? I have to tell the computer what the variable type is? What are all these funny characters and symbols? I can actually crash the computer with this? What is this compiler thing and all these funny options?

I'm (slowly) trying to re-learn C/C++ after getting decent with Ruby and honestly it just hurts. Few things I do need speed or system access on this level. The verbosity is painful, and you need so much (ugly code) to do so little.

[+] jallmann|15 years ago|reply
Front-end web work, at its basic level, is not hard in the traditional "let's go shopping!" sense, but what I find maddening is how unintuitive some of it is (CSS, I'm looking at you). Then you have to worry about how broken or incompatible browsers are. I think "pain in the ass" is a more apt term.

When you get deeper into the UX, things like A/B, optimizing load time, scaling the backend, etc, and that is where it becomes less tedious and the domain knowledge required is more respectable.

[+] jarin|15 years ago|reply
It seems like there's always been a little bit of a rivalry between compiled language developers, web developers, and network administrators. Having done all three at high levels, I can say that in my experience web development (top to bottom) requires the most diversity of knowledge, which also makes it the most enjoyable to me. At any rate, I'm going to show this to my game developer buddies next time they give me crap about "working with toy languages"!
[+] Aegean|15 years ago|reply
Web programming is hard but not necessarily intuitive. That's what I dislike about it. Its a heap of different technologies put together in unstructured fashion. This applies to individual technologies as well, take CSS and see how unintuitive a syntax it has.

Web programming is popular because it drives the web. I don't necessarily find it an intuitive programming experience. This way of thinking is the biggest obstacle ahead of the likes of me (hackers for pleasure) for business success, i.e. primarily focusing on what's interesting from an engineering perspective. This approach is sure to fail in business because people always care about the end result. Technology is just a tool to get there.

I guess part of the problem is when you study computer science or engineering in university, you try to learn the best engineering approaches out there and improve on your engineering skills. Then you face real life where the result matters. I cannot really say CSS or javascript is the best technology out there, but they play a crucial part driving the web.

[+] petercooper|15 years ago|reply
It was funny to read this as I now find client side development ridiculously hard (I last did it frequently > 10 years ago) and difficult to get my head into, whereas even large scale Web development feels quite surmountable or even easy to me. I think it merely proves that what you keep doing, you eventually find natural.