Ask HN: How did web development become so bizarrely complex?

[mathewsanders]

I’m recently joined a new team where we are tasked with building out a website with around 15-20 unique pages. There are 3 main types of pages, and the majority of them are lists of links to external PDF documents.

We’re using a headless CMS with content in the page broken into hundreds of individual elements, where a React client app renders the data from the CMS API to our users.

In addition we seem to be grabbing the PDFs from the server, converting to Base64 and then rendering as an object in the react app (rather than just linking to the pdf file directly).

The team (4 developers, 1 automation/QA, 3 designers, scrum expert, product owner and product manager) is in their second quarter working on the site, and aiming to release this MVP next quarter so around 9 months to build 15-to-20 page site.

When I started my career 20 years ago, this is the type of project where maybe 2-3 people would build out over a few weeks.

When Content Management Systems were first introduced, one of their selling points was that it reduced reliance on developers for ongoing BAU content updates as ‘business people’ could make content updates and potentially even some sort of workflow to have reviewed or signed off (perhaps by compliance etc) with our set up the CMS is so complex that we’ll have any requests for changes submitted to team and have a developer make the update.

What benefit do we get from this complexity compared to just having static HTML pages that someone would update manually?

There are no concerns from our project sponsors around how long this is taking or the costs involved.

How on earth did we get to this stage where such bizarre complexity is accepted as normal?!

[yongjik]

My half cynical take is "resume driven development." The more different tools one uses, the better they can pad their resume, and resumes don't say 90% of these tools weren't needed.

A more objective take would be simply that "they don't know better." Hire a bunch of junior developers right out of college, and throw them in a pool of layers, and they'll think this is how "a real job" is done - they've never seen anything better. It doesn't help that their go-to example of best, most polished websites (imagine google docs) are built by hundreds of engineers, with as many layers.

Once it becomes an ingrained culture, it would be extremely hard to turn around.

[tkiolp4]

I think it’s exactly “resume driven development” combined with “they don’t know better”. Let’s imagine I am one of those cynic developers and I am faced to work for 2 companies:

- company A is using WordPress to deliver the website OP is talking about. PHP and HTML. I have heard that was very used in the 2000s, but I don’t see any company out there hiring for such skills. Besides all the tech conferences I have attended haven’t talked about PHP at all

- company B is using cool Next.js, GraphQL and Amazon serverless. Now we are talking! Last summer I attended to a GraphQL conference and it was awesome! The host seemed very cool and showed us a demo that looked amazing. Besides, I just checked LinkedIn and there are tons of jobs that require Next.js skills. I think it’s better to work for company B; they seem to know their stuff because they use up to date tools. Also, company B pays way more!

The decision is straightforward, isn’t it?

[tcbasche]

Having worked somewhere that had a very boring tech stack there's a real drive among junior developers to pile on the cool thing they saw at a conference / read in a blog into their current work.

It becomes kind of a zealotry devoid of reality. They've been hired to make the company money by solving problems, and I think that's often the missing piece of the puzzle - that connection with the problem you are solving (and why!)

[swiley]

I've worked on things like this before, sometimes it's more because someone at the top demanded that the app get built that way and everyone building it knows it's a terrible idea but aren't empowered to make fundamental decisions like that.

[air7]

I wonder what people here would consider their best non "resume driven development" tool of choice for something like this?

[runawaybottle]

People have dynamic conceptions of what it means to be a good developer. It started with making composable components which led to an inflation of components in a codebase. Connecting all these little things led to global state management, which led to stuff like Redux, which led to a second abstraction layer on top of composition. Doing this meant you are a good developer.

Now, types in Typescript for every little thing means you are a good developer. Not necessarily judicious use of types and small type footprints, but insane obsessive, compulsive, exhaustive list of types for even the most basic function signature. This is what it means to be a good developer.

What we lack as a group is the ability to have a sit down in the current and say ‘some of this is not needed, we tried it, it’s always worth a trying, but perhaps we can cut down on some of this granular component composition, perhaps we can only use a little bit of typescript, perhaps we can reduce state management’.

We don’t have the mechanism to self-reflect, so our only form of healing is to keeping barreling forward. In other words, lie to ourselves, and run from the problem.

[risk000]

This is beautifully put and very true!

[kilroy123]

I can relate so much to this. I've been doing this since 2010, so over a decade.

Just yesterday, I needed a function that consumed some data from an endpoint, and did one of two things:

- Open a window with a PDF in so the user can print.

- Focus a god damn input field in a form.

That was it.

My manager, who isn't even a "front-end" guy, took my PR. Then changed it to some overly complex mess with an entire separate file. Huge comments. Overly built for a hundred different use cases and did it "the react" way.

We're literately opening up a new window or focusing an input.

I tried to push back but it went nowhere. He's right and I'm wrong. Never mind my 10+ years in the industry. He's the manager.

[nojvek]

That doesn’t seem very healthy team dynamics.

[kyawzazaw]

Your manager is coding and opening PR?

[bjourne]

As production becomes more and more specialized and more and more automated workers lose connection to the product they are producing. A shoemaker cares about the shoes they make because it is their person creation. A worker in a shoe factory who perhaps is just pulling a lever over and over again doesn't care at all about the end result. This is called alienation. Specialization causes workers to become alienated from their work.

We can assume that alienation is rife in the software industry. Someone writing a module for some extension interface for some in house editing tool for the accounting department may not care much. So they prioritize other things. An engineer wants to learn new programming languages because it looks good on their cv and a manager wants to lead large groups because it also looks good on their cv. Thus overcomplicated projects written in newfangled programming languages is in both their best interest. Especially if they are contractors. Then the simplest solution is almost always the wrong one.

[jimmygrapes]

'The specialists are profiting too well from the symptoms, evidently, to be concerned about cures—just as the myth of imminent cure (by some “breakthrough” of science or technology) is so lucrative and all-justifying as to foreclose any possibility of an interest in prevention.'

- Wendell Berry, The Unsettling of America: Culture & Agriculture

[tored]

Great comment.

Specialization led to splitting teams in frontend and backend teams, thus these teams started develop different coding culture & solutions, like separate build steps and whatnots. No longer could one team finish a feature instead it had to be a cross team cooperation if they could sync their schedules that is. If one team was failing their part the other team either couldn't or wouldn't fix it and the delays increased even more.

[jaredcwhite]

It's because some people think the way Google, Microsoft, Facebook, etc. build web pages is considered "best practice" and try to emulate them.

It's actually worst practice for all but the largest companies and teams. We as an industry need to start doing a much, much better job communicating to all relevant parties that the best way to build most web sites/apps most of the time will look radically different from how Big Tech does it, and that's actually expected and reasonable.

[tacostakohashi]

> In addition we seem to be grabbing the PDFs from the server, converting to Base64 and then rendering as an object in the react app (rather than just linking to the pdf file directly).

> What benefit do we get from this complexity compared to just having static HTML pages that someone would update manually?

In the good old days, HTML was simple because it was just text + <h1>, <p>, <b>, etc markup and HTML forms, and web authors were ok / expected that the web browser would do the heavy lifting and render it appropriately for the client OS using native toolkits / widgets.

These days, the publisher wants 100% control over everything, so all that rendering / toolkit that was previously left to the browser is re-invented in JS.

Ask your superiors why you need to embed a PDF viewer in your page instead of just linking the to PDFs and letting the client open it in their local browser / Acrobat / PDF-viewer of choice. They'll probably mumble something about a consistent experience, what if the user doesn't have acrobat installed, etc.

[david927]

You can use SVG which gives you "100% control over everything" and still is composed/rendered using the browser. Anything you see in a PDF can be done with SVG. Yet no one does. Why not?

[chrisrickard]

It all started with XMLHttpRequest (slightly serious).

Ajax spawned "Web 2.0", which in turn spawned the batteries-included-JS-frameworks (which overtook simpler libraries & plugins). Resume driven development then made them the standard, and we now have a generation of programmers who think a GET request only returns JSON, and were never taught it can also return HTML ;)

[AkshitGarg]

> we now have a generation of programmers who think a GET request only returns JSON, and were never taught it can also return HTML

Can't relate more to this. I sometimes find it difficult to accept that some of my peers in the college thought that fetch is the only way to submit a form. Those were the same people who fetched json to render a static blog post

[cblconfederate]

Kafkaesque is the word that describes it pretty well. Every few months i glance at that imposing black mountain of web development and it still makes an ominous impression. I 've stuck with PHP/Jquery and can still build whatever stupid idea comes in my head but always wonder what's behind the door.

> The gateway to the law is open as it always is, and the doorkeeper has stepped to one side, so the man bends over to try and see in. When the doorkeeper notices this he laughs and says, "If you're tempted give it a try, try and go in even though I say you can't. Careful though: I'm powerful. And I'm only the lowliest of all the doormen. But there’s a doorkeeper for each of the rooms and each of them is more powerful than the last. It's more than I can stand just to look at the third one.”

[alex504]

Without a better description of the requirements of the site it is hard to give an answer about your specific situation with regards to what is taking so long.

Technologies have improved, and requirements have gotten more complex. It sounds like the team you are working on is either doing things in an awful way or the requirements are such that there is a lot that needs to be done.

20 years ago it was very difficult to build anything remotely complex. 2-3 people could build a very simple, static site over a few weeks. If you showed web developers 20 years ago what a modern web app looks like today they would be impressed.

It is certainly possible to use a bunch of technologies that complicate things unnecessarily but even then it shouldn't take 9 months for four engineers to build a simple site.

[umbrellaguy]

In the OP: "When I started my career 20 years ago, this is the type of project where maybe 2-3 people would build out over a few weeks."

[ipaddr]

This is the type of project 20 years that was done by two or three people over a week or two.

Now a days it's a few hours.

[david927]

What I've seen is a loss in the ability to architect solutions that are simple. There seems to be a feeling that if doesn't include every technology available that it's missing something.

Good architectures (think both software and construction, both) only use what they need. They are complete not when there is nothing left to add, but nothing left to take away. Good architectures make changes and extensions effortless.

Instead, as Alan Kay says, "Most software today is very much like an Egyptian pyramid with millions of bricks piled on top of each other, with no structural integrity, but just done by brute force and thousands of slaves." He said that a while ago and it's getting worse.

[csomar]

The problem domain has become quite complex. Many people are complaining that developers are "complicating" front-end development but it's the other way around.

HTML has become complex to account for new features that users want. So did CSS (which makes you rely on SASS) and so did JavaScript (and now you complex apps require a strong-typed language like TypeScript).

A few years ago, HTTPS was used by prominent websites only. Now, it's required for even the simplest website. Implementing HTTPS can be simple (if you use Cloudflare) or a bit complex. (if you are running Nginx). But the complexity level did definitively increase.

[midrus]

I agree with most comments here. Just wanted to highlight this is not just a "frontend" thing. I'm seeing the same thing happening in the backend (when overcomplicating everything with Elixir or Go, etc) and infrastructure (Kubernetes, etc...) for what would be a single landing page or a very minimal "CMS" kind of site that with Rails or Django you'd be up and running in less than a week. Instead we spend months with "modern" stacks. Worst part is these stacks have a HUGE amount of custom glue code and what happens is that people eventually leave the company, leaving new joiners with a system which is impossible to understand in a reasonable amount of time. So these new joiners call this old system "legacy" and back to the start. Where if you had used a simpler, well documented, well tested, opinionated, popular solution (rails/django/laravel/etc) you'd at least have a good baseline despite any mess developers could create on top of it.

At my last job I was horribly looked down at when suggesting doing simpler things. People want to master a single gold hammer and use it for everything, no matter what the business problem is. React is THE WAY, Go is THE WAY, kubernetes is THE WAY and now it doesn't matter (we don't even think about it!! what's the actual f**ng business problem we're trying to solve). We just use THE WAY tools because otherwise we're doing "legacy".

It's sad, really sad.

[tcbasche]

> documented, well tested, opinionated, popular solution

The problem I found is when less experienced developers build hacks in and then all of the above attributes fly out the window. All the custom glue code kinda renders the use of a framework more of a hindrance.

But yes, fully agree. Don't start with the technology, start with the problem that is to be solved.

[is_true]

I know a bunch of people that can build web apps in react and they didn't know how to write html in a text editor. They all looked at me like I was some kind of wizard.

They all worked in marketing an their company had them do a react course.

[anakaine]

I find it difficult to believe that those learning javascript to build sites wouldn't realise that html could be edited in a text editor.

[BitwiseFool]

I'm not a web-dev and I have nothing to add, except how crazy it is that something as seemingly simple as a header, three columns, and a footer is so difficult. That, and centering things on a page.

https://en.wikipedia.org/wiki/Holy_grail_(web_design)

I'm kinda glad I'm back-end only.

[tcbasche]

I don't think this is an issue anymore. It's really straight-forward with flexboxes or a grid.

[ryalb]

I think Business decisions are less and less in touch with the technical realities.

I just quit a full time job in a Huge-Non Profit where my main project was to make a simple CRM, based on about 10 simple but intertwined tables. This CRM would have saved the hours of hundreds of overworked/burned out people.

The frontend was made entirely with Bootstrap. A functional, life-saving MVP without any dynamic content could have been made in a month with a team of 3 developers.

However, all of the feedback we constantly got was only to make it look more modern and dynamic (and debatably worse from a UX standpoint).

Mid-development we were forced to add Vue.js to our tech stack. I quit because my conflicts with the business side were becoming hopeless, the project has still not seen an MVP, 1.5 years and two avoidable burnouts in the making.

My thinking is that people feel the need to deliver something as modern and sexy as possible to get more credit. Completely losing track of the functional aspects of things.

[cosmodisk]

>The frontend was made entirely with Bootstrap. A functional, life-saving MVP without any dynamic content could have been made in a month with a team of 3 developers.

The correct answer should have been to go with Salesforce Non profit cloud, or select one of many competitors. To develop a CRM from scratch is always a bad idea.

[sbacic]

I think it comes down to one of two things; the first is ignorance. If the only tool you have is a hammer, everything starts to look like a nail. If you only know about React and Sass, any layer beneath that is a complete mystery and not something you'd even consider as a solution because you're not aware of it.

The second, and my preferred theory, is that the modern web stack has not adequately followed increasing user requirements, forcing us to take much longer to develop what seems like trivial new features. It's not that i18n, or larger images or maintaining state in the browser is some huge, insurmountable problem that can't be solved. It's just that our tools are so woefully immature that we spend more time wrestling with them than we do actually working on the features we're being paid to work on.

And for what? For some vague illusion of choice that for the vast majority of people doesn't even matter? I don't care if I use Redux or one of the bazillion different variations on the idea. Ditto for Sass, Tailwind or what have you. The web layer needs to solidify already, give us a decent bedrock to build our apps and websites on and just get out of the way.

[schwartzworld]

I don't think SASS/LESS/SCSS fall in the same category at all. The ability to nest selectors alone makes stylesheets more readable and easier to work on. If the rest of the team likes SASS but you don't, because I guess you are a masochist, you can just write regular CSS without using the SASS features.

Plus sass is a dev dependency. Even if my team wasn't using it, I could write SASS on my machine and compile before committing the code.

[relaunched]

The state of web development (and software development) is what it is because of the nature of many engineers. Ever piece of the code base is now a beautiful abstraction, that is scalable, extensible, configurable and... over-engineered.

There are probably a handful of sites that needs this level of engineering. But, for everyone else, that isn't running at a huge scale, it's hugely taxing.

[ipnon]

A whole generation has grown up floating effervescently a layer too high above the foundational stack. They have never written a website in "raw" HTML.[a] They have never configured Nginx, let alone Apache.[b] They have never read a complete request response, nor are they aware that one of the Ts in HTTP stands for text![c] They have been removed from the Internet.

[a] React, Angular, Vue [b] EC2, Droplets, VPS [c] Request, Axios

[midrus]

I'm amazed some of my younger coworkers don't even know you can submit a form post without making an ajax request or using JavaScript. I did this for a simple landing page we had to add a form to and they were so confused.

[sergiotapia]

I hear you, and we are actually in the beginning stages of replacing our create-react-app with Phoenix server side rendered templates.

We're also going to use esbuild to compile our very minimal alpinejs code. Most interactivity will be built using phoenix liveviews.

We're hitting the eject button on this whole ecosystem. There is a rot in node/npm that I think will never really go away as it's baked into the culture.

[tkiolp4]

Is this a joke, right? Pardon my ignorance but how is that Phoenix server side rendering is any simpler than React client side rendering when we have PHP and HTML that solves the problem just fine?

In a more serious tone: I like to write web apps in Go. I understand the appealing of using Go or Phoenix or React or whatever that smells like new. My decision to use Go is not totally rational, but hey, I like it!

[midrus]

You have to be kidding.

[Graffur]

Developing in React is using HTML, CSS and JavaScript. It's a declarative approach so the programmer states describes the end result and the React library does the work (code) to update to he DOM. This works really well and is not as hard as the comments make it out to be here. You could write all th JS out to achieve the same thing but it would be silly.

It forces the developer to break things into components which is essential for any maintainable UI.

Since React is the the most popular JS library it is easier to hire for.

SPAs that are done well are a better user experience because the page 'loads' once and then feels faster. That is what users expect.

If a company has multiple sites, it makes sense to be consistent with tech across those sites.

It's ready for future requirements. If you didn't use React and you kept getting requirements for the next few years you would be more likely to end up with a ball of mud if you were doing plain HTML and JavaScript.

One part that I agree seems complicated is the build and dev environment.

[ipaddr]

It's a 15 page static site with pdf links. What ball of mud would 15 html pages be?

2 years and a huge team cost this company a million dollars. Doing this in html would take $5,000 at most. Hiring for react is expensive. Hiring for basic html very cheap with large pools of candidates.